[RFC PATCH 00/10] Color Pipeline API w/ VKMS

[RFC PATCH 00/10] Color Pipeline API w/ VKMS

[Harry Wentland]

This is an early RFC set for a color pipeline API, along with a
sample implementation in VKMS. All the key API bits are here, but
I would like to show a larger variety of colorop types, as well
as examples of different possible color pipelines for a given plane.

The first patch is a doc patch that will explain the motivation
and reasoning behind this approach and give an overview over the
API.

IGT tests can be found at
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1

IGT patches are also being sent to the igt-dev mailing list.

libdrm changes to support the new IOCTLs are at
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1

If you prefer a gitlab MR for review you can find it at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5

A slightly different approach for a Color Pipeline API was sent by
Uma Shankar and can be found at
https://patchwork.freedesktop.org/series/123024/

The main difference is that his approach is not introducing a new DRM
core object but instead exposes color pipelines via blob properties.
There are pros and cons to both approaches.

Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>

Harry Wentland (10):
  drm/doc/rfc: Describe why prescriptive color pipeline is needed
  drm/colorop: Introduce new drm_colorop mode object
  drm/colorop: Add TYPE property
  drm/color: Add 1D Curve subtype
  drm/colorop: Add BYPASS property
  drm/colorop: Add NEXT property
  drm/colorop: Add atomic state print for drm_colorop
  drm/colorop: Add new IOCTLs to retrieve drm_colorop objects
  drm/plane: Add COLOR PIPELINE property
  drm/vkms: Add enumerated 1D curve colorop

 Documentation/gpu/rfc/color_pipeline.rst  | 278 ++++++++++++++++++
 drivers/gpu/drm/Makefile                  |   1 +
 drivers/gpu/drm/drm_atomic.c              | 154 ++++++++++
 drivers/gpu/drm/drm_atomic_helper.c       |  12 +
 drivers/gpu/drm/drm_atomic_state_helper.c |   5 +
 drivers/gpu/drm/drm_atomic_uapi.c         | 110 +++++++
 drivers/gpu/drm/drm_colorop.c             | 343 ++++++++++++++++++++++
 drivers/gpu/drm/drm_crtc_internal.h       |   4 +
 drivers/gpu/drm/drm_ioctl.c               |   5 +
 drivers/gpu/drm/drm_mode_config.c         |   7 +
 drivers/gpu/drm/drm_plane_helper.c        |   2 +-
 drivers/gpu/drm/vkms/Makefile             |   3 +-
 drivers/gpu/drm/vkms/vkms_colorop.c       | 108 +++++++
 drivers/gpu/drm/vkms/vkms_composer.c      | 316 ++++++++++++++++++++
 drivers/gpu/drm/vkms/vkms_drv.h           |   4 +
 drivers/gpu/drm/vkms/vkms_plane.c         |   2 +
 include/drm/drm_atomic.h                  |  82 ++++++
 include/drm/drm_atomic_uapi.h             |   3 +
 include/drm/drm_colorop.h                 | 233 +++++++++++++++
 include/drm/drm_mode_config.h             |  18 ++
 include/drm/drm_plane.h                   |  10 +
 include/uapi/drm/drm.h                    |   3 +
 include/uapi/drm/drm_mode.h               |  22 ++
 23 files changed, 1723 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
 create mode 100644 drivers/gpu/drm/drm_colorop.c
 create mode 100644 drivers/gpu/drm/vkms/vkms_colorop.c
 create mode 100644 include/drm/drm_colorop.h

--
2.42.0

[RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
 1 file changed, 278 insertions(+)
 create mode 100644 Documentation/gpu/rfc/color_pipeline.rst

diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
new file mode 100644
index 000000000000..bfa4a8f12087
--- /dev/null
+++ b/Documentation/gpu/rfc/color_pipeline.rst
@@ -0,0 +1,278 @@
+========================
+Linux Color Pipeline API
+========================
+
+What problem are we solving?
+============================
+
+We would like to support pre-, and post-blending complex color transformations
+in order to allow for HW-supported HDR use-cases, as well as to provide support
+to color-managed applications, such as video or image editors.
+
+While it is possible to support an HDR output on HW supporting the Colorspace
+and HDR Metadata drm_connector properties that requires the compositor or
+application to render and compose the content into one final buffer intended for
+display. Doing so is costly.
+
+Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
+operations to support color transformations. These operations are often
+implemented in fixed-function HW and therefore much more power efficient than
+performing similar operations via shaders or CPU.
+
+We would like to make use of this HW functionality to support complex color
+transformations with no, or minimal CPU or shader load.
+
+
+How are other OSes solving this problem?
+========================================
+
+The most widely supported use-cases regard HDR content, whether video or
+gaming.
+
+Most OSes will specify the source content format (color gamut, encoding transfer
+function, and other metadata, such as max and average light levels) to a driver.
+Drivers will then program their fixed-function HW accordingly to map from a
+source content buffer's space to a display's space.
+
+When fixed-function HW is not available the compositor will assemble a shader to
+ask the GPU to perform the transformation from the source content format to the
+display's format.
+
+A compositor's mapping function and a driver's mapping function are usually
+entirely separate concepts. On OSes where a HW vendor has no insight into
+closed-source compositor code such a vendor will tune their color management
+code to visually match the compositor's. On other OSes, where both mapping
+functions are open to an implementer they will ensure both mappings match.
+
+
+Why is Linux different?
+=======================
+
+Unlike other OSes, where there is one compositor for one or more drivers, on
+Linux we have a many-to-many relationship. Many compositors; many drivers.
+In addition each compositor vendor or community has their own view of how
+color management should be done. This is what makes Linux so beautiful.
+
+This means that a HW vendor can now no longer tune their driver to one
+compositor, as tuning it to one will almost inevitably make it look very
+different from another compositor's color mapping.
+
+We need a better solution.
+
+
+Descriptive API
+===============
+
+An API that describes the source and destination colorspaces is a descriptive
+API. It describes the input and output color spaces but does not describe
+how precisely they should be mapped. Such a mapping includes many minute
+design decision that can greatly affect the look of the final result.
+
+It is not feasible to describe such mapping with enough detail to ensure the
+same result from each implementation. In fact, these mappings are a very active
+research area.
+
+
+Prescriptive API
+================
+
+A prescriptive API describes not the source and destination colorspaces. It
+instead prescribes a recipe for how to manipulate pixel values to arrive at the
+desired outcome.
+
+This recipe is generally an order straight-forward operations, with clear
+mathematical definitions, such as 1D LUTs, 3D LUTs, matrices, or other
+operations that can be described in a precise manner.
+
+
+The Color Pipeline API
+======================
+
+HW color management pipelines can significantly differ between HW
+vendors in terms of availability, ordering, and capabilities of HW
+blocks. This makes a common definition of color management blocks and
+their ordering nigh impossible. Instead we are defining an API that
+allows user space to discover the HW capabilities.
+
+
+drm_colorop Object & IOCTLs
+===========================
+
+To support the definition of color pipelines we introduce a new DRM core
+object, a drm_colorop. Individual drm_colorop objects will be chained
+via the NEXT property of a drm_colorop to constitute a color pipeline.
+Each drm_colorop object is unique, i.e., even if multiple color
+pipelines have the same operation they won't share the same drm_colorop
+object to describe that operation.
+
+Just like other DRM objects the drm_colorop objects are discovered via
+IOCTLs:
+
+DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve the
+number of all drm_colorop objects.
+
+DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
+It includes the ID for the colorop object, as well as the plane_id of
+the associated plane. All other values should be registered as
+properties.
+
+Each drm_colorop has three core properties:
+
+TYPE: The type of transformation, such as
+* enumerated curve
+* custom (uniform) 1D LUT
+* 3x3 matrix
+* 3x4 matrix
+* 3D LUT
+* etc.
+
+Depending on the type of transformation other properties will describe
+more details.
+
+BYPASS: A boolean property that can be used to easily put a block into
+bypass mode. While setting other properties might fail atomic check,
+setting the BYPASS property to true should never fail. This allows DRM
+clients to fallback to other methods of color management if an atomic
+check for KMS color operations fails.
+
+NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
+drm_colorop is the last in the chain.
+
+An example of a drm_colorop object might look like one of these::
+
+    Color operation 42
+    ├─ "type": enum {Bypass, 1D curve} = 1D curve
+    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, BT.709, HLG, …} = LUT
+    ├─ "lut_size": immutable range = 4096
+    ├─ "lut_data": blob
+    └─ "next": immutable color operation ID = 43
+
+    Color operation 42
+    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
+    ├─ "lut_size": immutable range = 33
+    ├─ "lut_data": blob
+    └─ "next": immutable color operation ID = 43
+
+    Color operation 42
+    ├─ "type": enum {Bypass, Matrix} = Matrix
+    ├─ "matrix_data": blob
+    └─ "next": immutable color operation ID = 43
+
+
+COLOR_PIPELINE Plane Property
+=============================
+
+Because we don't have existing KMS color properties in the pre-blending
+portion of display pipelines (i.e. on drm_planes) we are introducing
+color pipelines here first. Eventually we'll want to use the same
+concept for the post-blending portion, i.e. drm_crtcs.
+
+Color Pipelines are created by a driver and advertised via a new
+COLOR_PIPELINE enum property on each plane. Values of the property
+always include '0', which is the default and means all color processing
+is disabled. Additional values will be the object IDs of the first
+drm_colorop in a pipeline. A driver can create and advertise none, one,
+or more possible color pipelines. A DRM client will select a color
+pipeline by setting the COLOR PIPELINE to the respective value.
+
+In the case where drivers have custom support for pre-blending color
+processing those drivers shall reject atomic commits that are trying to
+set both the custom color properties, as well as the COLOR_PIPELINE
+property.
+
+An example of a COLOR_PIPELINE property on a plane might look like this::
+
+    Plane 10
+    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
+    ├─ …
+    └─ "color_pipeline": enum {0, 42, 52} = 0
+
+
+Color Pipeline Discovery
+========================
+
+A DRM client wanting color management on a drm_plane will:
+
+1. Read all drm_colorop objects
+2. Get the COLOR_PIPELINE property of the plane
+3. iterate all COLOR_PIPELINE enum values
+4. for each enum value walk the color pipeline (via the NEXT pointers)
+   and see if the available color operations are suitable for the
+   desired color management operations
+
+An example of chained properties to define an AMD pre-blending color
+pipeline might look like this::
+
+    Plane 10
+    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
+    └─ "color_pipeline": enum {0, 42} = 0
+    Color operation 42 (input CSC)
+    ├─ "type": enum {Bypass, Matrix} = Matrix
+    ├─ "matrix_data": blob
+    └─ "next": immutable color operation ID = 43
+    Color operation 43
+    ├─ "type": enum {Scaling} = Scaling
+    └─ "next": immutable color operation ID = 44
+    Color operation 44 (DeGamma)
+    ├─ "type": enum {Bypass, 1D curve} = 1D curve
+    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
+    └─ "next": immutable color operation ID = 45
+    Color operation 45 (gamut remap)
+    ├─ "type": enum {Bypass, Matrix} = Matrix
+    ├─ "matrix_data": blob
+    └─ "next": immutable color operation ID = 46
+    Color operation 46 (shaper LUT RAM)
+    ├─ "type": enum {Bypass, 1D curve} = 1D curve
+    ├─ "1d_curve_type": enum {LUT} = LUT
+    ├─ "lut_size": immutable range = 4096
+    ├─ "lut_data": blob
+    └─ "next": immutable color operation ID = 47
+    Color operation 47 (3D LUT RAM)
+    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
+    ├─ "lut_size": immutable range = 17
+    ├─ "lut_data": blob
+    └─ "next": immutable color operation ID = 48
+    Color operation 48 (blend gamma)
+    ├─ "type": enum {Bypass, 1D curve} = 1D curve
+    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
+    ├─ "lut_size": immutable range = 4096
+    ├─ "lut_data": blob
+    └─ "next": immutable color operation ID = 0
+
+
+Color Pipeline Programming
+==========================
+
+Once a DRM client has found a suitable pipeline it will:
+
+1. Set the COLOR_PIPELINE enum value to the one pointing at the first
+   drm_colorop object of the desired pipeline
+2. Set the properties for all drm_colorop objects in the pipeline to the
+   desired values, setting BYPASS to true for unused drm_colorop blocks,
+   and false for enabled drm_colorop blocks
+3. Perform atomic_check/commit as desired
+
+To configure the pipeline for an HDR10 PQ plane and blending in linear
+space, a compositor might perform an atomic commit with the following
+property values::
+
+    Plane 10
+    └─ "color_pipeline" = 42
+    Color operation 42 (input CSC)
+    └─ "bypass" = true
+    Color operation 44 (DeGamma)
+    └─ "bypass" = true
+    Color operation 45 (gamut remap)
+    └─ "bypasse" = true
+    Color operation 46 (shaper LUT RAM)
+    └─ "bypass" = true
+    Color operation 47 (3D LUT RAM)
+    └─ "lut_data" = Gamut mapping + tone mapping + night mode
+    Color operation 48 (blend gamma)
+    └─ "1d_curve_type" = PQ inverse EOTF
+
+
+References
+==========
+
+1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
\ No newline at end of file
-- 
2.42.0

[RFC PATCH 02/10] drm/colorop: Introduce new drm_colorop mode object

[Harry Wentland]

This patches introduces a new drm_colorop mode object. This
object represents color transformations and can be used to
define color pipelines.

We also introduce the drm_colorop_state here, as well as
various helpers and state tracking bits.

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/Makefile            |   1 +
 drivers/gpu/drm/drm_atomic.c        |  79 +++++++++++++
 drivers/gpu/drm/drm_atomic_helper.c |  12 ++
 drivers/gpu/drm/drm_atomic_uapi.c   |  48 ++++++++
 drivers/gpu/drm/drm_colorop.c       | 169 ++++++++++++++++++++++++++++
 drivers/gpu/drm/drm_mode_config.c   |   7 ++
 drivers/gpu/drm/drm_plane_helper.c  |   2 +-
 include/drm/drm_atomic.h            |  82 ++++++++++++++
 include/drm/drm_atomic_uapi.h       |   1 +
 include/drm/drm_colorop.h           | 157 ++++++++++++++++++++++++++
 include/drm/drm_mode_config.h       |  18 +++
 include/drm/drm_plane.h             |   2 +
 include/uapi/drm/drm.h              |   3 +
 include/uapi/drm/drm_mode.h         |   1 +
 14 files changed, 581 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/drm_colorop.c
 create mode 100644 include/drm/drm_colorop.h

diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 1855863b4d7a..941de0269709 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -16,6 +16,7 @@ drm-y := \
 	drm_client.o \
 	drm_client_modeset.o \
 	drm_color_mgmt.o \
+	drm_colorop.o \
 	drm_connector.o \
 	drm_crtc.o \
 	drm_displayid.o \
diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 11f3a130f6f4..d734e9d5bfed 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -42,6 +42,7 @@
 #include <drm/drm_mode.h>
 #include <drm/drm_print.h>
 #include <drm/drm_writeback.h>
+#include <drm/drm_colorop.h>
 
 #include "drm_crtc_internal.h"
 #include "drm_internal.h"
@@ -108,6 +109,7 @@ void drm_atomic_state_default_release(struct drm_atomic_state *state)
 	kfree(state->connectors);
 	kfree(state->crtcs);
 	kfree(state->planes);
+	kfree(state->colorops);
 	kfree(state->private_objs);
 }
 EXPORT_SYMBOL(drm_atomic_state_default_release);
@@ -139,6 +141,10 @@ drm_atomic_state_init(struct drm_device *dev, struct drm_atomic_state *state)
 				sizeof(*state->planes), GFP_KERNEL);
 	if (!state->planes)
 		goto fail;
+	state->colorops = kcalloc(dev->mode_config.num_colorop,
+				  sizeof(*state->colorops), GFP_KERNEL);
+	if (!state->colorops)
+		goto fail;
 
 	state->dev = dev;
 
@@ -244,6 +250,20 @@ void drm_atomic_state_default_clear(struct drm_atomic_state *state)
 		state->planes[i].new_state = NULL;
 	}
 
+	for (i = 0; i < config->num_colorop; i++) {
+		struct drm_colorop *colorop = state->colorops[i].ptr;
+
+		if (!colorop)
+			continue;
+
+		drm_colorop_atomic_destroy_state(colorop,
+						 state->colorops[i].state);
+		state->colorops[i].ptr = NULL;
+		state->colorops[i].state = NULL;
+		state->colorops[i].old_state = NULL;
+		state->colorops[i].new_state = NULL;
+	}
+
 	for (i = 0; i < state->num_private_objs; i++) {
 		struct drm_private_obj *obj = state->private_objs[i].ptr;
 
@@ -562,6 +582,65 @@ drm_atomic_get_plane_state(struct drm_atomic_state *state,
 }
 EXPORT_SYMBOL(drm_atomic_get_plane_state);
 
+
+/**
+ * drm_atomic_get_colorop_state - get colorop state
+ * @state: global atomic state object
+ * @colorop: colorop to get state object for
+ *
+ * This function returns the colorop state for the given colorop, allocating it
+ * if needed. It will also grab the relevant plane lock to make sure that the
+ * state is consistent.
+ *
+ * Returns:
+ *
+ * Either the allocated state or the error code encoded into the pointer. When
+ * the error is EDEADLK then the w/w mutex code has detected a deadlock and the
+ * entire atomic sequence must be restarted. All other errors are fatal.
+ */
+struct drm_colorop_state *
+drm_atomic_get_colorop_state(struct drm_atomic_state *state,
+			     struct drm_colorop *colorop)
+{
+	int ret, index = drm_colorop_index(colorop);
+	struct drm_colorop_state *colorop_state;
+	struct drm_plane_state *plane_state;
+
+	WARN_ON(!state->acquire_ctx);
+
+	colorop_state = drm_atomic_get_existing_colorop_state(state, colorop);
+	if (colorop_state)
+		return colorop_state;
+
+	/* TODO where is the unlock? */
+	ret = drm_modeset_lock(&colorop->plane->mutex, state->acquire_ctx);
+	if (ret)
+		return ERR_PTR(ret);
+
+	colorop_state = drm_atomic_helper_colorop_duplicate_state(colorop);
+	if (!colorop_state)
+		return ERR_PTR(-ENOMEM);
+
+	state->colorops[index].state = colorop_state;
+	state->colorops[index].ptr = colorop;
+	state->colorops[index].old_state = colorop->state;
+	state->colorops[index].new_state = colorop_state;
+	colorop_state->state = state;
+
+	drm_dbg_atomic(colorop->dev, "Added [COLOROP:%d] %p state to %p\n",
+		       colorop->base.id, colorop_state, state);
+
+	/* TODO is this necessary? */
+
+	plane_state = drm_atomic_get_plane_state(state,
+						 colorop->plane);
+	if (IS_ERR(plane_state))
+		return ERR_CAST(plane_state);
+
+	return colorop_state;
+}
+EXPORT_SYMBOL(drm_atomic_get_colorop_state);
+
 static bool
 plane_switching_crtc(const struct drm_plane_state *old_plane_state,
 		     const struct drm_plane_state *new_plane_state)
diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
index d579fd8f7cb8..0472f6182c0a 100644
--- a/drivers/gpu/drm/drm_atomic_helper.c
+++ b/drivers/gpu/drm/drm_atomic_helper.c
@@ -2921,6 +2921,8 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 	struct drm_crtc_state *old_crtc_state, *new_crtc_state;
 	struct drm_plane *plane;
 	struct drm_plane_state *old_plane_state, *new_plane_state;
+	struct drm_colorop *colorop;
+	struct drm_colorop_state *old_colorop_state, *new_colorop_state;
 	struct drm_crtc_commit *commit;
 	struct drm_private_obj *obj;
 	struct drm_private_state *old_obj_state, *new_obj_state;
@@ -2998,6 +3000,16 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 		}
 	}
 
+	for_each_oldnew_colorop_in_state(state, colorop, old_colorop_state, new_colorop_state, i) {
+		WARN_ON(colorop->state != old_colorop_state);
+
+		old_colorop_state->state = state;
+		new_colorop_state->state = NULL;
+
+		state->colorops[i].state = old_colorop_state;
+		colorop->state = new_colorop_state;
+	}
+
 	for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
 		WARN_ON(plane->state != old_plane_state);
 
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index d867e7f9f2cd..b1aa752c1848 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -34,6 +34,7 @@
 #include <drm/drm_drv.h>
 #include <drm/drm_writeback.h>
 #include <drm/drm_vblank.h>
+#include <drm/drm_colorop.h>
 
 #include <linux/dma-fence.h>
 #include <linux/uaccess.h>
@@ -642,6 +643,26 @@ drm_atomic_plane_get_property(struct drm_plane *plane,
 	return 0;
 }
 
+
+static int drm_atomic_colorop_set_property(struct drm_colorop *colorop,
+		struct drm_colorop_state *state, struct drm_file *file_priv,
+		struct drm_property *property, uint64_t val)
+{
+	drm_dbg_atomic(colorop->dev,
+			"[COLOROP:%d] unknown property [PROP:%d:%s]]\n",
+			colorop->base.id,
+			property->base.id, property->name);
+	return -EINVAL;
+}
+
+static int
+drm_atomic_colorop_get_property(struct drm_colorop *colorop,
+		const struct drm_colorop_state *state,
+		struct drm_property *property, uint64_t *val)
+{
+	return -EINVAL;
+}
+
 static int drm_atomic_set_writeback_fb_for_connector(
 		struct drm_connector_state *conn_state,
 		struct drm_framebuffer *fb)
@@ -893,6 +914,16 @@ int drm_atomic_get_property(struct drm_mode_object *obj,
 				plane->state, property, val);
 		break;
 	}
+	case DRM_MODE_OBJECT_COLOROP: {
+		struct drm_colorop *colorop = obj_to_colorop(obj);
+
+		if (colorop->plane)
+			WARN_ON(!drm_modeset_is_locked(&colorop->plane->mutex));
+
+		ret = drm_atomic_colorop_get_property(colorop,
+				colorop->state, property, val);
+		break;
+	}
 	default:
 		ret = -EINVAL;
 		break;
@@ -1027,6 +1058,23 @@ int drm_atomic_set_property(struct drm_atomic_state *state,
 		ret = drm_atomic_plane_set_property(plane,
 				plane_state, file_priv,
 				prop, prop_value);
+
+		break;
+	}
+	case DRM_MODE_OBJECT_COLOROP: {
+		struct drm_colorop *colorop = obj_to_colorop(obj);
+		struct drm_colorop_state *colorop_state;
+
+		colorop_state = drm_atomic_get_colorop_state(state, colorop);
+		if (IS_ERR(colorop_state)) {
+			ret = PTR_ERR(colorop_state);
+			break;
+		}
+
+		ret = drm_atomic_colorop_set_property(colorop,
+				colorop_state, file_priv,
+				prop, prop_value);
+
 		break;
 	}
 	default:
diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
new file mode 100644
index 000000000000..78d6a0067f5b
--- /dev/null
+++ b/drivers/gpu/drm/drm_colorop.c
@@ -0,0 +1,169 @@
+/*
+ * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#include <drm/drm_colorop.h>
+#include <drm/drm_print.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_plane.h>
+
+#include "drm_crtc_internal.h"
+
+/* TODO big colorop doc, including properties, etc. */
+
+/* Init Helpers */
+
+int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
+		     struct drm_plane *plane)
+{
+	struct drm_mode_config *config = &dev->mode_config;
+	int ret = 0;
+
+	ret = drm_mode_object_add(dev, &colorop->base, DRM_MODE_OBJECT_COLOROP);
+	if (ret)
+		return ret;
+
+	colorop->base.properties = &colorop->properties;
+	colorop->dev = dev;
+	colorop->plane = plane;
+
+	list_add_tail(&colorop->head, &config->colorop_list);
+	colorop->index = config->num_colorop++;
+
+	/* add properties */
+	return ret;
+}
+EXPORT_SYMBOL(drm_colorop_init);
+
+void __drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop,
+						 struct drm_colorop_state *state)
+{
+	memcpy(state, colorop->state, sizeof(*state));
+}
+
+struct drm_colorop_state *
+drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop)
+{
+	struct drm_colorop_state *state;
+
+	if (WARN_ON(!colorop->state))
+		return NULL;
+
+	state = kmalloc(sizeof(*state), GFP_KERNEL);
+	if (state)
+		__drm_atomic_helper_colorop_duplicate_state(colorop, state);
+
+	return state;
+}
+
+
+void drm_colorop_atomic_destroy_state(struct drm_colorop *colorop,
+				      struct drm_colorop_state *state)
+{
+	kfree(state);
+}
+
+/**
+ * __drm_colorop_destroy_state - release colorop state
+ * @state: colorop state object to release
+ *
+ * Releases all resources stored in the colorop state without actually freeing
+ * the memory of the colorop state. This is useful for drivers that subclass the
+ * colorop state.
+ */
+void __drm_colorop_destroy_state(struct drm_colorop_state *state)
+{
+	/* TODO might need this later */
+}
+
+/**
+ * drm_colorop_destroy_state - default state destroy hook
+ * @colorop: drm colorop
+ * @state: colorop state object to release
+ *
+ * Default colorop state destroy hook for drivers which don't have their own
+ * subclassed colorop state structure.
+ */
+void drm_colorop_destroy_state(struct drm_colorop *colorop,
+					   struct drm_colorop_state *state)
+{
+	kfree(state);
+}
+EXPORT_SYMBOL(drm_colorop_destroy_state);
+
+/**
+ * __drm_colorop_state_reset - resets colorop state to default values
+ * @colorop_state: atomic colorop state, must not be NULL
+ * @colorop: colorop object, must not be NULL
+ *
+ * Initializes the newly allocated @colorop_state with default
+ * values. This is useful for drivers that subclass the CRTC state.
+ */
+void __drm_colorop_state_reset(struct drm_colorop_state *colorop_state,
+					   struct drm_colorop *colorop)
+{
+	colorop_state->colorop = colorop;
+}
+EXPORT_SYMBOL(__drm_colorop_state_reset);
+
+/**
+ * __drm_colorop_reset - reset state on colorop
+ * @colorop: drm colorop
+ * @colorop_state: colorop state to assign
+ *
+ * Initializes the newly allocated @colorop_state and assigns it to
+ * the &drm_crtc->state pointer of @colorop, usually required when
+ * initializing the drivers or when called from the &drm_colorop_funcs.reset
+ * hook.
+ *
+ * This is useful for drivers that subclass the colorop state.
+ */
+void __drm_colorop_reset(struct drm_colorop *colorop,
+				     struct drm_colorop_state *colorop_state)
+{
+	if (colorop_state)
+		__drm_colorop_state_reset(colorop_state, colorop);
+
+	colorop->state = colorop_state;
+}
+
+/**
+ * drm_colorop_reset - reset colorop atomic state
+ * @colorop: drm colorop
+ *
+ * Resets the atomic state for @colorop by freeing the state pointer (which might
+ * be NULL, e.g. at driver load time) and allocating a new empty state object.
+ */
+void drm_colorop_reset(struct drm_colorop *colorop)
+{
+	if (colorop->state)
+		__drm_colorop_destroy_state(colorop->state);
+
+	kfree(colorop->state);
+	colorop->state = kzalloc(sizeof(*colorop->state), GFP_KERNEL);
+
+	if (colorop->state)
+		__drm_colorop_reset(colorop, colorop->state);
+}
+EXPORT_SYMBOL(drm_colorop_reset);
diff --git a/drivers/gpu/drm/drm_mode_config.c b/drivers/gpu/drm/drm_mode_config.c
index 87eb591fe9b5..5bdcf71e1ae0 100644
--- a/drivers/gpu/drm/drm_mode_config.c
+++ b/drivers/gpu/drm/drm_mode_config.c
@@ -29,6 +29,7 @@
 #include <drm/drm_managed.h>
 #include <drm/drm_mode_config.h>
 #include <drm/drm_print.h>
+#include <drm/drm_colorop.h>
 #include <linux/dma-resv.h>
 
 #include "drm_crtc_internal.h"
@@ -184,11 +185,15 @@ int drm_mode_getresources(struct drm_device *dev, void *data,
 void drm_mode_config_reset(struct drm_device *dev)
 {
 	struct drm_crtc *crtc;
+	struct drm_colorop *colorop;
 	struct drm_plane *plane;
 	struct drm_encoder *encoder;
 	struct drm_connector *connector;
 	struct drm_connector_list_iter conn_iter;
 
+	drm_for_each_colorop(colorop, dev)
+		drm_colorop_reset(colorop);
+
 	drm_for_each_plane(plane, dev)
 		if (plane->funcs->reset)
 			plane->funcs->reset(plane);
@@ -415,6 +420,7 @@ int drmm_mode_config_init(struct drm_device *dev)
 	INIT_LIST_HEAD(&dev->mode_config.property_list);
 	INIT_LIST_HEAD(&dev->mode_config.property_blob_list);
 	INIT_LIST_HEAD(&dev->mode_config.plane_list);
+	INIT_LIST_HEAD(&dev->mode_config.colorop_list);
 	INIT_LIST_HEAD(&dev->mode_config.privobj_list);
 	idr_init_base(&dev->mode_config.object_idr, 1);
 	idr_init_base(&dev->mode_config.tile_idr, 1);
@@ -436,6 +442,7 @@ int drmm_mode_config_init(struct drm_device *dev)
 	dev->mode_config.num_crtc = 0;
 	dev->mode_config.num_encoder = 0;
 	dev->mode_config.num_total_plane = 0;
+	dev->mode_config.num_colorop = 0;
 
 	if (IS_ENABLED(CONFIG_LOCKDEP)) {
 		struct drm_modeset_acquire_ctx modeset_ctx;
diff --git a/drivers/gpu/drm/drm_plane_helper.c b/drivers/gpu/drm/drm_plane_helper.c
index c91e454eba09..69faa0eeb27f 100644
--- a/drivers/gpu/drm/drm_plane_helper.c
+++ b/drivers/gpu/drm/drm_plane_helper.c
@@ -318,4 +318,4 @@ int drm_plane_helper_atomic_check(struct drm_plane *plane, struct drm_atomic_sta
 						   DRM_PLANE_NO_SCALING,
 						   false, false);
 }
-EXPORT_SYMBOL(drm_plane_helper_atomic_check);
+EXPORT_SYMBOL(drm_plane_helper_atomic_check);
\ No newline at end of file
diff --git a/include/drm/drm_atomic.h b/include/drm/drm_atomic.h
index 92586ab55ef5..dca8fc26ad71 100644
--- a/include/drm/drm_atomic.h
+++ b/include/drm/drm_atomic.h
@@ -30,6 +30,7 @@
 
 #include <drm/drm_crtc.h>
 #include <drm/drm_util.h>
+#include <drm/drm_colorop.h>
 
 /**
  * struct drm_crtc_commit - track modeset commits on a CRTC
@@ -157,6 +158,11 @@ struct drm_crtc_commit {
 	bool abort_completion;
 };
 
+struct __drm_colorops_state {
+	struct drm_colorop *ptr;
+	struct drm_colorop_state *state, *old_state, *new_state;
+};
+
 struct __drm_planes_state {
 	struct drm_plane *ptr;
 	struct drm_plane_state *state, *old_state, *new_state;
@@ -398,6 +404,7 @@ struct drm_atomic_state {
 	 * states.
 	 */
 	bool duplicated : 1;
+	struct __drm_colorops_state *colorops;
 	struct __drm_planes_state *planes;
 	struct __drm_crtcs_state *crtcs;
 	int num_connector;
@@ -501,6 +508,9 @@ drm_atomic_get_crtc_state(struct drm_atomic_state *state,
 struct drm_plane_state * __must_check
 drm_atomic_get_plane_state(struct drm_atomic_state *state,
 			   struct drm_plane *plane);
+struct drm_colorop_state *
+drm_atomic_get_colorop_state(struct drm_atomic_state *state,
+			     struct drm_colorop *colorop);
 struct drm_connector_state * __must_check
 drm_atomic_get_connector_state(struct drm_atomic_state *state,
 			       struct drm_connector *connector);
@@ -623,6 +633,55 @@ drm_atomic_get_new_plane_state(const struct drm_atomic_state *state,
 	return state->planes[drm_plane_index(plane)].new_state;
 }
 
+
+/**
+ * drm_atomic_get_existing_colorop_state - get colorop state, if it exists
+ * @state: global atomic state object
+ * @colorop: colorop to grab
+ *
+ * This function returns the colorop state for the given colorop, or NULL
+ * if the colorop is not part of the global atomic state.
+ *
+ * This function is deprecated, @drm_atomic_get_old_colorop_state or
+ * @drm_atomic_get_new_colorop_state should be used instead.
+ */
+static inline struct drm_colorop_state *
+drm_atomic_get_existing_colorop_state(struct drm_atomic_state *state,
+				    struct drm_colorop *colorop)
+{
+	return state->colorops[drm_colorop_index(colorop)].state;
+}
+
+/**
+ * drm_atomic_get_old_colorop_state - get colorop state, if it exists
+ * @state: global atomic state object
+ * @colorop: colorop to grab
+ *
+ * This function returns the old colorop state for the given colorop, or
+ * NULL if the colorop is not part of the global atomic state.
+ */
+static inline struct drm_colorop_state *
+drm_atomic_get_old_colorop_state(struct drm_atomic_state *state,
+			       struct drm_colorop *colorop)
+{
+	return state->colorops[drm_colorop_index(colorop)].old_state;
+}
+
+/**
+ * drm_atomic_get_new_colorop_state - get colorop state, if it exists
+ * @state: global atomic state object
+ * @colorop: colorop to grab
+ *
+ * This function returns the new colorop state for the given colorop, or
+ * NULL if the colorop is not part of the global atomic state.
+ */
+static inline struct drm_colorop_state *
+drm_atomic_get_new_colorop_state(struct drm_atomic_state *state,
+			       struct drm_colorop *colorop)
+{
+	return state->colorops[drm_colorop_index(colorop)].new_state;
+}
+
 /**
  * drm_atomic_get_existing_connector_state - get connector state, if it exists
  * @state: global atomic state object
@@ -870,6 +929,29 @@ void drm_state_dump(struct drm_device *dev, struct drm_printer *p);
 			     (new_crtc_state) = (__state)->crtcs[__i].new_state, \
 			     (void)(new_crtc_state) /* Only to avoid unused-but-set-variable warning */, 1))
 
+/**
+ * for_each_oldnew_colorop_in_state - iterate over all colorops in an atomic update
+ * @__state: &struct drm_atomic_state pointer
+ * @colorop: &struct drm_colorop iteration cursor
+ * @old_colorop_state: &struct drm_colorop_state iteration cursor for the old state
+ * @new_colorop_state: &struct drm_colorop_state iteration cursor for the new state
+ * @__i: int iteration cursor, for macro-internal use
+ *
+ * This iterates over all colorops in an atomic update, tracking both old and
+ * new state. This is useful in places where the state delta needs to be
+ * considered, for example in atomic check functions.
+ */
+#define for_each_oldnew_colorop_in_state(__state, colorop, old_colorop_state, new_colorop_state, __i) \
+	for ((__i) = 0;							\
+	     (__i) < (__state)->dev->mode_config.num_colorop;	\
+	     (__i)++)							\
+		for_each_if ((__state)->colorops[__i].ptr &&		\
+			     ((colorop) = (__state)->colorops[__i].ptr,	\
+			      (void)(colorop) /* Only to avoid unused-but-set-variable warning */, \
+			      (old_colorop_state) = (__state)->colorops[__i].old_state,\
+			      (new_colorop_state) = (__state)->colorops[__i].new_state, 1))
+
+
 /**
  * for_each_oldnew_plane_in_state - iterate over all planes in an atomic update
  * @__state: &struct drm_atomic_state pointer
diff --git a/include/drm/drm_atomic_uapi.h b/include/drm/drm_atomic_uapi.h
index 4c6d39d7bdb2..70a115d523cd 100644
--- a/include/drm/drm_atomic_uapi.h
+++ b/include/drm/drm_atomic_uapi.h
@@ -37,6 +37,7 @@ struct drm_crtc;
 struct drm_connector_state;
 struct dma_fence;
 struct drm_framebuffer;
+struct drm_colorop;
 
 int __must_check
 drm_atomic_set_mode_for_crtc(struct drm_crtc_state *state,
diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
new file mode 100644
index 000000000000..3dd169b0317d
--- /dev/null
+++ b/include/drm/drm_colorop.h
@@ -0,0 +1,157 @@
+/*
+ * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef __DRM_COLOROP_H__
+#define __DRM_COLOROP_H__
+
+#include <drm/drm_mode_object.h>
+#include <drm/drm_mode.h>
+#include <drm/drm_property.h>
+
+/**
+ * struct drm_colorop_state - mutable colorop state
+ */
+struct drm_colorop_state {
+	/** @colorop: backpointer to the colorop */
+	struct drm_colorop *colorop;
+
+	/* colorop properties */
+
+	/** @state: backpointer to global drm_atomic_state */
+	struct drm_atomic_state *state;
+};
+
+/**
+ * struct drm_colorop - DRM color operation control structure
+ *
+ * A colorop represents one color operation. They can be chained via
+ * the 'next' pointer to build a color pipeline.
+ */
+struct drm_colorop {
+	/** @dev: parent DRM device */
+	struct drm_device *dev;
+
+	/**
+	 * @head:
+	 *
+	 * List of all colorops on @dev, linked from &drm_mode_config.colorop_list.
+	 * Invariant over the lifetime of @dev and therefore does not need
+	 * locking.
+	 */
+	struct list_head head;
+
+	/**
+	 * @index: Position inside the mode_config.list, can be used as an array
+	 * index. It is invariant over the lifetime of the plane.
+	 */
+	unsigned index;
+
+	/* TODO do we need a separate mutex or will we tag along with the plane mutex? */
+
+	/** @base base mode object*/
+	struct drm_mode_object base;
+
+	/**
+	 * @plane:
+	 *
+	 * The plane on which the colorop sits. A drm_colorop is always unique
+	 * to a plane.
+	 */
+	struct drm_plane *plane;
+
+	/**
+	 * @state:
+	 *
+	 * Current atomic state for this colorop.
+	 *
+	 * This is protected by @mutex. Note that nonblocking atomic commits
+	 * access the current colorop state without taking locks. Either by
+	 * going through the &struct drm_atomic_state pointers, see
+	 * for_each_oldnew_plane_in_state(), for_each_old_plane_in_state() and
+	 * for_each_new_plane_in_state(). Or through careful ordering of atomic
+	 * commit operations as implemented in the atomic helpers, see
+	 * &struct drm_crtc_commit.
+	 *
+	 * TODO keep, remove, or rewrite above plane references?
+	 */
+	struct drm_colorop_state *state;
+
+	/* colorop properties */
+
+	/** @properties: property tracking for this plane */
+	struct drm_object_properties properties;
+
+};
+
+#define obj_to_colorop(x) container_of(x, struct drm_colorop, base)
+
+/**
+ * drm_crtc_find - look up a Colorop object from its ID
+ * @dev: DRM device
+ * @file_priv: drm file to check for lease against.
+ * @id: &drm_mode_object ID
+ *
+ * This can be used to look up a Colorop from its userspace ID. Only used by
+ * drivers for legacy IOCTLs and interface, nowadays extensions to the KMS
+ * userspace interface should be done using &drm_property.
+ */
+static inline struct drm_colorop *drm_colorop_find(struct drm_device *dev,
+		struct drm_file *file_priv,
+		uint32_t id)
+{
+	struct drm_mode_object *mo;
+	mo = drm_mode_object_find(dev, file_priv, id, DRM_MODE_OBJECT_COLOROP);
+	return mo ? obj_to_colorop(mo) : NULL;
+}
+
+int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
+		     struct drm_plane *plane);
+
+struct drm_colorop_state *
+drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop);
+
+void drm_colorop_atomic_destroy_state(struct drm_colorop *colorop,
+				      struct drm_colorop_state *state);
+
+void drm_colorop_reset(struct drm_colorop *colorop);
+
+/**
+ * drm_colorop_index - find the index of a registered colorop
+ * @colorop: colorop to find index for
+ *
+ * Given a registered colorop, return the index of that colorop within a DRM
+ * device's list of colorops.
+ */
+static inline unsigned int drm_colorop_index(const struct drm_colorop *colorop)
+{
+	return colorop->index;
+}
+
+
+#define drm_for_each_colorop(colorop, dev) \
+	list_for_each_entry(colorop, &(dev)->mode_config.colorop_list, head)
+
+
+#endif /* __DRM_COLOROP_H__ */
diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h
index e5b053001d22..f56d21d93cf0 100644
--- a/include/drm/drm_mode_config.h
+++ b/include/drm/drm_mode_config.h
@@ -505,6 +505,24 @@ struct drm_mode_config {
 	 */
 	struct list_head plane_list;
 
+	/**
+	 * @num_colorop:
+	 *
+	 * Number of colorop objects on this device.
+	 * This is invariant over the lifetime of a device and hence doesn't
+	 * need any locks.
+	 */
+	int num_colorop;
+
+	/**
+	 * @colorops_list:
+	 *
+	 * List of colorop objects linked with &drm_colorop.head. This is
+	 * invariant over the lifetime of a device and hence doesn't need any
+	 * locks.
+	 */
+	struct list_head colorop_list;
+
 	/**
 	 * @num_crtc:
 	 *
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index 51291983ea44..cfb8f46b94ab 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -227,6 +227,8 @@ struct drm_plane_state {
 	 */
 	enum drm_scaling_filter scaling_filter;
 
+	struct drm_colorop *color_pipeline;
+
 	/**
 	 * @commit: Tracks the pending commit to prevent use-after-free conditions,
 	 * and for async plane updates.
diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h
index 642808520d92..dec498a44eae 100644
--- a/include/uapi/drm/drm.h
+++ b/include/uapi/drm/drm.h
@@ -1116,6 +1116,9 @@ extern "C" {
  */
 #define DRM_IOCTL_MODE_GETFB2		DRM_IOWR(0xCE, struct drm_mode_fb_cmd2)
 
+#define DRM_IOCTL_MODE_GETCOLOROPRESOURCES DRM_IOWR(0xD0, struct drm_mode_get_colorop_res)
+#define DRM_IOCTL_MODE_GETCOLOROP          DRM_IOWR(0xD1, struct drm_mode_get_colorop)
+
 /*
  * Device specific ioctls should only be in their respective headers
  * The device specific ioctl range is from 0x40 to 0x9f.
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index 46becedf5b2f..6dcf628def56 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -626,6 +626,7 @@ struct drm_mode_connector_set_property {
 #define DRM_MODE_OBJECT_FB 0xfbfbfbfb
 #define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb
 #define DRM_MODE_OBJECT_PLANE 0xeeeeeeee
+#define DRM_MODE_OBJECT_COLOROP 0xfafafafa
 #define DRM_MODE_OBJECT_ANY 0
 
 struct drm_mode_obj_get_properties {
-- 
2.42.0

[RFC PATCH 03/10] drm/colorop: Add TYPE property

[Harry Wentland]

Add a read-only TYPE property. The TYPE specifies the colorop
type, such as enumerated curve, 1D LUT, CTM, 3D LUT, PWL LUT,
etc.

For now we're only introducing an enumerated 1D LUT type to
illustrate the concept.

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/drm_atomic.c      |  4 +--
 drivers/gpu/drm/drm_atomic_uapi.c |  8 +++++-
 drivers/gpu/drm/drm_colorop.c     | 44 ++++++++++++++++++++++++++++++-
 include/drm/drm_colorop.h         | 21 ++++++++++++++-
 4 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index d734e9d5bfed..8a5f8cd22c8d 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -627,8 +627,8 @@ drm_atomic_get_colorop_state(struct drm_atomic_state *state,
 	state->colorops[index].new_state = colorop_state;
 	colorop_state->state = state;
 
-	drm_dbg_atomic(colorop->dev, "Added [COLOROP:%d] %p state to %p\n",
-		       colorop->base.id, colorop_state, state);
+	drm_dbg_atomic(colorop->dev, "Added [COLOROP:%d:%d] %p state to %p\n",
+		       colorop->base.id, colorop->type, colorop_state, state);
 
 	/* TODO is this necessary? */
 
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index b1aa752c1848..51072fe2b548 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -660,7 +660,13 @@ drm_atomic_colorop_get_property(struct drm_colorop *colorop,
 		const struct drm_colorop_state *state,
 		struct drm_property *property, uint64_t *val)
 {
-	return -EINVAL;
+	if (property == colorop->type_property) {
+		*val = colorop->type;
+	} else {
+		return -EINVAL;
+	}
+
+	return 0;
 }
 
 static int drm_atomic_set_writeback_fb_for_connector(
diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
index 78d6a0067f5b..c028d5426d42 100644
--- a/drivers/gpu/drm/drm_colorop.c
+++ b/drivers/gpu/drm/drm_colorop.c
@@ -32,12 +32,17 @@
 
 /* TODO big colorop doc, including properties, etc. */
 
+static const struct drm_prop_enum_list drm_colorop_type_enum_list[] = {
+	{ DRM_COLOROP_1D_CURVE, "1D Curve" },
+};
+
 /* Init Helpers */
 
 int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
-		     struct drm_plane *plane)
+		     struct drm_plane *plane, enum drm_colorop_type type)
 {
 	struct drm_mode_config *config = &dev->mode_config;
+	struct drm_property *prop;
 	int ret = 0;
 
 	ret = drm_mode_object_add(dev, &colorop->base, DRM_MODE_OBJECT_COLOROP);
@@ -46,12 +51,28 @@ int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
 
 	colorop->base.properties = &colorop->properties;
 	colorop->dev = dev;
+	colorop->type = type;
 	colorop->plane = plane;
 
 	list_add_tail(&colorop->head, &config->colorop_list);
 	colorop->index = config->num_colorop++;
 
 	/* add properties */
+
+	/* type */
+	prop = drm_property_create_enum(dev,
+					DRM_MODE_PROP_IMMUTABLE | DRM_MODE_PROP_ATOMIC,
+					"TYPE", drm_colorop_type_enum_list,
+					ARRAY_SIZE(drm_colorop_type_enum_list));
+	if (!prop)
+		return -ENOMEM;
+
+	colorop->type_property = prop;
+
+	drm_object_attach_property(&colorop->base,
+				   colorop->type_property,
+				   colorop->type);
+
 	return ret;
 }
 EXPORT_SYMBOL(drm_colorop_init);
@@ -167,3 +188,24 @@ void drm_colorop_reset(struct drm_colorop *colorop)
 		__drm_colorop_reset(colorop, colorop->state);
 }
 EXPORT_SYMBOL(drm_colorop_reset);
+
+
+static const char * const colorop_type_name[] = {
+	[DRM_COLOROP_1D_CURVE] = "1D Curve",
+};
+
+/**
+ * drm_get_colorop_type_name - return a string for colorop type
+ * @range: colorop type to compute name of
+ *
+ * In contrast to the other drm_get_*_name functions this one here returns a
+ * const pointer and hence is threadsafe.
+ */
+const char *drm_get_colorop_type_name(enum drm_colorop_type type)
+{
+	if (WARN_ON(type >= ARRAY_SIZE(colorop_type_name)))
+		return "unknown";
+
+	return colorop_type_name[type];
+}
+
diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
index 3dd169b0317d..22a217372428 100644
--- a/include/drm/drm_colorop.h
+++ b/include/drm/drm_colorop.h
@@ -30,6 +30,10 @@
 #include <drm/drm_mode.h>
 #include <drm/drm_property.h>
 
+enum drm_colorop_type {
+	DRM_COLOROP_1D_CURVE
+};
+
 /**
  * struct drm_colorop_state - mutable colorop state
  */
@@ -103,6 +107,21 @@ struct drm_colorop {
 	/** @properties: property tracking for this plane */
 	struct drm_object_properties properties;
 
+	/**
+	 * @type:
+	 *
+	 * Read-only
+	 * Type of color operation
+	 */
+	enum drm_colorop_type type;
+
+	/**
+	 * @type_property:
+	 *
+	 * Read-only "TYPE" enum property for specifying the type of
+	 * this color operation. The type is enum drm_colorop_type.
+	 */
+	struct drm_property *type_property;
 };
 
 #define obj_to_colorop(x) container_of(x, struct drm_colorop, base)
@@ -127,7 +146,7 @@ static inline struct drm_colorop *drm_colorop_find(struct drm_device *dev,
 }
 
 int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
-		     struct drm_plane *plane);
+		     struct drm_plane *plane, enum drm_colorop_type type);
 
 struct drm_colorop_state *
 drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop);
-- 
2.42.0

[RFC PATCH 04/10] drm/color: Add 1D Curve subtype

[Harry Wentland]

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/drm_atomic_uapi.c | 18 ++++++++++----
 drivers/gpu/drm/drm_colorop.c     | 39 +++++++++++++++++++++++++++++++
 include/drm/drm_colorop.h         | 20 ++++++++++++++++
 3 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 51072fe2b548..9b01f234b04e 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -648,11 +648,17 @@ static int drm_atomic_colorop_set_property(struct drm_colorop *colorop,
 		struct drm_colorop_state *state, struct drm_file *file_priv,
 		struct drm_property *property, uint64_t val)
 {
-	drm_dbg_atomic(colorop->dev,
-			"[COLOROP:%d] unknown property [PROP:%d:%s]]\n",
-			colorop->base.id,
-			property->base.id, property->name);
-	return -EINVAL;
+	if (property == colorop->curve_1d_type_property) {
+		state->curve_1d_type = val;
+	} else {
+		drm_dbg_atomic(colorop->dev,
+			       "[COLOROP:%d:%d] unknown property [PROP:%d:%s]]\n",
+			       colorop->base.id, colorop->type,
+			       property->base.id, property->name);
+		return -EINVAL;
+	}
+
+	return 0;
 }
 
 static int
@@ -662,6 +668,8 @@ drm_atomic_colorop_get_property(struct drm_colorop *colorop,
 {
 	if (property == colorop->type_property) {
 		*val = colorop->type;
+	} else if (property == colorop->curve_1d_type_property) {
+		*val = state->curve_1d_type;
 	} else {
 		return -EINVAL;
 	}
diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
index c028d5426d42..f665a12a214e 100644
--- a/drivers/gpu/drm/drm_colorop.c
+++ b/drivers/gpu/drm/drm_colorop.c
@@ -36,6 +36,11 @@ static const struct drm_prop_enum_list drm_colorop_type_enum_list[] = {
 	{ DRM_COLOROP_1D_CURVE, "1D Curve" },
 };
 
+static const struct drm_prop_enum_list drm_colorop_curve_1d_type_enum_list[] = {
+	{ DRM_COLOROP_1D_CURVE_SRGB_EOTF, "sRGB EOTF" },
+	{ DRM_COLOROP_1D_CURVE_SRGB_INV_EOTF, "sRGB Inverse EOTF" },
+};
+
 /* Init Helpers */
 
 int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
@@ -73,6 +78,20 @@ int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
 				   colorop->type_property,
 				   colorop->type);
 
+	/* curve_1d_type */
+	/* TODO move to mode_config? */
+	prop = drm_property_create_enum(dev, DRM_MODE_PROP_ATOMIC,
+					"CURVE_1D_TYPE",
+					drm_colorop_curve_1d_type_enum_list,
+					ARRAY_SIZE(drm_colorop_curve_1d_type_enum_list));
+	if (!prop)
+		return -ENOMEM;
+
+	colorop->curve_1d_type_property = prop;
+	drm_object_attach_property(&colorop->base,
+				   colorop->curve_1d_type_property,
+				   0);
+
 	return ret;
 }
 EXPORT_SYMBOL(drm_colorop_init);
@@ -194,6 +213,11 @@ static const char * const colorop_type_name[] = {
 	[DRM_COLOROP_1D_CURVE] = "1D Curve",
 };
 
+static const char * const colorop_curve_1d_type_name[] = {
+	[DRM_COLOROP_1D_CURVE_SRGB_EOTF] = "sRGB EOTF",
+	[DRM_COLOROP_1D_CURVE_SRGB_INV_EOTF] = "sRGB Inverse EOTF",
+};
+
 /**
  * drm_get_colorop_type_name - return a string for colorop type
  * @range: colorop type to compute name of
@@ -209,3 +233,18 @@ const char *drm_get_colorop_type_name(enum drm_colorop_type type)
 	return colorop_type_name[type];
 }
 
+/**
+ * drm_get_colorop_curve_1d_type_name - return a string for 1D curve type
+ * @range: 1d curve type to compute name of
+ *
+ * In contrast to the other drm_get_*_name functions this one here returns a
+ * const pointer and hence is threadsafe.
+ */
+const char *drm_get_colorop_curve_1d_type_name(enum drm_colorop_curve_1d_type type)
+{
+	if (WARN_ON(type >= ARRAY_SIZE(colorop_curve_1d_type_name)))
+		return "unknown";
+
+	return colorop_curve_1d_type_name[type];
+}
+
diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
index 22a217372428..7701b61ff7e9 100644
--- a/include/drm/drm_colorop.h
+++ b/include/drm/drm_colorop.h
@@ -34,6 +34,11 @@ enum drm_colorop_type {
 	DRM_COLOROP_1D_CURVE
 };
 
+enum drm_colorop_curve_1d_type {
+	DRM_COLOROP_1D_CURVE_SRGB_EOTF,
+	DRM_COLOROP_1D_CURVE_SRGB_INV_EOTF
+};
+
 /**
  * struct drm_colorop_state - mutable colorop state
  */
@@ -43,6 +48,13 @@ struct drm_colorop_state {
 
 	/* colorop properties */
 
+	/**
+	 * @curve_1d_type:
+	 *
+	 * Type of 1D curve.
+	 */
+	enum drm_colorop_curve_1d_type curve_1d_type;
+
 	/** @state: backpointer to global drm_atomic_state */
 	struct drm_atomic_state *state;
 };
@@ -122,6 +134,14 @@ struct drm_colorop {
 	 * this color operation. The type is enum drm_colorop_type.
 	 */
 	struct drm_property *type_property;
+
+	/**
+	 * @curve_1d_type:
+	 *
+	 * Sub-type for DRM_COLOROP_1D_CURVE type.
+	 */
+	struct drm_property *curve_1d_type_property;
+
 };
 
 #define obj_to_colorop(x) container_of(x, struct drm_colorop, base)
-- 
2.42.0

[RFC PATCH 05/10] drm/colorop: Add BYPASS property

[Harry Wentland]

We want to be able to bypass each colorop at all times.
Introduce a new BYPASS boolean property for this.

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/drm_atomic_uapi.c |  6 +++++-
 drivers/gpu/drm/drm_colorop.c     | 15 +++++++++++++++
 include/drm/drm_colorop.h         | 20 ++++++++++++++++++++
 3 files changed, 40 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 9b01f234b04e..ca3512038d4c 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -648,7 +648,9 @@ static int drm_atomic_colorop_set_property(struct drm_colorop *colorop,
 		struct drm_colorop_state *state, struct drm_file *file_priv,
 		struct drm_property *property, uint64_t val)
 {
-	if (property == colorop->curve_1d_type_property) {
+	if (property == colorop->bypass_property) {
+		state->bypass = val;
+	} else if (property == colorop->curve_1d_type_property) {
 		state->curve_1d_type = val;
 	} else {
 		drm_dbg_atomic(colorop->dev,
@@ -668,6 +670,8 @@ drm_atomic_colorop_get_property(struct drm_colorop *colorop,
 {
 	if (property == colorop->type_property) {
 		*val = colorop->type;
+	} else if (property == colorop->bypass_property) {
+		*val = state->bypass;
 	} else if (property == colorop->curve_1d_type_property) {
 		*val = state->curve_1d_type;
 	} else {
diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
index f665a12a214e..409df022b256 100644
--- a/drivers/gpu/drm/drm_colorop.c
+++ b/drivers/gpu/drm/drm_colorop.c
@@ -78,6 +78,18 @@ int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
 				   colorop->type_property,
 				   colorop->type);
 
+	/* bypass */
+	/* TODO can we reuse the mode_config->active_prop? */
+	prop = drm_property_create_bool(dev, DRM_MODE_PROP_ATOMIC,
+					"BYPASS");
+	if (!prop)
+		return -ENOMEM;
+
+	colorop->bypass_property = prop;
+	drm_object_attach_property(&colorop->base,
+				   colorop->bypass_property,
+				   1);
+
 	/* curve_1d_type */
 	/* TODO move to mode_config? */
 	prop = drm_property_create_enum(dev, DRM_MODE_PROP_ATOMIC,
@@ -100,6 +112,8 @@ void __drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop,
 						 struct drm_colorop_state *state)
 {
 	memcpy(state, colorop->state, sizeof(*state));
+
+	state->bypass = true;
 }
 
 struct drm_colorop_state *
@@ -164,6 +178,7 @@ void __drm_colorop_state_reset(struct drm_colorop_state *colorop_state,
 					   struct drm_colorop *colorop)
 {
 	colorop_state->colorop = colorop;
+	colorop_state->bypass = true;
 }
 EXPORT_SYMBOL(__drm_colorop_state_reset);
 
diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
index 7701b61ff7e9..69636f6752a0 100644
--- a/include/drm/drm_colorop.h
+++ b/include/drm/drm_colorop.h
@@ -48,6 +48,14 @@ struct drm_colorop_state {
 
 	/* colorop properties */
 
+	/**
+	 * @bypass:
+	 *
+	 * True if colorop shall be bypassed. False if colorop is
+	 * enabled.
+	 */
+	bool bypass;
+
 	/**
 	 * @curve_1d_type:
 	 *
@@ -135,6 +143,18 @@ struct drm_colorop {
 	 */
 	struct drm_property *type_property;
 
+	/**
+	 * @bypass_property:
+	 *
+	 * Boolean property to control enablement of the color
+	 * operation. Setting bypass to "true" shall always be supported
+	 * in order to allow compositors to quickly fall back to
+	 * alternate methods of color processing. This is important
+	 * since setting color operations can fail due to unique
+	 * HW constraints.
+	 */
+	struct drm_property *bypass_property;
+
 	/**
 	 * @curve_1d_type:
 	 *
-- 
2.42.0

[RFC PATCH 06/10] drm/colorop: Add NEXT property

[Harry Wentland]

We'll construct color pipelines out of drm_colorop by
chaining them via the NEXT pointer. NEXT will point to
the next drm_colorop in the pipeline, or by 0 if we're
at the end of the pipeline.

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/drm_colorop.c | 27 +++++++++++++++++++++++++++
 include/drm/drm_colorop.h     | 12 ++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
index 409df022b256..a92e170aed87 100644
--- a/drivers/gpu/drm/drm_colorop.c
+++ b/drivers/gpu/drm/drm_colorop.c
@@ -104,6 +104,15 @@ int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
 				   colorop->curve_1d_type_property,
 				   0);
 
+	prop = drm_property_create_object(dev, DRM_MODE_PROP_IMMUTABLE | DRM_MODE_PROP_ATOMIC,
+			"NEXT", DRM_MODE_OBJECT_COLOROP);
+	if (!prop)
+		return -ENOMEM;
+	colorop->next_property = prop;
+	drm_object_attach_property(&colorop->base,
+				   colorop->next_property,
+				   0);
+
 	return ret;
 }
 EXPORT_SYMBOL(drm_colorop_init);
@@ -263,3 +272,21 @@ const char *drm_get_colorop_curve_1d_type_name(enum drm_colorop_curve_1d_type ty
 	return colorop_curve_1d_type_name[type];
 }
 
+
+/**
+ * drm_colorop_set_next_property - sets the next pointer
+ * @colorop: drm colorop
+ * @next: next colorop
+ *
+ * Should be used when constructing the color pipeline
+ */
+void drm_colorop_set_next_property(struct drm_colorop *colorop, struct drm_colorop *next)
+{
+	if (!colorop->next_property)
+		return;
+
+	drm_object_property_set_value(&colorop->base,
+				      colorop->next_property,
+				      next->base.id);
+}
+EXPORT_SYMBOL(drm_colorop_set_next_property);
diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
index 69636f6752a0..1ddd0e65fe36 100644
--- a/include/drm/drm_colorop.h
+++ b/include/drm/drm_colorop.h
@@ -162,10 +162,20 @@ struct drm_colorop {
 	 */
 	struct drm_property *curve_1d_type_property;
 
+	/**
+	 * @next_property
+	 *
+	 * Read-only property to next colorop in the pipeline
+	 */
+	struct drm_property *next_property;
+
 };
 
 #define obj_to_colorop(x) container_of(x, struct drm_colorop, base)
 
+
+
+
 /**
  * drm_crtc_find - look up a Colorop object from its ID
  * @dev: DRM device
@@ -212,5 +222,7 @@ static inline unsigned int drm_colorop_index(const struct drm_colorop *colorop)
 #define drm_for_each_colorop(colorop, dev) \
 	list_for_each_entry(colorop, &(dev)->mode_config.colorop_list, head)
 
+void drm_colorop_set_next_property(struct drm_colorop *colorop, struct drm_colorop *next);
+
 
 #endif /* __DRM_COLOROP_H__ */
-- 
2.42.0

[RFC PATCH 07/10] drm/colorop: Add atomic state print for drm_colorop

[Harry Wentland]

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/drm_atomic.c | 29 +++++++++++++++++++++++++++++
 include/drm/drm_colorop.h    |  5 +++++
 2 files changed, 34 insertions(+)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 8a5f8cd22c8d..30308b8dec53 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -783,6 +783,19 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
 	return 0;
 }
 
+
+
+static void drm_atomic_colorop_print_state(struct drm_printer *p,
+		const struct drm_colorop_state *state)
+{
+	struct drm_colorop *colorop = state->colorop;
+
+	drm_printf(p, "colorop[%u]:\n", colorop->base.id);
+	drm_printf(p, "\ttype=%s\n", drm_get_colorop_type_name(colorop->type));
+	drm_printf(p, "\tbypass=%u\n", state->bypass);
+	drm_printf(p, "\tcurve_1d_type=%s\n", drm_get_colorop_curve_1d_type_name(state->curve_1d_type));
+}
+
 static void drm_atomic_plane_print_state(struct drm_printer *p,
 		const struct drm_plane_state *state)
 {
@@ -803,6 +816,13 @@ static void drm_atomic_plane_print_state(struct drm_printer *p,
 		   drm_get_color_encoding_name(state->color_encoding));
 	drm_printf(p, "\tcolor-range=%s\n",
 		   drm_get_color_range_name(state->color_range));
+#if 0
+	drm_printf(p, "\tcolor-pipeline=%s\n",
+		   drm_get_color_pipeline_name(state->color_pipeline));
+#else
+	drm_printf(p, "\tcolor-pipeline=%d\n",
+		   state->color_pipeline ? state->color_pipeline->base.id : 0);
+#endif
 
 	if (plane->funcs->atomic_print_state)
 		plane->funcs->atomic_print_state(p, state);
@@ -1779,6 +1799,7 @@ static void __drm_state_dump(struct drm_device *dev, struct drm_printer *p,
 			     bool take_locks)
 {
 	struct drm_mode_config *config = &dev->mode_config;
+	struct drm_colorop *colorop;
 	struct drm_plane *plane;
 	struct drm_crtc *crtc;
 	struct drm_connector *connector;
@@ -1787,6 +1808,14 @@ static void __drm_state_dump(struct drm_device *dev, struct drm_printer *p,
 	if (!drm_drv_uses_atomic_modeset(dev))
 		return;
 
+	list_for_each_entry(colorop, &config->colorop_list, head) {
+		if (take_locks)
+			drm_modeset_lock(&colorop->plane->mutex, NULL);
+		drm_atomic_colorop_print_state(p, colorop->state);
+		if (take_locks)
+			drm_modeset_unlock(&colorop->plane->mutex);
+	}
+
 	list_for_each_entry(plane, &config->plane_list, head) {
 		if (take_locks)
 			drm_modeset_lock(&plane->mutex, NULL);
diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
index 1ddd0e65fe36..622a671d2458 100644
--- a/include/drm/drm_colorop.h
+++ b/include/drm/drm_colorop.h
@@ -222,6 +222,11 @@ static inline unsigned int drm_colorop_index(const struct drm_colorop *colorop)
 #define drm_for_each_colorop(colorop, dev) \
 	list_for_each_entry(colorop, &(dev)->mode_config.colorop_list, head)
 
+const char *drm_get_color_pipeline_name(struct drm_colorop *colorop);
+
+const char *drm_get_colorop_type_name(enum drm_colorop_type type);
+const char *drm_get_colorop_curve_1d_type_name(enum drm_colorop_curve_1d_type type);
+
 void drm_colorop_set_next_property(struct drm_colorop *colorop, struct drm_colorop *next);
 
 
-- 
2.42.0

[RFC PATCH 08/10] drm/colorop: Add new IOCTLs to retrieve drm_colorop objects

[Harry Wentland]

Since we created a new DRM object we need new IOCTLs (and
new libdrm functions) to retrieve those objects.

TODO: Can we make these IOCTLs and libdrm functions generic
to allow for new DRM objects in the future without the need
for new IOCTLs and libdrm functions?

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/drm_colorop.c       | 51 +++++++++++++++++++++++++++++
 drivers/gpu/drm/drm_crtc_internal.h |  4 +++
 drivers/gpu/drm/drm_ioctl.c         |  5 +++
 include/uapi/drm/drm_mode.h         | 21 ++++++++++++
 4 files changed, 81 insertions(+)

diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
index a92e170aed87..fb85b5c41cc4 100644
--- a/drivers/gpu/drm/drm_colorop.c
+++ b/drivers/gpu/drm/drm_colorop.c
@@ -32,6 +32,57 @@
 
 /* TODO big colorop doc, including properties, etc. */
 
+/* IOCTLs */
+
+int drm_mode_getcolorop_res(struct drm_device *dev, void *data,
+			    struct drm_file *file_priv)
+{
+	struct drm_mode_get_colorop_res *colorop_resp = data;
+	struct drm_colorop *colorop;
+	uint32_t __user *colorop_ptr;
+	int count = 0;
+
+	if (!drm_core_check_feature(dev, DRIVER_MODESET))
+		return -EOPNOTSUPP;
+
+	colorop_ptr = u64_to_user_ptr(colorop_resp->colorop_id_ptr);
+
+	/*
+	 * This ioctl is called twice, once to determine how much space is
+	 * needed, and the 2nd time to fill it.
+	 */
+	drm_for_each_colorop(colorop, dev) {
+		if (drm_lease_held(file_priv, colorop->base.id)) {
+			if (count < colorop_resp->count_colorops &&
+			    put_user(colorop->base.id, colorop_ptr + count))
+				return -EFAULT;
+			count++;
+		}
+	}
+	colorop_resp->count_colorops = count;
+
+	return 0;
+}
+
+int drm_mode_getcolorop(struct drm_device *dev, void *data,
+		        struct drm_file *file_priv)
+{
+	struct drm_mode_get_colorop *colorop_resp = data;
+	struct drm_colorop *colorop;
+
+	if (!drm_core_check_feature(dev, DRIVER_MODESET))
+		return -EOPNOTSUPP;
+
+	colorop = drm_colorop_find(dev, file_priv, colorop_resp->colorop_id);
+	if (!colorop)
+		return -ENOENT;
+
+	colorop_resp->colorop_id = colorop->base.id;
+	colorop_resp->plane_id = colorop->plane ? colorop->plane->base.id : 0;
+
+	return 0;
+}
+
 static const struct drm_prop_enum_list drm_colorop_type_enum_list[] = {
 	{ DRM_COLOROP_1D_CURVE, "1D Curve" },
 };
diff --git a/drivers/gpu/drm/drm_crtc_internal.h b/drivers/gpu/drm/drm_crtc_internal.h
index 501a10edd0e1..b68e05c2cf57 100644
--- a/drivers/gpu/drm/drm_crtc_internal.h
+++ b/drivers/gpu/drm/drm_crtc_internal.h
@@ -278,6 +278,10 @@ int drm_mode_getplane(struct drm_device *dev,
 		      void *data, struct drm_file *file_priv);
 int drm_mode_setplane(struct drm_device *dev,
 		      void *data, struct drm_file *file_priv);
+int drm_mode_getcolorop_res(struct drm_device *dev, void *data,
+			    struct drm_file *file_priv);
+int drm_mode_getcolorop(struct drm_device *dev, void *data,
+		        struct drm_file *file_priv);
 int drm_mode_cursor_ioctl(struct drm_device *dev,
 			  void *data, struct drm_file *file_priv);
 int drm_mode_cursor2_ioctl(struct drm_device *dev,
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
index 7c9d66ee917d..a3c137ac88c6 100644
--- a/drivers/gpu/drm/drm_ioctl.c
+++ b/drivers/gpu/drm/drm_ioctl.c
@@ -716,6 +716,11 @@ static const struct drm_ioctl_desc drm_ioctls[] = {
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_LIST_LESSEES, drm_mode_list_lessees_ioctl, DRM_MASTER),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GET_LEASE, drm_mode_get_lease_ioctl, DRM_MASTER),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_REVOKE_LEASE, drm_mode_revoke_lease_ioctl, DRM_MASTER),
+
+	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETCOLOROPRESOURCES, drm_mode_getcolorop_res, 0),
+	/* TODO do we need GETCOLOROP? */
+	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETCOLOROP, drm_mode_getcolorop, 0),
+
 };
 
 #define DRM_CORE_IOCTL_COUNT	ARRAY_SIZE(drm_ioctls)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index 6dcf628def56..9e37eec55291 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -357,6 +357,27 @@ struct drm_mode_get_plane {
 	__u64 format_type_ptr;
 };
 
+struct drm_mode_get_colorop_res {
+	__u64 colorop_id_ptr;
+	__u32 count_colorops;
+};
+
+
+/**
+ * struct drm_mode_get_colorop - Get colorop metadata.
+ *
+ * Userspace can perform a GETCOLOROP ioctl to retrieve information about a
+ * colorop.
+ */
+struct drm_mode_get_colorop {
+	/**
+	 * @colorop_id: Object ID of the colorop whose information should be
+	 * retrieved. Set by caller.
+	 */
+	__u32 colorop_id;
+	__u32 plane_id;
+};
+
 struct drm_mode_get_plane_res {
 	__u64 plane_id_ptr;
 	__u32 count_planes;
-- 
2.42.0

[RFC PATCH 09/10] drm/plane: Add COLOR PIPELINE property

[Harry Wentland]

We're adding a new enum COLOR PIPELINE property. This
property will have entries for each COLOR PIPELINE by
referencing the DRM object ID of the first drm_colorop
of the pipeline. 0 disables the entire COLOR PIPELINE.

Userspace can use this to discover the available color
pipelines, as well as set the desired one. The color
pipelines are programmed via properties on the actual
drm_colorop objects.

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/drm_atomic.c              | 46 +++++++++++++++++++++++
 drivers/gpu/drm/drm_atomic_state_helper.c |  5 +++
 drivers/gpu/drm/drm_atomic_uapi.c         | 44 ++++++++++++++++++++++
 include/drm/drm_atomic_uapi.h             |  2 +
 include/drm/drm_plane.h                   |  8 ++++
 5 files changed, 105 insertions(+)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 30308b8dec53..a8b978e8f3eb 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -1403,6 +1403,52 @@ drm_atomic_add_affected_planes(struct drm_atomic_state *state,
 }
 EXPORT_SYMBOL(drm_atomic_add_affected_planes);
 
+/**
+ * drm_atomic_add_affected_colorops - add colorops for plane
+ * @state: atomic state
+ * @plane: DRM plane
+ *
+ * This function walks the current configuration and adds all colorops
+ * currently used by @plane to the atomic configuration @state. This is useful
+ * when an atomic commit also needs to check all currently enabled colorop on
+ * @plane, e.g. when changing the mode. It's also useful when re-enabling a plane
+ * to avoid special code to force-enable all colorops.
+ *
+ * Since acquiring a colorop state will always also acquire the w/w mutex of the
+ * current plane for that colorop (if there is any) adding all the colorop states for
+ * a plane will not reduce parallelism of atomic updates.
+ *
+ * Returns:
+ * 0 on success or can fail with -EDEADLK or -ENOMEM. When the error is EDEADLK
+ * then the w/w mutex code has detected a deadlock and the entire atomic
+ * sequence must be restarted. All other errors are fatal.
+ */
+int
+drm_atomic_add_affected_colorops(struct drm_atomic_state *state,
+				 struct drm_plane *plane)
+{
+	struct drm_colorop *colorop;
+	struct drm_colorop_state *colorop_state;
+
+	WARN_ON(!drm_atomic_get_new_plane_state(state, plane));
+
+	drm_dbg_atomic(plane->dev,
+		       "Adding all current colorops for [plane:%d:%s] to %p\n",
+		       plane->base.id, plane->name, state);
+
+	drm_for_each_colorop(colorop, plane->dev) {
+		if (colorop->plane != plane)
+			continue;
+
+		colorop_state = drm_atomic_get_colorop_state(state, colorop);
+		if (IS_ERR(colorop_state))
+			return PTR_ERR(colorop_state);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL(drm_atomic_add_affected_colorops);
+
 /**
  * drm_atomic_check_only - check whether a given config would work
  * @state: atomic configuration to check
diff --git a/drivers/gpu/drm/drm_atomic_state_helper.c b/drivers/gpu/drm/drm_atomic_state_helper.c
index 784e63d70a42..3c5f2c8e33d0 100644
--- a/drivers/gpu/drm/drm_atomic_state_helper.c
+++ b/drivers/gpu/drm/drm_atomic_state_helper.c
@@ -267,6 +267,11 @@ void __drm_atomic_helper_plane_state_reset(struct drm_plane_state *plane_state,
 			plane_state->color_range = val;
 	}
 
+	if (plane->color_pipeline_property) {
+		/* default is always NULL, i.e., bypass */
+		plane_state->color_pipeline = NULL;
+	}
+
 	if (plane->zpos_property) {
 		if (!drm_object_property_get_default_value(&plane->base,
 							   plane->zpos_property,
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index ca3512038d4c..44ceb10acb6f 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -256,6 +256,38 @@ drm_atomic_set_fb_for_plane(struct drm_plane_state *plane_state,
 }
 EXPORT_SYMBOL(drm_atomic_set_fb_for_plane);
 
+
+/**
+ * drm_atomic_set_colorop_for_plane - set colorop for plane
+ * @plane_state: atomic state object for the plane
+ * @colorop: colorop to use for the plane
+ *
+ * Changing the assigned framebuffer for a plane requires us to grab a reference
+ * to the new fb and drop the reference to the old fb, if there is one. This
+ * function takes care of all these details besides updating the pointer in the
+ * state object itself.
+ */
+void
+drm_atomic_set_colorop_for_plane(struct drm_plane_state *plane_state,
+				 struct drm_colorop *colorop)
+{
+	struct drm_plane *plane = plane_state->plane;
+
+	if (colorop)
+		drm_dbg_atomic(plane->dev,
+			       "Set [COLOROP:%d] for [PLANE:%d:%s] state %p\n",
+			       colorop->base.id, plane->base.id, plane->name,
+			       plane_state);
+	else
+		drm_dbg_atomic(plane->dev,
+			       "Set [NOCOLOROP] for [PLANE:%d:%s] state %p\n",
+			       plane->base.id, plane->name, plane_state);
+
+	plane_state->color_pipeline = colorop;
+}
+EXPORT_SYMBOL(drm_atomic_set_colorop_for_plane);
+
+
 /**
  * drm_atomic_set_crtc_for_connector - set CRTC for connector
  * @conn_state: atomic state object for the connector
@@ -563,6 +595,16 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
 		state->color_encoding = val;
 	} else if (property == plane->color_range_property) {
 		state->color_range = val;
+	} else if (property == plane->color_pipeline_property) {
+		/* find DRM colorop object */
+		struct drm_colorop *colorop = NULL;
+		colorop = drm_colorop_find(dev, file_priv, val);
+
+		if (val && !colorop)
+			return -EACCES;
+
+		/* set it on drm_plane_state */
+		drm_atomic_set_colorop_for_plane(state, colorop);
 	} else if (property == config->prop_fb_damage_clips) {
 		ret = drm_atomic_replace_property_blob_from_id(dev,
 					&state->fb_damage_clips,
@@ -629,6 +671,8 @@ drm_atomic_plane_get_property(struct drm_plane *plane,
 		*val = state->color_encoding;
 	} else if (property == plane->color_range_property) {
 		*val = state->color_range;
+	} else if (property == plane->color_pipeline_property) {
+		*val = (state->color_pipeline) ? state->color_pipeline->base.id : 0;
 	} else if (property == config->prop_fb_damage_clips) {
 		*val = (state->fb_damage_clips) ?
 			state->fb_damage_clips->base.id : 0;
diff --git a/include/drm/drm_atomic_uapi.h b/include/drm/drm_atomic_uapi.h
index 70a115d523cd..436315523326 100644
--- a/include/drm/drm_atomic_uapi.h
+++ b/include/drm/drm_atomic_uapi.h
@@ -50,6 +50,8 @@ drm_atomic_set_crtc_for_plane(struct drm_plane_state *plane_state,
 			      struct drm_crtc *crtc);
 void drm_atomic_set_fb_for_plane(struct drm_plane_state *plane_state,
 				 struct drm_framebuffer *fb);
+void drm_atomic_set_colorop_for_plane(struct drm_plane_state *plane_state,
+				      struct drm_colorop *colorop);
 int __must_check
 drm_atomic_set_crtc_for_connector(struct drm_connector_state *conn_state,
 				  struct drm_crtc *crtc);
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index cfb8f46b94ab..9c6961743a68 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -745,6 +745,14 @@ struct drm_plane {
 	 */
 	struct drm_property *color_range_property;
 
+	/**
+	 * @color_pipeline_property:
+	 *
+	 * Optional "COLOR_PIPELINE" enum property for specifying
+	 * a color pipeline to use on the plane.
+	 */
+	struct drm_property *color_pipeline_property;
+
 	/**
 	 * @scaling_filter_property: property to apply a particular filter while
 	 * scaling.
-- 
2.42.0

[RFC PATCH 10/10] drm/vkms: Add enumerated 1D curve colorop

[Harry Wentland]

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Cc: Joshua Ashton <joshua@froggi.es>
Cc: Michel Dänzer <mdaenzer@redhat.com>
Cc: Aleix Pol <aleixpol@kde.org>
Cc: Xaver Hugl <xaver.hugl@gmail.com>
Cc: Victoria Brekenfeld <victoria@system76.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Uma Shankar <uma.shankar@intel.com>
Cc: Naseer Ahmed <quic_naseer@quicinc.com>
Cc: Christopher Braga <quic_cbraga@quicinc.com>
---
 drivers/gpu/drm/vkms/Makefile        |   3 +-
 drivers/gpu/drm/vkms/vkms_colorop.c  | 108 +++++++++
 drivers/gpu/drm/vkms/vkms_composer.c | 316 +++++++++++++++++++++++++++
 drivers/gpu/drm/vkms/vkms_drv.h      |   4 +
 drivers/gpu/drm/vkms/vkms_plane.c    |   2 +
 5 files changed, 432 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/vkms/vkms_colorop.c

diff --git a/drivers/gpu/drm/vkms/Makefile b/drivers/gpu/drm/vkms/Makefile
index 1b28a6a32948..bcf508873622 100644
--- a/drivers/gpu/drm/vkms/Makefile
+++ b/drivers/gpu/drm/vkms/Makefile
@@ -6,6 +6,7 @@ vkms-y := \
 	vkms_formats.o \
 	vkms_crtc.o \
 	vkms_composer.o \
-	vkms_writeback.o
+	vkms_writeback.o \
+	vkms_colorop.o
 
 obj-$(CONFIG_DRM_VKMS) += vkms.o
diff --git a/drivers/gpu/drm/vkms/vkms_colorop.c b/drivers/gpu/drm/vkms/vkms_colorop.c
new file mode 100644
index 000000000000..b3da0705bca7
--- /dev/null
+++ b/drivers/gpu/drm/vkms/vkms_colorop.c
@@ -0,0 +1,108 @@
+/*
+ * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#include <linux/slab.h>
+#include <drm/drm_colorop.h>
+#include <drm/drm_print.h>
+#include <drm/drm_property.h>
+#include <drm/drm_plane.h>
+
+#define MAX_COLOR_PIPELINES 5
+
+const int vkms_initialize_tf_pipeline(struct drm_plane *plane, struct drm_prop_enum_list *list)
+{
+
+	struct drm_colorop *op, *prev_op;
+	struct drm_device *dev = plane->dev;
+	int ret;
+
+	/* 1st op: 1d curve */
+	op = kzalloc(sizeof(struct drm_colorop), GFP_KERNEL);
+	if (!op) {
+		DRM_ERROR("KMS: Failed to allocate colorop\n");
+		return -ENOMEM;
+	}
+
+	ret = drm_colorop_init(dev, op, plane, DRM_COLOROP_1D_CURVE);
+	if (ret)
+		return ret;
+
+	list->type = op->base.id;
+	list->name = kasprintf(GFP_KERNEL, "Color Pipeline %d", op->base.id);
+
+	prev_op = op;
+
+	/* 2nd op: 1d curve */
+	op = kzalloc(sizeof(struct drm_colorop), GFP_KERNEL);
+	if (!op) {
+		DRM_ERROR("KMS: Failed to allocate colorop\n");
+		return -ENOMEM;
+	}
+
+	ret = drm_colorop_init(dev, op, plane, DRM_COLOROP_1D_CURVE);
+	if (ret)
+		return ret;
+
+	drm_colorop_set_next_property(prev_op, op);
+
+	return 0;
+}
+
+int vkms_initialize_colorops(struct drm_plane *plane)
+{
+	struct drm_device *dev = plane->dev;
+	struct drm_property *prop;
+	struct drm_prop_enum_list pipelines[MAX_COLOR_PIPELINES];
+	int len = 0;
+	int ret;
+
+	/* Add "Bypass" (i.e. NULL) pipeline */
+	pipelines[len].type = 0;
+	pipelines[len].name = "Bypass";
+	len++;
+
+	/* Add pipeline consisting of transfer functions */
+	ret = vkms_initialize_tf_pipeline(plane, &(pipelines[len]));
+	if (ret)
+		return ret;
+	len++;
+
+	/* Create COLOR_PIPELINE property and attach */
+	prop = drm_property_create_enum(dev, DRM_MODE_PROP_ATOMIC,
+					"COLOR_PIPELINE",
+					pipelines, len);
+	if (!prop)
+		return -ENOMEM;
+
+	plane->color_pipeline_property = prop;
+
+	drm_object_attach_property(&plane->base, prop, 0);
+
+	/* TODO do we even need this? */
+	if (plane->state)
+		plane->state->color_pipeline = NULL;
+
+	return 0;
+}
diff --git a/drivers/gpu/drm/vkms/vkms_composer.c b/drivers/gpu/drm/vkms/vkms_composer.c
index f6c311e8a87c..92ab9c62a554 100644
--- a/drivers/gpu/drm/vkms/vkms_composer.c
+++ b/drivers/gpu/drm/vkms/vkms_composer.c
@@ -12,6 +12,284 @@
 
 #include "vkms_drv.h"
 
+#define LUT_SIZE 256
+
+struct drm_color_lut srgb_array[LUT_SIZE] = {
+	{ 0x13, 0x13, 0x13, 0 },
+	{ 0x27, 0x27, 0x27, 0 },
+	{ 0x3b, 0x3b, 0x3b, 0 },
+	{ 0x4f, 0x4f, 0x4f, 0 },
+	{ 0x63, 0x63, 0x63, 0 },
+	{ 0x76, 0x76, 0x76, 0 },
+	{ 0x8a, 0x8a, 0x8a, 0 },
+	{ 0x9e, 0x9e, 0x9e, 0 },
+	{ 0xb2, 0xb2, 0xb2, 0 },
+	{ 0xc6, 0xc6, 0xc6, 0 },
+	{ 0xda, 0xda, 0xda, 0 },
+	{ 0xef, 0xef, 0xef, 0 },
+	{ 0x106, 0x106, 0x106, 0 },
+	{ 0x11e, 0x11e, 0x11e, 0 },
+	{ 0x137, 0x137, 0x137, 0 },
+	{ 0x151, 0x151, 0x151, 0 },
+	{ 0x16d, 0x16d, 0x16d, 0 },
+	{ 0x18a, 0x18a, 0x18a, 0 },
+	{ 0x1a8, 0x1a8, 0x1a8, 0 },
+	{ 0x1c7, 0x1c7, 0x1c7, 0 },
+	{ 0x1e8, 0x1e8, 0x1e8, 0 },
+	{ 0x20a, 0x20a, 0x20a, 0 },
+	{ 0x22e, 0x22e, 0x22e, 0 },
+	{ 0x253, 0x253, 0x253, 0 },
+	{ 0x279, 0x279, 0x279, 0 },
+	{ 0x2a0, 0x2a0, 0x2a0, 0 },
+	{ 0x2c9, 0x2c9, 0x2c9, 0 },
+	{ 0x2f4, 0x2f4, 0x2f4, 0 },
+	{ 0x320, 0x320, 0x320, 0 },
+	{ 0x34d, 0x34d, 0x34d, 0 },
+	{ 0x37c, 0x37c, 0x37c, 0 },
+	{ 0x3ac, 0x3ac, 0x3ac, 0 },
+	{ 0x3de, 0x3de, 0x3de, 0 },
+	{ 0x411, 0x411, 0x411, 0 },
+	{ 0x446, 0x446, 0x446, 0 },
+	{ 0x47c, 0x47c, 0x47c, 0 },
+	{ 0x4b4, 0x4b4, 0x4b4, 0 },
+	{ 0x4ed, 0x4ed, 0x4ed, 0 },
+	{ 0x528, 0x528, 0x528, 0 },
+	{ 0x564, 0x564, 0x564, 0 },
+	{ 0x5a3, 0x5a3, 0x5a3, 0 },
+	{ 0x5e2, 0x5e2, 0x5e2, 0 },
+	{ 0x624, 0x624, 0x624, 0 },
+	{ 0x666, 0x666, 0x666, 0 },
+	{ 0x6ab, 0x6ab, 0x6ab, 0 },
+	{ 0x6f1, 0x6f1, 0x6f1, 0 },
+	{ 0x739, 0x739, 0x739, 0 },
+	{ 0x782, 0x782, 0x782, 0 },
+	{ 0x7ce, 0x7ce, 0x7ce, 0 },
+	{ 0x81b, 0x81b, 0x81b, 0 },
+	{ 0x869, 0x869, 0x869, 0 },
+	{ 0x8b9, 0x8b9, 0x8b9, 0 },
+	{ 0x90b, 0x90b, 0x90b, 0 },
+	{ 0x95f, 0x95f, 0x95f, 0 },
+	{ 0x9b5, 0x9b5, 0x9b5, 0 },
+	{ 0xa0c, 0xa0c, 0xa0c, 0 },
+	{ 0xa65, 0xa65, 0xa65, 0 },
+	{ 0xabf, 0xabf, 0xabf, 0 },
+	{ 0xb1c, 0xb1c, 0xb1c, 0 },
+	{ 0xb7a, 0xb7a, 0xb7a, 0 },
+	{ 0xbda, 0xbda, 0xbda, 0 },
+	{ 0xc3c, 0xc3c, 0xc3c, 0 },
+	{ 0xca0, 0xca0, 0xca0, 0 },
+	{ 0xd06, 0xd06, 0xd06, 0 },
+	{ 0xd6d, 0xd6d, 0xd6d, 0 },
+	{ 0xdd6, 0xdd6, 0xdd6, 0 },
+	{ 0xe41, 0xe41, 0xe41, 0 },
+	{ 0xeae, 0xeae, 0xeae, 0 },
+	{ 0xf1d, 0xf1d, 0xf1d, 0 },
+	{ 0xf8e, 0xf8e, 0xf8e, 0 },
+	{ 0x1001, 0x1001, 0x1001, 0 },
+	{ 0x1075, 0x1075, 0x1075, 0 },
+	{ 0x10ec, 0x10ec, 0x10ec, 0 },
+	{ 0x1164, 0x1164, 0x1164, 0 },
+	{ 0x11de, 0x11de, 0x11de, 0 },
+	{ 0x125a, 0x125a, 0x125a, 0 },
+	{ 0x12d9, 0x12d9, 0x12d9, 0 },
+	{ 0x1359, 0x1359, 0x1359, 0 },
+	{ 0x13db, 0x13db, 0x13db, 0 },
+	{ 0x145f, 0x145f, 0x145f, 0 },
+	{ 0x14e5, 0x14e5, 0x14e5, 0 },
+	{ 0x156d, 0x156d, 0x156d, 0 },
+	{ 0x15f7, 0x15f7, 0x15f7, 0 },
+	{ 0x1683, 0x1683, 0x1683, 0 },
+	{ 0x1711, 0x1711, 0x1711, 0 },
+	{ 0x17a1, 0x17a1, 0x17a1, 0 },
+	{ 0x1833, 0x1833, 0x1833, 0 },
+	{ 0x18c7, 0x18c7, 0x18c7, 0 },
+	{ 0x195d, 0x195d, 0x195d, 0 },
+	{ 0x19f6, 0x19f6, 0x19f6, 0 },
+	{ 0x1a90, 0x1a90, 0x1a90, 0 },
+	{ 0x1b2c, 0x1b2c, 0x1b2c, 0 },
+	{ 0x1bcb, 0x1bcb, 0x1bcb, 0 },
+	{ 0x1c6b, 0x1c6b, 0x1c6b, 0 },
+	{ 0x1d0e, 0x1d0e, 0x1d0e, 0 },
+	{ 0x1db3, 0x1db3, 0x1db3, 0 },
+	{ 0x1e59, 0x1e59, 0x1e59, 0 },
+	{ 0x1f02, 0x1f02, 0x1f02, 0 },
+	{ 0x1fad, 0x1fad, 0x1fad, 0 },
+	{ 0x205b, 0x205b, 0x205b, 0 },
+	{ 0x210a, 0x210a, 0x210a, 0 },
+	{ 0x21bb, 0x21bb, 0x21bb, 0 },
+	{ 0x226f, 0x226f, 0x226f, 0 },
+	{ 0x2325, 0x2325, 0x2325, 0 },
+	{ 0x23dd, 0x23dd, 0x23dd, 0 },
+	{ 0x2497, 0x2497, 0x2497, 0 },
+	{ 0x2553, 0x2553, 0x2553, 0 },
+	{ 0x2612, 0x2612, 0x2612, 0 },
+	{ 0x26d2, 0x26d2, 0x26d2, 0 },
+	{ 0x2795, 0x2795, 0x2795, 0 },
+	{ 0x285a, 0x285a, 0x285a, 0 },
+	{ 0x2922, 0x2922, 0x2922, 0 },
+	{ 0x29eb, 0x29eb, 0x29eb, 0 },
+	{ 0x2ab7, 0x2ab7, 0x2ab7, 0 },
+	{ 0x2b85, 0x2b85, 0x2b85, 0 },
+	{ 0x2c56, 0x2c56, 0x2c56, 0 },
+	{ 0x2d28, 0x2d28, 0x2d28, 0 },
+	{ 0x2dfd, 0x2dfd, 0x2dfd, 0 },
+	{ 0x2ed4, 0x2ed4, 0x2ed4, 0 },
+	{ 0x2fad, 0x2fad, 0x2fad, 0 },
+	{ 0x3089, 0x3089, 0x3089, 0 },
+	{ 0x3167, 0x3167, 0x3167, 0 },
+	{ 0x3247, 0x3247, 0x3247, 0 },
+	{ 0x332a, 0x332a, 0x332a, 0 },
+	{ 0x340e, 0x340e, 0x340e, 0 },
+	{ 0x34f5, 0x34f5, 0x34f5, 0 },
+	{ 0x35df, 0x35df, 0x35df, 0 },
+	{ 0x36cb, 0x36cb, 0x36cb, 0 },
+	{ 0x37b9, 0x37b9, 0x37b9, 0 },
+	{ 0x38a9, 0x38a9, 0x38a9, 0 },
+	{ 0x399c, 0x399c, 0x399c, 0 },
+	{ 0x3a91, 0x3a91, 0x3a91, 0 },
+	{ 0x3b89, 0x3b89, 0x3b89, 0 },
+	{ 0x3c83, 0x3c83, 0x3c83, 0 },
+	{ 0x3d7f, 0x3d7f, 0x3d7f, 0 },
+	{ 0x3e7e, 0x3e7e, 0x3e7e, 0 },
+	{ 0x3f7f, 0x3f7f, 0x3f7f, 0 },
+	{ 0x4082, 0x4082, 0x4082, 0 },
+	{ 0x4188, 0x4188, 0x4188, 0 },
+	{ 0x4290, 0x4290, 0x4290, 0 },
+	{ 0x439b, 0x439b, 0x439b, 0 },
+	{ 0x44a8, 0x44a8, 0x44a8, 0 },
+	{ 0x45b7, 0x45b7, 0x45b7, 0 },
+	{ 0x46c9, 0x46c9, 0x46c9, 0 },
+	{ 0x47dd, 0x47dd, 0x47dd, 0 },
+	{ 0x48f4, 0x48f4, 0x48f4, 0 },
+	{ 0x4a0d, 0x4a0d, 0x4a0d, 0 },
+	{ 0x4b29, 0x4b29, 0x4b29, 0 },
+	{ 0x4c47, 0x4c47, 0x4c47, 0 },
+	{ 0x4d68, 0x4d68, 0x4d68, 0 },
+	{ 0x4e8b, 0x4e8b, 0x4e8b, 0 },
+	{ 0x4fb1, 0x4fb1, 0x4fb1, 0 },
+	{ 0x50d9, 0x50d9, 0x50d9, 0 },
+	{ 0x5203, 0x5203, 0x5203, 0 },
+	{ 0x5330, 0x5330, 0x5330, 0 },
+	{ 0x5460, 0x5460, 0x5460, 0 },
+	{ 0x5592, 0x5592, 0x5592, 0 },
+	{ 0x56c6, 0x56c6, 0x56c6, 0 },
+	{ 0x57fd, 0x57fd, 0x57fd, 0 },
+	{ 0x5937, 0x5937, 0x5937, 0 },
+	{ 0x5a73, 0x5a73, 0x5a73, 0 },
+	{ 0x5bb2, 0x5bb2, 0x5bb2, 0 },
+	{ 0x5cf3, 0x5cf3, 0x5cf3, 0 },
+	{ 0x5e37, 0x5e37, 0x5e37, 0 },
+	{ 0x5f7d, 0x5f7d, 0x5f7d, 0 },
+	{ 0x60c6, 0x60c6, 0x60c6, 0 },
+	{ 0x6212, 0x6212, 0x6212, 0 },
+	{ 0x6360, 0x6360, 0x6360, 0 },
+	{ 0x64b0, 0x64b0, 0x64b0, 0 },
+	{ 0x6604, 0x6604, 0x6604, 0 },
+	{ 0x6759, 0x6759, 0x6759, 0 },
+	{ 0x68b2, 0x68b2, 0x68b2, 0 },
+	{ 0x6a0d, 0x6a0d, 0x6a0d, 0 },
+	{ 0x6b6a, 0x6b6a, 0x6b6a, 0 },
+	{ 0x6ccb, 0x6ccb, 0x6ccb, 0 },
+	{ 0x6e2d, 0x6e2d, 0x6e2d, 0 },
+	{ 0x6f93, 0x6f93, 0x6f93, 0 },
+	{ 0x70fb, 0x70fb, 0x70fb, 0 },
+	{ 0x7266, 0x7266, 0x7266, 0 },
+	{ 0x73d3, 0x73d3, 0x73d3, 0 },
+	{ 0x7543, 0x7543, 0x7543, 0 },
+	{ 0x76b6, 0x76b6, 0x76b6, 0 },
+	{ 0x782b, 0x782b, 0x782b, 0 },
+	{ 0x79a3, 0x79a3, 0x79a3, 0 },
+	{ 0x7b1d, 0x7b1d, 0x7b1d, 0 },
+	{ 0x7c9b, 0x7c9b, 0x7c9b, 0 },
+	{ 0x7e1b, 0x7e1b, 0x7e1b, 0 },
+	{ 0x7f9d, 0x7f9d, 0x7f9d, 0 },
+	{ 0x8123, 0x8123, 0x8123, 0 },
+	{ 0x82ab, 0x82ab, 0x82ab, 0 },
+	{ 0x8436, 0x8436, 0x8436, 0 },
+	{ 0x85c3, 0x85c3, 0x85c3, 0 },
+	{ 0x8753, 0x8753, 0x8753, 0 },
+	{ 0x88e6, 0x88e6, 0x88e6, 0 },
+	{ 0x8a7c, 0x8a7c, 0x8a7c, 0 },
+	{ 0x8c14, 0x8c14, 0x8c14, 0 },
+	{ 0x8daf, 0x8daf, 0x8daf, 0 },
+	{ 0x8f4d, 0x8f4d, 0x8f4d, 0 },
+	{ 0x90ed, 0x90ed, 0x90ed, 0 },
+	{ 0x9290, 0x9290, 0x9290, 0 },
+	{ 0x9436, 0x9436, 0x9436, 0 },
+	{ 0x95df, 0x95df, 0x95df, 0 },
+	{ 0x978b, 0x978b, 0x978b, 0 },
+	{ 0x9939, 0x9939, 0x9939, 0 },
+	{ 0x9aea, 0x9aea, 0x9aea, 0 },
+	{ 0x9c9e, 0x9c9e, 0x9c9e, 0 },
+	{ 0x9e55, 0x9e55, 0x9e55, 0 },
+	{ 0xa00e, 0xa00e, 0xa00e, 0 },
+	{ 0xa1ca, 0xa1ca, 0xa1ca, 0 },
+	{ 0xa389, 0xa389, 0xa389, 0 },
+	{ 0xa54b, 0xa54b, 0xa54b, 0 },
+	{ 0xa710, 0xa710, 0xa710, 0 },
+	{ 0xa8d7, 0xa8d7, 0xa8d7, 0 },
+	{ 0xaaa1, 0xaaa1, 0xaaa1, 0 },
+	{ 0xac6e, 0xac6e, 0xac6e, 0 },
+	{ 0xae3e, 0xae3e, 0xae3e, 0 },
+	{ 0xb011, 0xb011, 0xb011, 0 },
+	{ 0xb1e7, 0xb1e7, 0xb1e7, 0 },
+	{ 0xb3bf, 0xb3bf, 0xb3bf, 0 },
+	{ 0xb59a, 0xb59a, 0xb59a, 0 },
+	{ 0xb778, 0xb778, 0xb778, 0 },
+	{ 0xb959, 0xb959, 0xb959, 0 },
+	{ 0xbb3d, 0xbb3d, 0xbb3d, 0 },
+	{ 0xbd24, 0xbd24, 0xbd24, 0 },
+	{ 0xbf0d, 0xbf0d, 0xbf0d, 0 },
+	{ 0xc0fa, 0xc0fa, 0xc0fa, 0 },
+	{ 0xc2e9, 0xc2e9, 0xc2e9, 0 },
+	{ 0xc4db, 0xc4db, 0xc4db, 0 },
+	{ 0xc6d0, 0xc6d0, 0xc6d0, 0 },
+	{ 0xc8c8, 0xc8c8, 0xc8c8, 0 },
+	{ 0xcac3, 0xcac3, 0xcac3, 0 },
+	{ 0xccc1, 0xccc1, 0xccc1, 0 },
+	{ 0xcec1, 0xcec1, 0xcec1, 0 },
+	{ 0xd0c5, 0xd0c5, 0xd0c5, 0 },
+	{ 0xd2cc, 0xd2cc, 0xd2cc, 0 },
+	{ 0xd4d5, 0xd4d5, 0xd4d5, 0 },
+	{ 0xd6e1, 0xd6e1, 0xd6e1, 0 },
+	{ 0xd8f1, 0xd8f1, 0xd8f1, 0 },
+	{ 0xdb03, 0xdb03, 0xdb03, 0 },
+	{ 0xdd18, 0xdd18, 0xdd18, 0 },
+	{ 0xdf30, 0xdf30, 0xdf30, 0 },
+	{ 0xe14b, 0xe14b, 0xe14b, 0 },
+	{ 0xe369, 0xe369, 0xe369, 0 },
+	{ 0xe58a, 0xe58a, 0xe58a, 0 },
+	{ 0xe7ae, 0xe7ae, 0xe7ae, 0 },
+	{ 0xe9d5, 0xe9d5, 0xe9d5, 0 },
+	{ 0xebff, 0xebff, 0xebff, 0 },
+	{ 0xee2c, 0xee2c, 0xee2c, 0 },
+	{ 0xf05c, 0xf05c, 0xf05c, 0 },
+	{ 0xf28f, 0xf28f, 0xf28f, 0 },
+	{ 0xf4c4, 0xf4c4, 0xf4c4, 0 },
+	{ 0xf6fd, 0xf6fd, 0xf6fd, 0 },
+	{ 0xf939, 0xf939, 0xf939, 0 },
+	{ 0xfb78, 0xfb78, 0xfb78, 0 },
+	{ 0xfdba, 0xfdba, 0xfdba, 0 },
+	{ 0xffff, 0xffff, 0xffff, 0 }
+};
+
+#if 0
+struct vkms_color_lut srgb_eotf = {
+	.base = NULL,
+	. lut_length = LUT_SIZE,
+	.channel_value2index_ratio = drm_int2fixp(0xffff)
+	// .channel_value2index_ratio = 0 //drm_fixp_div(drm_int2fixp(0xffff), drm_int2fixp(LUT_SIZE))
+};
+
+#else
+const struct vkms_color_lut srgb_eotf = {
+	.base = srgb_array,
+	.lut_length = 256,
+	.channel_value2index_ratio = 16711935ll
+};
+
+#endif
+
 static u16 pre_mul_blend_channel(u16 src, u16 dst, u16 alpha)
 {
 	u32 new_color;
@@ -136,6 +414,39 @@ static void apply_lut(const struct vkms_crtc_state *crtc_state, struct line_buff
 	}
 }
 
+static void pre_blend_color_transform(const struct vkms_plane_state *plane_state, struct line_buffer *output_buffer)
+{
+	struct drm_colorop *pipeline = plane_state->base.base.color_pipeline;
+	/* TODO this is probably wrong */
+	struct drm_colorop_state *colorop_state;
+
+	if (!pipeline)
+		return;
+
+	colorop_state = pipeline->state;
+
+	if (!colorop_state)
+		return;
+
+	for (size_t x = 0; x < output_buffer->n_pixels; x++) {
+		struct pixel_argb_u16 *pixel = &output_buffer->pixels[x];
+
+		if (pipeline->type == DRM_COLOROP_1D_CURVE &&
+			colorop_state->bypass == false) {
+			switch (colorop_state->curve_1d_type) {
+				case DRM_COLOROP_1D_CURVE_SRGB_INV_EOTF:
+					break;
+				case DRM_COLOROP_1D_CURVE_SRGB_EOTF:
+				default:
+					pixel->r = apply_lut_to_channel_value(&srgb_eotf, pixel->r, LUT_RED);
+					pixel->g = apply_lut_to_channel_value(&srgb_eotf, pixel->g, LUT_GREEN);
+					pixel->b = apply_lut_to_channel_value(&srgb_eotf, pixel->b, LUT_BLUE);
+					break;
+			}
+		}
+	}
+}
+
 /**
  * @wb_frame_info: The writeback frame buffer metadata
  * @crtc_state: The crtc state
@@ -168,8 +479,13 @@ static void blend(struct vkms_writeback_job *wb,
 				continue;
 
 			plane[i]->plane_read(stage_buffer, plane[i]->frame_info, y);
+
+			/* do per-plane color transformations here */
+			// pre_blend_color_transform(plane[i], stage_buffer);
+
 			pre_mul_alpha_blend(plane[i]->frame_info, stage_buffer,
 					    output_buffer);
+			pre_blend_color_transform(plane[i], output_buffer);
 		}
 
 		apply_lut(crtc_state, output_buffer);
diff --git a/drivers/gpu/drm/vkms/vkms_drv.h b/drivers/gpu/drm/vkms/vkms_drv.h
index 310b31f47928..c04f714cd486 100644
--- a/drivers/gpu/drm/vkms/vkms_drv.h
+++ b/drivers/gpu/drm/vkms/vkms_drv.h
@@ -168,4 +168,8 @@ void vkms_set_composer(struct vkms_output *out, bool enabled);
 /* Writeback */
 int vkms_enable_writeback_connector(struct vkms_device *vkmsdev);
 
+/* Colorops */
+int vkms_initialize_colorops(struct drm_plane *plane);
+
+
 #endif /* _VKMS_DRV_H_ */
diff --git a/drivers/gpu/drm/vkms/vkms_plane.c b/drivers/gpu/drm/vkms/vkms_plane.c
index b3f8a115cc23..cbffbdd7cbf9 100644
--- a/drivers/gpu/drm/vkms/vkms_plane.c
+++ b/drivers/gpu/drm/vkms/vkms_plane.c
@@ -237,5 +237,7 @@ struct vkms_plane *vkms_plane_init(struct vkms_device *vkmsdev,
 
 	drm_plane_helper_add(&plane->base, funcs);
 
+	vkms_initialize_colorops(&plane->base);
+
 	return plane;
 }
-- 
2.42.0

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Sebastian Wick]

Hey Harry,

Thank you and Simon for this great document. Really happy about it, but
obviously I've got a few notes and questions inline.

On Fri, Sep 08, 2023 at 11:02:26AM -0400, Harry Wentland wrote:
> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: Jonas Ådahl <jadahl@redhat.com>
> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> Cc: Shashank Sharma <shashank.sharma@amd.com>
> Cc: Alexander Goins <agoins@nvidia.com>
> Cc: Joshua Ashton <joshua@froggi.es>
> Cc: Michel Dänzer <mdaenzer@redhat.com>
> Cc: Aleix Pol <aleixpol@kde.org>
> Cc: Xaver Hugl <xaver.hugl@gmail.com>
> Cc: Victoria Brekenfeld <victoria@system76.com>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Uma Shankar <uma.shankar@intel.com>
> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> Cc: Christopher Braga <quic_cbraga@quicinc.com>
> ---
>  Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
>  1 file changed, 278 insertions(+)
>  create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
> 
> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
> new file mode 100644
> index 000000000000..bfa4a8f12087
> --- /dev/null
> +++ b/Documentation/gpu/rfc/color_pipeline.rst
> @@ -0,0 +1,278 @@
> +========================
> +Linux Color Pipeline API
> +========================
> +
> +What problem are we solving?
> +============================
> +
> +We would like to support pre-, and post-blending complex color transformations
> +in order to allow for HW-supported HDR use-cases, as well as to provide support
> +to color-managed applications, such as video or image editors.
> +
> +While it is possible to support an HDR output on HW supporting the Colorspace
> +and HDR Metadata drm_connector properties that requires the compositor or
> +application to render and compose the content into one final buffer intended for
> +display. Doing so is costly.
> +
> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
> +operations to support color transformations. These operations are often
> +implemented in fixed-function HW and therefore much more power efficient than
> +performing similar operations via shaders or CPU.
> +
> +We would like to make use of this HW functionality to support complex color
> +transformations with no, or minimal CPU or shader load.
> +
> +
> +How are other OSes solving this problem?
> +========================================
> +
> +The most widely supported use-cases regard HDR content, whether video or
> +gaming.
> +
> +Most OSes will specify the source content format (color gamut, encoding transfer
> +function, and other metadata, such as max and average light levels) to a driver.
> +Drivers will then program their fixed-function HW accordingly to map from a
> +source content buffer's space to a display's space.
> +
> +When fixed-function HW is not available the compositor will assemble a shader to
> +ask the GPU to perform the transformation from the source content format to the
> +display's format.
> +
> +A compositor's mapping function and a driver's mapping function are usually
> +entirely separate concepts. On OSes where a HW vendor has no insight into
> +closed-source compositor code such a vendor will tune their color management
> +code to visually match the compositor's. On other OSes, where both mapping
> +functions are open to an implementer they will ensure both mappings match.
> +
> +
> +Why is Linux different?
> +=======================
> +
> +Unlike other OSes, where there is one compositor for one or more drivers, on
> +Linux we have a many-to-many relationship. Many compositors; many drivers.
> +In addition each compositor vendor or community has their own view of how
> +color management should be done. This is what makes Linux so beautiful.
> +
> +This means that a HW vendor can now no longer tune their driver to one
> +compositor, as tuning it to one will almost inevitably make it look very
> +different from another compositor's color mapping.
> +
> +We need a better solution.
> +
> +
> +Descriptive API
> +===============
> +
> +An API that describes the source and destination colorspaces is a descriptive
> +API. It describes the input and output color spaces but does not describe
> +how precisely they should be mapped. Such a mapping includes many minute
> +design decision that can greatly affect the look of the final result.
> +
> +It is not feasible to describe such mapping with enough detail to ensure the
> +same result from each implementation. In fact, these mappings are a very active
> +research area.
> +
> +
> +Prescriptive API
> +================
> +
> +A prescriptive API describes not the source and destination colorspaces. It
> +instead prescribes a recipe for how to manipulate pixel values to arrive at the
> +desired outcome.
> +
> +This recipe is generally an order straight-forward operations, with clear
> +mathematical definitions, such as 1D LUTs, 3D LUTs, matrices, or other
> +operations that can be described in a precise manner.
> +
> +
> +The Color Pipeline API
> +======================
> +
> +HW color management pipelines can significantly differ between HW
> +vendors in terms of availability, ordering, and capabilities of HW
> +blocks. This makes a common definition of color management blocks and
> +their ordering nigh impossible. Instead we are defining an API that
> +allows user space to discover the HW capabilities.
> +
> +
> +drm_colorop Object & IOCTLs
> +===========================
> +
> +To support the definition of color pipelines we introduce a new DRM core
> +object, a drm_colorop. Individual drm_colorop objects will be chained
> +via the NEXT property of a drm_colorop to constitute a color pipeline.
> +Each drm_colorop object is unique, i.e., even if multiple color
> +pipelines have the same operation they won't share the same drm_colorop
> +object to describe that operation.
> +
> +Just like other DRM objects the drm_colorop objects are discovered via
> +IOCTLs:
> +
> +DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve the
> +number of all drm_colorop objects.
> +
> +DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
> +It includes the ID for the colorop object, as well as the plane_id of
> +the associated plane. All other values should be registered as
> +properties.
> +
> +Each drm_colorop has three core properties:

We talked a bit about how forwards compatibility will look like. When a
driver introduces a new property that user space doesn't understand yet,
for user space to make use of the pipeline, the property either has to
have a bypass property, or some other kind of standard/core property
that promises that this property is purely informational and the
existance doesn't change the result of the color pipeline. Something
like "INFORMATIONAL" or "INFO" (better suggestions welcome!).

If a property doesn't fall into the two categories, the new pipeline
would be unusable for user space. So, if this new property is added on
an existing piece of hardware, it should be done on an entirely new
pipeline to avoid user space regressions.

I believe these compatibility considerations are very important to
document.

> +
> +TYPE: The type of transformation, such as
> +* enumerated curve
> +* custom (uniform) 1D LUT
> +* 3x3 matrix
> +* 3x4 matrix
> +* 3D LUT
> +* etc.
> +
> +Depending on the type of transformation other properties will describe
> +more details.
> +
> +BYPASS: A boolean property that can be used to easily put a block into
> +bypass mode. While setting other properties might fail atomic check,
> +setting the BYPASS property to true should never fail. This allows DRM
> +clients to fallback to other methods of color management if an atomic
> +check for KMS color operations fails.

This says BYPASS is a prop but in the example below the type property is
a mutable enum with Bypass being an enum variant.

Any reason why you chose to make the type property mutable instead of a
separate BYPASS property? Personally I like the simplicity of immutable
properties and would favor a BYPASS property.

> +
> +NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
> +drm_colorop is the last in the chain.
> +
> +An example of a drm_colorop object might look like one of these::
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, BT.709, HLG, …} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> +    ├─ "lut_size": immutable range = 33
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +
> +COLOR_PIPELINE Plane Property
> +=============================
> +
> +Because we don't have existing KMS color properties in the pre-blending
> +portion of display pipelines (i.e. on drm_planes) we are introducing
> +color pipelines here first. Eventually we'll want to use the same
> +concept for the post-blending portion, i.e. drm_crtcs.
> +
> +Color Pipelines are created by a driver and advertised via a new
> +COLOR_PIPELINE enum property on each plane. Values of the property
> +always include '0', which is the default and means all color processing
> +is disabled. Additional values will be the object IDs of the first

Regarding the requirement to never fail atomic checks which set a
colorop to bypass: I think for user space it would be sufficient if
setting the entire pipeline to bypass would not fail the atomic check.

I'm also wondering if there is/will be hardware out there which always
does some kind of transformation which cannot be turned off. How would
they implement the '0' pipeline?

> +drm_colorop in a pipeline. A driver can create and advertise none, one,
> +or more possible color pipelines. A DRM client will select a color
> +pipeline by setting the COLOR PIPELINE to the respective value.
> +
> +In the case where drivers have custom support for pre-blending color
> +processing those drivers shall reject atomic commits that are trying to
> +set both the custom color properties, as well as the COLOR_PIPELINE
> +property.

By 'not setting the COLOR_PIPELINE property' you mean any value other
than '0' I guess? Should be fine to a proper user space.

> +
> +An example of a COLOR_PIPELINE property on a plane might look like this::
> +
> +    Plane 10
> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> +    ├─ …
> +    └─ "color_pipeline": enum {0, 42, 52} = 0
> +
> +
> +Color Pipeline Discovery
> +========================
> +
> +A DRM client wanting color management on a drm_plane will:
> +
> +1. Read all drm_colorop objects
> +2. Get the COLOR_PIPELINE property of the plane
> +3. iterate all COLOR_PIPELINE enum values
> +4. for each enum value walk the color pipeline (via the NEXT pointers)
> +   and see if the available color operations are suitable for the
> +   desired color management operations
> +
> +An example of chained properties to define an AMD pre-blending color
> +pipeline might look like this::
> +
> +    Plane 10
> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> +    └─ "color_pipeline": enum {0, 42} = 0
> +    Color operation 42 (input CSC)
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 43
> +    Color operation 43
> +    ├─ "type": enum {Scaling} = Scaling
> +    └─ "next": immutable color operation ID = 44
> +    Color operation 44 (DeGamma)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
> +    └─ "next": immutable color operation ID = 45
> +    Color operation 45 (gamut remap)
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 46
> +    Color operation 46 (shaper LUT RAM)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 47
> +    Color operation 47 (3D LUT RAM)
> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> +    ├─ "lut_size": immutable range = 17
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 48
> +    Color operation 48 (blend gamma)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 0
> +
> +
> +Color Pipeline Programming
> +==========================
> +
> +Once a DRM client has found a suitable pipeline it will:
> +
> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
> +   drm_colorop object of the desired pipeline
> +2. Set the properties for all drm_colorop objects in the pipeline to the
> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
> +   and false for enabled drm_colorop blocks
> +3. Perform atomic_check/commit as desired
> +
> +To configure the pipeline for an HDR10 PQ plane and blending in linear
> +space, a compositor might perform an atomic commit with the following
> +property values::
> +
> +    Plane 10
> +    └─ "color_pipeline" = 42
> +    Color operation 42 (input CSC)
> +    └─ "bypass" = true
> +    Color operation 44 (DeGamma)
> +    └─ "bypass" = true
> +    Color operation 45 (gamut remap)
> +    └─ "bypasse" = true
> +    Color operation 46 (shaper LUT RAM)
> +    └─ "bypass" = true
> +    Color operation 47 (3D LUT RAM)
> +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
> +    Color operation 48 (blend gamma)
> +    └─ "1d_curve_type" = PQ inverse EOTF
> +
> +
> +References
> +==========
> +
> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> \ No newline at end of file
> -- 
> 2.42.0
> 

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-09-08 15:30, Sebastian Wick wrote:
> Hey Harry,
> 
> Thank you and Simon for this great document. Really happy about it, but
> obviously I've got a few notes and questions inline.
> 
> On Fri, Sep 08, 2023 at 11:02:26AM -0400, Harry Wentland wrote:
>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
>> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
>> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
>> Cc: Simon Ser <contact@emersion.fr>
>> Cc: Harry Wentland <harry.wentland@amd.com>
>> Cc: Melissa Wen <mwen@igalia.com>
>> Cc: Jonas Ådahl <jadahl@redhat.com>
>> Cc: Sebastian Wick <sebastian.wick@redhat.com>
>> Cc: Shashank Sharma <shashank.sharma@amd.com>
>> Cc: Alexander Goins <agoins@nvidia.com>
>> Cc: Joshua Ashton <joshua@froggi.es>
>> Cc: Michel Dänzer <mdaenzer@redhat.com>
>> Cc: Aleix Pol <aleixpol@kde.org>
>> Cc: Xaver Hugl <xaver.hugl@gmail.com>
>> Cc: Victoria Brekenfeld <victoria@system76.com>
>> Cc: Daniel Vetter <daniel@ffwll.ch>
>> Cc: Uma Shankar <uma.shankar@intel.com>
>> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
>> Cc: Christopher Braga <quic_cbraga@quicinc.com>
>> ---
>>   Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
>>   1 file changed, 278 insertions(+)
>>   create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
>>
>> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
>> new file mode 100644
>> index 000000000000..bfa4a8f12087
>> --- /dev/null
>> +++ b/Documentation/gpu/rfc/color_pipeline.rst
>> @@ -0,0 +1,278 @@
>> +========================
>> +Linux Color Pipeline API
>> +========================
>> +
>> +What problem are we solving?
>> +============================
>> +
>> +We would like to support pre-, and post-blending complex color transformations
>> +in order to allow for HW-supported HDR use-cases, as well as to provide support
>> +to color-managed applications, such as video or image editors.
>> +
>> +While it is possible to support an HDR output on HW supporting the Colorspace
>> +and HDR Metadata drm_connector properties that requires the compositor or
>> +application to render and compose the content into one final buffer intended for
>> +display. Doing so is costly.
>> +
>> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
>> +operations to support color transformations. These operations are often
>> +implemented in fixed-function HW and therefore much more power efficient than
>> +performing similar operations via shaders or CPU.
>> +
>> +We would like to make use of this HW functionality to support complex color
>> +transformations with no, or minimal CPU or shader load.
>> +
>> +
>> +How are other OSes solving this problem?
>> +========================================
>> +
>> +The most widely supported use-cases regard HDR content, whether video or
>> +gaming.
>> +
>> +Most OSes will specify the source content format (color gamut, encoding transfer
>> +function, and other metadata, such as max and average light levels) to a driver.
>> +Drivers will then program their fixed-function HW accordingly to map from a
>> +source content buffer's space to a display's space.
>> +
>> +When fixed-function HW is not available the compositor will assemble a shader to
>> +ask the GPU to perform the transformation from the source content format to the
>> +display's format.
>> +
>> +A compositor's mapping function and a driver's mapping function are usually
>> +entirely separate concepts. On OSes where a HW vendor has no insight into
>> +closed-source compositor code such a vendor will tune their color management
>> +code to visually match the compositor's. On other OSes, where both mapping
>> +functions are open to an implementer they will ensure both mappings match.
>> +
>> +
>> +Why is Linux different?
>> +=======================
>> +
>> +Unlike other OSes, where there is one compositor for one or more drivers, on
>> +Linux we have a many-to-many relationship. Many compositors; many drivers.
>> +In addition each compositor vendor or community has their own view of how
>> +color management should be done. This is what makes Linux so beautiful.
>> +
>> +This means that a HW vendor can now no longer tune their driver to one
>> +compositor, as tuning it to one will almost inevitably make it look very
>> +different from another compositor's color mapping.
>> +
>> +We need a better solution.
>> +
>> +
>> +Descriptive API
>> +===============
>> +
>> +An API that describes the source and destination colorspaces is a descriptive
>> +API. It describes the input and output color spaces but does not describe
>> +how precisely they should be mapped. Such a mapping includes many minute
>> +design decision that can greatly affect the look of the final result.
>> +
>> +It is not feasible to describe such mapping with enough detail to ensure the
>> +same result from each implementation. In fact, these mappings are a very active
>> +research area.
>> +
>> +
>> +Prescriptive API
>> +================
>> +
>> +A prescriptive API describes not the source and destination colorspaces. It
>> +instead prescribes a recipe for how to manipulate pixel values to arrive at the
>> +desired outcome.
>> +
>> +This recipe is generally an order straight-forward operations, with clear
>> +mathematical definitions, such as 1D LUTs, 3D LUTs, matrices, or other
>> +operations that can be described in a precise manner.
>> +
>> +
>> +The Color Pipeline API
>> +======================
>> +
>> +HW color management pipelines can significantly differ between HW
>> +vendors in terms of availability, ordering, and capabilities of HW
>> +blocks. This makes a common definition of color management blocks and
>> +their ordering nigh impossible. Instead we are defining an API that
>> +allows user space to discover the HW capabilities.
>> +
>> +
>> +drm_colorop Object & IOCTLs
>> +===========================
>> +
>> +To support the definition of color pipelines we introduce a new DRM core
>> +object, a drm_colorop. Individual drm_colorop objects will be chained
>> +via the NEXT property of a drm_colorop to constitute a color pipeline.
>> +Each drm_colorop object is unique, i.e., even if multiple color
>> +pipelines have the same operation they won't share the same drm_colorop
>> +object to describe that operation.
>> +
>> +Just like other DRM objects the drm_colorop objects are discovered via
>> +IOCTLs:
>> +
>> +DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve the
>> +number of all drm_colorop objects.
>> +
>> +DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
>> +It includes the ID for the colorop object, as well as the plane_id of
>> +the associated plane. All other values should be registered as
>> +properties.
>> +
>> +Each drm_colorop has three core properties:
> 
> We talked a bit about how forwards compatibility will look like. When a
> driver introduces a new property that user space doesn't understand yet,
> for user space to make use of the pipeline, the property either has to
> have a bypass property, or some other kind of standard/core property
> that promises that this property is purely informational and the
> existance doesn't change the result of the color pipeline. Something
> like "INFORMATIONAL" or "INFO" (better suggestions welcome!).
> 
> If a property doesn't fall into the two categories, the new pipeline
> would be unusable for user space. So, if this new property is added on
> an existing piece of hardware, it should be done on an entirely new
> pipeline to avoid user space regressions.
> 
> I believe these compatibility considerations are very important to
> document.
> 

Should a property have a "bypass" or would it be enough if a colorop
has a "bypass"?

I might need to digest what you said a bit more... Not sure I fully
understand yet, in particular how one would bypass a particular
property.

>> +
>> +TYPE: The type of transformation, such as
>> +* enumerated curve
>> +* custom (uniform) 1D LUT
>> +* 3x3 matrix
>> +* 3x4 matrix
>> +* 3D LUT
>> +* etc.
>> +
>> +Depending on the type of transformation other properties will describe
>> +more details.
>> +
>> +BYPASS: A boolean property that can be used to easily put a block into
>> +bypass mode. While setting other properties might fail atomic check,
>> +setting the BYPASS property to true should never fail. This allows DRM
>> +clients to fallback to other methods of color management if an atomic
>> +check for KMS color operations fails.
> 
> This says BYPASS is a prop but in the example below the type property is
> a mutable enum with Bypass being an enum variant.
> 
> Any reason why you chose to make the type property mutable instead of a
> separate BYPASS property? Personally I like the simplicity of immutable
> properties and would favor a BYPASS property.
> 

My bad, I copy-pasted from Simon's draft but my actual implementation
already looks different. Type is immutable. BYPASS is a separate mutable 
property.

>> +
>> +NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
>> +drm_colorop is the last in the chain.
>> +
>> +An example of a drm_colorop object might look like one of these::
>> +
>> +    Color operation 42
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, BT.709, HLG, …} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +
>> +    Color operation 42
>> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
>> +    ├─ "lut_size": immutable range = 33
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +
>> +    Color operation 42
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +
>> +
>> +COLOR_PIPELINE Plane Property
>> +=============================
>> +
>> +Because we don't have existing KMS color properties in the pre-blending
>> +portion of display pipelines (i.e. on drm_planes) we are introducing
>> +color pipelines here first. Eventually we'll want to use the same
>> +concept for the post-blending portion, i.e. drm_crtcs.
>> +
>> +Color Pipelines are created by a driver and advertised via a new
>> +COLOR_PIPELINE enum property on each plane. Values of the property
>> +always include '0', which is the default and means all color processing
>> +is disabled. Additional values will be the object IDs of the first
> 
> Regarding the requirement to never fail atomic checks which set a
> colorop to bypass: I think for user space it would be sufficient if
> setting the entire pipeline to bypass would not fail the atomic check.
> 
> I'm also wondering if there is/will be hardware out there which always
> does some kind of transformation which cannot be turned off. How would
> they implement the '0' pipeline?
> 

Interesting thought. Not sure if that exists. Maybe in that case
COLOR_PIPELINE doesn't ever expose 0 and we use an (immutable) 
drm_colorop to describe the operation.

>> +drm_colorop in a pipeline. A driver can create and advertise none, one,
>> +or more possible color pipelines. A DRM client will select a color
>> +pipeline by setting the COLOR PIPELINE to the respective value.
>> +
>> +In the case where drivers have custom support for pre-blending color
>> +processing those drivers shall reject atomic commits that are trying to
>> +set both the custom color properties, as well as the COLOR_PIPELINE
>> +property.
> 
> By 'not setting the COLOR_PIPELINE property' you mean any value other
> than '0' I guess? Should be fine to a proper user space.
> 

Correct, I should tighten up my language here.

Harry

>> +
>> +An example of a COLOR_PIPELINE property on a plane might look like this::
>> +
>> +    Plane 10
>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>> +    ├─ …
>> +    └─ "color_pipeline": enum {0, 42, 52} = 0
>> +
>> +
>> +Color Pipeline Discovery
>> +========================
>> +
>> +A DRM client wanting color management on a drm_plane will:
>> +
>> +1. Read all drm_colorop objects
>> +2. Get the COLOR_PIPELINE property of the plane
>> +3. iterate all COLOR_PIPELINE enum values
>> +4. for each enum value walk the color pipeline (via the NEXT pointers)
>> +   and see if the available color operations are suitable for the
>> +   desired color management operations
>> +
>> +An example of chained properties to define an AMD pre-blending color
>> +pipeline might look like this::
>> +
>> +    Plane 10
>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>> +    └─ "color_pipeline": enum {0, 42} = 0
>> +    Color operation 42 (input CSC)
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +    Color operation 43
>> +    ├─ "type": enum {Scaling} = Scaling
>> +    └─ "next": immutable color operation ID = 44
>> +    Color operation 44 (DeGamma)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
>> +    └─ "next": immutable color operation ID = 45
>> +    Color operation 45 (gamut remap)
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 46
>> +    Color operation 46 (shaper LUT RAM)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 47
>> +    Color operation 47 (3D LUT RAM)
>> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
>> +    ├─ "lut_size": immutable range = 17
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 48
>> +    Color operation 48 (blend gamma)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 0
>> +
>> +
>> +Color Pipeline Programming
>> +==========================
>> +
>> +Once a DRM client has found a suitable pipeline it will:
>> +
>> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
>> +   drm_colorop object of the desired pipeline
>> +2. Set the properties for all drm_colorop objects in the pipeline to the
>> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
>> +   and false for enabled drm_colorop blocks
>> +3. Perform atomic_check/commit as desired
>> +
>> +To configure the pipeline for an HDR10 PQ plane and blending in linear
>> +space, a compositor might perform an atomic commit with the following
>> +property values::
>> +
>> +    Plane 10
>> +    └─ "color_pipeline" = 42
>> +    Color operation 42 (input CSC)
>> +    └─ "bypass" = true
>> +    Color operation 44 (DeGamma)
>> +    └─ "bypass" = true
>> +    Color operation 45 (gamut remap)
>> +    └─ "bypasse" = true
>> +    Color operation 46 (shaper LUT RAM)
>> +    └─ "bypass" = true
>> +    Color operation 47 (3D LUT RAM)
>> +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
>> +    Color operation 48 (blend gamma)
>> +    └─ "1d_curve_type" = PQ inverse EOTF
>> +
>> +
>> +References
>> +==========
>> +
>> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>> \ No newline at end of file
>> -- 
>> 2.42.0
>>
> 

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 18066 bytes --]

On Fri, 8 Sep 2023 11:02:26 -0400
Harry Wentland <harry.wentland@amd.com> wrote:

> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: Jonas Ådahl <jadahl@redhat.com>
> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> Cc: Shashank Sharma <shashank.sharma@amd.com>
> Cc: Alexander Goins <agoins@nvidia.com>
> Cc: Joshua Ashton <joshua@froggi.es>
> Cc: Michel Dänzer <mdaenzer@redhat.com>
> Cc: Aleix Pol <aleixpol@kde.org>
> Cc: Xaver Hugl <xaver.hugl@gmail.com>
> Cc: Victoria Brekenfeld <victoria@system76.com>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Uma Shankar <uma.shankar@intel.com>
> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> Cc: Christopher Braga <quic_cbraga@quicinc.com>
> ---
>  Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
>  1 file changed, 278 insertions(+)
>  create mode 100644 Documentation/gpu/rfc/color_pipeline.rst

Hi Harry,

it's really nice to see this!

Sebastian started on the backward/forward compatibility, so I'll
comment on everything else here, and leave the compatibility for that
thread.

> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
> new file mode 100644
> index 000000000000..bfa4a8f12087
> --- /dev/null
> +++ b/Documentation/gpu/rfc/color_pipeline.rst
> @@ -0,0 +1,278 @@
> +========================
> +Linux Color Pipeline API
> +========================
> +
> +What problem are we solving?
> +============================
> +
> +We would like to support pre-, and post-blending complex color transformations

+in display controller hardware

> +in order to allow for HW-supported HDR use-cases, as well as to provide support
> +to color-managed applications, such as video or image editors.
> +
> +While it is possible to support an HDR output on HW supporting the Colorspace
> +and HDR Metadata drm_connector properties that requires the compositor or
> +application to render and compose the content into one final buffer intended for
> +display. Doing so is costly.

I think a tiny re-wording would make it easier to read:

+~While i~*I*t is possible to support an HDR output on HW supporting the Colorspace
+and HDR Metadata drm_connector properties*, but* that requires the compositor or
+application to render and compose the content into one final buffer intended for
+display. Doing so is costly.

deletion ~~
addition **


> +
> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
> +operations to support color transformations. These operations are often
> +implemented in fixed-function HW and therefore much more power efficient than
> +performing similar operations via shaders or CPU.
> +
> +We would like to make use of this HW functionality to support complex color
> +transformations with no, or minimal CPU or shader load.
> +
> +
> +How are other OSes solving this problem?
> +========================================
> +
> +The most widely supported use-cases regard HDR content, whether video or
> +gaming.
> +
> +Most OSes will specify the source content format (color gamut, encoding transfer
> +function, and other metadata, such as max and average light levels) to a driver.
> +Drivers will then program their fixed-function HW accordingly to map from a
> +source content buffer's space to a display's space.
> +
> +When fixed-function HW is not available the compositor will assemble a shader to
> +ask the GPU to perform the transformation from the source content format to the
> +display's format.
> +
> +A compositor's mapping function and a driver's mapping function are usually
> +entirely separate concepts. On OSes where a HW vendor has no insight into
> +closed-source compositor code such a vendor will tune their color management
> +code to visually match the compositor's. On other OSes, where both mapping
> +functions are open to an implementer they will ensure both mappings match.
> +

I'd add, assuming it's true:

This results in mapping algorithm lock-in, meaning that no-one alone can
experiment with or introduce new mapping algorithms and achieve
consistent results regardless of which implementation path is taken.

> +
> +Why is Linux different?
> +=======================
> +
> +Unlike other OSes, where there is one compositor for one or more drivers, on
> +Linux we have a many-to-many relationship. Many compositors; many drivers.
> +In addition each compositor vendor or community has their own view of how
> +color management should be done. This is what makes Linux so beautiful.
> +
> +This means that a HW vendor can now no longer tune their driver to one
> +compositor, as tuning it to one will almost inevitably make it look very
> +different from another compositor's color mapping.

This is easy to misunderstand as "all Linux desktops will get your
colors different so you cannot have a consistent look in an app". That
might trigger a few flamewars, even though it is a true goal in essence.

Maybe "almost inevitably" should be worded much more uncertain. Maybe
just "could make it look fairly different".

Much of color management is about user preferences. Different desktops
may have different sets of tunables, like different monitors and TV
have different tunables and color modes. It is still also an active
research area, and new image formats and new ways of driving displays
will surely emerge.

Different use cases have different fundamental goals with color
management, which warrants the "different colors". Linux should be
applicable to a wide range of use cases.

If these thoughts give you ideas how to rewrite this section, go for
it, but it could also be enough to change the couple words I suggested.


> +
> +We need a better solution.
> +
> +
> +Descriptive API
> +===============
> +
> +An API that describes the source and destination colorspaces is a descriptive
> +API. It describes the input and output color spaces but does not describe
> +how precisely they should be mapped. Such a mapping includes many minute
> +design decision that can greatly affect the look of the final result.
> +
> +It is not feasible to describe such mapping with enough detail to ensure the
> +same result from each implementation. In fact, these mappings are a very active
> +research area.
> +
> +
> +Prescriptive API
> +================
> +
> +A prescriptive API describes not the source and destination colorspaces. It
> +instead prescribes a recipe for how to manipulate pixel values to arrive at the
> +desired outcome.
> +
> +This recipe is generally an order straight-forward operations, with clear

Is this line missing some words?

> +mathematical definitions, such as 1D LUTs, 3D LUTs, matrices, or other
> +operations that can be described in a precise manner.
> +
> +
> +The Color Pipeline API
> +======================
> +
> +HW color management pipelines can significantly differ between HW
> +vendors in terms of availability, ordering, and capabilities of HW
> +blocks. This makes a common definition of color management blocks and
> +their ordering nigh impossible. Instead we are defining an API that
> +allows user space to discover the HW capabilities.

+in a generic manner, agnostic of specific drivers and hardware.

> +
> +
> +drm_colorop Object & IOCTLs
> +===========================
> +
> +To support the definition of color pipelines we introduce a new DRM core

"we define the DRM core object type drm_colorop."

I think that reads better after being merged upstream.

> +object, a drm_colorop. Individual drm_colorop objects will be chained
> +via the NEXT property of a drm_colorop to constitute a color pipeline.
> +Each drm_colorop object is unique, i.e., even if multiple color
> +pipelines have the same operation they won't share the same drm_colorop
> +object to describe that operation.

Maybe add some words here how drivers are not expected to map a
drm_colorop object statically to a specific HW block?

If someone was to assume the contrary, they would be wondering how they
can ever expose multiple pipelines.

The mapping between drm_colorop objects and HW blocks is completely a
driver internal detail, and can be as dynamic or static as the driver
needs it to be.

Speaking of drivers, I remember recently writing an email about what
would be a good way to expose HW functionality as drm_colorops.
Something about that would be good to document for driver writers,
maybe in a section of its own:

- Try to expose pipelines that use already defined colorops, even if
  your hardware pipeline is split differently. This allows existing
  userspace to immediately take advantage of the hardware.

- Additionally, try to expose your actual hardware blocks as colorops.
  Define new colorop types where you believe it can offer significant
  benefits if userspace learns to program them.

- Avoid defining new colorops for compound operations with very narrow
  scope. If you have a hardware block for a special operation that
  cannot be split further, you can expose that as a new colorop type.
  However, try to not define colorops for "use cases", especially if
  they require you to combine multiple hardware blocks.

- Design new colorops as prescriptive, not descriptive; by the
  mathematical formula, not by the assumed input and output.

A defined colorop type must be deterministic. Its operation can depend
only on its properties (and input?) and nothing else, allowed error
tolerance notwithstanding.

(By input I'm thinking of a block that maintains some state from
previous frame color statistics, and adjusts its behaviour.)

> +
> +Just like other DRM objects the drm_colorop objects are discovered via
> +IOCTLs:
> +
> +DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve the
> +number of all drm_colorop objects.

What is this useful for? Isn't the COLOR_PIPELINE plane property enough
to discover everything?

> +DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
> +It includes the ID for the colorop object, as well as the plane_id of
> +the associated plane. All other values should be registered as
> +properties.

Why should plane_id not be a property? Could we remove GETCOLOROP
completely?

> +
> +Each drm_colorop has three core properties:
> +
> +TYPE: The type of transformation, such as
> +* enumerated curve
> +* custom (uniform) 1D LUT
> +* 3x3 matrix
> +* 3x4 matrix
> +* 3D LUT
> +* etc.
> +
> +Depending on the type of transformation other properties will describe
> +more details.
> +
> +BYPASS: A boolean property that can be used to easily put a block into
> +bypass mode. While setting other properties might fail atomic check,
> +setting the BYPASS property to true should never fail. This allows DRM
> +clients to fallback to other methods of color management if an atomic
> +check for KMS color operations fails.
> +
> +NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
> +drm_colorop is the last in the chain.
> +
> +An example of a drm_colorop object might look like one of these::
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, BT.709, HLG, …} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> +    ├─ "lut_size": immutable range = 33
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +
> +COLOR_PIPELINE Plane Property
> +=============================
> +
> +Because we don't have existing KMS color properties in the pre-blending
> +portion of display pipelines (i.e. on drm_planes) we are introducing
> +color pipelines here first. Eventually we'll want to use the same
> +concept for the post-blending portion, i.e. drm_crtcs.

This paragraph might fit better in a cover letter.

> +
> +Color Pipelines are created by a driver and advertised via a new
> +COLOR_PIPELINE enum property on each plane. Values of the property
> +always include '0', which is the default and means all color processing
> +is disabled. Additional values will be the object IDs of the first
> +drm_colorop in a pipeline. A driver can create and advertise none, one,
> +or more possible color pipelines. A DRM client will select a color
> +pipeline by setting the COLOR PIPELINE to the respective value.
> +
> +In the case where drivers have custom support for pre-blending color
> +processing those drivers shall reject atomic commits that are trying to
> +set both the custom color properties, as well as the COLOR_PIPELINE

s/set/use/ because one of them could be carried-over state from
previous commits while not literally set in this one.

> +property.
> +
> +An example of a COLOR_PIPELINE property on a plane might look like this::
> +
> +    Plane 10
> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> +    ├─ …
> +    └─ "color_pipeline": enum {0, 42, 52} = 0

Enum values are string names. I presume the intention here is that the
strings will never need to be parsed, and the uint64_t is always equal
to the string representation, right?

That needs a statement here. It differs from all previous uses of
enums, and e.g. requires a little bit of new API in libweston's
DRM-backend to handle since it has its own enums referring to the
string names that get mapped to the uint64_t per owning KMS object.

> +
> +
> +Color Pipeline Discovery
> +========================
> +
> +A DRM client wanting color management on a drm_plane will:
> +
> +1. Read all drm_colorop objects

What does this do?

> +2. Get the COLOR_PIPELINE property of the plane
> +3. iterate all COLOR_PIPELINE enum values
> +4. for each enum value walk the color pipeline (via the NEXT pointers)
> +   and see if the available color operations are suitable for the
> +   desired color management operations
> +
> +An example of chained properties to define an AMD pre-blending color
> +pipeline might look like this::
> +
> +    Plane 10
> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> +    └─ "color_pipeline": enum {0, 42} = 0
> +    Color operation 42 (input CSC)

I presume the string "(input CSC)" does not come from KMS, and is
actually just a comment added here by hand?


Thanks,
pq

> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 43
> +    Color operation 43
> +    ├─ "type": enum {Scaling} = Scaling
> +    └─ "next": immutable color operation ID = 44
> +    Color operation 44 (DeGamma)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
> +    └─ "next": immutable color operation ID = 45
> +    Color operation 45 (gamut remap)
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 46
> +    Color operation 46 (shaper LUT RAM)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 47
> +    Color operation 47 (3D LUT RAM)
> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> +    ├─ "lut_size": immutable range = 17
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 48
> +    Color operation 48 (blend gamma)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 0
> +
> +
> +Color Pipeline Programming
> +==========================
> +
> +Once a DRM client has found a suitable pipeline it will:
> +
> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
> +   drm_colorop object of the desired pipeline
> +2. Set the properties for all drm_colorop objects in the pipeline to the
> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
> +   and false for enabled drm_colorop blocks
> +3. Perform atomic_check/commit as desired
> +
> +To configure the pipeline for an HDR10 PQ plane and blending in linear
> +space, a compositor might perform an atomic commit with the following
> +property values::
> +
> +    Plane 10
> +    └─ "color_pipeline" = 42
> +    Color operation 42 (input CSC)
> +    └─ "bypass" = true
> +    Color operation 44 (DeGamma)
> +    └─ "bypass" = true
> +    Color operation 45 (gamut remap)
> +    └─ "bypasse" = true
> +    Color operation 46 (shaper LUT RAM)
> +    └─ "bypass" = true
> +    Color operation 47 (3D LUT RAM)
> +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
> +    Color operation 48 (blend gamma)
> +    └─ "1d_curve_type" = PQ inverse EOTF
> +
> +
> +References
> +==========
> +
> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> \ No newline at end of file


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 18983 bytes --]

On Fri, 8 Sep 2023 16:38:44 -0400
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-09-08 15:30, Sebastian Wick wrote:
> > Hey Harry,
> > 
> > Thank you and Simon for this great document. Really happy about it, but
> > obviously I've got a few notes and questions inline.
> > 
> > On Fri, Sep 08, 2023 at 11:02:26AM -0400, Harry Wentland wrote:  
> >> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> >> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> >> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> >> Cc: Simon Ser <contact@emersion.fr>
> >> Cc: Harry Wentland <harry.wentland@amd.com>
> >> Cc: Melissa Wen <mwen@igalia.com>
> >> Cc: Jonas Ådahl <jadahl@redhat.com>
> >> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> >> Cc: Shashank Sharma <shashank.sharma@amd.com>
> >> Cc: Alexander Goins <agoins@nvidia.com>
> >> Cc: Joshua Ashton <joshua@froggi.es>
> >> Cc: Michel Dänzer <mdaenzer@redhat.com>
> >> Cc: Aleix Pol <aleixpol@kde.org>
> >> Cc: Xaver Hugl <xaver.hugl@gmail.com>
> >> Cc: Victoria Brekenfeld <victoria@system76.com>
> >> Cc: Daniel Vetter <daniel@ffwll.ch>
> >> Cc: Uma Shankar <uma.shankar@intel.com>
> >> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> >> Cc: Christopher Braga <quic_cbraga@quicinc.com>
> >> ---
> >>   Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
> >>   1 file changed, 278 insertions(+)
> >>   create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
> >>
> >> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
> >> new file mode 100644
> >> index 000000000000..bfa4a8f12087
> >> --- /dev/null
> >> +++ b/Documentation/gpu/rfc/color_pipeline.rst
> >> @@ -0,0 +1,278 @@
> >> +========================
> >> +Linux Color Pipeline API
> >> +========================
> >> +
> >> +What problem are we solving?
> >> +============================
> >> +
> >> +We would like to support pre-, and post-blending complex color transformations
> >> +in order to allow for HW-supported HDR use-cases, as well as to provide support
> >> +to color-managed applications, such as video or image editors.
> >> +
> >> +While it is possible to support an HDR output on HW supporting the Colorspace
> >> +and HDR Metadata drm_connector properties that requires the compositor or
> >> +application to render and compose the content into one final buffer intended for
> >> +display. Doing so is costly.
> >> +
> >> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
> >> +operations to support color transformations. These operations are often
> >> +implemented in fixed-function HW and therefore much more power efficient than
> >> +performing similar operations via shaders or CPU.
> >> +
> >> +We would like to make use of this HW functionality to support complex color
> >> +transformations with no, or minimal CPU or shader load.
> >> +
> >> +
> >> +How are other OSes solving this problem?
> >> +========================================
> >> +
> >> +The most widely supported use-cases regard HDR content, whether video or
> >> +gaming.
> >> +
> >> +Most OSes will specify the source content format (color gamut, encoding transfer
> >> +function, and other metadata, such as max and average light levels) to a driver.
> >> +Drivers will then program their fixed-function HW accordingly to map from a
> >> +source content buffer's space to a display's space.
> >> +
> >> +When fixed-function HW is not available the compositor will assemble a shader to
> >> +ask the GPU to perform the transformation from the source content format to the
> >> +display's format.
> >> +
> >> +A compositor's mapping function and a driver's mapping function are usually
> >> +entirely separate concepts. On OSes where a HW vendor has no insight into
> >> +closed-source compositor code such a vendor will tune their color management
> >> +code to visually match the compositor's. On other OSes, where both mapping
> >> +functions are open to an implementer they will ensure both mappings match.
> >> +
> >> +
> >> +Why is Linux different?
> >> +=======================
> >> +
> >> +Unlike other OSes, where there is one compositor for one or more drivers, on
> >> +Linux we have a many-to-many relationship. Many compositors; many drivers.
> >> +In addition each compositor vendor or community has their own view of how
> >> +color management should be done. This is what makes Linux so beautiful.
> >> +
> >> +This means that a HW vendor can now no longer tune their driver to one
> >> +compositor, as tuning it to one will almost inevitably make it look very
> >> +different from another compositor's color mapping.
> >> +
> >> +We need a better solution.
> >> +
> >> +
> >> +Descriptive API
> >> +===============
> >> +
> >> +An API that describes the source and destination colorspaces is a descriptive
> >> +API. It describes the input and output color spaces but does not describe
> >> +how precisely they should be mapped. Such a mapping includes many minute
> >> +design decision that can greatly affect the look of the final result.
> >> +
> >> +It is not feasible to describe such mapping with enough detail to ensure the
> >> +same result from each implementation. In fact, these mappings are a very active
> >> +research area.
> >> +
> >> +
> >> +Prescriptive API
> >> +================
> >> +
> >> +A prescriptive API describes not the source and destination colorspaces. It
> >> +instead prescribes a recipe for how to manipulate pixel values to arrive at the
> >> +desired outcome.
> >> +
> >> +This recipe is generally an order straight-forward operations, with clear
> >> +mathematical definitions, such as 1D LUTs, 3D LUTs, matrices, or other
> >> +operations that can be described in a precise manner.
> >> +
> >> +
> >> +The Color Pipeline API
> >> +======================
> >> +
> >> +HW color management pipelines can significantly differ between HW
> >> +vendors in terms of availability, ordering, and capabilities of HW
> >> +blocks. This makes a common definition of color management blocks and
> >> +their ordering nigh impossible. Instead we are defining an API that
> >> +allows user space to discover the HW capabilities.
> >> +
> >> +
> >> +drm_colorop Object & IOCTLs
> >> +===========================
> >> +
> >> +To support the definition of color pipelines we introduce a new DRM core
> >> +object, a drm_colorop. Individual drm_colorop objects will be chained
> >> +via the NEXT property of a drm_colorop to constitute a color pipeline.
> >> +Each drm_colorop object is unique, i.e., even if multiple color
> >> +pipelines have the same operation they won't share the same drm_colorop
> >> +object to describe that operation.
> >> +
> >> +Just like other DRM objects the drm_colorop objects are discovered via
> >> +IOCTLs:
> >> +
> >> +DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve the
> >> +number of all drm_colorop objects.
> >> +
> >> +DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
> >> +It includes the ID for the colorop object, as well as the plane_id of
> >> +the associated plane. All other values should be registered as
> >> +properties.
> >> +
> >> +Each drm_colorop has three core properties:  
> > 
> > We talked a bit about how forwards compatibility will look like. When a
> > driver introduces a new property that user space doesn't understand yet,
> > for user space to make use of the pipeline, the property either has to

A new property or colorop?

> > have a bypass property, or some other kind of standard/core property
> > that promises that this property is purely informational and the
> > existance doesn't change the result of the color pipeline. Something
> > like "INFORMATIONAL" or "INFO" (better suggestions welcome!).
> > 
> > If a property doesn't fall into the two categories, the new pipeline
> > would be unusable for user space. So, if this new property is added on
> > an existing piece of hardware, it should be done on an entirely new
> > pipeline to avoid user space regressions.
> > 
> > I believe these compatibility considerations are very important to
> > document.
> >   
> 
> Should a property have a "bypass" or would it be enough if a colorop
> has a "bypass"?
> 
> I might need to digest what you said a bit more... Not sure I fully
> understand yet, in particular how one would bypass a particular
> property.

I suspect new colorops in an old pipeline need different handling than
a new property in an old colorop.

What if bypass is not what happened before a new colorop was added in a
pipeline?

Let's have an example: the CRTC automatic RGB->YUV conversion to the
wire format applied as needed. Previously it was all implicit,
automatic, and not exposed. Now one wants to expose it as a new colorop
in the pipeline. Let's say you also expose the old pipeline without
this.

How will userspace choose between a pipeline without the colorop, and a
pipeline with the colorop set to bypass?

The definition of bypass is that it does not alter the values passing
through, but that's not what happens without the colorop.

Maybe this colorop must not have the standard bypass property.

Maybe "bypass" should be "default" instead. But then the color pipeline
results become undefined from userspace perspective, so that doesn't
work.

Maybe we need a rule that everything that modifies pixel values in a
pipeline MUST be exposed from the very moment the COLOR_PIPELINE
property is added to a KMS object. HW blocks that a driver hardcodes to
pass-through and so do not modify any values do not need to be exposed;
they can be trivially exposed later with "bypass" property.

That last "maybe" is actually a hard requirement for the whole color
pipeline design to work, because otherwise the results are unknown to
userspace.

What should a driver writer do if hardware implicitly clamps values at
some point? That potentially modifies values, so it must be taken into
account, especially when it is possible to exceed the nominal unit
range [0.0, 1.0].

What if a driver writer misses something or makes a mistake? Can that
be fixed a few releases later without regressing userspace? I feel like
that last "maybe" might be difficult to ensure.

> >> +
> >> +TYPE: The type of transformation, such as
> >> +* enumerated curve
> >> +* custom (uniform) 1D LUT
> >> +* 3x3 matrix
> >> +* 3x4 matrix
> >> +* 3D LUT
> >> +* etc.
> >> +
> >> +Depending on the type of transformation other properties will describe
> >> +more details.
> >> +
> >> +BYPASS: A boolean property that can be used to easily put a block into
> >> +bypass mode. While setting other properties might fail atomic check,
> >> +setting the BYPASS property to true should never fail. This allows DRM
> >> +clients to fallback to other methods of color management if an atomic
> >> +check for KMS color operations fails.  
> > 
> > This says BYPASS is a prop but in the example below the type property is
> > a mutable enum with Bypass being an enum variant.
> > 
> > Any reason why you chose to make the type property mutable instead of a
> > separate BYPASS property? Personally I like the simplicity of immutable
> > properties and would favor a BYPASS property.
> >   
> 
> My bad, I copy-pasted from Simon's draft but my actual implementation
> already looks different. Type is immutable. BYPASS is a separate mutable 
> property.
> 
> >> +
> >> +NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
> >> +drm_colorop is the last in the chain.
> >> +
> >> +An example of a drm_colorop object might look like one of these::
> >> +
> >> +    Color operation 42
> >> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> >> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, BT.709, HLG, …} = LUT
> >> +    ├─ "lut_size": immutable range = 4096
> >> +    ├─ "lut_data": blob
> >> +    └─ "next": immutable color operation ID = 43
> >> +
> >> +    Color operation 42
> >> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> >> +    ├─ "lut_size": immutable range = 33
> >> +    ├─ "lut_data": blob
> >> +    └─ "next": immutable color operation ID = 43
> >> +
> >> +    Color operation 42
> >> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> >> +    ├─ "matrix_data": blob
> >> +    └─ "next": immutable color operation ID = 43
> >> +
> >> +
> >> +COLOR_PIPELINE Plane Property
> >> +=============================
> >> +
> >> +Because we don't have existing KMS color properties in the pre-blending
> >> +portion of display pipelines (i.e. on drm_planes) we are introducing
> >> +color pipelines here first. Eventually we'll want to use the same
> >> +concept for the post-blending portion, i.e. drm_crtcs.
> >> +
> >> +Color Pipelines are created by a driver and advertised via a new
> >> +COLOR_PIPELINE enum property on each plane. Values of the property
> >> +always include '0', which is the default and means all color processing
> >> +is disabled. Additional values will be the object IDs of the first  
> > 
> > Regarding the requirement to never fail atomic checks which set a
> > colorop to bypass: I think for user space it would be sufficient if
> > setting the entire pipeline to bypass would not fail the atomic check.

I agree.

> > 
> > I'm also wondering if there is/will be hardware out there which always
> > does some kind of transformation which cannot be turned off. How would
> > they implement the '0' pipeline?
> >   
> 
> Interesting thought. Not sure if that exists. Maybe in that case
> COLOR_PIPELINE doesn't ever expose 0 and we use an (immutable) 
> drm_colorop to describe the operation.

Right.


Thanks,
pq

> 
> >> +drm_colorop in a pipeline. A driver can create and advertise none, one,
> >> +or more possible color pipelines. A DRM client will select a color
> >> +pipeline by setting the COLOR PIPELINE to the respective value.
> >> +
> >> +In the case where drivers have custom support for pre-blending color
> >> +processing those drivers shall reject atomic commits that are trying to
> >> +set both the custom color properties, as well as the COLOR_PIPELINE
> >> +property.  
> > 
> > By 'not setting the COLOR_PIPELINE property' you mean any value other
> > than '0' I guess? Should be fine to a proper user space.
> >   
> 
> Correct, I should tighten up my language here.
> 
> Harry
> 
> >> +
> >> +An example of a COLOR_PIPELINE property on a plane might look like this::
> >> +
> >> +    Plane 10
> >> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> >> +    ├─ …
> >> +    └─ "color_pipeline": enum {0, 42, 52} = 0
> >> +
> >> +
> >> +Color Pipeline Discovery
> >> +========================
> >> +
> >> +A DRM client wanting color management on a drm_plane will:
> >> +
> >> +1. Read all drm_colorop objects
> >> +2. Get the COLOR_PIPELINE property of the plane
> >> +3. iterate all COLOR_PIPELINE enum values
> >> +4. for each enum value walk the color pipeline (via the NEXT pointers)
> >> +   and see if the available color operations are suitable for the
> >> +   desired color management operations
> >> +
> >> +An example of chained properties to define an AMD pre-blending color
> >> +pipeline might look like this::
> >> +
> >> +    Plane 10
> >> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> >> +    └─ "color_pipeline": enum {0, 42} = 0
> >> +    Color operation 42 (input CSC)
> >> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> >> +    ├─ "matrix_data": blob
> >> +    └─ "next": immutable color operation ID = 43
> >> +    Color operation 43
> >> +    ├─ "type": enum {Scaling} = Scaling
> >> +    └─ "next": immutable color operation ID = 44
> >> +    Color operation 44 (DeGamma)
> >> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> >> +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
> >> +    └─ "next": immutable color operation ID = 45
> >> +    Color operation 45 (gamut remap)
> >> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> >> +    ├─ "matrix_data": blob
> >> +    └─ "next": immutable color operation ID = 46
> >> +    Color operation 46 (shaper LUT RAM)
> >> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> >> +    ├─ "1d_curve_type": enum {LUT} = LUT
> >> +    ├─ "lut_size": immutable range = 4096
> >> +    ├─ "lut_data": blob
> >> +    └─ "next": immutable color operation ID = 47
> >> +    Color operation 47 (3D LUT RAM)
> >> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> >> +    ├─ "lut_size": immutable range = 17
> >> +    ├─ "lut_data": blob
> >> +    └─ "next": immutable color operation ID = 48
> >> +    Color operation 48 (blend gamma)
> >> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> >> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
> >> +    ├─ "lut_size": immutable range = 4096
> >> +    ├─ "lut_data": blob
> >> +    └─ "next": immutable color operation ID = 0
> >> +
> >> +
> >> +Color Pipeline Programming
> >> +==========================
> >> +
> >> +Once a DRM client has found a suitable pipeline it will:
> >> +
> >> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
> >> +   drm_colorop object of the desired pipeline
> >> +2. Set the properties for all drm_colorop objects in the pipeline to the
> >> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
> >> +   and false for enabled drm_colorop blocks
> >> +3. Perform atomic_check/commit as desired
> >> +
> >> +To configure the pipeline for an HDR10 PQ plane and blending in linear
> >> +space, a compositor might perform an atomic commit with the following
> >> +property values::
> >> +
> >> +    Plane 10
> >> +    └─ "color_pipeline" = 42
> >> +    Color operation 42 (input CSC)
> >> +    └─ "bypass" = true
> >> +    Color operation 44 (DeGamma)
> >> +    └─ "bypass" = true
> >> +    Color operation 45 (gamut remap)
> >> +    └─ "bypasse" = true
> >> +    Color operation 46 (shaper LUT RAM)
> >> +    └─ "bypass" = true
> >> +    Color operation 47 (3D LUT RAM)
> >> +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
> >> +    Color operation 48 (blend gamma)
> >> +    └─ "1d_curve_type" = PQ inverse EOTF
> >> +
> >> +
> >> +References
> >> +==========
> >> +
> >> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> >> \ No newline at end of file
> >> -- 
> >> 2.42.0
> >>  
> >   


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Melissa Wen]

O 09/08, Harry Wentland wrote:
> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: Jonas Ådahl <jadahl@redhat.com>
> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> Cc: Shashank Sharma <shashank.sharma@amd.com>
> Cc: Alexander Goins <agoins@nvidia.com>
> Cc: Joshua Ashton <joshua@froggi.es>
> Cc: Michel Dänzer <mdaenzer@redhat.com>
> Cc: Aleix Pol <aleixpol@kde.org>
> Cc: Xaver Hugl <xaver.hugl@gmail.com>
> Cc: Victoria Brekenfeld <victoria@system76.com>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Uma Shankar <uma.shankar@intel.com>
> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> Cc: Christopher Braga <quic_cbraga@quicinc.com>
> ---
>  Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
>  1 file changed, 278 insertions(+)
>  create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
> 
> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
> new file mode 100644
> index 000000000000..bfa4a8f12087
> --- /dev/null
> +++ b/Documentation/gpu/rfc/color_pipeline.rst
> @@ -0,0 +1,278 @@
> +========================
> +Linux Color Pipeline API
> +========================
> +
> +What problem are we solving?
> +============================
> +
> +We would like to support pre-, and post-blending complex color transformations
> +in order to allow for HW-supported HDR use-cases, as well as to provide support
> +to color-managed applications, such as video or image editors.
> +
> +While it is possible to support an HDR output on HW supporting the Colorspace
> +and HDR Metadata drm_connector properties that requires the compositor or
> +application to render and compose the content into one final buffer intended for
> +display. Doing so is costly.
> +
> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
> +operations to support color transformations. These operations are often
> +implemented in fixed-function HW and therefore much more power efficient than
> +performing similar operations via shaders or CPU.
> +
> +We would like to make use of this HW functionality to support complex color
> +transformations with no, or minimal CPU or shader load.
> +
> +
> +How are other OSes solving this problem?
> +========================================
> +
> +The most widely supported use-cases regard HDR content, whether video or
> +gaming.
> +
> +Most OSes will specify the source content format (color gamut, encoding transfer
> +function, and other metadata, such as max and average light levels) to a driver.
> +Drivers will then program their fixed-function HW accordingly to map from a
> +source content buffer's space to a display's space.
> +
> +When fixed-function HW is not available the compositor will assemble a shader to
> +ask the GPU to perform the transformation from the source content format to the
> +display's format.
> +
> +A compositor's mapping function and a driver's mapping function are usually
> +entirely separate concepts. On OSes where a HW vendor has no insight into
> +closed-source compositor code such a vendor will tune their color management
> +code to visually match the compositor's. On other OSes, where both mapping
> +functions are open to an implementer they will ensure both mappings match.
> +
> +
> +Why is Linux different?
> +=======================
> +
> +Unlike other OSes, where there is one compositor for one or more drivers, on
> +Linux we have a many-to-many relationship. Many compositors; many drivers.
> +In addition each compositor vendor or community has their own view of how
> +color management should be done. This is what makes Linux so beautiful.
> +
> +This means that a HW vendor can now no longer tune their driver to one
> +compositor, as tuning it to one will almost inevitably make it look very
> +different from another compositor's color mapping.
> +
> +We need a better solution.
> +
> +
> +Descriptive API
> +===============
> +
> +An API that describes the source and destination colorspaces is a descriptive
> +API. It describes the input and output color spaces but does not describe
> +how precisely they should be mapped. Such a mapping includes many minute
> +design decision that can greatly affect the look of the final result.
> +
> +It is not feasible to describe such mapping with enough detail to ensure the
> +same result from each implementation. In fact, these mappings are a very active
> +research area.
> +
> +
> +Prescriptive API
> +================
> +
> +A prescriptive API describes not the source and destination colorspaces. It
> +instead prescribes a recipe for how to manipulate pixel values to arrive at the
> +desired outcome.
> +
> +This recipe is generally an order straight-forward operations, with clear
> +mathematical definitions, such as 1D LUTs, 3D LUTs, matrices, or other
> +operations that can be described in a precise manner.
> +
> +
> +The Color Pipeline API
> +======================
> +
> +HW color management pipelines can significantly differ between HW
> +vendors in terms of availability, ordering, and capabilities of HW
> +blocks. This makes a common definition of color management blocks and
> +their ordering nigh impossible. Instead we are defining an API that
> +allows user space to discover the HW capabilities.
> +
> +
> +drm_colorop Object & IOCTLs
> +===========================
> +
> +To support the definition of color pipelines we introduce a new DRM core
> +object, a drm_colorop. Individual drm_colorop objects will be chained
> +via the NEXT property of a drm_colorop to constitute a color pipeline.
> +Each drm_colorop object is unique, i.e., even if multiple color
> +pipelines have the same operation they won't share the same drm_colorop
> +object to describe that operation.
> +
> +Just like other DRM objects the drm_colorop objects are discovered via
> +IOCTLs:
> +
> +DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve the
> +number of all drm_colorop objects.
> +
> +DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
> +It includes the ID for the colorop object, as well as the plane_id of
> +the associated plane. All other values should be registered as
> +properties.
> +
> +Each drm_colorop has three core properties:
> +
> +TYPE: The type of transformation, such as
> +* enumerated curve
> +* custom (uniform) 1D LUT
> +* 3x3 matrix
> +* 3x4 matrix
> +* 3D LUT
> +* etc.
> +
> +Depending on the type of transformation other properties will describe
> +more details.
> +
> +BYPASS: A boolean property that can be used to easily put a block into
> +bypass mode. While setting other properties might fail atomic check,
> +setting the BYPASS property to true should never fail. This allows DRM
> +clients to fallback to other methods of color management if an atomic
> +check for KMS color operations fails.
> +
> +NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
> +drm_colorop is the last in the chain.
> +
> +An example of a drm_colorop object might look like one of these::
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, BT.709, HLG, …} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> +    ├─ "lut_size": immutable range = 33
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +    Color operation 42
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 43
> +
> +
> +COLOR_PIPELINE Plane Property
> +=============================
> +
> +Because we don't have existing KMS color properties in the pre-blending
> +portion of display pipelines (i.e. on drm_planes) we are introducing
> +color pipelines here first. Eventually we'll want to use the same
> +concept for the post-blending portion, i.e. drm_crtcs.
> +
> +Color Pipelines are created by a driver and advertised via a new
> +COLOR_PIPELINE enum property on each plane. Values of the property
> +always include '0', which is the default and means all color processing
> +is disabled. Additional values will be the object IDs of the first
> +drm_colorop in a pipeline. A driver can create and advertise none, one,
> +or more possible color pipelines. A DRM client will select a color
> +pipeline by setting the COLOR PIPELINE to the respective value.
> +
> +In the case where drivers have custom support for pre-blending color
> +processing those drivers shall reject atomic commits that are trying to
> +set both the custom color properties, as well as the COLOR_PIPELINE
> +property.
> +
> +An example of a COLOR_PIPELINE property on a plane might look like this::
> +
> +    Plane 10
> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> +    ├─ …
> +    └─ "color_pipeline": enum {0, 42, 52} = 0
> +
> +
> +Color Pipeline Discovery
> +========================
> +
> +A DRM client wanting color management on a drm_plane will:
> +
> +1. Read all drm_colorop objects
> +2. Get the COLOR_PIPELINE property of the plane
> +3. iterate all COLOR_PIPELINE enum values
> +4. for each enum value walk the color pipeline (via the NEXT pointers)
> +   and see if the available color operations are suitable for the
> +   desired color management operations
> +
> +An example of chained properties to define an AMD pre-blending color
> +pipeline might look like this::

Hi Harry,

Thanks for sharing this proposal. Overall I think it's very aligned with
Simon's description of the generic KMS color API. I think it's a good
start point and we can refine over iterations. My general questions have
already been pointed out by Sebastian and Pekka (mainly regarding the
BYPASS property).

I still have some doubts on how to fit these set of colorops with some
AMD corners cases as below:

> +
> +    Plane 10
> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> +    └─ "color_pipeline": enum {0, 42} = 0
> +    Color operation 42 (input CSC)
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 43

IIUC, for input CSC, there are currently two possiblities, or we use
`drm_plane_color_encoding` and `drm_plane_color range` to get
pre-defined coefficients or we set a custom matrix here, right? If so, I
think we need some kind of pre-defined matrix option?

Also, with this new plane API in place, I understand that we will
already need think on how to deal with the mixing between old drm color
properties (color encoding and color range) and these new way of setting
plane color properties. IIUC, Pekka asked a related question about it
when talking about CRTC automatic RGB->YUV (?) 

> +    Color operation 43
> +    ├─ "type": enum {Scaling} = Scaling
> +    └─ "next": immutable color operation ID = 44
> +    Color operation 44 (DeGamma)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
> +    └─ "next": immutable color operation ID = 45
> +    Color operation 45 (gamut remap)
> +    ├─ "type": enum {Bypass, Matrix} = Matrix
> +    ├─ "matrix_data": blob
> +    └─ "next": immutable color operation ID = 46
> +    Color operation 46 (shaper LUT RAM)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 47

For shaper and blend LUT RAM, that the driver supports pre-defined
curves and custom LUT at the same time but the resulted LUT is a
combination of those, how to make this behavior clear? Could this
behavior be described by two colorop in a row: for example, one for
shaper TF and,just after, another for shaper LUT or would it not be the
right representation?

> +    Color operation 47 (3D LUT RAM)
> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> +    ├─ "lut_size": immutable range = 17
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 48
> +    Color operation 48 (blend gamma)
> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
> +    ├─ "lut_size": immutable range = 4096
> +    ├─ "lut_data": blob
> +    └─ "next": immutable color operation ID = 0
> +
> +
> +Color Pipeline Programming
> +==========================
> +
> +Once a DRM client has found a suitable pipeline it will:
> +
> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
> +   drm_colorop object of the desired pipeline
> +2. Set the properties for all drm_colorop objects in the pipeline to the
> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
> +   and false for enabled drm_colorop blocks
> +3. Perform atomic_check/commit as desired
> +
> +To configure the pipeline for an HDR10 PQ plane and blending in linear
> +space, a compositor might perform an atomic commit with the following
> +property values::
> +
> +    Plane 10
> +    └─ "color_pipeline" = 42
> +    Color operation 42 (input CSC)
> +    └─ "bypass" = true
> +    Color operation 44 (DeGamma)
> +    └─ "bypass" = true
> +    Color operation 45 (gamut remap)
> +    └─ "bypasse" = true
> +    Color operation 46 (shaper LUT RAM)
> +    └─ "bypass" = true
> +    Color operation 47 (3D LUT RAM)
> +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
> +    Color operation 48 (blend gamma)
> +    └─ "1d_curve_type" = PQ inverse EOTF

Isn't it a PQ EOTF for blend gamma?

Best Regards,

Melissa

> +
> +
> +References
> +==========
> +
> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> \ No newline at end of file
> -- 
> 2.42.0
> 

Re: [RFC PATCH 02/10] drm/colorop: Introduce new drm_colorop mode object

[Melissa Wen]

On 09/08, Harry Wentland wrote:
> This patches introduces a new drm_colorop mode object. This
> object represents color transformations and can be used to
> define color pipelines.
> 
> We also introduce the drm_colorop_state here, as well as
> various helpers and state tracking bits.
> 
> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: Jonas Ådahl <jadahl@redhat.com>
> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> Cc: Shashank Sharma <shashank.sharma@amd.com>
> Cc: Alexander Goins <agoins@nvidia.com>
> Cc: Joshua Ashton <joshua@froggi.es>
> Cc: Michel Dänzer <mdaenzer@redhat.com>
> Cc: Aleix Pol <aleixpol@kde.org>
> Cc: Xaver Hugl <xaver.hugl@gmail.com>
> Cc: Victoria Brekenfeld <victoria@system76.com>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Uma Shankar <uma.shankar@intel.com>
> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> Cc: Christopher Braga <quic_cbraga@quicinc.com>
> ---
>  drivers/gpu/drm/Makefile            |   1 +
>  drivers/gpu/drm/drm_atomic.c        |  79 +++++++++++++
>  drivers/gpu/drm/drm_atomic_helper.c |  12 ++
>  drivers/gpu/drm/drm_atomic_uapi.c   |  48 ++++++++
>  drivers/gpu/drm/drm_colorop.c       | 169 ++++++++++++++++++++++++++++
>  drivers/gpu/drm/drm_mode_config.c   |   7 ++
>  drivers/gpu/drm/drm_plane_helper.c  |   2 +-
>  include/drm/drm_atomic.h            |  82 ++++++++++++++
>  include/drm/drm_atomic_uapi.h       |   1 +
>  include/drm/drm_colorop.h           | 157 ++++++++++++++++++++++++++
>  include/drm/drm_mode_config.h       |  18 +++
>  include/drm/drm_plane.h             |   2 +
>  include/uapi/drm/drm.h              |   3 +
>  include/uapi/drm/drm_mode.h         |   1 +
>  14 files changed, 581 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/gpu/drm/drm_colorop.c
>  create mode 100644 include/drm/drm_colorop.h
> 
> diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
> index 1855863b4d7a..941de0269709 100644
> --- a/drivers/gpu/drm/Makefile
> +++ b/drivers/gpu/drm/Makefile
> @@ -16,6 +16,7 @@ drm-y := \
>  	drm_client.o \
>  	drm_client_modeset.o \
>  	drm_color_mgmt.o \
> +	drm_colorop.o \
>  	drm_connector.o \
>  	drm_crtc.o \
>  	drm_displayid.o \
> diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
> index 11f3a130f6f4..d734e9d5bfed 100644
> --- a/drivers/gpu/drm/drm_atomic.c
> +++ b/drivers/gpu/drm/drm_atomic.c
> @@ -42,6 +42,7 @@
>  #include <drm/drm_mode.h>
>  #include <drm/drm_print.h>
>  #include <drm/drm_writeback.h>
> +#include <drm/drm_colorop.h>
>  
>  #include "drm_crtc_internal.h"
>  #include "drm_internal.h"
> @@ -108,6 +109,7 @@ void drm_atomic_state_default_release(struct drm_atomic_state *state)
>  	kfree(state->connectors);
>  	kfree(state->crtcs);
>  	kfree(state->planes);
> +	kfree(state->colorops);
>  	kfree(state->private_objs);
>  }
>  EXPORT_SYMBOL(drm_atomic_state_default_release);
> @@ -139,6 +141,10 @@ drm_atomic_state_init(struct drm_device *dev, struct drm_atomic_state *state)
>  				sizeof(*state->planes), GFP_KERNEL);
>  	if (!state->planes)
>  		goto fail;
> +	state->colorops = kcalloc(dev->mode_config.num_colorop,
> +				  sizeof(*state->colorops), GFP_KERNEL);
> +	if (!state->colorops)
> +		goto fail;
>  
>  	state->dev = dev;
>  
> @@ -244,6 +250,20 @@ void drm_atomic_state_default_clear(struct drm_atomic_state *state)
>  		state->planes[i].new_state = NULL;
>  	}
>  
> +	for (i = 0; i < config->num_colorop; i++) {
> +		struct drm_colorop *colorop = state->colorops[i].ptr;
> +
> +		if (!colorop)
> +			continue;
> +
> +		drm_colorop_atomic_destroy_state(colorop,
> +						 state->colorops[i].state);
> +		state->colorops[i].ptr = NULL;
> +		state->colorops[i].state = NULL;
> +		state->colorops[i].old_state = NULL;
> +		state->colorops[i].new_state = NULL;
> +	}
> +
>  	for (i = 0; i < state->num_private_objs; i++) {
>  		struct drm_private_obj *obj = state->private_objs[i].ptr;
>  
> @@ -562,6 +582,65 @@ drm_atomic_get_plane_state(struct drm_atomic_state *state,
>  }
>  EXPORT_SYMBOL(drm_atomic_get_plane_state);
>  
> +
> +/**
> + * drm_atomic_get_colorop_state - get colorop state
> + * @state: global atomic state object
> + * @colorop: colorop to get state object for
> + *
> + * This function returns the colorop state for the given colorop, allocating it
> + * if needed. It will also grab the relevant plane lock to make sure that the
> + * state is consistent.
> + *
> + * Returns:
> + *
> + * Either the allocated state or the error code encoded into the pointer. When
> + * the error is EDEADLK then the w/w mutex code has detected a deadlock and the
> + * entire atomic sequence must be restarted. All other errors are fatal.
> + */
> +struct drm_colorop_state *
> +drm_atomic_get_colorop_state(struct drm_atomic_state *state,
> +			     struct drm_colorop *colorop)
> +{
> +	int ret, index = drm_colorop_index(colorop);
> +	struct drm_colorop_state *colorop_state;
> +	struct drm_plane_state *plane_state;
> +
> +	WARN_ON(!state->acquire_ctx);
> +
> +	colorop_state = drm_atomic_get_existing_colorop_state(state, colorop);
> +	if (colorop_state)
> +		return colorop_state;
> +
> +	/* TODO where is the unlock? */

I understand that this is tracked by acquire-ctx and unlocked by
`drm_modeset_drop_locks()` (?)

> +	ret = drm_modeset_lock(&colorop->plane->mutex, state->acquire_ctx);
> +	if (ret)
> +		return ERR_PTR(ret);
> +
> +	colorop_state = drm_atomic_helper_colorop_duplicate_state(colorop);
> +	if (!colorop_state)
> +		return ERR_PTR(-ENOMEM);
> +
> +	state->colorops[index].state = colorop_state;
> +	state->colorops[index].ptr = colorop;
> +	state->colorops[index].old_state = colorop->state;
> +	state->colorops[index].new_state = colorop_state;
> +	colorop_state->state = state;
> +
> +	drm_dbg_atomic(colorop->dev, "Added [COLOROP:%d] %p state to %p\n",
> +		       colorop->base.id, colorop_state, state);
> +
> +	/* TODO is this necessary? */
> +
> +	plane_state = drm_atomic_get_plane_state(state,
> +						 colorop->plane);
> +	if (IS_ERR(plane_state))
> +		return ERR_CAST(plane_state);
> +
> +	return colorop_state;
> +}
> +EXPORT_SYMBOL(drm_atomic_get_colorop_state);
> +
>  static bool
>  plane_switching_crtc(const struct drm_plane_state *old_plane_state,
>  		     const struct drm_plane_state *new_plane_state)
> diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
> index d579fd8f7cb8..0472f6182c0a 100644
> --- a/drivers/gpu/drm/drm_atomic_helper.c
> +++ b/drivers/gpu/drm/drm_atomic_helper.c
> @@ -2921,6 +2921,8 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
>  	struct drm_crtc_state *old_crtc_state, *new_crtc_state;
>  	struct drm_plane *plane;
>  	struct drm_plane_state *old_plane_state, *new_plane_state;
> +	struct drm_colorop *colorop;
> +	struct drm_colorop_state *old_colorop_state, *new_colorop_state;
>  	struct drm_crtc_commit *commit;
>  	struct drm_private_obj *obj;
>  	struct drm_private_state *old_obj_state, *new_obj_state;
> @@ -2998,6 +3000,16 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
>  		}
>  	}
>  
> +	for_each_oldnew_colorop_in_state(state, colorop, old_colorop_state, new_colorop_state, i) {
> +		WARN_ON(colorop->state != old_colorop_state);
> +
> +		old_colorop_state->state = state;
> +		new_colorop_state->state = NULL;
> +
> +		state->colorops[i].state = old_colorop_state;
> +		colorop->state = new_colorop_state;
> +	}
> +
>  	for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
>  		WARN_ON(plane->state != old_plane_state);
>  
> diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
> index d867e7f9f2cd..b1aa752c1848 100644
> --- a/drivers/gpu/drm/drm_atomic_uapi.c
> +++ b/drivers/gpu/drm/drm_atomic_uapi.c
> @@ -34,6 +34,7 @@
>  #include <drm/drm_drv.h>
>  #include <drm/drm_writeback.h>
>  #include <drm/drm_vblank.h>
> +#include <drm/drm_colorop.h>
>  
>  #include <linux/dma-fence.h>
>  #include <linux/uaccess.h>
> @@ -642,6 +643,26 @@ drm_atomic_plane_get_property(struct drm_plane *plane,
>  	return 0;
>  }
>  
> +
> +static int drm_atomic_colorop_set_property(struct drm_colorop *colorop,
> +		struct drm_colorop_state *state, struct drm_file *file_priv,
> +		struct drm_property *property, uint64_t val)
> +{
> +	drm_dbg_atomic(colorop->dev,
> +			"[COLOROP:%d] unknown property [PROP:%d:%s]]\n",
> +			colorop->base.id,
> +			property->base.id, property->name);
> +	return -EINVAL;
> +}
> +
> +static int
> +drm_atomic_colorop_get_property(struct drm_colorop *colorop,
> +		const struct drm_colorop_state *state,
> +		struct drm_property *property, uint64_t *val)
> +{
> +	return -EINVAL;
> +}
> +
>  static int drm_atomic_set_writeback_fb_for_connector(
>  		struct drm_connector_state *conn_state,
>  		struct drm_framebuffer *fb)
> @@ -893,6 +914,16 @@ int drm_atomic_get_property(struct drm_mode_object *obj,
>  				plane->state, property, val);
>  		break;
>  	}
> +	case DRM_MODE_OBJECT_COLOROP: {
> +		struct drm_colorop *colorop = obj_to_colorop(obj);
> +
> +		if (colorop->plane)
> +			WARN_ON(!drm_modeset_is_locked(&colorop->plane->mutex));
> +
> +		ret = drm_atomic_colorop_get_property(colorop,
> +				colorop->state, property, val);
> +		break;
> +	}
>  	default:
>  		ret = -EINVAL;
>  		break;
> @@ -1027,6 +1058,23 @@ int drm_atomic_set_property(struct drm_atomic_state *state,
>  		ret = drm_atomic_plane_set_property(plane,
>  				plane_state, file_priv,
>  				prop, prop_value);
> +
> +		break;
> +	}
> +	case DRM_MODE_OBJECT_COLOROP: {
> +		struct drm_colorop *colorop = obj_to_colorop(obj);
> +		struct drm_colorop_state *colorop_state;
> +
> +		colorop_state = drm_atomic_get_colorop_state(state, colorop);
> +		if (IS_ERR(colorop_state)) {
> +			ret = PTR_ERR(colorop_state);
> +			break;
> +		}
> +
> +		ret = drm_atomic_colorop_set_property(colorop,
> +				colorop_state, file_priv,
> +				prop, prop_value);
> +
>  		break;
>  	}
>  	default:
> diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
> new file mode 100644
> index 000000000000..78d6a0067f5b
> --- /dev/null
> +++ b/drivers/gpu/drm/drm_colorop.c
> @@ -0,0 +1,169 @@
> +/*
> + * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + *
> + * Authors: AMD
> + *
> + */
> +
> +#include <drm/drm_colorop.h>
> +#include <drm/drm_print.h>
> +#include <drm/drm_drv.h>
> +#include <drm/drm_plane.h>
> +
> +#include "drm_crtc_internal.h"
> +
> +/* TODO big colorop doc, including properties, etc. */
> +
> +/* Init Helpers */
> +
> +int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
> +		     struct drm_plane *plane)
> +{
> +	struct drm_mode_config *config = &dev->mode_config;
> +	int ret = 0;
> +
> +	ret = drm_mode_object_add(dev, &colorop->base, DRM_MODE_OBJECT_COLOROP);
> +	if (ret)
> +		return ret;
> +
> +	colorop->base.properties = &colorop->properties;
> +	colorop->dev = dev;
> +	colorop->plane = plane;
> +
> +	list_add_tail(&colorop->head, &config->colorop_list);
> +	colorop->index = config->num_colorop++;
> +
> +	/* add properties */
> +	return ret;
> +}
> +EXPORT_SYMBOL(drm_colorop_init);
> +
> +void __drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop,
> +						 struct drm_colorop_state *state)
> +{
> +	memcpy(state, colorop->state, sizeof(*state));
> +}
> +
> +struct drm_colorop_state *
> +drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop)
> +{
> +	struct drm_colorop_state *state;
> +
> +	if (WARN_ON(!colorop->state))
> +		return NULL;
> +
> +	state = kmalloc(sizeof(*state), GFP_KERNEL);
> +	if (state)
> +		__drm_atomic_helper_colorop_duplicate_state(colorop, state);
> +
> +	return state;
> +}
> +
> +
> +void drm_colorop_atomic_destroy_state(struct drm_colorop *colorop,
> +				      struct drm_colorop_state *state)
> +{
> +	kfree(state);
> +}
> +
> +/**
> + * __drm_colorop_destroy_state - release colorop state
> + * @state: colorop state object to release
> + *
> + * Releases all resources stored in the colorop state without actually freeing
> + * the memory of the colorop state. This is useful for drivers that subclass the
> + * colorop state.
> + */
> +void __drm_colorop_destroy_state(struct drm_colorop_state *state)
> +{
> +	/* TODO might need this later */
> +}
> +
> +/**
> + * drm_colorop_destroy_state - default state destroy hook
> + * @colorop: drm colorop
> + * @state: colorop state object to release
> + *
> + * Default colorop state destroy hook for drivers which don't have their own
> + * subclassed colorop state structure.
> + */
> +void drm_colorop_destroy_state(struct drm_colorop *colorop,
> +					   struct drm_colorop_state *state)
> +{
> +	kfree(state);
> +}
> +EXPORT_SYMBOL(drm_colorop_destroy_state);
> +
> +/**
> + * __drm_colorop_state_reset - resets colorop state to default values
> + * @colorop_state: atomic colorop state, must not be NULL
> + * @colorop: colorop object, must not be NULL
> + *
> + * Initializes the newly allocated @colorop_state with default
> + * values. This is useful for drivers that subclass the CRTC state.
> + */
> +void __drm_colorop_state_reset(struct drm_colorop_state *colorop_state,
> +					   struct drm_colorop *colorop)
> +{
> +	colorop_state->colorop = colorop;
> +}
> +EXPORT_SYMBOL(__drm_colorop_state_reset);
> +
> +/**
> + * __drm_colorop_reset - reset state on colorop
> + * @colorop: drm colorop
> + * @colorop_state: colorop state to assign
> + *
> + * Initializes the newly allocated @colorop_state and assigns it to
> + * the &drm_crtc->state pointer of @colorop, usually required when
> + * initializing the drivers or when called from the &drm_colorop_funcs.reset
> + * hook.
> + *
> + * This is useful for drivers that subclass the colorop state.
> + */
> +void __drm_colorop_reset(struct drm_colorop *colorop,
> +				     struct drm_colorop_state *colorop_state)
> +{
> +	if (colorop_state)
> +		__drm_colorop_state_reset(colorop_state, colorop);
> +
> +	colorop->state = colorop_state;
> +}
> +
> +/**
> + * drm_colorop_reset - reset colorop atomic state
> + * @colorop: drm colorop
> + *
> + * Resets the atomic state for @colorop by freeing the state pointer (which might
> + * be NULL, e.g. at driver load time) and allocating a new empty state object.
> + */
> +void drm_colorop_reset(struct drm_colorop *colorop)
> +{
> +	if (colorop->state)
> +		__drm_colorop_destroy_state(colorop->state);
> +
> +	kfree(colorop->state);
> +	colorop->state = kzalloc(sizeof(*colorop->state), GFP_KERNEL);
> +
> +	if (colorop->state)
> +		__drm_colorop_reset(colorop, colorop->state);
> +}
> +EXPORT_SYMBOL(drm_colorop_reset);
> diff --git a/drivers/gpu/drm/drm_mode_config.c b/drivers/gpu/drm/drm_mode_config.c
> index 87eb591fe9b5..5bdcf71e1ae0 100644
> --- a/drivers/gpu/drm/drm_mode_config.c
> +++ b/drivers/gpu/drm/drm_mode_config.c
> @@ -29,6 +29,7 @@
>  #include <drm/drm_managed.h>
>  #include <drm/drm_mode_config.h>
>  #include <drm/drm_print.h>
> +#include <drm/drm_colorop.h>
>  #include <linux/dma-resv.h>
>  
>  #include "drm_crtc_internal.h"
> @@ -184,11 +185,15 @@ int drm_mode_getresources(struct drm_device *dev, void *data,
>  void drm_mode_config_reset(struct drm_device *dev)
>  {
>  	struct drm_crtc *crtc;
> +	struct drm_colorop *colorop;
>  	struct drm_plane *plane;
>  	struct drm_encoder *encoder;
>  	struct drm_connector *connector;
>  	struct drm_connector_list_iter conn_iter;
>  
> +	drm_for_each_colorop(colorop, dev)
> +		drm_colorop_reset(colorop);
> +
>  	drm_for_each_plane(plane, dev)
>  		if (plane->funcs->reset)
>  			plane->funcs->reset(plane);
> @@ -415,6 +420,7 @@ int drmm_mode_config_init(struct drm_device *dev)
>  	INIT_LIST_HEAD(&dev->mode_config.property_list);
>  	INIT_LIST_HEAD(&dev->mode_config.property_blob_list);
>  	INIT_LIST_HEAD(&dev->mode_config.plane_list);
> +	INIT_LIST_HEAD(&dev->mode_config.colorop_list);
>  	INIT_LIST_HEAD(&dev->mode_config.privobj_list);
>  	idr_init_base(&dev->mode_config.object_idr, 1);
>  	idr_init_base(&dev->mode_config.tile_idr, 1);
> @@ -436,6 +442,7 @@ int drmm_mode_config_init(struct drm_device *dev)
>  	dev->mode_config.num_crtc = 0;
>  	dev->mode_config.num_encoder = 0;
>  	dev->mode_config.num_total_plane = 0;
> +	dev->mode_config.num_colorop = 0;
>  
>  	if (IS_ENABLED(CONFIG_LOCKDEP)) {
>  		struct drm_modeset_acquire_ctx modeset_ctx;
> diff --git a/drivers/gpu/drm/drm_plane_helper.c b/drivers/gpu/drm/drm_plane_helper.c
> index c91e454eba09..69faa0eeb27f 100644
> --- a/drivers/gpu/drm/drm_plane_helper.c
> +++ b/drivers/gpu/drm/drm_plane_helper.c
> @@ -318,4 +318,4 @@ int drm_plane_helper_atomic_check(struct drm_plane *plane, struct drm_atomic_sta
>  						   DRM_PLANE_NO_SCALING,
>  						   false, false);
>  }
> -EXPORT_SYMBOL(drm_plane_helper_atomic_check);
> +EXPORT_SYMBOL(drm_plane_helper_atomic_check);
> \ No newline at end of file
> diff --git a/include/drm/drm_atomic.h b/include/drm/drm_atomic.h
> index 92586ab55ef5..dca8fc26ad71 100644
> --- a/include/drm/drm_atomic.h
> +++ b/include/drm/drm_atomic.h
> @@ -30,6 +30,7 @@
>  
>  #include <drm/drm_crtc.h>
>  #include <drm/drm_util.h>
> +#include <drm/drm_colorop.h>
>  
>  /**
>   * struct drm_crtc_commit - track modeset commits on a CRTC
> @@ -157,6 +158,11 @@ struct drm_crtc_commit {
>  	bool abort_completion;
>  };
>  
> +struct __drm_colorops_state {
> +	struct drm_colorop *ptr;
> +	struct drm_colorop_state *state, *old_state, *new_state;
> +};
> +
>  struct __drm_planes_state {
>  	struct drm_plane *ptr;
>  	struct drm_plane_state *state, *old_state, *new_state;
> @@ -398,6 +404,7 @@ struct drm_atomic_state {
>  	 * states.
>  	 */
>  	bool duplicated : 1;
> +	struct __drm_colorops_state *colorops;
>  	struct __drm_planes_state *planes;
>  	struct __drm_crtcs_state *crtcs;
>  	int num_connector;
> @@ -501,6 +508,9 @@ drm_atomic_get_crtc_state(struct drm_atomic_state *state,
>  struct drm_plane_state * __must_check
>  drm_atomic_get_plane_state(struct drm_atomic_state *state,
>  			   struct drm_plane *plane);
> +struct drm_colorop_state *
> +drm_atomic_get_colorop_state(struct drm_atomic_state *state,
> +			     struct drm_colorop *colorop);
>  struct drm_connector_state * __must_check
>  drm_atomic_get_connector_state(struct drm_atomic_state *state,
>  			       struct drm_connector *connector);
> @@ -623,6 +633,55 @@ drm_atomic_get_new_plane_state(const struct drm_atomic_state *state,
>  	return state->planes[drm_plane_index(plane)].new_state;
>  }
>  
> +
> +/**
> + * drm_atomic_get_existing_colorop_state - get colorop state, if it exists
> + * @state: global atomic state object
> + * @colorop: colorop to grab
> + *
> + * This function returns the colorop state for the given colorop, or NULL
> + * if the colorop is not part of the global atomic state.
> + *
> + * This function is deprecated, @drm_atomic_get_old_colorop_state or
> + * @drm_atomic_get_new_colorop_state should be used instead.
> + */
> +static inline struct drm_colorop_state *
> +drm_atomic_get_existing_colorop_state(struct drm_atomic_state *state,
> +				    struct drm_colorop *colorop)
> +{
> +	return state->colorops[drm_colorop_index(colorop)].state;
> +}
> +
> +/**
> + * drm_atomic_get_old_colorop_state - get colorop state, if it exists
> + * @state: global atomic state object
> + * @colorop: colorop to grab
> + *
> + * This function returns the old colorop state for the given colorop, or
> + * NULL if the colorop is not part of the global atomic state.
> + */
> +static inline struct drm_colorop_state *
> +drm_atomic_get_old_colorop_state(struct drm_atomic_state *state,
> +			       struct drm_colorop *colorop)
> +{
> +	return state->colorops[drm_colorop_index(colorop)].old_state;
> +}
> +
> +/**
> + * drm_atomic_get_new_colorop_state - get colorop state, if it exists
> + * @state: global atomic state object
> + * @colorop: colorop to grab
> + *
> + * This function returns the new colorop state for the given colorop, or
> + * NULL if the colorop is not part of the global atomic state.
> + */
> +static inline struct drm_colorop_state *
> +drm_atomic_get_new_colorop_state(struct drm_atomic_state *state,
> +			       struct drm_colorop *colorop)
> +{
> +	return state->colorops[drm_colorop_index(colorop)].new_state;
> +}
> +
>  /**
>   * drm_atomic_get_existing_connector_state - get connector state, if it exists
>   * @state: global atomic state object
> @@ -870,6 +929,29 @@ void drm_state_dump(struct drm_device *dev, struct drm_printer *p);
>  			     (new_crtc_state) = (__state)->crtcs[__i].new_state, \
>  			     (void)(new_crtc_state) /* Only to avoid unused-but-set-variable warning */, 1))
>  
> +/**
> + * for_each_oldnew_colorop_in_state - iterate over all colorops in an atomic update
> + * @__state: &struct drm_atomic_state pointer
> + * @colorop: &struct drm_colorop iteration cursor
> + * @old_colorop_state: &struct drm_colorop_state iteration cursor for the old state
> + * @new_colorop_state: &struct drm_colorop_state iteration cursor for the new state
> + * @__i: int iteration cursor, for macro-internal use
> + *
> + * This iterates over all colorops in an atomic update, tracking both old and
> + * new state. This is useful in places where the state delta needs to be
> + * considered, for example in atomic check functions.
> + */
> +#define for_each_oldnew_colorop_in_state(__state, colorop, old_colorop_state, new_colorop_state, __i) \
> +	for ((__i) = 0;							\
> +	     (__i) < (__state)->dev->mode_config.num_colorop;	\
> +	     (__i)++)							\
> +		for_each_if ((__state)->colorops[__i].ptr &&		\
> +			     ((colorop) = (__state)->colorops[__i].ptr,	\
> +			      (void)(colorop) /* Only to avoid unused-but-set-variable warning */, \
> +			      (old_colorop_state) = (__state)->colorops[__i].old_state,\
> +			      (new_colorop_state) = (__state)->colorops[__i].new_state, 1))
> +
> +
>  /**
>   * for_each_oldnew_plane_in_state - iterate over all planes in an atomic update
>   * @__state: &struct drm_atomic_state pointer
> diff --git a/include/drm/drm_atomic_uapi.h b/include/drm/drm_atomic_uapi.h
> index 4c6d39d7bdb2..70a115d523cd 100644
> --- a/include/drm/drm_atomic_uapi.h
> +++ b/include/drm/drm_atomic_uapi.h
> @@ -37,6 +37,7 @@ struct drm_crtc;
>  struct drm_connector_state;
>  struct dma_fence;
>  struct drm_framebuffer;
> +struct drm_colorop;
>  
>  int __must_check
>  drm_atomic_set_mode_for_crtc(struct drm_crtc_state *state,
> diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
> new file mode 100644
> index 000000000000..3dd169b0317d
> --- /dev/null
> +++ b/include/drm/drm_colorop.h
> @@ -0,0 +1,157 @@
> +/*
> + * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + *
> + * Authors: AMD
> + *
> + */
> +
> +#ifndef __DRM_COLOROP_H__
> +#define __DRM_COLOROP_H__
> +
> +#include <drm/drm_mode_object.h>
> +#include <drm/drm_mode.h>
> +#include <drm/drm_property.h>
> +
> +/**
> + * struct drm_colorop_state - mutable colorop state
> + */
> +struct drm_colorop_state {
> +	/** @colorop: backpointer to the colorop */
> +	struct drm_colorop *colorop;
> +
> +	/* colorop properties */
> +
> +	/** @state: backpointer to global drm_atomic_state */
> +	struct drm_atomic_state *state;
> +};
> +
> +/**
> + * struct drm_colorop - DRM color operation control structure
> + *
> + * A colorop represents one color operation. They can be chained via
> + * the 'next' pointer to build a color pipeline.
> + */
> +struct drm_colorop {
> +	/** @dev: parent DRM device */
> +	struct drm_device *dev;
> +
> +	/**
> +	 * @head:
> +	 *
> +	 * List of all colorops on @dev, linked from &drm_mode_config.colorop_list.
> +	 * Invariant over the lifetime of @dev and therefore does not need
> +	 * locking.
> +	 */
> +	struct list_head head;
> +
> +	/**
> +	 * @index: Position inside the mode_config.list, can be used as an array
> +	 * index. It is invariant over the lifetime of the plane.
> +	 */
> +	unsigned index;
> +
> +	/* TODO do we need a separate mutex or will we tag along with the plane mutex? */
> +
> +	/** @base base mode object*/
> +	struct drm_mode_object base;
> +
> +	/**
> +	 * @plane:
> +	 *
> +	 * The plane on which the colorop sits. A drm_colorop is always unique
> +	 * to a plane.
> +	 */
> +	struct drm_plane *plane;
> +
> +	/**
> +	 * @state:
> +	 *
> +	 * Current atomic state for this colorop.
> +	 *
> +	 * This is protected by @mutex. Note that nonblocking atomic commits
> +	 * access the current colorop state without taking locks. Either by
> +	 * going through the &struct drm_atomic_state pointers, see
> +	 * for_each_oldnew_plane_in_state(), for_each_old_plane_in_state() and
> +	 * for_each_new_plane_in_state(). Or through careful ordering of atomic
> +	 * commit operations as implemented in the atomic helpers, see
> +	 * &struct drm_crtc_commit.
> +	 *
> +	 * TODO keep, remove, or rewrite above plane references?
> +	 */
> +	struct drm_colorop_state *state;
> +
> +	/* colorop properties */
> +
> +	/** @properties: property tracking for this plane */
> +	struct drm_object_properties properties;
> +
> +};
> +
> +#define obj_to_colorop(x) container_of(x, struct drm_colorop, base)
> +
> +/**
> + * drm_crtc_find - look up a Colorop object from its ID
> + * @dev: DRM device
> + * @file_priv: drm file to check for lease against.
> + * @id: &drm_mode_object ID
> + *
> + * This can be used to look up a Colorop from its userspace ID. Only used by
> + * drivers for legacy IOCTLs and interface, nowadays extensions to the KMS
> + * userspace interface should be done using &drm_property.
> + */
> +static inline struct drm_colorop *drm_colorop_find(struct drm_device *dev,
> +		struct drm_file *file_priv,
> +		uint32_t id)
> +{
> +	struct drm_mode_object *mo;
> +	mo = drm_mode_object_find(dev, file_priv, id, DRM_MODE_OBJECT_COLOROP);
> +	return mo ? obj_to_colorop(mo) : NULL;
> +}
> +
> +int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
> +		     struct drm_plane *plane);
> +
> +struct drm_colorop_state *
> +drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop);
> +
> +void drm_colorop_atomic_destroy_state(struct drm_colorop *colorop,
> +				      struct drm_colorop_state *state);
> +
> +void drm_colorop_reset(struct drm_colorop *colorop);
> +
> +/**
> + * drm_colorop_index - find the index of a registered colorop
> + * @colorop: colorop to find index for
> + *
> + * Given a registered colorop, return the index of that colorop within a DRM
> + * device's list of colorops.
> + */
> +static inline unsigned int drm_colorop_index(const struct drm_colorop *colorop)
> +{
> +	return colorop->index;
> +}
> +
> +
> +#define drm_for_each_colorop(colorop, dev) \
> +	list_for_each_entry(colorop, &(dev)->mode_config.colorop_list, head)
> +
> +
> +#endif /* __DRM_COLOROP_H__ */
> diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h
> index e5b053001d22..f56d21d93cf0 100644
> --- a/include/drm/drm_mode_config.h
> +++ b/include/drm/drm_mode_config.h
> @@ -505,6 +505,24 @@ struct drm_mode_config {
>  	 */
>  	struct list_head plane_list;
>  
> +	/**
> +	 * @num_colorop:
> +	 *
> +	 * Number of colorop objects on this device.
> +	 * This is invariant over the lifetime of a device and hence doesn't
> +	 * need any locks.
> +	 */
> +	int num_colorop;
> +
> +	/**
> +	 * @colorops_list:
> +	 *
> +	 * List of colorop objects linked with &drm_colorop.head. This is
> +	 * invariant over the lifetime of a device and hence doesn't need any
> +	 * locks.
> +	 */
> +	struct list_head colorop_list;
> +
>  	/**
>  	 * @num_crtc:
>  	 *
> diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
> index 51291983ea44..cfb8f46b94ab 100644
> --- a/include/drm/drm_plane.h
> +++ b/include/drm/drm_plane.h
> @@ -227,6 +227,8 @@ struct drm_plane_state {
>  	 */
>  	enum drm_scaling_filter scaling_filter;
>  
> +	struct drm_colorop *color_pipeline;
> +
>  	/**
>  	 * @commit: Tracks the pending commit to prevent use-after-free conditions,
>  	 * and for async plane updates.
> diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h
> index 642808520d92..dec498a44eae 100644
> --- a/include/uapi/drm/drm.h
> +++ b/include/uapi/drm/drm.h
> @@ -1116,6 +1116,9 @@ extern "C" {
>   */
>  #define DRM_IOCTL_MODE_GETFB2		DRM_IOWR(0xCE, struct drm_mode_fb_cmd2)
>  
> +#define DRM_IOCTL_MODE_GETCOLOROPRESOURCES DRM_IOWR(0xD0, struct drm_mode_get_colorop_res)
> +#define DRM_IOCTL_MODE_GETCOLOROP          DRM_IOWR(0xD1, struct drm_mode_get_colorop)
> +
>  /*
>   * Device specific ioctls should only be in their respective headers
>   * The device specific ioctl range is from 0x40 to 0x9f.
> diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> index 46becedf5b2f..6dcf628def56 100644
> --- a/include/uapi/drm/drm_mode.h
> +++ b/include/uapi/drm/drm_mode.h
> @@ -626,6 +626,7 @@ struct drm_mode_connector_set_property {
>  #define DRM_MODE_OBJECT_FB 0xfbfbfbfb
>  #define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb
>  #define DRM_MODE_OBJECT_PLANE 0xeeeeeeee
> +#define DRM_MODE_OBJECT_COLOROP 0xfafafafa
>  #define DRM_MODE_OBJECT_ANY 0
>  
>  struct drm_mode_obj_get_properties {
> -- 
> 2.42.0
> 

Re: [RFC PATCH 10/10] drm/vkms: Add enumerated 1D curve colorop

[Melissa Wen]

On 09/08, Harry Wentland wrote:
> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: Jonas Ådahl <jadahl@redhat.com>
> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> Cc: Shashank Sharma <shashank.sharma@amd.com>
> Cc: Alexander Goins <agoins@nvidia.com>
> Cc: Joshua Ashton <joshua@froggi.es>
> Cc: Michel Dänzer <mdaenzer@redhat.com>
> Cc: Aleix Pol <aleixpol@kde.org>
> Cc: Xaver Hugl <xaver.hugl@gmail.com>
> Cc: Victoria Brekenfeld <victoria@system76.com>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Uma Shankar <uma.shankar@intel.com>
> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> Cc: Christopher Braga <quic_cbraga@quicinc.com>
> ---
>  drivers/gpu/drm/vkms/Makefile        |   3 +-
>  drivers/gpu/drm/vkms/vkms_colorop.c  | 108 +++++++++
>  drivers/gpu/drm/vkms/vkms_composer.c | 316 +++++++++++++++++++++++++++
>  drivers/gpu/drm/vkms/vkms_drv.h      |   4 +
>  drivers/gpu/drm/vkms/vkms_plane.c    |   2 +
>  5 files changed, 432 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/gpu/drm/vkms/vkms_colorop.c
> 
> diff --git a/drivers/gpu/drm/vkms/Makefile b/drivers/gpu/drm/vkms/Makefile
> index 1b28a6a32948..bcf508873622 100644
> --- a/drivers/gpu/drm/vkms/Makefile
> +++ b/drivers/gpu/drm/vkms/Makefile
> @@ -6,6 +6,7 @@ vkms-y := \
>  	vkms_formats.o \
>  	vkms_crtc.o \
>  	vkms_composer.o \
> -	vkms_writeback.o
> +	vkms_writeback.o \
> +	vkms_colorop.o
>  
>  obj-$(CONFIG_DRM_VKMS) += vkms.o
> diff --git a/drivers/gpu/drm/vkms/vkms_colorop.c b/drivers/gpu/drm/vkms/vkms_colorop.c
> new file mode 100644
> index 000000000000..b3da0705bca7
> --- /dev/null
> +++ b/drivers/gpu/drm/vkms/vkms_colorop.c
> @@ -0,0 +1,108 @@
> +/*
> + * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + *
> + * Authors: AMD
> + *
> + */
> +
> +#include <linux/slab.h>
> +#include <drm/drm_colorop.h>
> +#include <drm/drm_print.h>
> +#include <drm/drm_property.h>
> +#include <drm/drm_plane.h>
> +
> +#define MAX_COLOR_PIPELINES 5
> +
> +const int vkms_initialize_tf_pipeline(struct drm_plane *plane, struct drm_prop_enum_list *list)
> +{
> +
> +	struct drm_colorop *op, *prev_op;
> +	struct drm_device *dev = plane->dev;
> +	int ret;
> +
> +	/* 1st op: 1d curve */
> +	op = kzalloc(sizeof(struct drm_colorop), GFP_KERNEL);
> +	if (!op) {
> +		DRM_ERROR("KMS: Failed to allocate colorop\n");
> +		return -ENOMEM;
> +	}
> +
> +	ret = drm_colorop_init(dev, op, plane, DRM_COLOROP_1D_CURVE);
> +	if (ret)
> +		return ret;
> +
> +	list->type = op->base.id;
> +	list->name = kasprintf(GFP_KERNEL, "Color Pipeline %d", op->base.id);
> +
> +	prev_op = op;
> +
> +	/* 2nd op: 1d curve */
> +	op = kzalloc(sizeof(struct drm_colorop), GFP_KERNEL);
> +	if (!op) {
> +		DRM_ERROR("KMS: Failed to allocate colorop\n");
> +		return -ENOMEM;
> +	}
> +
> +	ret = drm_colorop_init(dev, op, plane, DRM_COLOROP_1D_CURVE);
> +	if (ret)
> +		return ret;
> +
> +	drm_colorop_set_next_property(prev_op, op);
> +
> +	return 0;
> +}
> +
> +int vkms_initialize_colorops(struct drm_plane *plane)
> +{
> +	struct drm_device *dev = plane->dev;
> +	struct drm_property *prop;
> +	struct drm_prop_enum_list pipelines[MAX_COLOR_PIPELINES];
> +	int len = 0;
> +	int ret;
> +
> +	/* Add "Bypass" (i.e. NULL) pipeline */
> +	pipelines[len].type = 0;
> +	pipelines[len].name = "Bypass";
> +	len++;
> +
> +	/* Add pipeline consisting of transfer functions */
> +	ret = vkms_initialize_tf_pipeline(plane, &(pipelines[len]));
> +	if (ret)
> +		return ret;
> +	len++;
> +
> +	/* Create COLOR_PIPELINE property and attach */
> +	prop = drm_property_create_enum(dev, DRM_MODE_PROP_ATOMIC,
> +					"COLOR_PIPELINE",
> +					pipelines, len);
> +	if (!prop)
> +		return -ENOMEM;
> +
> +	plane->color_pipeline_property = prop;
> +
> +	drm_object_attach_property(&plane->base, prop, 0);
> +
> +	/* TODO do we even need this? */
> +	if (plane->state)
> +		plane->state->color_pipeline = NULL;
> +
> +	return 0;
> +}
> diff --git a/drivers/gpu/drm/vkms/vkms_composer.c b/drivers/gpu/drm/vkms/vkms_composer.c
> index f6c311e8a87c..92ab9c62a554 100644
> --- a/drivers/gpu/drm/vkms/vkms_composer.c
> +++ b/drivers/gpu/drm/vkms/vkms_composer.c
> @@ -12,6 +12,284 @@
>  
>  #include "vkms_drv.h"
>  
> +#define LUT_SIZE 256
> +
> +struct drm_color_lut srgb_array[LUT_SIZE] = {
> +	{ 0x13, 0x13, 0x13, 0 },
> +	{ 0x27, 0x27, 0x27, 0 },
> +	{ 0x3b, 0x3b, 0x3b, 0 },
> +	{ 0x4f, 0x4f, 0x4f, 0 },
> +	{ 0x63, 0x63, 0x63, 0 },
> +	{ 0x76, 0x76, 0x76, 0 },
> +	{ 0x8a, 0x8a, 0x8a, 0 },
> +	{ 0x9e, 0x9e, 0x9e, 0 },
> +	{ 0xb2, 0xb2, 0xb2, 0 },
> +	{ 0xc6, 0xc6, 0xc6, 0 },
> +	{ 0xda, 0xda, 0xda, 0 },
> +	{ 0xef, 0xef, 0xef, 0 },
> +	{ 0x106, 0x106, 0x106, 0 },
> +	{ 0x11e, 0x11e, 0x11e, 0 },
> +	{ 0x137, 0x137, 0x137, 0 },
> +	{ 0x151, 0x151, 0x151, 0 },
> +	{ 0x16d, 0x16d, 0x16d, 0 },
> +	{ 0x18a, 0x18a, 0x18a, 0 },
> +	{ 0x1a8, 0x1a8, 0x1a8, 0 },
> +	{ 0x1c7, 0x1c7, 0x1c7, 0 },
> +	{ 0x1e8, 0x1e8, 0x1e8, 0 },
> +	{ 0x20a, 0x20a, 0x20a, 0 },
> +	{ 0x22e, 0x22e, 0x22e, 0 },
> +	{ 0x253, 0x253, 0x253, 0 },
> +	{ 0x279, 0x279, 0x279, 0 },
> +	{ 0x2a0, 0x2a0, 0x2a0, 0 },
> +	{ 0x2c9, 0x2c9, 0x2c9, 0 },
> +	{ 0x2f4, 0x2f4, 0x2f4, 0 },
> +	{ 0x320, 0x320, 0x320, 0 },
> +	{ 0x34d, 0x34d, 0x34d, 0 },
> +	{ 0x37c, 0x37c, 0x37c, 0 },
> +	{ 0x3ac, 0x3ac, 0x3ac, 0 },
> +	{ 0x3de, 0x3de, 0x3de, 0 },
> +	{ 0x411, 0x411, 0x411, 0 },
> +	{ 0x446, 0x446, 0x446, 0 },
> +	{ 0x47c, 0x47c, 0x47c, 0 },
> +	{ 0x4b4, 0x4b4, 0x4b4, 0 },
> +	{ 0x4ed, 0x4ed, 0x4ed, 0 },
> +	{ 0x528, 0x528, 0x528, 0 },
> +	{ 0x564, 0x564, 0x564, 0 },
> +	{ 0x5a3, 0x5a3, 0x5a3, 0 },
> +	{ 0x5e2, 0x5e2, 0x5e2, 0 },
> +	{ 0x624, 0x624, 0x624, 0 },
> +	{ 0x666, 0x666, 0x666, 0 },
> +	{ 0x6ab, 0x6ab, 0x6ab, 0 },
> +	{ 0x6f1, 0x6f1, 0x6f1, 0 },
> +	{ 0x739, 0x739, 0x739, 0 },
> +	{ 0x782, 0x782, 0x782, 0 },
> +	{ 0x7ce, 0x7ce, 0x7ce, 0 },
> +	{ 0x81b, 0x81b, 0x81b, 0 },
> +	{ 0x869, 0x869, 0x869, 0 },
> +	{ 0x8b9, 0x8b9, 0x8b9, 0 },
> +	{ 0x90b, 0x90b, 0x90b, 0 },
> +	{ 0x95f, 0x95f, 0x95f, 0 },
> +	{ 0x9b5, 0x9b5, 0x9b5, 0 },
> +	{ 0xa0c, 0xa0c, 0xa0c, 0 },
> +	{ 0xa65, 0xa65, 0xa65, 0 },
> +	{ 0xabf, 0xabf, 0xabf, 0 },
> +	{ 0xb1c, 0xb1c, 0xb1c, 0 },
> +	{ 0xb7a, 0xb7a, 0xb7a, 0 },
> +	{ 0xbda, 0xbda, 0xbda, 0 },
> +	{ 0xc3c, 0xc3c, 0xc3c, 0 },
> +	{ 0xca0, 0xca0, 0xca0, 0 },
> +	{ 0xd06, 0xd06, 0xd06, 0 },
> +	{ 0xd6d, 0xd6d, 0xd6d, 0 },
> +	{ 0xdd6, 0xdd6, 0xdd6, 0 },
> +	{ 0xe41, 0xe41, 0xe41, 0 },
> +	{ 0xeae, 0xeae, 0xeae, 0 },
> +	{ 0xf1d, 0xf1d, 0xf1d, 0 },
> +	{ 0xf8e, 0xf8e, 0xf8e, 0 },
> +	{ 0x1001, 0x1001, 0x1001, 0 },
> +	{ 0x1075, 0x1075, 0x1075, 0 },
> +	{ 0x10ec, 0x10ec, 0x10ec, 0 },
> +	{ 0x1164, 0x1164, 0x1164, 0 },
> +	{ 0x11de, 0x11de, 0x11de, 0 },
> +	{ 0x125a, 0x125a, 0x125a, 0 },
> +	{ 0x12d9, 0x12d9, 0x12d9, 0 },
> +	{ 0x1359, 0x1359, 0x1359, 0 },
> +	{ 0x13db, 0x13db, 0x13db, 0 },
> +	{ 0x145f, 0x145f, 0x145f, 0 },
> +	{ 0x14e5, 0x14e5, 0x14e5, 0 },
> +	{ 0x156d, 0x156d, 0x156d, 0 },
> +	{ 0x15f7, 0x15f7, 0x15f7, 0 },
> +	{ 0x1683, 0x1683, 0x1683, 0 },
> +	{ 0x1711, 0x1711, 0x1711, 0 },
> +	{ 0x17a1, 0x17a1, 0x17a1, 0 },
> +	{ 0x1833, 0x1833, 0x1833, 0 },
> +	{ 0x18c7, 0x18c7, 0x18c7, 0 },
> +	{ 0x195d, 0x195d, 0x195d, 0 },
> +	{ 0x19f6, 0x19f6, 0x19f6, 0 },
> +	{ 0x1a90, 0x1a90, 0x1a90, 0 },
> +	{ 0x1b2c, 0x1b2c, 0x1b2c, 0 },
> +	{ 0x1bcb, 0x1bcb, 0x1bcb, 0 },
> +	{ 0x1c6b, 0x1c6b, 0x1c6b, 0 },
> +	{ 0x1d0e, 0x1d0e, 0x1d0e, 0 },
> +	{ 0x1db3, 0x1db3, 0x1db3, 0 },
> +	{ 0x1e59, 0x1e59, 0x1e59, 0 },
> +	{ 0x1f02, 0x1f02, 0x1f02, 0 },
> +	{ 0x1fad, 0x1fad, 0x1fad, 0 },
> +	{ 0x205b, 0x205b, 0x205b, 0 },
> +	{ 0x210a, 0x210a, 0x210a, 0 },
> +	{ 0x21bb, 0x21bb, 0x21bb, 0 },
> +	{ 0x226f, 0x226f, 0x226f, 0 },
> +	{ 0x2325, 0x2325, 0x2325, 0 },
> +	{ 0x23dd, 0x23dd, 0x23dd, 0 },
> +	{ 0x2497, 0x2497, 0x2497, 0 },
> +	{ 0x2553, 0x2553, 0x2553, 0 },
> +	{ 0x2612, 0x2612, 0x2612, 0 },
> +	{ 0x26d2, 0x26d2, 0x26d2, 0 },
> +	{ 0x2795, 0x2795, 0x2795, 0 },
> +	{ 0x285a, 0x285a, 0x285a, 0 },
> +	{ 0x2922, 0x2922, 0x2922, 0 },
> +	{ 0x29eb, 0x29eb, 0x29eb, 0 },
> +	{ 0x2ab7, 0x2ab7, 0x2ab7, 0 },
> +	{ 0x2b85, 0x2b85, 0x2b85, 0 },
> +	{ 0x2c56, 0x2c56, 0x2c56, 0 },
> +	{ 0x2d28, 0x2d28, 0x2d28, 0 },
> +	{ 0x2dfd, 0x2dfd, 0x2dfd, 0 },
> +	{ 0x2ed4, 0x2ed4, 0x2ed4, 0 },
> +	{ 0x2fad, 0x2fad, 0x2fad, 0 },
> +	{ 0x3089, 0x3089, 0x3089, 0 },
> +	{ 0x3167, 0x3167, 0x3167, 0 },
> +	{ 0x3247, 0x3247, 0x3247, 0 },
> +	{ 0x332a, 0x332a, 0x332a, 0 },
> +	{ 0x340e, 0x340e, 0x340e, 0 },
> +	{ 0x34f5, 0x34f5, 0x34f5, 0 },
> +	{ 0x35df, 0x35df, 0x35df, 0 },
> +	{ 0x36cb, 0x36cb, 0x36cb, 0 },
> +	{ 0x37b9, 0x37b9, 0x37b9, 0 },
> +	{ 0x38a9, 0x38a9, 0x38a9, 0 },
> +	{ 0x399c, 0x399c, 0x399c, 0 },
> +	{ 0x3a91, 0x3a91, 0x3a91, 0 },
> +	{ 0x3b89, 0x3b89, 0x3b89, 0 },
> +	{ 0x3c83, 0x3c83, 0x3c83, 0 },
> +	{ 0x3d7f, 0x3d7f, 0x3d7f, 0 },
> +	{ 0x3e7e, 0x3e7e, 0x3e7e, 0 },
> +	{ 0x3f7f, 0x3f7f, 0x3f7f, 0 },
> +	{ 0x4082, 0x4082, 0x4082, 0 },
> +	{ 0x4188, 0x4188, 0x4188, 0 },
> +	{ 0x4290, 0x4290, 0x4290, 0 },
> +	{ 0x439b, 0x439b, 0x439b, 0 },
> +	{ 0x44a8, 0x44a8, 0x44a8, 0 },
> +	{ 0x45b7, 0x45b7, 0x45b7, 0 },
> +	{ 0x46c9, 0x46c9, 0x46c9, 0 },
> +	{ 0x47dd, 0x47dd, 0x47dd, 0 },
> +	{ 0x48f4, 0x48f4, 0x48f4, 0 },
> +	{ 0x4a0d, 0x4a0d, 0x4a0d, 0 },
> +	{ 0x4b29, 0x4b29, 0x4b29, 0 },
> +	{ 0x4c47, 0x4c47, 0x4c47, 0 },
> +	{ 0x4d68, 0x4d68, 0x4d68, 0 },
> +	{ 0x4e8b, 0x4e8b, 0x4e8b, 0 },
> +	{ 0x4fb1, 0x4fb1, 0x4fb1, 0 },
> +	{ 0x50d9, 0x50d9, 0x50d9, 0 },
> +	{ 0x5203, 0x5203, 0x5203, 0 },
> +	{ 0x5330, 0x5330, 0x5330, 0 },
> +	{ 0x5460, 0x5460, 0x5460, 0 },
> +	{ 0x5592, 0x5592, 0x5592, 0 },
> +	{ 0x56c6, 0x56c6, 0x56c6, 0 },
> +	{ 0x57fd, 0x57fd, 0x57fd, 0 },
> +	{ 0x5937, 0x5937, 0x5937, 0 },
> +	{ 0x5a73, 0x5a73, 0x5a73, 0 },
> +	{ 0x5bb2, 0x5bb2, 0x5bb2, 0 },
> +	{ 0x5cf3, 0x5cf3, 0x5cf3, 0 },
> +	{ 0x5e37, 0x5e37, 0x5e37, 0 },
> +	{ 0x5f7d, 0x5f7d, 0x5f7d, 0 },
> +	{ 0x60c6, 0x60c6, 0x60c6, 0 },
> +	{ 0x6212, 0x6212, 0x6212, 0 },
> +	{ 0x6360, 0x6360, 0x6360, 0 },
> +	{ 0x64b0, 0x64b0, 0x64b0, 0 },
> +	{ 0x6604, 0x6604, 0x6604, 0 },
> +	{ 0x6759, 0x6759, 0x6759, 0 },
> +	{ 0x68b2, 0x68b2, 0x68b2, 0 },
> +	{ 0x6a0d, 0x6a0d, 0x6a0d, 0 },
> +	{ 0x6b6a, 0x6b6a, 0x6b6a, 0 },
> +	{ 0x6ccb, 0x6ccb, 0x6ccb, 0 },
> +	{ 0x6e2d, 0x6e2d, 0x6e2d, 0 },
> +	{ 0x6f93, 0x6f93, 0x6f93, 0 },
> +	{ 0x70fb, 0x70fb, 0x70fb, 0 },
> +	{ 0x7266, 0x7266, 0x7266, 0 },
> +	{ 0x73d3, 0x73d3, 0x73d3, 0 },
> +	{ 0x7543, 0x7543, 0x7543, 0 },
> +	{ 0x76b6, 0x76b6, 0x76b6, 0 },
> +	{ 0x782b, 0x782b, 0x782b, 0 },
> +	{ 0x79a3, 0x79a3, 0x79a3, 0 },
> +	{ 0x7b1d, 0x7b1d, 0x7b1d, 0 },
> +	{ 0x7c9b, 0x7c9b, 0x7c9b, 0 },
> +	{ 0x7e1b, 0x7e1b, 0x7e1b, 0 },
> +	{ 0x7f9d, 0x7f9d, 0x7f9d, 0 },
> +	{ 0x8123, 0x8123, 0x8123, 0 },
> +	{ 0x82ab, 0x82ab, 0x82ab, 0 },
> +	{ 0x8436, 0x8436, 0x8436, 0 },
> +	{ 0x85c3, 0x85c3, 0x85c3, 0 },
> +	{ 0x8753, 0x8753, 0x8753, 0 },
> +	{ 0x88e6, 0x88e6, 0x88e6, 0 },
> +	{ 0x8a7c, 0x8a7c, 0x8a7c, 0 },
> +	{ 0x8c14, 0x8c14, 0x8c14, 0 },
> +	{ 0x8daf, 0x8daf, 0x8daf, 0 },
> +	{ 0x8f4d, 0x8f4d, 0x8f4d, 0 },
> +	{ 0x90ed, 0x90ed, 0x90ed, 0 },
> +	{ 0x9290, 0x9290, 0x9290, 0 },
> +	{ 0x9436, 0x9436, 0x9436, 0 },
> +	{ 0x95df, 0x95df, 0x95df, 0 },
> +	{ 0x978b, 0x978b, 0x978b, 0 },
> +	{ 0x9939, 0x9939, 0x9939, 0 },
> +	{ 0x9aea, 0x9aea, 0x9aea, 0 },
> +	{ 0x9c9e, 0x9c9e, 0x9c9e, 0 },
> +	{ 0x9e55, 0x9e55, 0x9e55, 0 },
> +	{ 0xa00e, 0xa00e, 0xa00e, 0 },
> +	{ 0xa1ca, 0xa1ca, 0xa1ca, 0 },
> +	{ 0xa389, 0xa389, 0xa389, 0 },
> +	{ 0xa54b, 0xa54b, 0xa54b, 0 },
> +	{ 0xa710, 0xa710, 0xa710, 0 },
> +	{ 0xa8d7, 0xa8d7, 0xa8d7, 0 },
> +	{ 0xaaa1, 0xaaa1, 0xaaa1, 0 },
> +	{ 0xac6e, 0xac6e, 0xac6e, 0 },
> +	{ 0xae3e, 0xae3e, 0xae3e, 0 },
> +	{ 0xb011, 0xb011, 0xb011, 0 },
> +	{ 0xb1e7, 0xb1e7, 0xb1e7, 0 },
> +	{ 0xb3bf, 0xb3bf, 0xb3bf, 0 },
> +	{ 0xb59a, 0xb59a, 0xb59a, 0 },
> +	{ 0xb778, 0xb778, 0xb778, 0 },
> +	{ 0xb959, 0xb959, 0xb959, 0 },
> +	{ 0xbb3d, 0xbb3d, 0xbb3d, 0 },
> +	{ 0xbd24, 0xbd24, 0xbd24, 0 },
> +	{ 0xbf0d, 0xbf0d, 0xbf0d, 0 },
> +	{ 0xc0fa, 0xc0fa, 0xc0fa, 0 },
> +	{ 0xc2e9, 0xc2e9, 0xc2e9, 0 },
> +	{ 0xc4db, 0xc4db, 0xc4db, 0 },
> +	{ 0xc6d0, 0xc6d0, 0xc6d0, 0 },
> +	{ 0xc8c8, 0xc8c8, 0xc8c8, 0 },
> +	{ 0xcac3, 0xcac3, 0xcac3, 0 },
> +	{ 0xccc1, 0xccc1, 0xccc1, 0 },
> +	{ 0xcec1, 0xcec1, 0xcec1, 0 },
> +	{ 0xd0c5, 0xd0c5, 0xd0c5, 0 },
> +	{ 0xd2cc, 0xd2cc, 0xd2cc, 0 },
> +	{ 0xd4d5, 0xd4d5, 0xd4d5, 0 },
> +	{ 0xd6e1, 0xd6e1, 0xd6e1, 0 },
> +	{ 0xd8f1, 0xd8f1, 0xd8f1, 0 },
> +	{ 0xdb03, 0xdb03, 0xdb03, 0 },
> +	{ 0xdd18, 0xdd18, 0xdd18, 0 },
> +	{ 0xdf30, 0xdf30, 0xdf30, 0 },
> +	{ 0xe14b, 0xe14b, 0xe14b, 0 },
> +	{ 0xe369, 0xe369, 0xe369, 0 },
> +	{ 0xe58a, 0xe58a, 0xe58a, 0 },
> +	{ 0xe7ae, 0xe7ae, 0xe7ae, 0 },
> +	{ 0xe9d5, 0xe9d5, 0xe9d5, 0 },
> +	{ 0xebff, 0xebff, 0xebff, 0 },
> +	{ 0xee2c, 0xee2c, 0xee2c, 0 },
> +	{ 0xf05c, 0xf05c, 0xf05c, 0 },
> +	{ 0xf28f, 0xf28f, 0xf28f, 0 },
> +	{ 0xf4c4, 0xf4c4, 0xf4c4, 0 },
> +	{ 0xf6fd, 0xf6fd, 0xf6fd, 0 },
> +	{ 0xf939, 0xf939, 0xf939, 0 },
> +	{ 0xfb78, 0xfb78, 0xfb78, 0 },
> +	{ 0xfdba, 0xfdba, 0xfdba, 0 },
> +	{ 0xffff, 0xffff, 0xffff, 0 }
> +};
> +
> +#if 0
> +struct vkms_color_lut srgb_eotf = {
> +	.base = NULL,
> +	. lut_length = LUT_SIZE,
> +	.channel_value2index_ratio = drm_int2fixp(0xffff)
> +	// .channel_value2index_ratio = 0 //drm_fixp_div(drm_int2fixp(0xffff), drm_int2fixp(LUT_SIZE))
> +};
> +
> +#else
> +const struct vkms_color_lut srgb_eotf = {
> +	.base = srgb_array,
> +	.lut_length = 256,
> +	.channel_value2index_ratio = 16711935ll
> +};
> +
> +#endif
> +
>  static u16 pre_mul_blend_channel(u16 src, u16 dst, u16 alpha)
>  {
>  	u32 new_color;
> @@ -136,6 +414,39 @@ static void apply_lut(const struct vkms_crtc_state *crtc_state, struct line_buff
>  	}
>  }
>  
> +static void pre_blend_color_transform(const struct vkms_plane_state *plane_state, struct line_buffer *output_buffer)
> +{
> +	struct drm_colorop *pipeline = plane_state->base.base.color_pipeline;
> +	/* TODO this is probably wrong */
> +	struct drm_colorop_state *colorop_state;
> +
> +	if (!pipeline)
> +		return;
> +
> +	colorop_state = pipeline->state;
> +
> +	if (!colorop_state)
> +		return;
> +
> +	for (size_t x = 0; x < output_buffer->n_pixels; x++) {
> +		struct pixel_argb_u16 *pixel = &output_buffer->pixels[x];
> +
> +		if (pipeline->type == DRM_COLOROP_1D_CURVE &&
> +			colorop_state->bypass == false) {
> +			switch (colorop_state->curve_1d_type) {
> +				case DRM_COLOROP_1D_CURVE_SRGB_INV_EOTF:
> +					break;
> +				case DRM_COLOROP_1D_CURVE_SRGB_EOTF:
> +				default:
> +					pixel->r = apply_lut_to_channel_value(&srgb_eotf, pixel->r, LUT_RED);
> +					pixel->g = apply_lut_to_channel_value(&srgb_eotf, pixel->g, LUT_GREEN);
> +					pixel->b = apply_lut_to_channel_value(&srgb_eotf, pixel->b, LUT_BLUE);
> +					break;
> +			}
> +		}
> +	}
> +}
> +
>  /**
>   * @wb_frame_info: The writeback frame buffer metadata
>   * @crtc_state: The crtc state
> @@ -168,8 +479,13 @@ static void blend(struct vkms_writeback_job *wb,
>  				continue;
>  
>  			plane[i]->plane_read(stage_buffer, plane[i]->frame_info, y);
> +
> +			/* do per-plane color transformations here */
> +			// pre_blend_color_transform(plane[i], stage_buffer);
> +
>  			pre_mul_alpha_blend(plane[i]->frame_info, stage_buffer,
>  					    output_buffer);
> +			pre_blend_color_transform(plane[i], output_buffer);

I see it should be applied to the `stage_buffer` before blending (i.e.
pre_mul_alpha_blend()) and in the lines commented above. Were you
getting any unexpected result?

Otherwise, having this VKMS implementation looks very nice. Thank you!

Melissa

>  		}
>  
>  		apply_lut(crtc_state, output_buffer);
> diff --git a/drivers/gpu/drm/vkms/vkms_drv.h b/drivers/gpu/drm/vkms/vkms_drv.h
> index 310b31f47928..c04f714cd486 100644
> --- a/drivers/gpu/drm/vkms/vkms_drv.h
> +++ b/drivers/gpu/drm/vkms/vkms_drv.h
> @@ -168,4 +168,8 @@ void vkms_set_composer(struct vkms_output *out, bool enabled);
>  /* Writeback */
>  int vkms_enable_writeback_connector(struct vkms_device *vkmsdev);
>  
> +/* Colorops */
> +int vkms_initialize_colorops(struct drm_plane *plane);
> +
> +
>  #endif /* _VKMS_DRV_H_ */
> diff --git a/drivers/gpu/drm/vkms/vkms_plane.c b/drivers/gpu/drm/vkms/vkms_plane.c
> index b3f8a115cc23..cbffbdd7cbf9 100644
> --- a/drivers/gpu/drm/vkms/vkms_plane.c
> +++ b/drivers/gpu/drm/vkms/vkms_plane.c
> @@ -237,5 +237,7 @@ struct vkms_plane *vkms_plane_init(struct vkms_device *vkmsdev,
>  
>  	drm_plane_helper_add(&plane->base, funcs);
>  
> +	vkms_initialize_colorops(&plane->base);
> +
>  	return plane;
>  }
> -- 
> 2.42.0
> 

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 7914 bytes --]

On Tue, 10 Oct 2023 15:13:46 -0100
Melissa Wen <mwen@igalia.com> wrote:

> O 09/08, Harry Wentland wrote:
> > Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > Cc: Simon Ser <contact@emersion.fr>
> > Cc: Harry Wentland <harry.wentland@amd.com>
> > Cc: Melissa Wen <mwen@igalia.com>
> > Cc: Jonas Ådahl <jadahl@redhat.com>
> > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > Cc: Alexander Goins <agoins@nvidia.com>
> > Cc: Joshua Ashton <joshua@froggi.es>
> > Cc: Michel Dänzer <mdaenzer@redhat.com>
> > Cc: Aleix Pol <aleixpol@kde.org>
> > Cc: Xaver Hugl <xaver.hugl@gmail.com>
> > Cc: Victoria Brekenfeld <victoria@system76.com>
> > Cc: Daniel Vetter <daniel@ffwll.ch>
> > Cc: Uma Shankar <uma.shankar@intel.com>
> > Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> > Cc: Christopher Braga <quic_cbraga@quicinc.com>
> > ---
> >  Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
> >  1 file changed, 278 insertions(+)
> >  create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
> > 
> > diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
> > new file mode 100644
> > index 000000000000..bfa4a8f12087
> > --- /dev/null
> > +++ b/Documentation/gpu/rfc/color_pipeline.rst

...

> > +Color Pipeline Discovery
> > +========================
> > +
> > +A DRM client wanting color management on a drm_plane will:
> > +
> > +1. Read all drm_colorop objects
> > +2. Get the COLOR_PIPELINE property of the plane
> > +3. iterate all COLOR_PIPELINE enum values
> > +4. for each enum value walk the color pipeline (via the NEXT pointers)
> > +   and see if the available color operations are suitable for the
> > +   desired color management operations
> > +
> > +An example of chained properties to define an AMD pre-blending color
> > +pipeline might look like this::  
> 
> Hi Harry,
> 
> Thanks for sharing this proposal. Overall I think it's very aligned with
> Simon's description of the generic KMS color API. I think it's a good
> start point and we can refine over iterations. My general questions have
> already been pointed out by Sebastian and Pekka (mainly regarding the
> BYPASS property).
> 
> I still have some doubts on how to fit these set of colorops with some
> AMD corners cases as below:
> 
> > +
> > +    Plane 10
> > +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> > +    └─ "color_pipeline": enum {0, 42} = 0
> > +    Color operation 42 (input CSC)
> > +    ├─ "type": enum {Bypass, Matrix} = Matrix
> > +    ├─ "matrix_data": blob
> > +    └─ "next": immutable color operation ID = 43  
> 
> IIUC, for input CSC, there are currently two possiblities, or we use
> `drm_plane_color_encoding` and `drm_plane_color range` to get
> pre-defined coefficients or we set a custom matrix here, right? If so, I
> think we need some kind of pre-defined matrix option?
> 
> Also, with this new plane API in place, I understand that we will
> already need think on how to deal with the mixing between old drm color
> properties (color encoding and color range) and these new way of setting
> plane color properties. IIUC, Pekka asked a related question about it
> when talking about CRTC automatic RGB->YUV (?) 

I didn't realize color encoding and color range KMS plane properties
even existed. There is even color space on rockchip!

https://drmdb.emersion.fr/properties?object-type=4008636142

That list has even more conflicts: DEGAMMA_MODE, EOTF, FEATURE,
NV_INPUT_COLORSPACE, SCALING_FILTER, WATERMARK, alpha, GLOBAL_ALPHA,
brightness, colorkey, contrast, and more. I hope most of them are
actually from downstream drivers.

I think they should be forbidden to be used together with the new
pipeline UAPI. Mixing does not work in the long run, it would be
undefined at what position do the old properties apply in a pipeline.

Apparently, we already need a DRM client cap for the new color pipeline
UAPI to hide these legacy things.


This is different from "CRTC automatic RGB->YUV", because the CRTC
thing is chosen silently by the driver and there is nothing after it.
Those old plane properties are explicitly programmed by userspace.


Thanks,
pq

> > +    Color operation 43
> > +    ├─ "type": enum {Scaling} = Scaling
> > +    └─ "next": immutable color operation ID = 44
> > +    Color operation 44 (DeGamma)
> > +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> > +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
> > +    └─ "next": immutable color operation ID = 45
> > +    Color operation 45 (gamut remap)
> > +    ├─ "type": enum {Bypass, Matrix} = Matrix
> > +    ├─ "matrix_data": blob
> > +    └─ "next": immutable color operation ID = 46
> > +    Color operation 46 (shaper LUT RAM)
> > +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> > +    ├─ "1d_curve_type": enum {LUT} = LUT
> > +    ├─ "lut_size": immutable range = 4096
> > +    ├─ "lut_data": blob
> > +    └─ "next": immutable color operation ID = 47  
> 
> For shaper and blend LUT RAM, that the driver supports pre-defined
> curves and custom LUT at the same time but the resulted LUT is a
> combination of those, how to make this behavior clear? Could this
> behavior be described by two colorop in a row: for example, one for
> shaper TF and,just after, another for shaper LUT or would it not be the
> right representation?
> 
> > +    Color operation 47 (3D LUT RAM)
> > +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> > +    ├─ "lut_size": immutable range = 17
> > +    ├─ "lut_data": blob
> > +    └─ "next": immutable color operation ID = 48
> > +    Color operation 48 (blend gamma)
> > +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> > +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
> > +    ├─ "lut_size": immutable range = 4096
> > +    ├─ "lut_data": blob
> > +    └─ "next": immutable color operation ID = 0
> > +
> > +
> > +Color Pipeline Programming
> > +==========================
> > +
> > +Once a DRM client has found a suitable pipeline it will:
> > +
> > +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
> > +   drm_colorop object of the desired pipeline
> > +2. Set the properties for all drm_colorop objects in the pipeline to the
> > +   desired values, setting BYPASS to true for unused drm_colorop blocks,
> > +   and false for enabled drm_colorop blocks
> > +3. Perform atomic_check/commit as desired
> > +
> > +To configure the pipeline for an HDR10 PQ plane and blending in linear
> > +space, a compositor might perform an atomic commit with the following
> > +property values::
> > +
> > +    Plane 10
> > +    └─ "color_pipeline" = 42
> > +    Color operation 42 (input CSC)
> > +    └─ "bypass" = true
> > +    Color operation 44 (DeGamma)
> > +    └─ "bypass" = true
> > +    Color operation 45 (gamut remap)
> > +    └─ "bypasse" = true
> > +    Color operation 46 (shaper LUT RAM)
> > +    └─ "bypass" = true
> > +    Color operation 47 (3D LUT RAM)
> > +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
> > +    Color operation 48 (blend gamma)
> > +    └─ "1d_curve_type" = PQ inverse EOTF  
> 
> Isn't it a PQ EOTF for blend gamma?
> 
> Best Regards,
> 
> Melissa
> 
> > +
> > +
> > +References
> > +==========
> > +
> > +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> > \ No newline at end of file
> > -- 
> > 2.42.0
> >   


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Sebastian Wick]

On Wed, Oct 11, 2023 at 12:20 PM Pekka Paalanen <ppaalanen@gmail.com> wrote:
>
> On Tue, 10 Oct 2023 15:13:46 -0100
> Melissa Wen <mwen@igalia.com> wrote:
>
> > O 09/08, Harry Wentland wrote:
> > > Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> > > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > > Cc: Simon Ser <contact@emersion.fr>
> > > Cc: Harry Wentland <harry.wentland@amd.com>
> > > Cc: Melissa Wen <mwen@igalia.com>
> > > Cc: Jonas Ådahl <jadahl@redhat.com>
> > > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > > Cc: Alexander Goins <agoins@nvidia.com>
> > > Cc: Joshua Ashton <joshua@froggi.es>
> > > Cc: Michel Dänzer <mdaenzer@redhat.com>
> > > Cc: Aleix Pol <aleixpol@kde.org>
> > > Cc: Xaver Hugl <xaver.hugl@gmail.com>
> > > Cc: Victoria Brekenfeld <victoria@system76.com>
> > > Cc: Daniel Vetter <daniel@ffwll.ch>
> > > Cc: Uma Shankar <uma.shankar@intel.com>
> > > Cc: Naseer Ahmed <quic_naseer@quicinc.com>
> > > Cc: Christopher Braga <quic_cbraga@quicinc.com>
> > > ---
> > >  Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
> > >  1 file changed, 278 insertions(+)
> > >  create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
> > >
> > > diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
> > > new file mode 100644
> > > index 000000000000..bfa4a8f12087
> > > --- /dev/null
> > > +++ b/Documentation/gpu/rfc/color_pipeline.rst
>
> ...
>
> > > +Color Pipeline Discovery
> > > +========================
> > > +
> > > +A DRM client wanting color management on a drm_plane will:
> > > +
> > > +1. Read all drm_colorop objects
> > > +2. Get the COLOR_PIPELINE property of the plane
> > > +3. iterate all COLOR_PIPELINE enum values
> > > +4. for each enum value walk the color pipeline (via the NEXT pointers)
> > > +   and see if the available color operations are suitable for the
> > > +   desired color management operations
> > > +
> > > +An example of chained properties to define an AMD pre-blending color
> > > +pipeline might look like this::
> >
> > Hi Harry,
> >
> > Thanks for sharing this proposal. Overall I think it's very aligned with
> > Simon's description of the generic KMS color API. I think it's a good
> > start point and we can refine over iterations. My general questions have
> > already been pointed out by Sebastian and Pekka (mainly regarding the
> > BYPASS property).
> >
> > I still have some doubts on how to fit these set of colorops with some
> > AMD corners cases as below:
> >
> > > +
> > > +    Plane 10
> > > +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> > > +    └─ "color_pipeline": enum {0, 42} = 0
> > > +    Color operation 42 (input CSC)
> > > +    ├─ "type": enum {Bypass, Matrix} = Matrix
> > > +    ├─ "matrix_data": blob
> > > +    └─ "next": immutable color operation ID = 43
> >
> > IIUC, for input CSC, there are currently two possiblities, or we use
> > `drm_plane_color_encoding` and `drm_plane_color range` to get
> > pre-defined coefficients or we set a custom matrix here, right? If so, I
> > think we need some kind of pre-defined matrix option?

Seems reasonable. If they are mutually exclusive you might want to
expose 2 different pipelines for it.

> > Also, with this new plane API in place, I understand that we will
> > already need think on how to deal with the mixing between old drm color
> > properties (color encoding and color range) and these new way of setting
> > plane color properties. IIUC, Pekka asked a related question about it
> > when talking about CRTC automatic RGB->YUV (?)

Indeed, good catch! I listed some of them in my proposal more than one
year ago but completely forgot about them already.

>
> I didn't realize color encoding and color range KMS plane properties
> even existed. There is even color space on rockchip!
>
> https://drmdb.emersion.fr/properties?object-type=4008636142
>
> That list has even more conflicts: DEGAMMA_MODE, EOTF, FEATURE,
> NV_INPUT_COLORSPACE, SCALING_FILTER, WATERMARK, alpha, GLOBAL_ALPHA,
> brightness, colorkey, contrast, and more. I hope most of them are
> actually from downstream drivers.
>
> I think they should be forbidden to be used together with the new
> pipeline UAPI. Mixing does not work in the long run, it would be
> undefined at what position do the old properties apply in a pipeline.
>
> Apparently, we already need a DRM client cap for the new color pipeline
> UAPI to hide these legacy things.

Agreed. We'll need one cap for planes and one in the future for CRTCs then.

>
> This is different from "CRTC automatic RGB->YUV", because the CRTC
> thing is chosen silently by the driver and there is nothing after it.
> Those old plane properties are explicitly programmed by userspace.
>
>
> Thanks,
> pq
>
> > > +    Color operation 43
> > > +    ├─ "type": enum {Scaling} = Scaling
> > > +    └─ "next": immutable color operation ID = 44
> > > +    Color operation 44 (DeGamma)
> > > +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> > > +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
> > > +    └─ "next": immutable color operation ID = 45
> > > +    Color operation 45 (gamut remap)
> > > +    ├─ "type": enum {Bypass, Matrix} = Matrix
> > > +    ├─ "matrix_data": blob
> > > +    └─ "next": immutable color operation ID = 46
> > > +    Color operation 46 (shaper LUT RAM)
> > > +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> > > +    ├─ "1d_curve_type": enum {LUT} = LUT
> > > +    ├─ "lut_size": immutable range = 4096
> > > +    ├─ "lut_data": blob
> > > +    └─ "next": immutable color operation ID = 47
> >
> > For shaper and blend LUT RAM, that the driver supports pre-defined
> > curves and custom LUT at the same time but the resulted LUT is a
> > combination of those, how to make this behavior clear? Could this
> > behavior be described by two colorop in a row: for example, one for
> > shaper TF and,just after, another for shaper LUT or would it not be the
> > right representation?
> >
> > > +    Color operation 47 (3D LUT RAM)
> > > +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
> > > +    ├─ "lut_size": immutable range = 17
> > > +    ├─ "lut_data": blob
> > > +    └─ "next": immutable color operation ID = 48
> > > +    Color operation 48 (blend gamma)
> > > +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
> > > +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
> > > +    ├─ "lut_size": immutable range = 4096
> > > +    ├─ "lut_data": blob
> > > +    └─ "next": immutable color operation ID = 0
> > > +
> > > +
> > > +Color Pipeline Programming
> > > +==========================
> > > +
> > > +Once a DRM client has found a suitable pipeline it will:
> > > +
> > > +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
> > > +   drm_colorop object of the desired pipeline
> > > +2. Set the properties for all drm_colorop objects in the pipeline to the
> > > +   desired values, setting BYPASS to true for unused drm_colorop blocks,
> > > +   and false for enabled drm_colorop blocks
> > > +3. Perform atomic_check/commit as desired
> > > +
> > > +To configure the pipeline for an HDR10 PQ plane and blending in linear
> > > +space, a compositor might perform an atomic commit with the following
> > > +property values::
> > > +
> > > +    Plane 10
> > > +    └─ "color_pipeline" = 42
> > > +    Color operation 42 (input CSC)
> > > +    └─ "bypass" = true
> > > +    Color operation 44 (DeGamma)
> > > +    └─ "bypass" = true
> > > +    Color operation 45 (gamut remap)
> > > +    └─ "bypasse" = true
> > > +    Color operation 46 (shaper LUT RAM)
> > > +    └─ "bypass" = true
> > > +    Color operation 47 (3D LUT RAM)
> > > +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
> > > +    Color operation 48 (blend gamma)
> > > +    └─ "1d_curve_type" = PQ inverse EOTF
> >
> > Isn't it a PQ EOTF for blend gamma?
> >
> > Best Regards,
> >
> > Melissa
> >
> > > +
> > > +
> > > +References
> > > +==========
> > > +
> > > +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> > > \ No newline at end of file
> > > --
> > > 2.42.0
> > >
>

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-09-13 07:29, Pekka Paalanen wrote:
> On Fri, 8 Sep 2023 11:02:26 -0400
> Harry Wentland <harry.wentland@amd.com> wrote:
> 
>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
>> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
>> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
>> Cc: Simon Ser <contact@emersion.fr>
>> Cc: Harry Wentland <harry.wentland@amd.com>
>> Cc: Melissa Wen <mwen@igalia.com>
>> Cc: Jonas Ådahl <jadahl@redhat.com>
>> Cc: Sebastian Wick <sebastian.wick@redhat.com>
>> Cc: Shashank Sharma <shashank.sharma@amd.com>
>> Cc: Alexander Goins <agoins@nvidia.com>
>> Cc: Joshua Ashton <joshua@froggi.es>
>> Cc: Michel Dänzer <mdaenzer@redhat.com>
>> Cc: Aleix Pol <aleixpol@kde.org>
>> Cc: Xaver Hugl <xaver.hugl@gmail.com>
>> Cc: Victoria Brekenfeld <victoria@system76.com>
>> Cc: Daniel Vetter <daniel@ffwll.ch>
>> Cc: Uma Shankar <uma.shankar@intel.com>
>> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
>> Cc: Christopher Braga <quic_cbraga@quicinc.com>
>> ---
>>  Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
>>  1 file changed, 278 insertions(+)
>>  create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
> 
> Hi Harry,
> 
> it's really nice to see this!
> 

Thanks for the feedback. I'm just putting a v2 together with comments
(partially) addressed.

> Sebastian started on the backward/forward compatibility, so I'll
> comment on everything else here, and leave the compatibility for that
> thread.
> 
>> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
>> new file mode 100644
>> index 000000000000..bfa4a8f12087
>> --- /dev/null
>> +++ b/Documentation/gpu/rfc/color_pipeline.rst
...
>> +COLOR_PIPELINE Plane Property
>> +=============================
>> +
>> +Because we don't have existing KMS color properties in the pre-blending
>> +portion of display pipelines (i.e. on drm_planes) we are introducing
>> +color pipelines here first. Eventually we'll want to use the same
>> +concept for the post-blending portion, i.e. drm_crtcs.
> 
> This paragraph might fit better in a cover letter.
> 
>> +
>> +Color Pipelines are created by a driver and advertised via a new
>> +COLOR_PIPELINE enum property on each plane. Values of the property
>> +always include '0', which is the default and means all color processing
>> +is disabled. Additional values will be the object IDs of the first
>> +drm_colorop in a pipeline. A driver can create and advertise none, one,
>> +or more possible color pipelines. A DRM client will select a color
>> +pipeline by setting the COLOR PIPELINE to the respective value.
>> +
>> +In the case where drivers have custom support for pre-blending color
>> +processing those drivers shall reject atomic commits that are trying to
>> +set both the custom color properties, as well as the COLOR_PIPELINE
> 
> s/set/use/ because one of them could be carried-over state from
> previous commits while not literally set in this one.
> 
>> +property.
>> +
>> +An example of a COLOR_PIPELINE property on a plane might look like this::
>> +
>> +    Plane 10
>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>> +    ├─ …
>> +    └─ "color_pipeline": enum {0, 42, 52} = 0
> 
> Enum values are string names. I presume the intention here is that the
> strings will never need to be parsed, and the uint64_t is always equal
> to the string representation, right?
> 
> That needs a statement here. It differs from all previous uses of
> enums, and e.g. requires a little bit of new API in libweston's
> DRM-backend to handle since it has its own enums referring to the
> string names that get mapped to the uint64_t per owning KMS object.
> 

I'm currently putting the DRM object ID in the "value" and use the
"name" as a descriptive name.

> struct drm_mode_property_enum {
> 	__u64 value;
> 	char name[DRM_PROP_NAME_LEN];
> };

This works well in IGT and gives us a nice descriptive name for
debugging, but I could consider changing this if it'd simplify
people's lives.

>> +
>> +
>> +Color Pipeline Discovery
>> +========================
>> +
>> +A DRM client wanting color management on a drm_plane will:
>> +
>> +1. Read all drm_colorop objects
> 
> What does this do?

We probably don't need this, and with it we probably don't need
the new IOCTLs. I added this to align with IGT's current init
procedure where it reads all DRM core objects, like planes, etc.,
before using them. But realistically we can just look at the
colorop ID from the COLOR_PIPELINE property and then retrieve
the other colorops through the NEXT pointer.

> 
>> +2. Get the COLOR_PIPELINE property of the plane
>> +3. iterate all COLOR_PIPELINE enum values
>> +4. for each enum value walk the color pipeline (via the NEXT pointers)
>> +   and see if the available color operations are suitable for the
>> +   desired color management operations
>> +
>> +An example of chained properties to define an AMD pre-blending color
>> +pipeline might look like this::
>> +
>> +    Plane 10
>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>> +    └─ "color_pipeline": enum {0, 42} = 0
>> +    Color operation 42 (input CSC)
> 
> I presume the string "(input CSC)" does not come from KMS, and is
> actually just a comment added here by hand?
> 

Exactly. It only exists as a comment here. I'll remove it.

Harry

> 
> Thanks,
> pq
> 
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +    Color operation 43
>> +    ├─ "type": enum {Scaling} = Scaling
>> +    └─ "next": immutable color operation ID = 44
>> +    Color operation 44 (DeGamma)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
>> +    └─ "next": immutable color operation ID = 45
>> +    Color operation 45 (gamut remap)
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 46
>> +    Color operation 46 (shaper LUT RAM)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 47
>> +    Color operation 47 (3D LUT RAM)
>> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
>> +    ├─ "lut_size": immutable range = 17
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 48
>> +    Color operation 48 (blend gamma)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 0
>> +
>> +
>> +Color Pipeline Programming
>> +==========================
>> +
>> +Once a DRM client has found a suitable pipeline it will:
>> +
>> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
>> +   drm_colorop object of the desired pipeline
>> +2. Set the properties for all drm_colorop objects in the pipeline to the
>> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
>> +   and false for enabled drm_colorop blocks
>> +3. Perform atomic_check/commit as desired
>> +
>> +To configure the pipeline for an HDR10 PQ plane and blending in linear
>> +space, a compositor might perform an atomic commit with the following
>> +property values::
>> +
>> +    Plane 10
>> +    └─ "color_pipeline" = 42
>> +    Color operation 42 (input CSC)
>> +    └─ "bypass" = true
>> +    Color operation 44 (DeGamma)
>> +    └─ "bypass" = true
>> +    Color operation 45 (gamut remap)
>> +    └─ "bypasse" = true
>> +    Color operation 46 (shaper LUT RAM)
>> +    └─ "bypass" = true
>> +    Color operation 47 (3D LUT RAM)
>> +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
>> +    Color operation 48 (blend gamma)
>> +    └─ "1d_curve_type" = PQ inverse EOTF
>> +
>> +
>> +References
>> +==========
>> +
>> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>> \ No newline at end of file
> 

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-10-10 12:13, Melissa Wen wrote:
> O 09/08, Harry Wentland wrote:
>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
>> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
>> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
>> Cc: Simon Ser <contact@emersion.fr>
>> Cc: Harry Wentland <harry.wentland@amd.com>
>> Cc: Melissa Wen <mwen@igalia.com>
>> Cc: Jonas Ådahl <jadahl@redhat.com>
>> Cc: Sebastian Wick <sebastian.wick@redhat.com>
>> Cc: Shashank Sharma <shashank.sharma@amd.com>
>> Cc: Alexander Goins <agoins@nvidia.com>
>> Cc: Joshua Ashton <joshua@froggi.es>
>> Cc: Michel Dänzer <mdaenzer@redhat.com>
>> Cc: Aleix Pol <aleixpol@kde.org>
>> Cc: Xaver Hugl <xaver.hugl@gmail.com>
>> Cc: Victoria Brekenfeld <victoria@system76.com>
>> Cc: Daniel Vetter <daniel@ffwll.ch>
>> Cc: Uma Shankar <uma.shankar@intel.com>
>> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
>> Cc: Christopher Braga <quic_cbraga@quicinc.com>
>> ---
>>  Documentation/gpu/rfc/color_pipeline.rst | 278 +++++++++++++++++++++++
>>  1 file changed, 278 insertions(+)
>>  create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
>>
>> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/gpu/rfc/color_pipeline.rst
>> new file mode 100644
>> index 000000000000..bfa4a8f12087
>> --- /dev/null
>> +++ b/Documentation/gpu/rfc/color_pipeline.rst
>> @@ -0,0 +1,278 @@
>> +========================
>> +Linux Color Pipeline API
>> +========================
>> +
>> +What problem are we solving?
>> +============================
>> +
>> +We would like to support pre-, and post-blending complex color transformations
>> +in order to allow for HW-supported HDR use-cases, as well as to provide support
>> +to color-managed applications, such as video or image editors.
>> +
>> +While it is possible to support an HDR output on HW supporting the Colorspace
>> +and HDR Metadata drm_connector properties that requires the compositor or
>> +application to render and compose the content into one final buffer intended for
>> +display. Doing so is costly.
>> +
>> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
>> +operations to support color transformations. These operations are often
>> +implemented in fixed-function HW and therefore much more power efficient than
>> +performing similar operations via shaders or CPU.
>> +
>> +We would like to make use of this HW functionality to support complex color
>> +transformations with no, or minimal CPU or shader load.
>> +
>> +
>> +How are other OSes solving this problem?
>> +========================================
>> +
>> +The most widely supported use-cases regard HDR content, whether video or
>> +gaming.
>> +
>> +Most OSes will specify the source content format (color gamut, encoding transfer
>> +function, and other metadata, such as max and average light levels) to a driver.
>> +Drivers will then program their fixed-function HW accordingly to map from a
>> +source content buffer's space to a display's space.
>> +
>> +When fixed-function HW is not available the compositor will assemble a shader to
>> +ask the GPU to perform the transformation from the source content format to the
>> +display's format.
>> +
>> +A compositor's mapping function and a driver's mapping function are usually
>> +entirely separate concepts. On OSes where a HW vendor has no insight into
>> +closed-source compositor code such a vendor will tune their color management
>> +code to visually match the compositor's. On other OSes, where both mapping
>> +functions are open to an implementer they will ensure both mappings match.
>> +
>> +
>> +Why is Linux different?
>> +=======================
>> +
>> +Unlike other OSes, where there is one compositor for one or more drivers, on
>> +Linux we have a many-to-many relationship. Many compositors; many drivers.
>> +In addition each compositor vendor or community has their own view of how
>> +color management should be done. This is what makes Linux so beautiful.
>> +
>> +This means that a HW vendor can now no longer tune their driver to one
>> +compositor, as tuning it to one will almost inevitably make it look very
>> +different from another compositor's color mapping.
>> +
>> +We need a better solution.
>> +
>> +
>> +Descriptive API
>> +===============
>> +
>> +An API that describes the source and destination colorspaces is a descriptive
>> +API. It describes the input and output color spaces but does not describe
>> +how precisely they should be mapped. Such a mapping includes many minute
>> +design decision that can greatly affect the look of the final result.
>> +
>> +It is not feasible to describe such mapping with enough detail to ensure the
>> +same result from each implementation. In fact, these mappings are a very active
>> +research area.
>> +
>> +
>> +Prescriptive API
>> +================
>> +
>> +A prescriptive API describes not the source and destination colorspaces. It
>> +instead prescribes a recipe for how to manipulate pixel values to arrive at the
>> +desired outcome.
>> +
>> +This recipe is generally an order straight-forward operations, with clear
>> +mathematical definitions, such as 1D LUTs, 3D LUTs, matrices, or other
>> +operations that can be described in a precise manner.
>> +
>> +
>> +The Color Pipeline API
>> +======================
>> +
>> +HW color management pipelines can significantly differ between HW
>> +vendors in terms of availability, ordering, and capabilities of HW
>> +blocks. This makes a common definition of color management blocks and
>> +their ordering nigh impossible. Instead we are defining an API that
>> +allows user space to discover the HW capabilities.
>> +
>> +
>> +drm_colorop Object & IOCTLs
>> +===========================
>> +
>> +To support the definition of color pipelines we introduce a new DRM core
>> +object, a drm_colorop. Individual drm_colorop objects will be chained
>> +via the NEXT property of a drm_colorop to constitute a color pipeline.
>> +Each drm_colorop object is unique, i.e., even if multiple color
>> +pipelines have the same operation they won't share the same drm_colorop
>> +object to describe that operation.
>> +
>> +Just like other DRM objects the drm_colorop objects are discovered via
>> +IOCTLs:
>> +
>> +DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve the
>> +number of all drm_colorop objects.
>> +
>> +DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
>> +It includes the ID for the colorop object, as well as the plane_id of
>> +the associated plane. All other values should be registered as
>> +properties.
>> +
>> +Each drm_colorop has three core properties:
>> +
>> +TYPE: The type of transformation, such as
>> +* enumerated curve
>> +* custom (uniform) 1D LUT
>> +* 3x3 matrix
>> +* 3x4 matrix
>> +* 3D LUT
>> +* etc.
>> +
>> +Depending on the type of transformation other properties will describe
>> +more details.
>> +
>> +BYPASS: A boolean property that can be used to easily put a block into
>> +bypass mode. While setting other properties might fail atomic check,
>> +setting the BYPASS property to true should never fail. This allows DRM
>> +clients to fallback to other methods of color management if an atomic
>> +check for KMS color operations fails.
>> +
>> +NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
>> +drm_colorop is the last in the chain.
>> +
>> +An example of a drm_colorop object might look like one of these::
>> +
>> +    Color operation 42
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, BT.709, HLG, …} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +
>> +    Color operation 42
>> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
>> +    ├─ "lut_size": immutable range = 33
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +
>> +    Color operation 42
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 43
>> +
>> +
>> +COLOR_PIPELINE Plane Property
>> +=============================
>> +
>> +Because we don't have existing KMS color properties in the pre-blending
>> +portion of display pipelines (i.e. on drm_planes) we are introducing
>> +color pipelines here first. Eventually we'll want to use the same
>> +concept for the post-blending portion, i.e. drm_crtcs.
>> +
>> +Color Pipelines are created by a driver and advertised via a new
>> +COLOR_PIPELINE enum property on each plane. Values of the property
>> +always include '0', which is the default and means all color processing
>> +is disabled. Additional values will be the object IDs of the first
>> +drm_colorop in a pipeline. A driver can create and advertise none, one,
>> +or more possible color pipelines. A DRM client will select a color
>> +pipeline by setting the COLOR PIPELINE to the respective value.
>> +
>> +In the case where drivers have custom support for pre-blending color
>> +processing those drivers shall reject atomic commits that are trying to
>> +set both the custom color properties, as well as the COLOR_PIPELINE
>> +property.
>> +
>> +An example of a COLOR_PIPELINE property on a plane might look like this::
>> +
>> +    Plane 10
>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>> +    ├─ …
>> +    └─ "color_pipeline": enum {0, 42, 52} = 0
>> +
>> +
>> +Color Pipeline Discovery
>> +========================
>> +
>> +A DRM client wanting color management on a drm_plane will:
>> +
>> +1. Read all drm_colorop objects
>> +2. Get the COLOR_PIPELINE property of the plane
>> +3. iterate all COLOR_PIPELINE enum values
>> +4. for each enum value walk the color pipeline (via the NEXT pointers)
>> +   and see if the available color operations are suitable for the
>> +   desired color management operations
>> +
>> +An example of chained properties to define an AMD pre-blending color
>> +pipeline might look like this::
> 
> Hi Harry,
> 
> Thanks for sharing this proposal. Overall I think it's very aligned with
> Simon's description of the generic KMS color API. I think it's a good
> start point and we can refine over iterations. My general questions have
> already been pointed out by Sebastian and Pekka (mainly regarding the
> BYPASS property).
> 
> I still have some doubts on how to fit these set of colorops with some
> AMD corners cases as below:
> 

These aren't the final ones. This is intentionally presented as how
an AMD pre-blending color pipeline "might look like". An actual one
will look different and problem align a bit more to your AMD driver-
specific properties.

>> +
>> +    Plane 10
>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>> +    └─ "color_pipeline": enum {0, 42} = 0
>> +    Color operation 42 (input CSC)
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 43
> 
> IIUC, for input CSC, there are currently two possiblities, or we use
> `drm_plane_color_encoding` and `drm_plane_color range` to get
> pre-defined coefficients or we set a custom matrix here, right? If so, I
> think we need some kind of pre-defined matrix option?
> 

Agreed.

> Also, with this new plane API in place, I understand that we will
> already need think on how to deal with the mixing between old drm color
> properties (color encoding and color range) and these new way of setting
> plane color properties. IIUC, Pekka asked a related question about it
> when talking about CRTC automatic RGB->YUV (?) 
> 

We'll still need to confirm whether we'll want to deprecate these
existing properties. If we do that we'd want a client prop. Things
should still work without deprecating them, if drivers just pick up
after the initial encoding and range CSC.

But realistically it might be better to deprecate them and turn them
into explicit colorops.

>> +    Color operation 43
>> +    ├─ "type": enum {Scaling} = Scaling
>> +    └─ "next": immutable color operation ID = 44
>> +    Color operation 44 (DeGamma)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {sRGB, PQ, …} = sRGB
>> +    └─ "next": immutable color operation ID = 45
>> +    Color operation 45 (gamut remap)
>> +    ├─ "type": enum {Bypass, Matrix} = Matrix
>> +    ├─ "matrix_data": blob
>> +    └─ "next": immutable color operation ID = 46
>> +    Color operation 46 (shaper LUT RAM)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 47
> 
> For shaper and blend LUT RAM, that the driver supports pre-defined
> curves and custom LUT at the same time but the resulted LUT is a
> combination of those, how to make this behavior clear? Could this
> behavior be described by two colorop in a row: for example, one for
> shaper TF and,just after, another for shaper LUT or would it not be the
> right representation?
> 

Yes. Again, this is only a (simplified) example in order to show how
things could look like on real HW.

>> +    Color operation 47 (3D LUT RAM)
>> +    ├─ "type": enum {Bypass, 3D LUT} = 3D LUT
>> +    ├─ "lut_size": immutable range = 17
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 48
>> +    Color operation 48 (blend gamma)
>> +    ├─ "type": enum {Bypass, 1D curve} = 1D curve
>> +    ├─ "1d_curve_type": enum {LUT, sRGB, PQ, …} = LUT
>> +    ├─ "lut_size": immutable range = 4096
>> +    ├─ "lut_data": blob
>> +    └─ "next": immutable color operation ID = 0
>> +
>> +
>> +Color Pipeline Programming
>> +==========================
>> +
>> +Once a DRM client has found a suitable pipeline it will:
>> +
>> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
>> +   drm_colorop object of the desired pipeline
>> +2. Set the properties for all drm_colorop objects in the pipeline to the
>> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
>> +   and false for enabled drm_colorop blocks
>> +3. Perform atomic_check/commit as desired
>> +
>> +To configure the pipeline for an HDR10 PQ plane and blending in linear
>> +space, a compositor might perform an atomic commit with the following
>> +property values::
>> +
>> +    Plane 10
>> +    └─ "color_pipeline" = 42
>> +    Color operation 42 (input CSC)
>> +    └─ "bypass" = true
>> +    Color operation 44 (DeGamma)
>> +    └─ "bypass" = true
>> +    Color operation 45 (gamut remap)
>> +    └─ "bypasse" = true
>> +    Color operation 46 (shaper LUT RAM)
>> +    └─ "bypass" = true
>> +    Color operation 47 (3D LUT RAM)
>> +    └─ "lut_data" = Gamut mapping + tone mapping + night mode
>> +    Color operation 48 (blend gamma)
>> +    └─ "1d_curve_type" = PQ inverse EOTF
> 
> Isn't it a PQ EOTF for blend gamma?
> 

Of course.

Harry

> Best Regards,
> 
> Melissa
> 
>> +
>> +
>> +References
>> +==========
>> +
>> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>> \ No newline at end of file
>> -- 
>> 2.42.0
>>

Re: [RFC PATCH 02/10] drm/colorop: Introduce new drm_colorop mode object

[Harry Wentland]

On 2023-10-10 12:19, Melissa Wen wrote:
> On 09/08, Harry Wentland wrote:
>> This patches introduces a new drm_colorop mode object. This
>> object represents color transformations and can be used to
>> define color pipelines.
>>
>> We also introduce the drm_colorop_state here, as well as
>> various helpers and state tracking bits.
>>
>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
>> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
>> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
>> Cc: Simon Ser <contact@emersion.fr>
>> Cc: Harry Wentland <harry.wentland@amd.com>
>> Cc: Melissa Wen <mwen@igalia.com>
>> Cc: Jonas Ådahl <jadahl@redhat.com>
>> Cc: Sebastian Wick <sebastian.wick@redhat.com>
>> Cc: Shashank Sharma <shashank.sharma@amd.com>
>> Cc: Alexander Goins <agoins@nvidia.com>
>> Cc: Joshua Ashton <joshua@froggi.es>
>> Cc: Michel Dänzer <mdaenzer@redhat.com>
>> Cc: Aleix Pol <aleixpol@kde.org>
>> Cc: Xaver Hugl <xaver.hugl@gmail.com>
>> Cc: Victoria Brekenfeld <victoria@system76.com>
>> Cc: Daniel Vetter <daniel@ffwll.ch>
>> Cc: Uma Shankar <uma.shankar@intel.com>
>> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
>> Cc: Christopher Braga <quic_cbraga@quicinc.com>
>> ---
>>  drivers/gpu/drm/Makefile            |   1 +
>>  drivers/gpu/drm/drm_atomic.c        |  79 +++++++++++++
>>  drivers/gpu/drm/drm_atomic_helper.c |  12 ++
>>  drivers/gpu/drm/drm_atomic_uapi.c   |  48 ++++++++
>>  drivers/gpu/drm/drm_colorop.c       | 169 ++++++++++++++++++++++++++++
>>  drivers/gpu/drm/drm_mode_config.c   |   7 ++
>>  drivers/gpu/drm/drm_plane_helper.c  |   2 +-
>>  include/drm/drm_atomic.h            |  82 ++++++++++++++
>>  include/drm/drm_atomic_uapi.h       |   1 +
>>  include/drm/drm_colorop.h           | 157 ++++++++++++++++++++++++++
>>  include/drm/drm_mode_config.h       |  18 +++
>>  include/drm/drm_plane.h             |   2 +
>>  include/uapi/drm/drm.h              |   3 +
>>  include/uapi/drm/drm_mode.h         |   1 +
>>  14 files changed, 581 insertions(+), 1 deletion(-)
>>  create mode 100644 drivers/gpu/drm/drm_colorop.c
>>  create mode 100644 include/drm/drm_colorop.h
>>
>> diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
>> index 1855863b4d7a..941de0269709 100644
>> --- a/drivers/gpu/drm/Makefile
>> +++ b/drivers/gpu/drm/Makefile
>> @@ -16,6 +16,7 @@ drm-y := \
>>  	drm_client.o \
>>  	drm_client_modeset.o \
>>  	drm_color_mgmt.o \
>> +	drm_colorop.o \
>>  	drm_connector.o \
>>  	drm_crtc.o \
>>  	drm_displayid.o \
>> diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
>> index 11f3a130f6f4..d734e9d5bfed 100644
>> --- a/drivers/gpu/drm/drm_atomic.c
>> +++ b/drivers/gpu/drm/drm_atomic.c
>> @@ -42,6 +42,7 @@
>>  #include <drm/drm_mode.h>
>>  #include <drm/drm_print.h>
>>  #include <drm/drm_writeback.h>
>> +#include <drm/drm_colorop.h>
>>  
>>  #include "drm_crtc_internal.h"
>>  #include "drm_internal.h"
>> @@ -108,6 +109,7 @@ void drm_atomic_state_default_release(struct drm_atomic_state *state)
>>  	kfree(state->connectors);
>>  	kfree(state->crtcs);
>>  	kfree(state->planes);
>> +	kfree(state->colorops);
>>  	kfree(state->private_objs);
>>  }
>>  EXPORT_SYMBOL(drm_atomic_state_default_release);
>> @@ -139,6 +141,10 @@ drm_atomic_state_init(struct drm_device *dev, struct drm_atomic_state *state)
>>  				sizeof(*state->planes), GFP_KERNEL);
>>  	if (!state->planes)
>>  		goto fail;
>> +	state->colorops = kcalloc(dev->mode_config.num_colorop,
>> +				  sizeof(*state->colorops), GFP_KERNEL);
>> +	if (!state->colorops)
>> +		goto fail;
>>  
>>  	state->dev = dev;
>>  
>> @@ -244,6 +250,20 @@ void drm_atomic_state_default_clear(struct drm_atomic_state *state)
>>  		state->planes[i].new_state = NULL;
>>  	}
>>  
>> +	for (i = 0; i < config->num_colorop; i++) {
>> +		struct drm_colorop *colorop = state->colorops[i].ptr;
>> +
>> +		if (!colorop)
>> +			continue;
>> +
>> +		drm_colorop_atomic_destroy_state(colorop,
>> +						 state->colorops[i].state);
>> +		state->colorops[i].ptr = NULL;
>> +		state->colorops[i].state = NULL;
>> +		state->colorops[i].old_state = NULL;
>> +		state->colorops[i].new_state = NULL;
>> +	}
>> +
>>  	for (i = 0; i < state->num_private_objs; i++) {
>>  		struct drm_private_obj *obj = state->private_objs[i].ptr;
>>  
>> @@ -562,6 +582,65 @@ drm_atomic_get_plane_state(struct drm_atomic_state *state,
>>  }
>>  EXPORT_SYMBOL(drm_atomic_get_plane_state);
>>  
>> +
>> +/**
>> + * drm_atomic_get_colorop_state - get colorop state
>> + * @state: global atomic state object
>> + * @colorop: colorop to get state object for
>> + *
>> + * This function returns the colorop state for the given colorop, allocating it
>> + * if needed. It will also grab the relevant plane lock to make sure that the
>> + * state is consistent.
>> + *
>> + * Returns:
>> + *
>> + * Either the allocated state or the error code encoded into the pointer. When
>> + * the error is EDEADLK then the w/w mutex code has detected a deadlock and the
>> + * entire atomic sequence must be restarted. All other errors are fatal.
>> + */
>> +struct drm_colorop_state *
>> +drm_atomic_get_colorop_state(struct drm_atomic_state *state,
>> +			     struct drm_colorop *colorop)
>> +{
>> +	int ret, index = drm_colorop_index(colorop);
>> +	struct drm_colorop_state *colorop_state;
>> +	struct drm_plane_state *plane_state;
>> +
>> +	WARN_ON(!state->acquire_ctx);
>> +
>> +	colorop_state = drm_atomic_get_existing_colorop_state(state, colorop);
>> +	if (colorop_state)
>> +		return colorop_state;
>> +
>> +	/* TODO where is the unlock? */
> 
> I understand that this is tracked by acquire-ctx and unlocked by
> `drm_modeset_drop_locks()` (?)
> 

I think you're right. I've left it here as a reminder to review the locking.

Harry

>> +	ret = drm_modeset_lock(&colorop->plane->mutex, state->acquire_ctx);
>> +	if (ret)
>> +		return ERR_PTR(ret);
>> +
>> +	colorop_state = drm_atomic_helper_colorop_duplicate_state(colorop);
>> +	if (!colorop_state)
>> +		return ERR_PTR(-ENOMEM);
>> +
>> +	state->colorops[index].state = colorop_state;
>> +	state->colorops[index].ptr = colorop;
>> +	state->colorops[index].old_state = colorop->state;
>> +	state->colorops[index].new_state = colorop_state;
>> +	colorop_state->state = state;
>> +
>> +	drm_dbg_atomic(colorop->dev, "Added [COLOROP:%d] %p state to %p\n",
>> +		       colorop->base.id, colorop_state, state);
>> +
>> +	/* TODO is this necessary? */
>> +
>> +	plane_state = drm_atomic_get_plane_state(state,
>> +						 colorop->plane);
>> +	if (IS_ERR(plane_state))
>> +		return ERR_CAST(plane_state);
>> +
>> +	return colorop_state;
>> +}
>> +EXPORT_SYMBOL(drm_atomic_get_colorop_state);
>> +
>>  static bool
>>  plane_switching_crtc(const struct drm_plane_state *old_plane_state,
>>  		     const struct drm_plane_state *new_plane_state)
>> diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
>> index d579fd8f7cb8..0472f6182c0a 100644
>> --- a/drivers/gpu/drm/drm_atomic_helper.c
>> +++ b/drivers/gpu/drm/drm_atomic_helper.c
>> @@ -2921,6 +2921,8 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
>>  	struct drm_crtc_state *old_crtc_state, *new_crtc_state;
>>  	struct drm_plane *plane;
>>  	struct drm_plane_state *old_plane_state, *new_plane_state;
>> +	struct drm_colorop *colorop;
>> +	struct drm_colorop_state *old_colorop_state, *new_colorop_state;
>>  	struct drm_crtc_commit *commit;
>>  	struct drm_private_obj *obj;
>>  	struct drm_private_state *old_obj_state, *new_obj_state;
>> @@ -2998,6 +3000,16 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
>>  		}
>>  	}
>>  
>> +	for_each_oldnew_colorop_in_state(state, colorop, old_colorop_state, new_colorop_state, i) {
>> +		WARN_ON(colorop->state != old_colorop_state);
>> +
>> +		old_colorop_state->state = state;
>> +		new_colorop_state->state = NULL;
>> +
>> +		state->colorops[i].state = old_colorop_state;
>> +		colorop->state = new_colorop_state;
>> +	}
>> +
>>  	for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
>>  		WARN_ON(plane->state != old_plane_state);
>>  
>> diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
>> index d867e7f9f2cd..b1aa752c1848 100644
>> --- a/drivers/gpu/drm/drm_atomic_uapi.c
>> +++ b/drivers/gpu/drm/drm_atomic_uapi.c
>> @@ -34,6 +34,7 @@
>>  #include <drm/drm_drv.h>
>>  #include <drm/drm_writeback.h>
>>  #include <drm/drm_vblank.h>
>> +#include <drm/drm_colorop.h>
>>  
>>  #include <linux/dma-fence.h>
>>  #include <linux/uaccess.h>
>> @@ -642,6 +643,26 @@ drm_atomic_plane_get_property(struct drm_plane *plane,
>>  	return 0;
>>  }
>>  
>> +
>> +static int drm_atomic_colorop_set_property(struct drm_colorop *colorop,
>> +		struct drm_colorop_state *state, struct drm_file *file_priv,
>> +		struct drm_property *property, uint64_t val)
>> +{
>> +	drm_dbg_atomic(colorop->dev,
>> +			"[COLOROP:%d] unknown property [PROP:%d:%s]]\n",
>> +			colorop->base.id,
>> +			property->base.id, property->name);
>> +	return -EINVAL;
>> +}
>> +
>> +static int
>> +drm_atomic_colorop_get_property(struct drm_colorop *colorop,
>> +		const struct drm_colorop_state *state,
>> +		struct drm_property *property, uint64_t *val)
>> +{
>> +	return -EINVAL;
>> +}
>> +
>>  static int drm_atomic_set_writeback_fb_for_connector(
>>  		struct drm_connector_state *conn_state,
>>  		struct drm_framebuffer *fb)
>> @@ -893,6 +914,16 @@ int drm_atomic_get_property(struct drm_mode_object *obj,
>>  				plane->state, property, val);
>>  		break;
>>  	}
>> +	case DRM_MODE_OBJECT_COLOROP: {
>> +		struct drm_colorop *colorop = obj_to_colorop(obj);
>> +
>> +		if (colorop->plane)
>> +			WARN_ON(!drm_modeset_is_locked(&colorop->plane->mutex));
>> +
>> +		ret = drm_atomic_colorop_get_property(colorop,
>> +				colorop->state, property, val);
>> +		break;
>> +	}
>>  	default:
>>  		ret = -EINVAL;
>>  		break;
>> @@ -1027,6 +1058,23 @@ int drm_atomic_set_property(struct drm_atomic_state *state,
>>  		ret = drm_atomic_plane_set_property(plane,
>>  				plane_state, file_priv,
>>  				prop, prop_value);
>> +
>> +		break;
>> +	}
>> +	case DRM_MODE_OBJECT_COLOROP: {
>> +		struct drm_colorop *colorop = obj_to_colorop(obj);
>> +		struct drm_colorop_state *colorop_state;
>> +
>> +		colorop_state = drm_atomic_get_colorop_state(state, colorop);
>> +		if (IS_ERR(colorop_state)) {
>> +			ret = PTR_ERR(colorop_state);
>> +			break;
>> +		}
>> +
>> +		ret = drm_atomic_colorop_set_property(colorop,
>> +				colorop_state, file_priv,
>> +				prop, prop_value);
>> +
>>  		break;
>>  	}
>>  	default:
>> diff --git a/drivers/gpu/drm/drm_colorop.c b/drivers/gpu/drm/drm_colorop.c
>> new file mode 100644
>> index 000000000000..78d6a0067f5b
>> --- /dev/null
>> +++ b/drivers/gpu/drm/drm_colorop.c
>> @@ -0,0 +1,169 @@
>> +/*
>> + * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a
>> + * copy of this software and associated documentation files (the "Software"),
>> + * to deal in the Software without restriction, including without limitation
>> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
>> + * and/or sell copies of the Software, and to permit persons to whom the
>> + * Software is furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
>> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
>> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
>> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
>> + * OTHER DEALINGS IN THE SOFTWARE.
>> + *
>> + * Authors: AMD
>> + *
>> + */
>> +
>> +#include <drm/drm_colorop.h>
>> +#include <drm/drm_print.h>
>> +#include <drm/drm_drv.h>
>> +#include <drm/drm_plane.h>
>> +
>> +#include "drm_crtc_internal.h"
>> +
>> +/* TODO big colorop doc, including properties, etc. */
>> +
>> +/* Init Helpers */
>> +
>> +int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
>> +		     struct drm_plane *plane)
>> +{
>> +	struct drm_mode_config *config = &dev->mode_config;
>> +	int ret = 0;
>> +
>> +	ret = drm_mode_object_add(dev, &colorop->base, DRM_MODE_OBJECT_COLOROP);
>> +	if (ret)
>> +		return ret;
>> +
>> +	colorop->base.properties = &colorop->properties;
>> +	colorop->dev = dev;
>> +	colorop->plane = plane;
>> +
>> +	list_add_tail(&colorop->head, &config->colorop_list);
>> +	colorop->index = config->num_colorop++;
>> +
>> +	/* add properties */
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL(drm_colorop_init);
>> +
>> +void __drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop,
>> +						 struct drm_colorop_state *state)
>> +{
>> +	memcpy(state, colorop->state, sizeof(*state));
>> +}
>> +
>> +struct drm_colorop_state *
>> +drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop)
>> +{
>> +	struct drm_colorop_state *state;
>> +
>> +	if (WARN_ON(!colorop->state))
>> +		return NULL;
>> +
>> +	state = kmalloc(sizeof(*state), GFP_KERNEL);
>> +	if (state)
>> +		__drm_atomic_helper_colorop_duplicate_state(colorop, state);
>> +
>> +	return state;
>> +}
>> +
>> +
>> +void drm_colorop_atomic_destroy_state(struct drm_colorop *colorop,
>> +				      struct drm_colorop_state *state)
>> +{
>> +	kfree(state);
>> +}
>> +
>> +/**
>> + * __drm_colorop_destroy_state - release colorop state
>> + * @state: colorop state object to release
>> + *
>> + * Releases all resources stored in the colorop state without actually freeing
>> + * the memory of the colorop state. This is useful for drivers that subclass the
>> + * colorop state.
>> + */
>> +void __drm_colorop_destroy_state(struct drm_colorop_state *state)
>> +{
>> +	/* TODO might need this later */
>> +}
>> +
>> +/**
>> + * drm_colorop_destroy_state - default state destroy hook
>> + * @colorop: drm colorop
>> + * @state: colorop state object to release
>> + *
>> + * Default colorop state destroy hook for drivers which don't have their own
>> + * subclassed colorop state structure.
>> + */
>> +void drm_colorop_destroy_state(struct drm_colorop *colorop,
>> +					   struct drm_colorop_state *state)
>> +{
>> +	kfree(state);
>> +}
>> +EXPORT_SYMBOL(drm_colorop_destroy_state);
>> +
>> +/**
>> + * __drm_colorop_state_reset - resets colorop state to default values
>> + * @colorop_state: atomic colorop state, must not be NULL
>> + * @colorop: colorop object, must not be NULL
>> + *
>> + * Initializes the newly allocated @colorop_state with default
>> + * values. This is useful for drivers that subclass the CRTC state.
>> + */
>> +void __drm_colorop_state_reset(struct drm_colorop_state *colorop_state,
>> +					   struct drm_colorop *colorop)
>> +{
>> +	colorop_state->colorop = colorop;
>> +}
>> +EXPORT_SYMBOL(__drm_colorop_state_reset);
>> +
>> +/**
>> + * __drm_colorop_reset - reset state on colorop
>> + * @colorop: drm colorop
>> + * @colorop_state: colorop state to assign
>> + *
>> + * Initializes the newly allocated @colorop_state and assigns it to
>> + * the &drm_crtc->state pointer of @colorop, usually required when
>> + * initializing the drivers or when called from the &drm_colorop_funcs.reset
>> + * hook.
>> + *
>> + * This is useful for drivers that subclass the colorop state.
>> + */
>> +void __drm_colorop_reset(struct drm_colorop *colorop,
>> +				     struct drm_colorop_state *colorop_state)
>> +{
>> +	if (colorop_state)
>> +		__drm_colorop_state_reset(colorop_state, colorop);
>> +
>> +	colorop->state = colorop_state;
>> +}
>> +
>> +/**
>> + * drm_colorop_reset - reset colorop atomic state
>> + * @colorop: drm colorop
>> + *
>> + * Resets the atomic state for @colorop by freeing the state pointer (which might
>> + * be NULL, e.g. at driver load time) and allocating a new empty state object.
>> + */
>> +void drm_colorop_reset(struct drm_colorop *colorop)
>> +{
>> +	if (colorop->state)
>> +		__drm_colorop_destroy_state(colorop->state);
>> +
>> +	kfree(colorop->state);
>> +	colorop->state = kzalloc(sizeof(*colorop->state), GFP_KERNEL);
>> +
>> +	if (colorop->state)
>> +		__drm_colorop_reset(colorop, colorop->state);
>> +}
>> +EXPORT_SYMBOL(drm_colorop_reset);
>> diff --git a/drivers/gpu/drm/drm_mode_config.c b/drivers/gpu/drm/drm_mode_config.c
>> index 87eb591fe9b5..5bdcf71e1ae0 100644
>> --- a/drivers/gpu/drm/drm_mode_config.c
>> +++ b/drivers/gpu/drm/drm_mode_config.c
>> @@ -29,6 +29,7 @@
>>  #include <drm/drm_managed.h>
>>  #include <drm/drm_mode_config.h>
>>  #include <drm/drm_print.h>
>> +#include <drm/drm_colorop.h>
>>  #include <linux/dma-resv.h>
>>  
>>  #include "drm_crtc_internal.h"
>> @@ -184,11 +185,15 @@ int drm_mode_getresources(struct drm_device *dev, void *data,
>>  void drm_mode_config_reset(struct drm_device *dev)
>>  {
>>  	struct drm_crtc *crtc;
>> +	struct drm_colorop *colorop;
>>  	struct drm_plane *plane;
>>  	struct drm_encoder *encoder;
>>  	struct drm_connector *connector;
>>  	struct drm_connector_list_iter conn_iter;
>>  
>> +	drm_for_each_colorop(colorop, dev)
>> +		drm_colorop_reset(colorop);
>> +
>>  	drm_for_each_plane(plane, dev)
>>  		if (plane->funcs->reset)
>>  			plane->funcs->reset(plane);
>> @@ -415,6 +420,7 @@ int drmm_mode_config_init(struct drm_device *dev)
>>  	INIT_LIST_HEAD(&dev->mode_config.property_list);
>>  	INIT_LIST_HEAD(&dev->mode_config.property_blob_list);
>>  	INIT_LIST_HEAD(&dev->mode_config.plane_list);
>> +	INIT_LIST_HEAD(&dev->mode_config.colorop_list);
>>  	INIT_LIST_HEAD(&dev->mode_config.privobj_list);
>>  	idr_init_base(&dev->mode_config.object_idr, 1);
>>  	idr_init_base(&dev->mode_config.tile_idr, 1);
>> @@ -436,6 +442,7 @@ int drmm_mode_config_init(struct drm_device *dev)
>>  	dev->mode_config.num_crtc = 0;
>>  	dev->mode_config.num_encoder = 0;
>>  	dev->mode_config.num_total_plane = 0;
>> +	dev->mode_config.num_colorop = 0;
>>  
>>  	if (IS_ENABLED(CONFIG_LOCKDEP)) {
>>  		struct drm_modeset_acquire_ctx modeset_ctx;
>> diff --git a/drivers/gpu/drm/drm_plane_helper.c b/drivers/gpu/drm/drm_plane_helper.c
>> index c91e454eba09..69faa0eeb27f 100644
>> --- a/drivers/gpu/drm/drm_plane_helper.c
>> +++ b/drivers/gpu/drm/drm_plane_helper.c
>> @@ -318,4 +318,4 @@ int drm_plane_helper_atomic_check(struct drm_plane *plane, struct drm_atomic_sta
>>  						   DRM_PLANE_NO_SCALING,
>>  						   false, false);
>>  }
>> -EXPORT_SYMBOL(drm_plane_helper_atomic_check);
>> +EXPORT_SYMBOL(drm_plane_helper_atomic_check);
>> \ No newline at end of file
>> diff --git a/include/drm/drm_atomic.h b/include/drm/drm_atomic.h
>> index 92586ab55ef5..dca8fc26ad71 100644
>> --- a/include/drm/drm_atomic.h
>> +++ b/include/drm/drm_atomic.h
>> @@ -30,6 +30,7 @@
>>  
>>  #include <drm/drm_crtc.h>
>>  #include <drm/drm_util.h>
>> +#include <drm/drm_colorop.h>
>>  
>>  /**
>>   * struct drm_crtc_commit - track modeset commits on a CRTC
>> @@ -157,6 +158,11 @@ struct drm_crtc_commit {
>>  	bool abort_completion;
>>  };
>>  
>> +struct __drm_colorops_state {
>> +	struct drm_colorop *ptr;
>> +	struct drm_colorop_state *state, *old_state, *new_state;
>> +};
>> +
>>  struct __drm_planes_state {
>>  	struct drm_plane *ptr;
>>  	struct drm_plane_state *state, *old_state, *new_state;
>> @@ -398,6 +404,7 @@ struct drm_atomic_state {
>>  	 * states.
>>  	 */
>>  	bool duplicated : 1;
>> +	struct __drm_colorops_state *colorops;
>>  	struct __drm_planes_state *planes;
>>  	struct __drm_crtcs_state *crtcs;
>>  	int num_connector;
>> @@ -501,6 +508,9 @@ drm_atomic_get_crtc_state(struct drm_atomic_state *state,
>>  struct drm_plane_state * __must_check
>>  drm_atomic_get_plane_state(struct drm_atomic_state *state,
>>  			   struct drm_plane *plane);
>> +struct drm_colorop_state *
>> +drm_atomic_get_colorop_state(struct drm_atomic_state *state,
>> +			     struct drm_colorop *colorop);
>>  struct drm_connector_state * __must_check
>>  drm_atomic_get_connector_state(struct drm_atomic_state *state,
>>  			       struct drm_connector *connector);
>> @@ -623,6 +633,55 @@ drm_atomic_get_new_plane_state(const struct drm_atomic_state *state,
>>  	return state->planes[drm_plane_index(plane)].new_state;
>>  }
>>  
>> +
>> +/**
>> + * drm_atomic_get_existing_colorop_state - get colorop state, if it exists
>> + * @state: global atomic state object
>> + * @colorop: colorop to grab
>> + *
>> + * This function returns the colorop state for the given colorop, or NULL
>> + * if the colorop is not part of the global atomic state.
>> + *
>> + * This function is deprecated, @drm_atomic_get_old_colorop_state or
>> + * @drm_atomic_get_new_colorop_state should be used instead.
>> + */
>> +static inline struct drm_colorop_state *
>> +drm_atomic_get_existing_colorop_state(struct drm_atomic_state *state,
>> +				    struct drm_colorop *colorop)
>> +{
>> +	return state->colorops[drm_colorop_index(colorop)].state;
>> +}
>> +
>> +/**
>> + * drm_atomic_get_old_colorop_state - get colorop state, if it exists
>> + * @state: global atomic state object
>> + * @colorop: colorop to grab
>> + *
>> + * This function returns the old colorop state for the given colorop, or
>> + * NULL if the colorop is not part of the global atomic state.
>> + */
>> +static inline struct drm_colorop_state *
>> +drm_atomic_get_old_colorop_state(struct drm_atomic_state *state,
>> +			       struct drm_colorop *colorop)
>> +{
>> +	return state->colorops[drm_colorop_index(colorop)].old_state;
>> +}
>> +
>> +/**
>> + * drm_atomic_get_new_colorop_state - get colorop state, if it exists
>> + * @state: global atomic state object
>> + * @colorop: colorop to grab
>> + *
>> + * This function returns the new colorop state for the given colorop, or
>> + * NULL if the colorop is not part of the global atomic state.
>> + */
>> +static inline struct drm_colorop_state *
>> +drm_atomic_get_new_colorop_state(struct drm_atomic_state *state,
>> +			       struct drm_colorop *colorop)
>> +{
>> +	return state->colorops[drm_colorop_index(colorop)].new_state;
>> +}
>> +
>>  /**
>>   * drm_atomic_get_existing_connector_state - get connector state, if it exists
>>   * @state: global atomic state object
>> @@ -870,6 +929,29 @@ void drm_state_dump(struct drm_device *dev, struct drm_printer *p);
>>  			     (new_crtc_state) = (__state)->crtcs[__i].new_state, \
>>  			     (void)(new_crtc_state) /* Only to avoid unused-but-set-variable warning */, 1))
>>  
>> +/**
>> + * for_each_oldnew_colorop_in_state - iterate over all colorops in an atomic update
>> + * @__state: &struct drm_atomic_state pointer
>> + * @colorop: &struct drm_colorop iteration cursor
>> + * @old_colorop_state: &struct drm_colorop_state iteration cursor for the old state
>> + * @new_colorop_state: &struct drm_colorop_state iteration cursor for the new state
>> + * @__i: int iteration cursor, for macro-internal use
>> + *
>> + * This iterates over all colorops in an atomic update, tracking both old and
>> + * new state. This is useful in places where the state delta needs to be
>> + * considered, for example in atomic check functions.
>> + */
>> +#define for_each_oldnew_colorop_in_state(__state, colorop, old_colorop_state, new_colorop_state, __i) \
>> +	for ((__i) = 0;							\
>> +	     (__i) < (__state)->dev->mode_config.num_colorop;	\
>> +	     (__i)++)							\
>> +		for_each_if ((__state)->colorops[__i].ptr &&		\
>> +			     ((colorop) = (__state)->colorops[__i].ptr,	\
>> +			      (void)(colorop) /* Only to avoid unused-but-set-variable warning */, \
>> +			      (old_colorop_state) = (__state)->colorops[__i].old_state,\
>> +			      (new_colorop_state) = (__state)->colorops[__i].new_state, 1))
>> +
>> +
>>  /**
>>   * for_each_oldnew_plane_in_state - iterate over all planes in an atomic update
>>   * @__state: &struct drm_atomic_state pointer
>> diff --git a/include/drm/drm_atomic_uapi.h b/include/drm/drm_atomic_uapi.h
>> index 4c6d39d7bdb2..70a115d523cd 100644
>> --- a/include/drm/drm_atomic_uapi.h
>> +++ b/include/drm/drm_atomic_uapi.h
>> @@ -37,6 +37,7 @@ struct drm_crtc;
>>  struct drm_connector_state;
>>  struct dma_fence;
>>  struct drm_framebuffer;
>> +struct drm_colorop;
>>  
>>  int __must_check
>>  drm_atomic_set_mode_for_crtc(struct drm_crtc_state *state,
>> diff --git a/include/drm/drm_colorop.h b/include/drm/drm_colorop.h
>> new file mode 100644
>> index 000000000000..3dd169b0317d
>> --- /dev/null
>> +++ b/include/drm/drm_colorop.h
>> @@ -0,0 +1,157 @@
>> +/*
>> + * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a
>> + * copy of this software and associated documentation files (the "Software"),
>> + * to deal in the Software without restriction, including without limitation
>> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
>> + * and/or sell copies of the Software, and to permit persons to whom the
>> + * Software is furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
>> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
>> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
>> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
>> + * OTHER DEALINGS IN THE SOFTWARE.
>> + *
>> + * Authors: AMD
>> + *
>> + */
>> +
>> +#ifndef __DRM_COLOROP_H__
>> +#define __DRM_COLOROP_H__
>> +
>> +#include <drm/drm_mode_object.h>
>> +#include <drm/drm_mode.h>
>> +#include <drm/drm_property.h>
>> +
>> +/**
>> + * struct drm_colorop_state - mutable colorop state
>> + */
>> +struct drm_colorop_state {
>> +	/** @colorop: backpointer to the colorop */
>> +	struct drm_colorop *colorop;
>> +
>> +	/* colorop properties */
>> +
>> +	/** @state: backpointer to global drm_atomic_state */
>> +	struct drm_atomic_state *state;
>> +};
>> +
>> +/**
>> + * struct drm_colorop - DRM color operation control structure
>> + *
>> + * A colorop represents one color operation. They can be chained via
>> + * the 'next' pointer to build a color pipeline.
>> + */
>> +struct drm_colorop {
>> +	/** @dev: parent DRM device */
>> +	struct drm_device *dev;
>> +
>> +	/**
>> +	 * @head:
>> +	 *
>> +	 * List of all colorops on @dev, linked from &drm_mode_config.colorop_list.
>> +	 * Invariant over the lifetime of @dev and therefore does not need
>> +	 * locking.
>> +	 */
>> +	struct list_head head;
>> +
>> +	/**
>> +	 * @index: Position inside the mode_config.list, can be used as an array
>> +	 * index. It is invariant over the lifetime of the plane.
>> +	 */
>> +	unsigned index;
>> +
>> +	/* TODO do we need a separate mutex or will we tag along with the plane mutex? */
>> +
>> +	/** @base base mode object*/
>> +	struct drm_mode_object base;
>> +
>> +	/**
>> +	 * @plane:
>> +	 *
>> +	 * The plane on which the colorop sits. A drm_colorop is always unique
>> +	 * to a plane.
>> +	 */
>> +	struct drm_plane *plane;
>> +
>> +	/**
>> +	 * @state:
>> +	 *
>> +	 * Current atomic state for this colorop.
>> +	 *
>> +	 * This is protected by @mutex. Note that nonblocking atomic commits
>> +	 * access the current colorop state without taking locks. Either by
>> +	 * going through the &struct drm_atomic_state pointers, see
>> +	 * for_each_oldnew_plane_in_state(), for_each_old_plane_in_state() and
>> +	 * for_each_new_plane_in_state(). Or through careful ordering of atomic
>> +	 * commit operations as implemented in the atomic helpers, see
>> +	 * &struct drm_crtc_commit.
>> +	 *
>> +	 * TODO keep, remove, or rewrite above plane references?
>> +	 */
>> +	struct drm_colorop_state *state;
>> +
>> +	/* colorop properties */
>> +
>> +	/** @properties: property tracking for this plane */
>> +	struct drm_object_properties properties;
>> +
>> +};
>> +
>> +#define obj_to_colorop(x) container_of(x, struct drm_colorop, base)
>> +
>> +/**
>> + * drm_crtc_find - look up a Colorop object from its ID
>> + * @dev: DRM device
>> + * @file_priv: drm file to check for lease against.
>> + * @id: &drm_mode_object ID
>> + *
>> + * This can be used to look up a Colorop from its userspace ID. Only used by
>> + * drivers for legacy IOCTLs and interface, nowadays extensions to the KMS
>> + * userspace interface should be done using &drm_property.
>> + */
>> +static inline struct drm_colorop *drm_colorop_find(struct drm_device *dev,
>> +		struct drm_file *file_priv,
>> +		uint32_t id)
>> +{
>> +	struct drm_mode_object *mo;
>> +	mo = drm_mode_object_find(dev, file_priv, id, DRM_MODE_OBJECT_COLOROP);
>> +	return mo ? obj_to_colorop(mo) : NULL;
>> +}
>> +
>> +int drm_colorop_init(struct drm_device *dev, struct drm_colorop *colorop,
>> +		     struct drm_plane *plane);
>> +
>> +struct drm_colorop_state *
>> +drm_atomic_helper_colorop_duplicate_state(struct drm_colorop *colorop);
>> +
>> +void drm_colorop_atomic_destroy_state(struct drm_colorop *colorop,
>> +				      struct drm_colorop_state *state);
>> +
>> +void drm_colorop_reset(struct drm_colorop *colorop);
>> +
>> +/**
>> + * drm_colorop_index - find the index of a registered colorop
>> + * @colorop: colorop to find index for
>> + *
>> + * Given a registered colorop, return the index of that colorop within a DRM
>> + * device's list of colorops.
>> + */
>> +static inline unsigned int drm_colorop_index(const struct drm_colorop *colorop)
>> +{
>> +	return colorop->index;
>> +}
>> +
>> +
>> +#define drm_for_each_colorop(colorop, dev) \
>> +	list_for_each_entry(colorop, &(dev)->mode_config.colorop_list, head)
>> +
>> +
>> +#endif /* __DRM_COLOROP_H__ */
>> diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h
>> index e5b053001d22..f56d21d93cf0 100644
>> --- a/include/drm/drm_mode_config.h
>> +++ b/include/drm/drm_mode_config.h
>> @@ -505,6 +505,24 @@ struct drm_mode_config {
>>  	 */
>>  	struct list_head plane_list;
>>  
>> +	/**
>> +	 * @num_colorop:
>> +	 *
>> +	 * Number of colorop objects on this device.
>> +	 * This is invariant over the lifetime of a device and hence doesn't
>> +	 * need any locks.
>> +	 */
>> +	int num_colorop;
>> +
>> +	/**
>> +	 * @colorops_list:
>> +	 *
>> +	 * List of colorop objects linked with &drm_colorop.head. This is
>> +	 * invariant over the lifetime of a device and hence doesn't need any
>> +	 * locks.
>> +	 */
>> +	struct list_head colorop_list;
>> +
>>  	/**
>>  	 * @num_crtc:
>>  	 *
>> diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
>> index 51291983ea44..cfb8f46b94ab 100644
>> --- a/include/drm/drm_plane.h
>> +++ b/include/drm/drm_plane.h
>> @@ -227,6 +227,8 @@ struct drm_plane_state {
>>  	 */
>>  	enum drm_scaling_filter scaling_filter;
>>  
>> +	struct drm_colorop *color_pipeline;
>> +
>>  	/**
>>  	 * @commit: Tracks the pending commit to prevent use-after-free conditions,
>>  	 * and for async plane updates.
>> diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h
>> index 642808520d92..dec498a44eae 100644
>> --- a/include/uapi/drm/drm.h
>> +++ b/include/uapi/drm/drm.h
>> @@ -1116,6 +1116,9 @@ extern "C" {
>>   */
>>  #define DRM_IOCTL_MODE_GETFB2		DRM_IOWR(0xCE, struct drm_mode_fb_cmd2)
>>  
>> +#define DRM_IOCTL_MODE_GETCOLOROPRESOURCES DRM_IOWR(0xD0, struct drm_mode_get_colorop_res)
>> +#define DRM_IOCTL_MODE_GETCOLOROP          DRM_IOWR(0xD1, struct drm_mode_get_colorop)
>> +
>>  /*
>>   * Device specific ioctls should only be in their respective headers
>>   * The device specific ioctl range is from 0x40 to 0x9f.
>> diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
>> index 46becedf5b2f..6dcf628def56 100644
>> --- a/include/uapi/drm/drm_mode.h
>> +++ b/include/uapi/drm/drm_mode.h
>> @@ -626,6 +626,7 @@ struct drm_mode_connector_set_property {
>>  #define DRM_MODE_OBJECT_FB 0xfbfbfbfb
>>  #define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb
>>  #define DRM_MODE_OBJECT_PLANE 0xeeeeeeee
>> +#define DRM_MODE_OBJECT_COLOROP 0xfafafafa
>>  #define DRM_MODE_OBJECT_ANY 0
>>  
>>  struct drm_mode_obj_get_properties {
>> -- 
>> 2.42.0
>>

Re: [RFC PATCH 10/10] drm/vkms: Add enumerated 1D curve colorop

[Harry Wentland]

On 2023-10-10 12:27, Melissa Wen wrote:
> On 09/08, Harry Wentland wrote:
>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
>> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
>> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
>> Cc: Simon Ser <contact@emersion.fr>
>> Cc: Harry Wentland <harry.wentland@amd.com>
>> Cc: Melissa Wen <mwen@igalia.com>
>> Cc: Jonas Ådahl <jadahl@redhat.com>
>> Cc: Sebastian Wick <sebastian.wick@redhat.com>
>> Cc: Shashank Sharma <shashank.sharma@amd.com>
>> Cc: Alexander Goins <agoins@nvidia.com>
>> Cc: Joshua Ashton <joshua@froggi.es>
>> Cc: Michel Dänzer <mdaenzer@redhat.com>
>> Cc: Aleix Pol <aleixpol@kde.org>
>> Cc: Xaver Hugl <xaver.hugl@gmail.com>
>> Cc: Victoria Brekenfeld <victoria@system76.com>
>> Cc: Daniel Vetter <daniel@ffwll.ch>
>> Cc: Uma Shankar <uma.shankar@intel.com>
>> Cc: Naseer Ahmed <quic_naseer@quicinc.com>
>> Cc: Christopher Braga <quic_cbraga@quicinc.com>
>> ---
>>  drivers/gpu/drm/vkms/Makefile        |   3 +-
>>  drivers/gpu/drm/vkms/vkms_colorop.c  | 108 +++++++++
>>  drivers/gpu/drm/vkms/vkms_composer.c | 316 +++++++++++++++++++++++++++
>>  drivers/gpu/drm/vkms/vkms_drv.h      |   4 +
>>  drivers/gpu/drm/vkms/vkms_plane.c    |   2 +
>>  5 files changed, 432 insertions(+), 1 deletion(-)
>>  create mode 100644 drivers/gpu/drm/vkms/vkms_colorop.c
>>
>> diff --git a/drivers/gpu/drm/vkms/Makefile b/drivers/gpu/drm/vkms/Makefile
>> index 1b28a6a32948..bcf508873622 100644
>> --- a/drivers/gpu/drm/vkms/Makefile
>> +++ b/drivers/gpu/drm/vkms/Makefile
>> @@ -6,6 +6,7 @@ vkms-y := \
>>  	vkms_formats.o \
>>  	vkms_crtc.o \
>>  	vkms_composer.o \
>> -	vkms_writeback.o
>> +	vkms_writeback.o \
>> +	vkms_colorop.o
>>  
>>  obj-$(CONFIG_DRM_VKMS) += vkms.o
>> diff --git a/drivers/gpu/drm/vkms/vkms_colorop.c b/drivers/gpu/drm/vkms/vkms_colorop.c
>> new file mode 100644
>> index 000000000000..b3da0705bca7
>> --- /dev/null
>> +++ b/drivers/gpu/drm/vkms/vkms_colorop.c
>> @@ -0,0 +1,108 @@
>> +/*
>> + * Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a
>> + * copy of this software and associated documentation files (the "Software"),
>> + * to deal in the Software without restriction, including without limitation
>> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
>> + * and/or sell copies of the Software, and to permit persons to whom the
>> + * Software is furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
>> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
>> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
>> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
>> + * OTHER DEALINGS IN THE SOFTWARE.
>> + *
>> + * Authors: AMD
>> + *
>> + */
>> +
>> +#include <linux/slab.h>
>> +#include <drm/drm_colorop.h>
>> +#include <drm/drm_print.h>
>> +#include <drm/drm_property.h>
>> +#include <drm/drm_plane.h>
>> +
>> +#define MAX_COLOR_PIPELINES 5
>> +
>> +const int vkms_initialize_tf_pipeline(struct drm_plane *plane, struct drm_prop_enum_list *list)
>> +{
>> +
>> +	struct drm_colorop *op, *prev_op;
>> +	struct drm_device *dev = plane->dev;
>> +	int ret;
>> +
>> +	/* 1st op: 1d curve */
>> +	op = kzalloc(sizeof(struct drm_colorop), GFP_KERNEL);
>> +	if (!op) {
>> +		DRM_ERROR("KMS: Failed to allocate colorop\n");
>> +		return -ENOMEM;
>> +	}
>> +
>> +	ret = drm_colorop_init(dev, op, plane, DRM_COLOROP_1D_CURVE);
>> +	if (ret)
>> +		return ret;
>> +
>> +	list->type = op->base.id;
>> +	list->name = kasprintf(GFP_KERNEL, "Color Pipeline %d", op->base.id);
>> +
>> +	prev_op = op;
>> +
>> +	/* 2nd op: 1d curve */
>> +	op = kzalloc(sizeof(struct drm_colorop), GFP_KERNEL);
>> +	if (!op) {
>> +		DRM_ERROR("KMS: Failed to allocate colorop\n");
>> +		return -ENOMEM;
>> +	}
>> +
>> +	ret = drm_colorop_init(dev, op, plane, DRM_COLOROP_1D_CURVE);
>> +	if (ret)
>> +		return ret;
>> +
>> +	drm_colorop_set_next_property(prev_op, op);
>> +
>> +	return 0;
>> +}
>> +
>> +int vkms_initialize_colorops(struct drm_plane *plane)
>> +{
>> +	struct drm_device *dev = plane->dev;
>> +	struct drm_property *prop;
>> +	struct drm_prop_enum_list pipelines[MAX_COLOR_PIPELINES];
>> +	int len = 0;
>> +	int ret;
>> +
>> +	/* Add "Bypass" (i.e. NULL) pipeline */
>> +	pipelines[len].type = 0;
>> +	pipelines[len].name = "Bypass";
>> +	len++;
>> +
>> +	/* Add pipeline consisting of transfer functions */
>> +	ret = vkms_initialize_tf_pipeline(plane, &(pipelines[len]));
>> +	if (ret)
>> +		return ret;
>> +	len++;
>> +
>> +	/* Create COLOR_PIPELINE property and attach */
>> +	prop = drm_property_create_enum(dev, DRM_MODE_PROP_ATOMIC,
>> +					"COLOR_PIPELINE",
>> +					pipelines, len);
>> +	if (!prop)
>> +		return -ENOMEM;
>> +
>> +	plane->color_pipeline_property = prop;
>> +
>> +	drm_object_attach_property(&plane->base, prop, 0);
>> +
>> +	/* TODO do we even need this? */
>> +	if (plane->state)
>> +		plane->state->color_pipeline = NULL;
>> +
>> +	return 0;
>> +}
>> diff --git a/drivers/gpu/drm/vkms/vkms_composer.c b/drivers/gpu/drm/vkms/vkms_composer.c
>> index f6c311e8a87c..92ab9c62a554 100644
>> --- a/drivers/gpu/drm/vkms/vkms_composer.c
>> +++ b/drivers/gpu/drm/vkms/vkms_composer.c
>> @@ -12,6 +12,284 @@
>>  
>>  #include "vkms_drv.h"
>>  
>> +#define LUT_SIZE 256
>> +
>> +struct drm_color_lut srgb_array[LUT_SIZE] = {
>> +	{ 0x13, 0x13, 0x13, 0 },
>> +	{ 0x27, 0x27, 0x27, 0 },
>> +	{ 0x3b, 0x3b, 0x3b, 0 },
>> +	{ 0x4f, 0x4f, 0x4f, 0 },
>> +	{ 0x63, 0x63, 0x63, 0 },
>> +	{ 0x76, 0x76, 0x76, 0 },
>> +	{ 0x8a, 0x8a, 0x8a, 0 },
>> +	{ 0x9e, 0x9e, 0x9e, 0 },
>> +	{ 0xb2, 0xb2, 0xb2, 0 },
>> +	{ 0xc6, 0xc6, 0xc6, 0 },
>> +	{ 0xda, 0xda, 0xda, 0 },
>> +	{ 0xef, 0xef, 0xef, 0 },
>> +	{ 0x106, 0x106, 0x106, 0 },
>> +	{ 0x11e, 0x11e, 0x11e, 0 },
>> +	{ 0x137, 0x137, 0x137, 0 },
>> +	{ 0x151, 0x151, 0x151, 0 },
>> +	{ 0x16d, 0x16d, 0x16d, 0 },
>> +	{ 0x18a, 0x18a, 0x18a, 0 },
>> +	{ 0x1a8, 0x1a8, 0x1a8, 0 },
>> +	{ 0x1c7, 0x1c7, 0x1c7, 0 },
>> +	{ 0x1e8, 0x1e8, 0x1e8, 0 },
>> +	{ 0x20a, 0x20a, 0x20a, 0 },
>> +	{ 0x22e, 0x22e, 0x22e, 0 },
>> +	{ 0x253, 0x253, 0x253, 0 },
>> +	{ 0x279, 0x279, 0x279, 0 },
>> +	{ 0x2a0, 0x2a0, 0x2a0, 0 },
>> +	{ 0x2c9, 0x2c9, 0x2c9, 0 },
>> +	{ 0x2f4, 0x2f4, 0x2f4, 0 },
>> +	{ 0x320, 0x320, 0x320, 0 },
>> +	{ 0x34d, 0x34d, 0x34d, 0 },
>> +	{ 0x37c, 0x37c, 0x37c, 0 },
>> +	{ 0x3ac, 0x3ac, 0x3ac, 0 },
>> +	{ 0x3de, 0x3de, 0x3de, 0 },
>> +	{ 0x411, 0x411, 0x411, 0 },
>> +	{ 0x446, 0x446, 0x446, 0 },
>> +	{ 0x47c, 0x47c, 0x47c, 0 },
>> +	{ 0x4b4, 0x4b4, 0x4b4, 0 },
>> +	{ 0x4ed, 0x4ed, 0x4ed, 0 },
>> +	{ 0x528, 0x528, 0x528, 0 },
>> +	{ 0x564, 0x564, 0x564, 0 },
>> +	{ 0x5a3, 0x5a3, 0x5a3, 0 },
>> +	{ 0x5e2, 0x5e2, 0x5e2, 0 },
>> +	{ 0x624, 0x624, 0x624, 0 },
>> +	{ 0x666, 0x666, 0x666, 0 },
>> +	{ 0x6ab, 0x6ab, 0x6ab, 0 },
>> +	{ 0x6f1, 0x6f1, 0x6f1, 0 },
>> +	{ 0x739, 0x739, 0x739, 0 },
>> +	{ 0x782, 0x782, 0x782, 0 },
>> +	{ 0x7ce, 0x7ce, 0x7ce, 0 },
>> +	{ 0x81b, 0x81b, 0x81b, 0 },
>> +	{ 0x869, 0x869, 0x869, 0 },
>> +	{ 0x8b9, 0x8b9, 0x8b9, 0 },
>> +	{ 0x90b, 0x90b, 0x90b, 0 },
>> +	{ 0x95f, 0x95f, 0x95f, 0 },
>> +	{ 0x9b5, 0x9b5, 0x9b5, 0 },
>> +	{ 0xa0c, 0xa0c, 0xa0c, 0 },
>> +	{ 0xa65, 0xa65, 0xa65, 0 },
>> +	{ 0xabf, 0xabf, 0xabf, 0 },
>> +	{ 0xb1c, 0xb1c, 0xb1c, 0 },
>> +	{ 0xb7a, 0xb7a, 0xb7a, 0 },
>> +	{ 0xbda, 0xbda, 0xbda, 0 },
>> +	{ 0xc3c, 0xc3c, 0xc3c, 0 },
>> +	{ 0xca0, 0xca0, 0xca0, 0 },
>> +	{ 0xd06, 0xd06, 0xd06, 0 },
>> +	{ 0xd6d, 0xd6d, 0xd6d, 0 },
>> +	{ 0xdd6, 0xdd6, 0xdd6, 0 },
>> +	{ 0xe41, 0xe41, 0xe41, 0 },
>> +	{ 0xeae, 0xeae, 0xeae, 0 },
>> +	{ 0xf1d, 0xf1d, 0xf1d, 0 },
>> +	{ 0xf8e, 0xf8e, 0xf8e, 0 },
>> +	{ 0x1001, 0x1001, 0x1001, 0 },
>> +	{ 0x1075, 0x1075, 0x1075, 0 },
>> +	{ 0x10ec, 0x10ec, 0x10ec, 0 },
>> +	{ 0x1164, 0x1164, 0x1164, 0 },
>> +	{ 0x11de, 0x11de, 0x11de, 0 },
>> +	{ 0x125a, 0x125a, 0x125a, 0 },
>> +	{ 0x12d9, 0x12d9, 0x12d9, 0 },
>> +	{ 0x1359, 0x1359, 0x1359, 0 },
>> +	{ 0x13db, 0x13db, 0x13db, 0 },
>> +	{ 0x145f, 0x145f, 0x145f, 0 },
>> +	{ 0x14e5, 0x14e5, 0x14e5, 0 },
>> +	{ 0x156d, 0x156d, 0x156d, 0 },
>> +	{ 0x15f7, 0x15f7, 0x15f7, 0 },
>> +	{ 0x1683, 0x1683, 0x1683, 0 },
>> +	{ 0x1711, 0x1711, 0x1711, 0 },
>> +	{ 0x17a1, 0x17a1, 0x17a1, 0 },
>> +	{ 0x1833, 0x1833, 0x1833, 0 },
>> +	{ 0x18c7, 0x18c7, 0x18c7, 0 },
>> +	{ 0x195d, 0x195d, 0x195d, 0 },
>> +	{ 0x19f6, 0x19f6, 0x19f6, 0 },
>> +	{ 0x1a90, 0x1a90, 0x1a90, 0 },
>> +	{ 0x1b2c, 0x1b2c, 0x1b2c, 0 },
>> +	{ 0x1bcb, 0x1bcb, 0x1bcb, 0 },
>> +	{ 0x1c6b, 0x1c6b, 0x1c6b, 0 },
>> +	{ 0x1d0e, 0x1d0e, 0x1d0e, 0 },
>> +	{ 0x1db3, 0x1db3, 0x1db3, 0 },
>> +	{ 0x1e59, 0x1e59, 0x1e59, 0 },
>> +	{ 0x1f02, 0x1f02, 0x1f02, 0 },
>> +	{ 0x1fad, 0x1fad, 0x1fad, 0 },
>> +	{ 0x205b, 0x205b, 0x205b, 0 },
>> +	{ 0x210a, 0x210a, 0x210a, 0 },
>> +	{ 0x21bb, 0x21bb, 0x21bb, 0 },
>> +	{ 0x226f, 0x226f, 0x226f, 0 },
>> +	{ 0x2325, 0x2325, 0x2325, 0 },
>> +	{ 0x23dd, 0x23dd, 0x23dd, 0 },
>> +	{ 0x2497, 0x2497, 0x2497, 0 },
>> +	{ 0x2553, 0x2553, 0x2553, 0 },
>> +	{ 0x2612, 0x2612, 0x2612, 0 },
>> +	{ 0x26d2, 0x26d2, 0x26d2, 0 },
>> +	{ 0x2795, 0x2795, 0x2795, 0 },
>> +	{ 0x285a, 0x285a, 0x285a, 0 },
>> +	{ 0x2922, 0x2922, 0x2922, 0 },
>> +	{ 0x29eb, 0x29eb, 0x29eb, 0 },
>> +	{ 0x2ab7, 0x2ab7, 0x2ab7, 0 },
>> +	{ 0x2b85, 0x2b85, 0x2b85, 0 },
>> +	{ 0x2c56, 0x2c56, 0x2c56, 0 },
>> +	{ 0x2d28, 0x2d28, 0x2d28, 0 },
>> +	{ 0x2dfd, 0x2dfd, 0x2dfd, 0 },
>> +	{ 0x2ed4, 0x2ed4, 0x2ed4, 0 },
>> +	{ 0x2fad, 0x2fad, 0x2fad, 0 },
>> +	{ 0x3089, 0x3089, 0x3089, 0 },
>> +	{ 0x3167, 0x3167, 0x3167, 0 },
>> +	{ 0x3247, 0x3247, 0x3247, 0 },
>> +	{ 0x332a, 0x332a, 0x332a, 0 },
>> +	{ 0x340e, 0x340e, 0x340e, 0 },
>> +	{ 0x34f5, 0x34f5, 0x34f5, 0 },
>> +	{ 0x35df, 0x35df, 0x35df, 0 },
>> +	{ 0x36cb, 0x36cb, 0x36cb, 0 },
>> +	{ 0x37b9, 0x37b9, 0x37b9, 0 },
>> +	{ 0x38a9, 0x38a9, 0x38a9, 0 },
>> +	{ 0x399c, 0x399c, 0x399c, 0 },
>> +	{ 0x3a91, 0x3a91, 0x3a91, 0 },
>> +	{ 0x3b89, 0x3b89, 0x3b89, 0 },
>> +	{ 0x3c83, 0x3c83, 0x3c83, 0 },
>> +	{ 0x3d7f, 0x3d7f, 0x3d7f, 0 },
>> +	{ 0x3e7e, 0x3e7e, 0x3e7e, 0 },
>> +	{ 0x3f7f, 0x3f7f, 0x3f7f, 0 },
>> +	{ 0x4082, 0x4082, 0x4082, 0 },
>> +	{ 0x4188, 0x4188, 0x4188, 0 },
>> +	{ 0x4290, 0x4290, 0x4290, 0 },
>> +	{ 0x439b, 0x439b, 0x439b, 0 },
>> +	{ 0x44a8, 0x44a8, 0x44a8, 0 },
>> +	{ 0x45b7, 0x45b7, 0x45b7, 0 },
>> +	{ 0x46c9, 0x46c9, 0x46c9, 0 },
>> +	{ 0x47dd, 0x47dd, 0x47dd, 0 },
>> +	{ 0x48f4, 0x48f4, 0x48f4, 0 },
>> +	{ 0x4a0d, 0x4a0d, 0x4a0d, 0 },
>> +	{ 0x4b29, 0x4b29, 0x4b29, 0 },
>> +	{ 0x4c47, 0x4c47, 0x4c47, 0 },
>> +	{ 0x4d68, 0x4d68, 0x4d68, 0 },
>> +	{ 0x4e8b, 0x4e8b, 0x4e8b, 0 },
>> +	{ 0x4fb1, 0x4fb1, 0x4fb1, 0 },
>> +	{ 0x50d9, 0x50d9, 0x50d9, 0 },
>> +	{ 0x5203, 0x5203, 0x5203, 0 },
>> +	{ 0x5330, 0x5330, 0x5330, 0 },
>> +	{ 0x5460, 0x5460, 0x5460, 0 },
>> +	{ 0x5592, 0x5592, 0x5592, 0 },
>> +	{ 0x56c6, 0x56c6, 0x56c6, 0 },
>> +	{ 0x57fd, 0x57fd, 0x57fd, 0 },
>> +	{ 0x5937, 0x5937, 0x5937, 0 },
>> +	{ 0x5a73, 0x5a73, 0x5a73, 0 },
>> +	{ 0x5bb2, 0x5bb2, 0x5bb2, 0 },
>> +	{ 0x5cf3, 0x5cf3, 0x5cf3, 0 },
>> +	{ 0x5e37, 0x5e37, 0x5e37, 0 },
>> +	{ 0x5f7d, 0x5f7d, 0x5f7d, 0 },
>> +	{ 0x60c6, 0x60c6, 0x60c6, 0 },
>> +	{ 0x6212, 0x6212, 0x6212, 0 },
>> +	{ 0x6360, 0x6360, 0x6360, 0 },
>> +	{ 0x64b0, 0x64b0, 0x64b0, 0 },
>> +	{ 0x6604, 0x6604, 0x6604, 0 },
>> +	{ 0x6759, 0x6759, 0x6759, 0 },
>> +	{ 0x68b2, 0x68b2, 0x68b2, 0 },
>> +	{ 0x6a0d, 0x6a0d, 0x6a0d, 0 },
>> +	{ 0x6b6a, 0x6b6a, 0x6b6a, 0 },
>> +	{ 0x6ccb, 0x6ccb, 0x6ccb, 0 },
>> +	{ 0x6e2d, 0x6e2d, 0x6e2d, 0 },
>> +	{ 0x6f93, 0x6f93, 0x6f93, 0 },
>> +	{ 0x70fb, 0x70fb, 0x70fb, 0 },
>> +	{ 0x7266, 0x7266, 0x7266, 0 },
>> +	{ 0x73d3, 0x73d3, 0x73d3, 0 },
>> +	{ 0x7543, 0x7543, 0x7543, 0 },
>> +	{ 0x76b6, 0x76b6, 0x76b6, 0 },
>> +	{ 0x782b, 0x782b, 0x782b, 0 },
>> +	{ 0x79a3, 0x79a3, 0x79a3, 0 },
>> +	{ 0x7b1d, 0x7b1d, 0x7b1d, 0 },
>> +	{ 0x7c9b, 0x7c9b, 0x7c9b, 0 },
>> +	{ 0x7e1b, 0x7e1b, 0x7e1b, 0 },
>> +	{ 0x7f9d, 0x7f9d, 0x7f9d, 0 },
>> +	{ 0x8123, 0x8123, 0x8123, 0 },
>> +	{ 0x82ab, 0x82ab, 0x82ab, 0 },
>> +	{ 0x8436, 0x8436, 0x8436, 0 },
>> +	{ 0x85c3, 0x85c3, 0x85c3, 0 },
>> +	{ 0x8753, 0x8753, 0x8753, 0 },
>> +	{ 0x88e6, 0x88e6, 0x88e6, 0 },
>> +	{ 0x8a7c, 0x8a7c, 0x8a7c, 0 },
>> +	{ 0x8c14, 0x8c14, 0x8c14, 0 },
>> +	{ 0x8daf, 0x8daf, 0x8daf, 0 },
>> +	{ 0x8f4d, 0x8f4d, 0x8f4d, 0 },
>> +	{ 0x90ed, 0x90ed, 0x90ed, 0 },
>> +	{ 0x9290, 0x9290, 0x9290, 0 },
>> +	{ 0x9436, 0x9436, 0x9436, 0 },
>> +	{ 0x95df, 0x95df, 0x95df, 0 },
>> +	{ 0x978b, 0x978b, 0x978b, 0 },
>> +	{ 0x9939, 0x9939, 0x9939, 0 },
>> +	{ 0x9aea, 0x9aea, 0x9aea, 0 },
>> +	{ 0x9c9e, 0x9c9e, 0x9c9e, 0 },
>> +	{ 0x9e55, 0x9e55, 0x9e55, 0 },
>> +	{ 0xa00e, 0xa00e, 0xa00e, 0 },
>> +	{ 0xa1ca, 0xa1ca, 0xa1ca, 0 },
>> +	{ 0xa389, 0xa389, 0xa389, 0 },
>> +	{ 0xa54b, 0xa54b, 0xa54b, 0 },
>> +	{ 0xa710, 0xa710, 0xa710, 0 },
>> +	{ 0xa8d7, 0xa8d7, 0xa8d7, 0 },
>> +	{ 0xaaa1, 0xaaa1, 0xaaa1, 0 },
>> +	{ 0xac6e, 0xac6e, 0xac6e, 0 },
>> +	{ 0xae3e, 0xae3e, 0xae3e, 0 },
>> +	{ 0xb011, 0xb011, 0xb011, 0 },
>> +	{ 0xb1e7, 0xb1e7, 0xb1e7, 0 },
>> +	{ 0xb3bf, 0xb3bf, 0xb3bf, 0 },
>> +	{ 0xb59a, 0xb59a, 0xb59a, 0 },
>> +	{ 0xb778, 0xb778, 0xb778, 0 },
>> +	{ 0xb959, 0xb959, 0xb959, 0 },
>> +	{ 0xbb3d, 0xbb3d, 0xbb3d, 0 },
>> +	{ 0xbd24, 0xbd24, 0xbd24, 0 },
>> +	{ 0xbf0d, 0xbf0d, 0xbf0d, 0 },
>> +	{ 0xc0fa, 0xc0fa, 0xc0fa, 0 },
>> +	{ 0xc2e9, 0xc2e9, 0xc2e9, 0 },
>> +	{ 0xc4db, 0xc4db, 0xc4db, 0 },
>> +	{ 0xc6d0, 0xc6d0, 0xc6d0, 0 },
>> +	{ 0xc8c8, 0xc8c8, 0xc8c8, 0 },
>> +	{ 0xcac3, 0xcac3, 0xcac3, 0 },
>> +	{ 0xccc1, 0xccc1, 0xccc1, 0 },
>> +	{ 0xcec1, 0xcec1, 0xcec1, 0 },
>> +	{ 0xd0c5, 0xd0c5, 0xd0c5, 0 },
>> +	{ 0xd2cc, 0xd2cc, 0xd2cc, 0 },
>> +	{ 0xd4d5, 0xd4d5, 0xd4d5, 0 },
>> +	{ 0xd6e1, 0xd6e1, 0xd6e1, 0 },
>> +	{ 0xd8f1, 0xd8f1, 0xd8f1, 0 },
>> +	{ 0xdb03, 0xdb03, 0xdb03, 0 },
>> +	{ 0xdd18, 0xdd18, 0xdd18, 0 },
>> +	{ 0xdf30, 0xdf30, 0xdf30, 0 },
>> +	{ 0xe14b, 0xe14b, 0xe14b, 0 },
>> +	{ 0xe369, 0xe369, 0xe369, 0 },
>> +	{ 0xe58a, 0xe58a, 0xe58a, 0 },
>> +	{ 0xe7ae, 0xe7ae, 0xe7ae, 0 },
>> +	{ 0xe9d5, 0xe9d5, 0xe9d5, 0 },
>> +	{ 0xebff, 0xebff, 0xebff, 0 },
>> +	{ 0xee2c, 0xee2c, 0xee2c, 0 },
>> +	{ 0xf05c, 0xf05c, 0xf05c, 0 },
>> +	{ 0xf28f, 0xf28f, 0xf28f, 0 },
>> +	{ 0xf4c4, 0xf4c4, 0xf4c4, 0 },
>> +	{ 0xf6fd, 0xf6fd, 0xf6fd, 0 },
>> +	{ 0xf939, 0xf939, 0xf939, 0 },
>> +	{ 0xfb78, 0xfb78, 0xfb78, 0 },
>> +	{ 0xfdba, 0xfdba, 0xfdba, 0 },
>> +	{ 0xffff, 0xffff, 0xffff, 0 }
>> +};
>> +
>> +#if 0
>> +struct vkms_color_lut srgb_eotf = {
>> +	.base = NULL,
>> +	. lut_length = LUT_SIZE,
>> +	.channel_value2index_ratio = drm_int2fixp(0xffff)
>> +	// .channel_value2index_ratio = 0 //drm_fixp_div(drm_int2fixp(0xffff), drm_int2fixp(LUT_SIZE))
>> +};
>> +
>> +#else
>> +const struct vkms_color_lut srgb_eotf = {
>> +	.base = srgb_array,
>> +	.lut_length = 256,
>> +	.channel_value2index_ratio = 16711935ll
>> +};
>> +
>> +#endif
>> +
>>  static u16 pre_mul_blend_channel(u16 src, u16 dst, u16 alpha)
>>  {
>>  	u32 new_color;
>> @@ -136,6 +414,39 @@ static void apply_lut(const struct vkms_crtc_state *crtc_state, struct line_buff
>>  	}
>>  }
>>  
>> +static void pre_blend_color_transform(const struct vkms_plane_state *plane_state, struct line_buffer *output_buffer)
>> +{
>> +	struct drm_colorop *pipeline = plane_state->base.base.color_pipeline;
>> +	/* TODO this is probably wrong */
>> +	struct drm_colorop_state *colorop_state;
>> +
>> +	if (!pipeline)
>> +		return;
>> +
>> +	colorop_state = pipeline->state;
>> +
>> +	if (!colorop_state)
>> +		return;
>> +
>> +	for (size_t x = 0; x < output_buffer->n_pixels; x++) {
>> +		struct pixel_argb_u16 *pixel = &output_buffer->pixels[x];
>> +
>> +		if (pipeline->type == DRM_COLOROP_1D_CURVE &&
>> +			colorop_state->bypass == false) {
>> +			switch (colorop_state->curve_1d_type) {
>> +				case DRM_COLOROP_1D_CURVE_SRGB_INV_EOTF:
>> +					break;
>> +				case DRM_COLOROP_1D_CURVE_SRGB_EOTF:
>> +				default:
>> +					pixel->r = apply_lut_to_channel_value(&srgb_eotf, pixel->r, LUT_RED);
>> +					pixel->g = apply_lut_to_channel_value(&srgb_eotf, pixel->g, LUT_GREEN);
>> +					pixel->b = apply_lut_to_channel_value(&srgb_eotf, pixel->b, LUT_BLUE);
>> +					break;
>> +			}
>> +		}
>> +	}
>> +}
>> +
>>  /**
>>   * @wb_frame_info: The writeback frame buffer metadata
>>   * @crtc_state: The crtc state
>> @@ -168,8 +479,13 @@ static void blend(struct vkms_writeback_job *wb,
>>  				continue;
>>  
>>  			plane[i]->plane_read(stage_buffer, plane[i]->frame_info, y);
>> +
>> +			/* do per-plane color transformations here */
>> +			// pre_blend_color_transform(plane[i], stage_buffer);
>> +
>>  			pre_mul_alpha_blend(plane[i]->frame_info, stage_buffer,
>>  					    output_buffer);
>> +			pre_blend_color_transform(plane[i], output_buffer);
> 
> I see it should be applied to the `stage_buffer` before blending (i.e.
> pre_mul_alpha_blend()) and in the lines commented above. Were you
> getting any unexpected result?
> 

v2 fixes this. You're absolutely right. I did see some issues and was
experimenting with moving things around, but it needs to be before the
blend call.

> Otherwise, having this VKMS implementation looks very nice. Thank you!
> 

Thanks :)
Harry

> Melissa
> 
>>  		}
>>  
>>  		apply_lut(crtc_state, output_buffer);
>> diff --git a/drivers/gpu/drm/vkms/vkms_drv.h b/drivers/gpu/drm/vkms/vkms_drv.h
>> index 310b31f47928..c04f714cd486 100644
>> --- a/drivers/gpu/drm/vkms/vkms_drv.h
>> +++ b/drivers/gpu/drm/vkms/vkms_drv.h
>> @@ -168,4 +168,8 @@ void vkms_set_composer(struct vkms_output *out, bool enabled);
>>  /* Writeback */
>>  int vkms_enable_writeback_connector(struct vkms_device *vkmsdev);
>>  
>> +/* Colorops */
>> +int vkms_initialize_colorops(struct drm_plane *plane);
>> +
>> +
>>  #endif /* _VKMS_DRV_H_ */
>> diff --git a/drivers/gpu/drm/vkms/vkms_plane.c b/drivers/gpu/drm/vkms/vkms_plane.c
>> index b3f8a115cc23..cbffbdd7cbf9 100644
>> --- a/drivers/gpu/drm/vkms/vkms_plane.c
>> +++ b/drivers/gpu/drm/vkms/vkms_plane.c
>> @@ -237,5 +237,7 @@ struct vkms_plane *vkms_plane_init(struct vkms_device *vkmsdev,
>>  
>>  	drm_plane_helper_add(&plane->base, funcs);
>>  
>> +	vkms_initialize_colorops(&plane->base);
>> +
>>  	return plane;
>>  }
>> -- 
>> 2.42.0
>>

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 3933 bytes --]

On Thu, 19 Oct 2023 10:56:29 -0400
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-09-13 07:29, Pekka Paalanen wrote:
> > On Fri, 8 Sep 2023 11:02:26 -0400
> > Harry Wentland <harry.wentland@amd.com> wrote:
> >   
> >> Signed-off-by: Harry Wentland <harry.wentland@amd.com>

...

> >> +COLOR_PIPELINE Plane Property
> >> +=============================
> >> +
> >> +Because we don't have existing KMS color properties in the pre-blending
> >> +portion of display pipelines (i.e. on drm_planes) we are introducing
> >> +color pipelines here first. Eventually we'll want to use the same
> >> +concept for the post-blending portion, i.e. drm_crtcs.  
> > 
> > This paragraph might fit better in a cover letter.
> >   
> >> +
> >> +Color Pipelines are created by a driver and advertised via a new
> >> +COLOR_PIPELINE enum property on each plane. Values of the property
> >> +always include '0', which is the default and means all color processing
> >> +is disabled. Additional values will be the object IDs of the first
> >> +drm_colorop in a pipeline. A driver can create and advertise none, one,
> >> +or more possible color pipelines. A DRM client will select a color
> >> +pipeline by setting the COLOR PIPELINE to the respective value.
> >> +
> >> +In the case where drivers have custom support for pre-blending color
> >> +processing those drivers shall reject atomic commits that are trying to
> >> +set both the custom color properties, as well as the COLOR_PIPELINE  
> > 
> > s/set/use/ because one of them could be carried-over state from
> > previous commits while not literally set in this one.
> >   
> >> +property.
> >> +
> >> +An example of a COLOR_PIPELINE property on a plane might look like this::
> >> +
> >> +    Plane 10
> >> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> >> +    ├─ …
> >> +    └─ "color_pipeline": enum {0, 42, 52} = 0  
> > 
> > Enum values are string names. I presume the intention here is that the
> > strings will never need to be parsed, and the uint64_t is always equal
> > to the string representation, right?
> > 
> > That needs a statement here. It differs from all previous uses of
> > enums, and e.g. requires a little bit of new API in libweston's
> > DRM-backend to handle since it has its own enums referring to the
> > string names that get mapped to the uint64_t per owning KMS object.
> >   
> 
> I'm currently putting the DRM object ID in the "value" and use the
> "name" as a descriptive name.

Would that string name be UAPI? I mean, if userspace hardcodes and
looks for that name, will that keep working? If it's descriptive then I
would assume not, but for every enum existing so far the string name is
UAPI.

> > struct drm_mode_property_enum {
> > 	__u64 value;
> > 	char name[DRM_PROP_NAME_LEN];
> > };  
> 
> This works well in IGT and gives us a nice descriptive name for
> debugging, but I could consider changing this if it'd simplify
> people's lives.

It's nice if we can have a description string for each pipeline, but
according to KMS UAPI conventions so far, userspace would look for the
string name. I'm worried that could backfire to the kernel.

Or, maybe we do want to make the string UAPI and accept that some
userspace might look for it rather than an arrangement of colorops?

Matching colorop sequences is "hard". A developer looking at pipelines,
picking one, and hardcoding its name as a preferred choice would be too
easy. "Works for my cards." IOW, if there is a useful looking string
name, we can be sure that someone will depend on it.


Thanks,
pq

> >> +References
> >> +==========
> >> +
> >> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> >> \ No newline at end of file  
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 2751 bytes --]

On Thu, 19 Oct 2023 10:56:40 -0400
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-10-10 12:13, Melissa Wen wrote:
> > O 09/08, Harry Wentland wrote:  
> >> Signed-off-by: Harry Wentland <harry.wentland@amd.com>

...

> > Also, with this new plane API in place, I understand that we will
> > already need think on how to deal with the mixing between old drm color
> > properties (color encoding and color range) and these new way of setting
> > plane color properties. IIUC, Pekka asked a related question about it
> > when talking about CRTC automatic RGB->YUV (?) 
> >   
> 
> We'll still need to confirm whether we'll want to deprecate these
> existing properties. If we do that we'd want a client prop. Things
> should still work without deprecating them, if drivers just pick up
> after the initial encoding and range CSC.
> 
> But realistically it might be better to deprecate them and turn them
> into explicit colorops.

The existing properties would need to be explicitly reflected in the
new pipelines anyway, otherwise there would always be doubt at which
point of a pipeline the old properties apply, and they might even
need to change positions between pipelines.

I think it is simply easier to just hide all old color related
properties when userspace sets the client-cap to enable pipelines. The
problem is to make sure to hide all old properties on all drivers that
support the client-cap.

As a pipeline must be complete (describe everything that happens to
pixel values), it's going to be a flag day per driver.

Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
as well. Maybe it's purely informative and non-configurable, keyed by
FB pixel format, but still.

We also need a colorop to represent sample filtering, e.g. bilinear,
like I think Sebastian may have mentioned in the past. Everything
before the sample filter happens "per tap" as Joshua Ashton put it, and
everything after it happens on the sample that was computed as a
weighted average of the filter tap inputs (texels).

There could be colorops other than sample filtering that operate on
more than one sample at a time, like blur or sharpness. There could
even be colorops that change the image size like adding padding that
the following colorop hardware requires, and then yet another colorop
that clips that padding away. For an example, see
https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html

If that padding and its color can affect the pipeline results of the
pixels near the padding (e.g. some convolution is applied with them,
which may be the reason why padding is necessary to begin with), then
it would be best to model it.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-10-20 06:36, Pekka Paalanen wrote:
> On Thu, 19 Oct 2023 10:56:40 -0400
> Harry Wentland <harry.wentland@amd.com> wrote:
> 
>> On 2023-10-10 12:13, Melissa Wen wrote:
>>> O 09/08, Harry Wentland wrote:  
>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> 
> ...
> 
>>> Also, with this new plane API in place, I understand that we will
>>> already need think on how to deal with the mixing between old drm color
>>> properties (color encoding and color range) and these new way of setting
>>> plane color properties. IIUC, Pekka asked a related question about it
>>> when talking about CRTC automatic RGB->YUV (?) 
>>>   
>>
>> We'll still need to confirm whether we'll want to deprecate these
>> existing properties. If we do that we'd want a client prop. Things
>> should still work without deprecating them, if drivers just pick up
>> after the initial encoding and range CSC.
>>
>> But realistically it might be better to deprecate them and turn them
>> into explicit colorops.
> 
> The existing properties would need to be explicitly reflected in the
> new pipelines anyway, otherwise there would always be doubt at which
> point of a pipeline the old properties apply, and they might even
> need to change positions between pipelines.
> 
> I think it is simply easier to just hide all old color related
> properties when userspace sets the client-cap to enable pipelines. The
> problem is to make sure to hide all old properties on all drivers that
> support the client-cap.
> 
> As a pipeline must be complete (describe everything that happens to
> pixel values), it's going to be a flag day per driver.
> 
> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
> as well. Maybe it's purely informative and non-configurable, keyed by
> FB pixel format, but still.
> 
> We also need a colorop to represent sample filtering, e.g. bilinear,
> like I think Sebastian may have mentioned in the past. Everything
> before the sample filter happens "per tap" as Joshua Ashton put it, and
> everything after it happens on the sample that was computed as a
> weighted average of the filter tap inputs (texels).
> 
> There could be colorops other than sample filtering that operate on
> more than one sample at a time, like blur or sharpness. There could
> even be colorops that change the image size like adding padding that
> the following colorop hardware requires, and then yet another colorop
> that clips that padding away. For an example, see
> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
> 
> If that padding and its color can affect the pipeline results of the
> pixels near the padding (e.g. some convolution is applied with them,
> which may be the reason why padding is necessary to begin with), then
> it would be best to model it.
> 

I hear you but I'm also somewhat shying away from defining this at this point.
There are already too many things that need to happen and I will focus on the
actual color blocks (LUTs, matrices) first. We'll always be able to add a new
(read-only) colorop type to define scaling and tap behavior at any point and
a client is free to ignore a color pipeline if it doesn't find any tap/scale
info in it.

Harry

> 
> Thanks,
> pq

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-10-20 06:17, Pekka Paalanen wrote:
> On Thu, 19 Oct 2023 10:56:29 -0400
> Harry Wentland <harry.wentland@amd.com> wrote:
> 
>> On 2023-09-13 07:29, Pekka Paalanen wrote:
>>> On Fri, 8 Sep 2023 11:02:26 -0400
>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>   
>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> 
> ...
> 
>>>> +COLOR_PIPELINE Plane Property
>>>> +=============================
>>>> +
>>>> +Because we don't have existing KMS color properties in the pre-blending
>>>> +portion of display pipelines (i.e. on drm_planes) we are introducing
>>>> +color pipelines here first. Eventually we'll want to use the same
>>>> +concept for the post-blending portion, i.e. drm_crtcs.  
>>>
>>> This paragraph might fit better in a cover letter.
>>>   
>>>> +
>>>> +Color Pipelines are created by a driver and advertised via a new
>>>> +COLOR_PIPELINE enum property on each plane. Values of the property
>>>> +always include '0', which is the default and means all color processing
>>>> +is disabled. Additional values will be the object IDs of the first
>>>> +drm_colorop in a pipeline. A driver can create and advertise none, one,
>>>> +or more possible color pipelines. A DRM client will select a color
>>>> +pipeline by setting the COLOR PIPELINE to the respective value.
>>>> +
>>>> +In the case where drivers have custom support for pre-blending color
>>>> +processing those drivers shall reject atomic commits that are trying to
>>>> +set both the custom color properties, as well as the COLOR_PIPELINE  
>>>
>>> s/set/use/ because one of them could be carried-over state from
>>> previous commits while not literally set in this one.
>>>   
>>>> +property.
>>>> +
>>>> +An example of a COLOR_PIPELINE property on a plane might look like this::
>>>> +
>>>> +    Plane 10
>>>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>>>> +    ├─ …
>>>> +    └─ "color_pipeline": enum {0, 42, 52} = 0  
>>>
>>> Enum values are string names. I presume the intention here is that the
>>> strings will never need to be parsed, and the uint64_t is always equal
>>> to the string representation, right?
>>>
>>> That needs a statement here. It differs from all previous uses of
>>> enums, and e.g. requires a little bit of new API in libweston's
>>> DRM-backend to handle since it has its own enums referring to the
>>> string names that get mapped to the uint64_t per owning KMS object.
>>>   
>>
>> I'm currently putting the DRM object ID in the "value" and use the
>> "name" as a descriptive name.
> 
> Would that string name be UAPI? I mean, if userspace hardcodes and
> looks for that name, will that keep working? If it's descriptive then I
> would assume not, but for every enum existing so far the string name is
> UAPI.
> 

Yes, it's UAPI, as that's how userspace will set the property. The value
is still important to be able to find out which is the first colorop in
the pipeline.

Harry

>>> struct drm_mode_property_enum {
>>> 	__u64 value;
>>> 	char name[DRM_PROP_NAME_LEN];
>>> };  
>>
>> This works well in IGT and gives us a nice descriptive name for
>> debugging, but I could consider changing this if it'd simplify
>> people's lives.
> 
> It's nice if we can have a description string for each pipeline, but
> according to KMS UAPI conventions so far, userspace would look for the
> string name. I'm worried that could backfire to the kernel.
> 
> Or, maybe we do want to make the string UAPI and accept that some
> userspace might look for it rather than an arrangement of colorops?
> 
> Matching colorop sequences is "hard". A developer looking at pipelines,
> picking one, and hardcoding its name as a preferred choice would be too
> easy. "Works for my cards." IOW, if there is a useful looking string
> name, we can be sure that someone will depend on it.
> 
> 
> Thanks,
> pq
> 
>>>> +References
>>>> +==========
>>>> +
>>>> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>>>> \ No newline at end of file  
>>>   
>>
> 

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 5533 bytes --]

On Mon, 6 Nov 2023 11:24:50 -0500
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-10-20 06:17, Pekka Paalanen wrote:
> > On Thu, 19 Oct 2023 10:56:29 -0400
> > Harry Wentland <harry.wentland@amd.com> wrote:
> >   
> >> On 2023-09-13 07:29, Pekka Paalanen wrote:  
> >>> On Fri, 8 Sep 2023 11:02:26 -0400
> >>> Harry Wentland <harry.wentland@amd.com> wrote:
> >>>     
> >>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>  
> > 
> > ...
> >   
> >>>> +COLOR_PIPELINE Plane Property
> >>>> +=============================
> >>>> +
> >>>> +Because we don't have existing KMS color properties in the pre-blending
> >>>> +portion of display pipelines (i.e. on drm_planes) we are introducing
> >>>> +color pipelines here first. Eventually we'll want to use the same
> >>>> +concept for the post-blending portion, i.e. drm_crtcs.    
> >>>
> >>> This paragraph might fit better in a cover letter.
> >>>     
> >>>> +
> >>>> +Color Pipelines are created by a driver and advertised via a new
> >>>> +COLOR_PIPELINE enum property on each plane. Values of the property
> >>>> +always include '0', which is the default and means all color processing
> >>>> +is disabled. Additional values will be the object IDs of the first
> >>>> +drm_colorop in a pipeline. A driver can create and advertise none, one,
> >>>> +or more possible color pipelines. A DRM client will select a color
> >>>> +pipeline by setting the COLOR PIPELINE to the respective value.
> >>>> +
> >>>> +In the case where drivers have custom support for pre-blending color
> >>>> +processing those drivers shall reject atomic commits that are trying to
> >>>> +set both the custom color properties, as well as the COLOR_PIPELINE    
> >>>
> >>> s/set/use/ because one of them could be carried-over state from
> >>> previous commits while not literally set in this one.
> >>>     
> >>>> +property.
> >>>> +
> >>>> +An example of a COLOR_PIPELINE property on a plane might look like this::
> >>>> +
> >>>> +    Plane 10
> >>>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
> >>>> +    ├─ …
> >>>> +    └─ "color_pipeline": enum {0, 42, 52} = 0    
> >>>
> >>> Enum values are string names. I presume the intention here is that the
> >>> strings will never need to be parsed, and the uint64_t is always equal
> >>> to the string representation, right?
> >>>
> >>> That needs a statement here. It differs from all previous uses of
> >>> enums, and e.g. requires a little bit of new API in libweston's
> >>> DRM-backend to handle since it has its own enums referring to the
> >>> string names that get mapped to the uint64_t per owning KMS object.
> >>>     
> >>
> >> I'm currently putting the DRM object ID in the "value" and use the
> >> "name" as a descriptive name.  
> > 
> > Would that string name be UAPI? I mean, if userspace hardcodes and
> > looks for that name, will that keep working? If it's descriptive then I
> > would assume not, but for every enum existing so far the string name is
> > UAPI.
> >   
> 
> Yes, it's UAPI, as that's how userspace will set the property. The value
> is still important to be able to find out which is the first colorop in
> the pipeline.

Userspace will hardcode string names, look up the KMS uint64_t
corresponding to it, and then use the uint64_t to program KMS.

But for color pipeline objects, the initial idea was that we expect
userspace to look through all available pipelines and see if any of
them can express what userspace wants. This does not need the string
name to be UAPI per se.

Of course, it is easier if userspace can be hardcoded for a specific
color pipeline, so all that matching and searching is avoided, but as a
driver developer, do you want that?

Or maybe the practical end result is the same regardless, because if a
driver removes a pipeline on specific hardware and userspace cannot
find another, that would be a kernel regression anyway.

Then again, if userspace doesn't do the matching and searching, it'll
likely struggle to work on different hardware. Driver developers would
get requests to expose this and that specific pipeline. Is that an ok
prospect?


Thanks,
pq


> >>> struct drm_mode_property_enum {
> >>> 	__u64 value;
> >>> 	char name[DRM_PROP_NAME_LEN];
> >>> };    
> >>
> >> This works well in IGT and gives us a nice descriptive name for
> >> debugging, but I could consider changing this if it'd simplify
> >> people's lives.  
> > 
> > It's nice if we can have a description string for each pipeline, but
> > according to KMS UAPI conventions so far, userspace would look for the
> > string name. I'm worried that could backfire to the kernel.
> > 
> > Or, maybe we do want to make the string UAPI and accept that some
> > userspace might look for it rather than an arrangement of colorops?
> > 
> > Matching colorop sequences is "hard". A developer looking at pipelines,
> > picking one, and hardcoding its name as a preferred choice would be too
> > easy. "Works for my cards." IOW, if there is a useful looking string
> > name, we can be sure that someone will depend on it.
> > 
> > 
> > Thanks,
> > pq
> >   
> >>>> +References
> >>>> +==========
> >>>> +
> >>>> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
> >>>> \ No newline at end of file    
> >>>     
> >>  
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 4801 bytes --]

On Mon, 6 Nov 2023 11:19:27 -0500
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-10-20 06:36, Pekka Paalanen wrote:
> > On Thu, 19 Oct 2023 10:56:40 -0400
> > Harry Wentland <harry.wentland@amd.com> wrote:
> >   
> >> On 2023-10-10 12:13, Melissa Wen wrote:  
> >>> O 09/08, Harry Wentland wrote:    
> >>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>  
> > 
> > ...
> >   
> >>> Also, with this new plane API in place, I understand that we will
> >>> already need think on how to deal with the mixing between old drm color
> >>> properties (color encoding and color range) and these new way of setting
> >>> plane color properties. IIUC, Pekka asked a related question about it
> >>> when talking about CRTC automatic RGB->YUV (?) 
> >>>     
> >>
> >> We'll still need to confirm whether we'll want to deprecate these
> >> existing properties. If we do that we'd want a client prop. Things
> >> should still work without deprecating them, if drivers just pick up
> >> after the initial encoding and range CSC.
> >>
> >> But realistically it might be better to deprecate them and turn them
> >> into explicit colorops.  
> > 
> > The existing properties would need to be explicitly reflected in the
> > new pipelines anyway, otherwise there would always be doubt at which
> > point of a pipeline the old properties apply, and they might even
> > need to change positions between pipelines.
> > 
> > I think it is simply easier to just hide all old color related
> > properties when userspace sets the client-cap to enable pipelines. The
> > problem is to make sure to hide all old properties on all drivers that
> > support the client-cap.
> > 
> > As a pipeline must be complete (describe everything that happens to
> > pixel values), it's going to be a flag day per driver.
> > 
> > Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
> > as well. Maybe it's purely informative and non-configurable, keyed by
> > FB pixel format, but still.
> > 
> > We also need a colorop to represent sample filtering, e.g. bilinear,
> > like I think Sebastian may have mentioned in the past. Everything
> > before the sample filter happens "per tap" as Joshua Ashton put it, and
> > everything after it happens on the sample that was computed as a
> > weighted average of the filter tap inputs (texels).
> > 
> > There could be colorops other than sample filtering that operate on
> > more than one sample at a time, like blur or sharpness. There could
> > even be colorops that change the image size like adding padding that
> > the following colorop hardware requires, and then yet another colorop
> > that clips that padding away. For an example, see
> > https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
> > 
> > If that padding and its color can affect the pipeline results of the
> > pixels near the padding (e.g. some convolution is applied with them,
> > which may be the reason why padding is necessary to begin with), then
> > it would be best to model it.
> >   
> 
> I hear you but I'm also somewhat shying away from defining this at this point.

Would you define them before the new UAPI is released though?

I agree there is no need to have them in this patch series, but I think
we'd hit the below problems if the UAPI is released without them.

> There are already too many things that need to happen and I will focus on the
> actual color blocks (LUTs, matrices) first. We'll always be able to add a new
> (read-only) colorop type to define scaling and tap behavior at any point and
> a client is free to ignore a color pipeline if it doesn't find any tap/scale
> info in it.

How would userspace know to look for tap/scale info, if there is no
upstream definition even on paper?

And the opposite case, if someone writes userspace without tap/scale
colorops, and then drivers add those, and there is no pipeline without
them, because they always exist. Would that userspace disregard all
those pipelines because it does not understand tap/scale colorops,
leaving no usable pipelines? Would that not be kernel regressing
userspace?

If the kernel keeps on exposing pipelines without the colorops, it
fails the basic promise of the whole design: that all pixel value
affecting operations are at least listed if not controllable.

How will we avoid painting ourselves in a corner?

Maybe we need a colorop for "here be dragons" documented as having
unknown and unreliable effects, until driver authors are sure that
everything has been modelled in the pipeline and there are no unknowns?
Or a flag on the pipelines, if we can have that. Then we can at least
tell when the pipeline does not fulfil the basic promise.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-11-07 04:38, Pekka Paalanen wrote:
> On Mon, 6 Nov 2023 11:24:50 -0500
> Harry Wentland <harry.wentland@amd.com> wrote:
> 
>> On 2023-10-20 06:17, Pekka Paalanen wrote:
>>> On Thu, 19 Oct 2023 10:56:29 -0400
>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>   
>>>> On 2023-09-13 07:29, Pekka Paalanen wrote:  
>>>>> On Fri, 8 Sep 2023 11:02:26 -0400
>>>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>>>     
>>>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>  
>>>
>>> ...
>>>   
>>>>>> +COLOR_PIPELINE Plane Property
>>>>>> +=============================
>>>>>> +
>>>>>> +Because we don't have existing KMS color properties in the pre-blending
>>>>>> +portion of display pipelines (i.e. on drm_planes) we are introducing
>>>>>> +color pipelines here first. Eventually we'll want to use the same
>>>>>> +concept for the post-blending portion, i.e. drm_crtcs.    
>>>>>
>>>>> This paragraph might fit better in a cover letter.
>>>>>     
>>>>>> +
>>>>>> +Color Pipelines are created by a driver and advertised via a new
>>>>>> +COLOR_PIPELINE enum property on each plane. Values of the property
>>>>>> +always include '0', which is the default and means all color processing
>>>>>> +is disabled. Additional values will be the object IDs of the first
>>>>>> +drm_colorop in a pipeline. A driver can create and advertise none, one,
>>>>>> +or more possible color pipelines. A DRM client will select a color
>>>>>> +pipeline by setting the COLOR PIPELINE to the respective value.
>>>>>> +
>>>>>> +In the case where drivers have custom support for pre-blending color
>>>>>> +processing those drivers shall reject atomic commits that are trying to
>>>>>> +set both the custom color properties, as well as the COLOR_PIPELINE    
>>>>>
>>>>> s/set/use/ because one of them could be carried-over state from
>>>>> previous commits while not literally set in this one.
>>>>>     
>>>>>> +property.
>>>>>> +
>>>>>> +An example of a COLOR_PIPELINE property on a plane might look like this::
>>>>>> +
>>>>>> +    Plane 10
>>>>>> +    ├─ "type": immutable enum {Overlay, Primary, Cursor} = Primary
>>>>>> +    ├─ …
>>>>>> +    └─ "color_pipeline": enum {0, 42, 52} = 0    
>>>>>
>>>>> Enum values are string names. I presume the intention here is that the
>>>>> strings will never need to be parsed, and the uint64_t is always equal
>>>>> to the string representation, right?
>>>>>
>>>>> That needs a statement here. It differs from all previous uses of
>>>>> enums, and e.g. requires a little bit of new API in libweston's
>>>>> DRM-backend to handle since it has its own enums referring to the
>>>>> string names that get mapped to the uint64_t per owning KMS object.
>>>>>     
>>>>
>>>> I'm currently putting the DRM object ID in the "value" and use the
>>>> "name" as a descriptive name.  
>>>
>>> Would that string name be UAPI? I mean, if userspace hardcodes and
>>> looks for that name, will that keep working? If it's descriptive then I
>>> would assume not, but for every enum existing so far the string name is
>>> UAPI.
>>>   
>>
>> Yes, it's UAPI, as that's how userspace will set the property. The value
>> is still important to be able to find out which is the first colorop in
>> the pipeline.
> 
> Userspace will hardcode string names, look up the KMS uint64_t
> corresponding to it, and then use the uint64_t to program KMS.
> 
> But for color pipeline objects, the initial idea was that we expect
> userspace to look through all available pipelines and see if any of
> them can express what userspace wants. This does not need the string
> name to be UAPI per se.
> 
> Of course, it is easier if userspace can be hardcoded for a specific
> color pipeline, so all that matching and searching is avoided, but as a
> driver developer, do you want that?
> 
> Or maybe the practical end result is the same regardless, because if a
> driver removes a pipeline on specific hardware and userspace cannot
> find another, that would be a kernel regression anyway.
> 
> Then again, if userspace doesn't do the matching and searching, it'll
> likely struggle to work on different hardware. Driver developers would
> get requests to expose this and that specific pipeline. Is that an ok
> prospect?
> 

IMO in Linux land there will always be two types of DRM clients.

Ones that aim to work on all systems (within reasons) and use as
much of the DRM functionality in a driver-agnostic fashion. These
are things like Weston, Gnome, KDE, sway, etc., aka your "canonical
upstream project[s]" as mentioned in the "Userland Interfaces"
section of the "GPU Driver Developer's Guide".

Then there are people that build custom solutions for custom systems
with no desire or incentive to have them work in a driver-agnostic
fashion. These are often built on top of existing projects, such as
Weston, with customizations that are never designed to go upstream.

In the latter case I think it's entirely fair to hard-code the desired
color pipelines. In the former case I think that would be extremely
short-sighted and go against the driver-agnostic design philosophies
of these compositors.

As a driver developer I would like canonical upstream projects to have
an implementation that does not break with each new HW generation,
meaning these projects should parse the color pipelines.

Harry

> 
> Thanks,
> pq
> 
> 
>>>>> struct drm_mode_property_enum {
>>>>> 	__u64 value;
>>>>> 	char name[DRM_PROP_NAME_LEN];
>>>>> };    
>>>>
>>>> This works well in IGT and gives us a nice descriptive name for
>>>> debugging, but I could consider changing this if it'd simplify
>>>> people's lives.  
>>>
>>> It's nice if we can have a description string for each pipeline, but
>>> according to KMS UAPI conventions so far, userspace would look for the
>>> string name. I'm worried that could backfire to the kernel.
>>>
>>> Or, maybe we do want to make the string UAPI and accept that some
>>> userspace might look for it rather than an arrangement of colorops?
>>>
>>> Matching colorop sequences is "hard". A developer looking at pipelines,
>>> picking one, and hardcoding its name as a preferred choice would be too
>>> easy. "Works for my cards." IOW, if there is a useful looking string
>>> name, we can be sure that someone will depend on it.
>>>
>>>
>>> Thanks,
>>> pq
>>>   
>>>>>> +References
>>>>>> +==========
>>>>>> +
>>>>>> +1. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>>>>>> \ No newline at end of file    
>>>>>     
>>>>  
>>>   
>>
> 

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-11-07 04:55, Pekka Paalanen wrote:
> On Mon, 6 Nov 2023 11:19:27 -0500
> Harry Wentland <harry.wentland@amd.com> wrote:
> 
>> On 2023-10-20 06:36, Pekka Paalanen wrote:
>>> On Thu, 19 Oct 2023 10:56:40 -0400
>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>   
>>>> On 2023-10-10 12:13, Melissa Wen wrote:  
>>>>> O 09/08, Harry Wentland wrote:    
>>>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>  
>>>
>>> ...
>>>   
>>>>> Also, with this new plane API in place, I understand that we will
>>>>> already need think on how to deal with the mixing between old drm color
>>>>> properties (color encoding and color range) and these new way of setting
>>>>> plane color properties. IIUC, Pekka asked a related question about it
>>>>> when talking about CRTC automatic RGB->YUV (?) 
>>>>>     
>>>>
>>>> We'll still need to confirm whether we'll want to deprecate these
>>>> existing properties. If we do that we'd want a client prop. Things
>>>> should still work without deprecating them, if drivers just pick up
>>>> after the initial encoding and range CSC.
>>>>
>>>> But realistically it might be better to deprecate them and turn them
>>>> into explicit colorops.  
>>>
>>> The existing properties would need to be explicitly reflected in the
>>> new pipelines anyway, otherwise there would always be doubt at which
>>> point of a pipeline the old properties apply, and they might even
>>> need to change positions between pipelines.
>>>
>>> I think it is simply easier to just hide all old color related
>>> properties when userspace sets the client-cap to enable pipelines. The
>>> problem is to make sure to hide all old properties on all drivers that
>>> support the client-cap.
>>>
>>> As a pipeline must be complete (describe everything that happens to
>>> pixel values), it's going to be a flag day per driver.
>>>
>>> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
>>> as well. Maybe it's purely informative and non-configurable, keyed by
>>> FB pixel format, but still.
>>>
>>> We also need a colorop to represent sample filtering, e.g. bilinear,
>>> like I think Sebastian may have mentioned in the past. Everything
>>> before the sample filter happens "per tap" as Joshua Ashton put it, and
>>> everything after it happens on the sample that was computed as a
>>> weighted average of the filter tap inputs (texels).
>>>
>>> There could be colorops other than sample filtering that operate on
>>> more than one sample at a time, like blur or sharpness. There could
>>> even be colorops that change the image size like adding padding that
>>> the following colorop hardware requires, and then yet another colorop
>>> that clips that padding away. For an example, see
>>> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
>>>
>>> If that padding and its color can affect the pipeline results of the
>>> pixels near the padding (e.g. some convolution is applied with them,
>>> which may be the reason why padding is necessary to begin with), then
>>> it would be best to model it.
>>>   
>>
>> I hear you but I'm also somewhat shying away from defining this at this point.
> 
> Would you define them before the new UAPI is released though?
> 
> I agree there is no need to have them in this patch series, but I think
> we'd hit the below problems if the UAPI is released without them.
> 
>> There are already too many things that need to happen and I will focus on the
>> actual color blocks (LUTs, matrices) first. We'll always be able to add a new
>> (read-only) colorop type to define scaling and tap behavior at any point and
>> a client is free to ignore a color pipeline if it doesn't find any tap/scale
>> info in it.
> 
> How would userspace know to look for tap/scale info, if there is no
> upstream definition even on paper?
> 

So far OSes did not care about this. Whether that's good or bad is
something everyone can answer for themselves.

If you write a compositor and really need this you can just ignore
color pipelines that don't have this, i.e., you'll probably want
to wait with implementing color pipeline support until you have what
you need from DRM/KMS.

This is not to say I don't want to have support for Weston. But I'm
wondering if we place too much importance on getting every little
thing figured out whereas we could be making forward progress and
address more aspects of a color pipeline in the future. There is a
reason gamescope has made a huge difference in driving the color
management work forward.

> And the opposite case, if someone writes userspace without tap/scale
> colorops, and then drivers add those, and there is no pipeline without
> them, because they always exist. Would that userspace disregard all
> those pipelines because it does not understand tap/scale colorops,
> leaving no usable pipelines? Would that not be kernel regressing
> userspace?
> 

The simple solution is to leave previously advertised pipelines
untouched and add a new version that does include scaling information.

> If the kernel keeps on exposing pipelines without the colorops, it
> fails the basic promise of the whole design: that all pixel value
> affecting operations are at least listed if not controllable.
> 
> How will we avoid painting ourselves in a corner?
> 
> Maybe we need a colorop for "here be dragons" documented as having
> unknown and unreliable effects, until driver authors are sure that
> everything has been modelled in the pipeline and there are no unknowns?
> Or a flag on the pipelines, if we can have that. Then we can at least
> tell when the pipeline does not fulfil the basic promise.
> 

The will always be dragons at some level.

Harry

> 
> Thanks,
> pq

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 7257 bytes --]

On Tue, 7 Nov 2023 11:58:26 -0500
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-11-07 04:55, Pekka Paalanen wrote:
> > On Mon, 6 Nov 2023 11:19:27 -0500
> > Harry Wentland <harry.wentland@amd.com> wrote:
> >   
> >> On 2023-10-20 06:36, Pekka Paalanen wrote:  
> >>> On Thu, 19 Oct 2023 10:56:40 -0400
> >>> Harry Wentland <harry.wentland@amd.com> wrote:
> >>>     
> >>>> On 2023-10-10 12:13, Melissa Wen wrote:    
> >>>>> O 09/08, Harry Wentland wrote:      
> >>>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>    
> >>>
> >>> ...
> >>>     
> >>>>> Also, with this new plane API in place, I understand that we will
> >>>>> already need think on how to deal with the mixing between old drm color
> >>>>> properties (color encoding and color range) and these new way of setting
> >>>>> plane color properties. IIUC, Pekka asked a related question about it
> >>>>> when talking about CRTC automatic RGB->YUV (?) 
> >>>>>       
> >>>>
> >>>> We'll still need to confirm whether we'll want to deprecate these
> >>>> existing properties. If we do that we'd want a client prop. Things
> >>>> should still work without deprecating them, if drivers just pick up
> >>>> after the initial encoding and range CSC.
> >>>>
> >>>> But realistically it might be better to deprecate them and turn them
> >>>> into explicit colorops.    
> >>>
> >>> The existing properties would need to be explicitly reflected in the
> >>> new pipelines anyway, otherwise there would always be doubt at which
> >>> point of a pipeline the old properties apply, and they might even
> >>> need to change positions between pipelines.
> >>>
> >>> I think it is simply easier to just hide all old color related
> >>> properties when userspace sets the client-cap to enable pipelines. The
> >>> problem is to make sure to hide all old properties on all drivers that
> >>> support the client-cap.
> >>>
> >>> As a pipeline must be complete (describe everything that happens to
> >>> pixel values), it's going to be a flag day per driver.
> >>>
> >>> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
> >>> as well. Maybe it's purely informative and non-configurable, keyed by
> >>> FB pixel format, but still.
> >>>
> >>> We also need a colorop to represent sample filtering, e.g. bilinear,
> >>> like I think Sebastian may have mentioned in the past. Everything
> >>> before the sample filter happens "per tap" as Joshua Ashton put it, and
> >>> everything after it happens on the sample that was computed as a
> >>> weighted average of the filter tap inputs (texels).
> >>>
> >>> There could be colorops other than sample filtering that operate on
> >>> more than one sample at a time, like blur or sharpness. There could
> >>> even be colorops that change the image size like adding padding that
> >>> the following colorop hardware requires, and then yet another colorop
> >>> that clips that padding away. For an example, see
> >>> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
> >>>
> >>> If that padding and its color can affect the pipeline results of the
> >>> pixels near the padding (e.g. some convolution is applied with them,
> >>> which may be the reason why padding is necessary to begin with), then
> >>> it would be best to model it.
> >>>     
> >>
> >> I hear you but I'm also somewhat shying away from defining this at this point.  
> > 
> > Would you define them before the new UAPI is released though?
> > 
> > I agree there is no need to have them in this patch series, but I think
> > we'd hit the below problems if the UAPI is released without them.
> >   
> >> There are already too many things that need to happen and I will focus on the
> >> actual color blocks (LUTs, matrices) first. We'll always be able to add a new
> >> (read-only) colorop type to define scaling and tap behavior at any point and
> >> a client is free to ignore a color pipeline if it doesn't find any tap/scale
> >> info in it.  
> > 
> > How would userspace know to look for tap/scale info, if there is no
> > upstream definition even on paper?
> >   
> 
> So far OSes did not care about this. Whether that's good or bad is
> something everyone can answer for themselves.
> 
> If you write a compositor and really need this you can just ignore
> color pipelines that don't have this, i.e., you'll probably want
> to wait with implementing color pipeline support until you have what
> you need from DRM/KMS.
> 
> This is not to say I don't want to have support for Weston. But I'm
> wondering if we place too much importance on getting every little
> thing figured out whereas we could be making forward progress and
> address more aspects of a color pipeline in the future. There is a
> reason gamescope has made a huge difference in driving the color
> management work forward.
> 
> > And the opposite case, if someone writes userspace without tap/scale
> > colorops, and then drivers add those, and there is no pipeline without
> > them, because they always exist. Would that userspace disregard all
> > those pipelines because it does not understand tap/scale colorops,
> > leaving no usable pipelines? Would that not be kernel regressing
> > userspace?
> >   
> 
> The simple solution is to leave previously advertised pipelines
> untouched and add a new version that does include scaling information.
> 
> > If the kernel keeps on exposing pipelines without the colorops, it
> > fails the basic promise of the whole design: that all pixel value
> > affecting operations are at least listed if not controllable.
> > 
> > How will we avoid painting ourselves in a corner?
> > 
> > Maybe we need a colorop for "here be dragons" documented as having
> > unknown and unreliable effects, until driver authors are sure that
> > everything has been modelled in the pipeline and there are no unknowns?
> > Or a flag on the pipelines, if we can have that. Then we can at least
> > tell when the pipeline does not fulfil the basic promise.
> >   
> 
> The will always be dragons at some level.

Do I understand right that the goal of fully understood color pipelines
is a lost cause?

That every pipeline might always have something unknown and there is no
way for userspace to know if it does? Maybe because driver developers
don't know either?

By something unknown I refer to anything outside of basic precision
issues. Doing interpolation or mixing of inputs on the wrong side of a
known non-linear colorop, for example.

An incremental UAPI development approach is fine by me, meaning that
pipelines might not be complete at first, but I believe that requires
telling userspace whether the driver developers consider the pipeline
complete (no undescribed operations that would significantly change
results from the expected results given the UAPI exposed pipeline).

The prime example of what I would like to know is that if a FB
contains PQ-encoded image and I use a color pipeline to scale that
image up, will the interpolation happen before or after the non-linear
colorop that decodes PQ. That is a significant difference as pointed
out by Joshua.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Sebastian Wick]

On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:
>
> On Tue, 7 Nov 2023 11:58:26 -0500
> Harry Wentland <harry.wentland@amd.com> wrote:
>
> > On 2023-11-07 04:55, Pekka Paalanen wrote:
> > > On Mon, 6 Nov 2023 11:19:27 -0500
> > > Harry Wentland <harry.wentland@amd.com> wrote:
> > >
> > >> On 2023-10-20 06:36, Pekka Paalanen wrote:
> > >>> On Thu, 19 Oct 2023 10:56:40 -0400
> > >>> Harry Wentland <harry.wentland@amd.com> wrote:
> > >>>
> > >>>> On 2023-10-10 12:13, Melissa Wen wrote:
> > >>>>> O 09/08, Harry Wentland wrote:
> > >>>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
> > >>>
> > >>> ...
> > >>>
> > >>>>> Also, with this new plane API in place, I understand that we will
> > >>>>> already need think on how to deal with the mixing between old drm color
> > >>>>> properties (color encoding and color range) and these new way of setting
> > >>>>> plane color properties. IIUC, Pekka asked a related question about it
> > >>>>> when talking about CRTC automatic RGB->YUV (?)
> > >>>>>
> > >>>>
> > >>>> We'll still need to confirm whether we'll want to deprecate these
> > >>>> existing properties. If we do that we'd want a client prop. Things
> > >>>> should still work without deprecating them, if drivers just pick up
> > >>>> after the initial encoding and range CSC.
> > >>>>
> > >>>> But realistically it might be better to deprecate them and turn them
> > >>>> into explicit colorops.
> > >>>
> > >>> The existing properties would need to be explicitly reflected in the
> > >>> new pipelines anyway, otherwise there would always be doubt at which
> > >>> point of a pipeline the old properties apply, and they might even
> > >>> need to change positions between pipelines.
> > >>>
> > >>> I think it is simply easier to just hide all old color related
> > >>> properties when userspace sets the client-cap to enable pipelines. The
> > >>> problem is to make sure to hide all old properties on all drivers that
> > >>> support the client-cap.
> > >>>
> > >>> As a pipeline must be complete (describe everything that happens to
> > >>> pixel values), it's going to be a flag day per driver.
> > >>>
> > >>> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
> > >>> as well. Maybe it's purely informative and non-configurable, keyed by
> > >>> FB pixel format, but still.
> > >>>
> > >>> We also need a colorop to represent sample filtering, e.g. bilinear,
> > >>> like I think Sebastian may have mentioned in the past. Everything
> > >>> before the sample filter happens "per tap" as Joshua Ashton put it, and
> > >>> everything after it happens on the sample that was computed as a
> > >>> weighted average of the filter tap inputs (texels).
> > >>>
> > >>> There could be colorops other than sample filtering that operate on
> > >>> more than one sample at a time, like blur or sharpness. There could
> > >>> even be colorops that change the image size like adding padding that
> > >>> the following colorop hardware requires, and then yet another colorop
> > >>> that clips that padding away. For an example, see
> > >>> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
> > >>>
> > >>> If that padding and its color can affect the pipeline results of the
> > >>> pixels near the padding (e.g. some convolution is applied with them,
> > >>> which may be the reason why padding is necessary to begin with), then
> > >>> it would be best to model it.
> > >>>
> > >>
> > >> I hear you but I'm also somewhat shying away from defining this at this point.
> > >
> > > Would you define them before the new UAPI is released though?
> > >
> > > I agree there is no need to have them in this patch series, but I think
> > > we'd hit the below problems if the UAPI is released without them.
> > >
> > >> There are already too many things that need to happen and I will focus on the
> > >> actual color blocks (LUTs, matrices) first. We'll always be able to add a new
> > >> (read-only) colorop type to define scaling and tap behavior at any point and
> > >> a client is free to ignore a color pipeline if it doesn't find any tap/scale
> > >> info in it.
> > >
> > > How would userspace know to look for tap/scale info, if there is no
> > > upstream definition even on paper?
> > >
> >
> > So far OSes did not care about this. Whether that's good or bad is
> > something everyone can answer for themselves.
> >
> > If you write a compositor and really need this you can just ignore
> > color pipelines that don't have this, i.e., you'll probably want
> > to wait with implementing color pipeline support until you have what
> > you need from DRM/KMS.
> >
> > This is not to say I don't want to have support for Weston. But I'm
> > wondering if we place too much importance on getting every little
> > thing figured out whereas we could be making forward progress and
> > address more aspects of a color pipeline in the future. There is a
> > reason gamescope has made a huge difference in driving the color
> > management work forward.
> >
> > > And the opposite case, if someone writes userspace without tap/scale
> > > colorops, and then drivers add those, and there is no pipeline without
> > > them, because they always exist. Would that userspace disregard all
> > > those pipelines because it does not understand tap/scale colorops,
> > > leaving no usable pipelines? Would that not be kernel regressing
> > > userspace?
> > >
> >
> > The simple solution is to leave previously advertised pipelines
> > untouched and add a new version that does include scaling information.
> >
> > > If the kernel keeps on exposing pipelines without the colorops, it
> > > fails the basic promise of the whole design: that all pixel value
> > > affecting operations are at least listed if not controllable.
> > >
> > > How will we avoid painting ourselves in a corner?
> > >
> > > Maybe we need a colorop for "here be dragons" documented as having
> > > unknown and unreliable effects, until driver authors are sure that
> > > everything has been modelled in the pipeline and there are no unknowns?
> > > Or a flag on the pipelines, if we can have that. Then we can at least
> > > tell when the pipeline does not fulfil the basic promise.
> > >
> >
> > The will always be dragons at some level.
>
> Do I understand right that the goal of fully understood color pipelines
> is a lost cause?
>
> That every pipeline might always have something unknown and there is no
> way for userspace to know if it does? Maybe because driver developers
> don't know either?
>
> By something unknown I refer to anything outside of basic precision
> issues. Doing interpolation or mixing of inputs on the wrong side of a
> known non-linear colorop, for example.

I don't think that's the case. Hardware vendors should understand the
hardware and exposing everything that affects the values is the goal
here. There will be a transitional period where the pipelines might
not expose every detail yet but that's fine. It's better than what we
have now and should become even better with time. It would maybe be
helpful in the future to have a cap, or property, or whatever, to
indicate that the pipelines are "complete" descriptions of what
happens to the values but we can discuss it when it becomes relevant.

> An incremental UAPI development approach is fine by me, meaning that
> pipelines might not be complete at first, but I believe that requires
> telling userspace whether the driver developers consider the pipeline
> complete (no undescribed operations that would significantly change
> results from the expected results given the UAPI exposed pipeline).
>
> The prime example of what I would like to know is that if a FB
> contains PQ-encoded image and I use a color pipeline to scale that
> image up, will the interpolation happen before or after the non-linear
> colorop that decodes PQ. That is a significant difference as pointed
> out by Joshua.
>
>
> Thanks,
> pq

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-11-08 06:40, Sebastian Wick wrote:
> On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:
>>
>> On Tue, 7 Nov 2023 11:58:26 -0500
>> Harry Wentland <harry.wentland@amd.com> wrote:
>>
>>> On 2023-11-07 04:55, Pekka Paalanen wrote:
>>>> On Mon, 6 Nov 2023 11:19:27 -0500
>>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>>
>>>>> On 2023-10-20 06:36, Pekka Paalanen wrote:
>>>>>> On Thu, 19 Oct 2023 10:56:40 -0400
>>>>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>>>>
>>>>>>> On 2023-10-10 12:13, Melissa Wen wrote:
>>>>>>>> O 09/08, Harry Wentland wrote:
>>>>>>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>
>>>>>>
>>>>>> ...
>>>>>>
>>>>>>>> Also, with this new plane API in place, I understand that we will
>>>>>>>> already need think on how to deal with the mixing between old drm color
>>>>>>>> properties (color encoding and color range) and these new way of setting
>>>>>>>> plane color properties. IIUC, Pekka asked a related question about it
>>>>>>>> when talking about CRTC automatic RGB->YUV (?)
>>>>>>>>
>>>>>>>
>>>>>>> We'll still need to confirm whether we'll want to deprecate these
>>>>>>> existing properties. If we do that we'd want a client prop. Things
>>>>>>> should still work without deprecating them, if drivers just pick up
>>>>>>> after the initial encoding and range CSC.
>>>>>>>
>>>>>>> But realistically it might be better to deprecate them and turn them
>>>>>>> into explicit colorops.
>>>>>>
>>>>>> The existing properties would need to be explicitly reflected in the
>>>>>> new pipelines anyway, otherwise there would always be doubt at which
>>>>>> point of a pipeline the old properties apply, and they might even
>>>>>> need to change positions between pipelines.
>>>>>>
>>>>>> I think it is simply easier to just hide all old color related
>>>>>> properties when userspace sets the client-cap to enable pipelines. The
>>>>>> problem is to make sure to hide all old properties on all drivers that
>>>>>> support the client-cap.
>>>>>>
>>>>>> As a pipeline must be complete (describe everything that happens to
>>>>>> pixel values), it's going to be a flag day per driver.
>>>>>>
>>>>>> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
>>>>>> as well. Maybe it's purely informative and non-configurable, keyed by
>>>>>> FB pixel format, but still.
>>>>>>
>>>>>> We also need a colorop to represent sample filtering, e.g. bilinear,
>>>>>> like I think Sebastian may have mentioned in the past. Everything
>>>>>> before the sample filter happens "per tap" as Joshua Ashton put it, and
>>>>>> everything after it happens on the sample that was computed as a
>>>>>> weighted average of the filter tap inputs (texels).
>>>>>>
>>>>>> There could be colorops other than sample filtering that operate on
>>>>>> more than one sample at a time, like blur or sharpness. There could
>>>>>> even be colorops that change the image size like adding padding that
>>>>>> the following colorop hardware requires, and then yet another colorop
>>>>>> that clips that padding away. For an example, see
>>>>>> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
>>>>>>
>>>>>> If that padding and its color can affect the pipeline results of the
>>>>>> pixels near the padding (e.g. some convolution is applied with them,
>>>>>> which may be the reason why padding is necessary to begin with), then
>>>>>> it would be best to model it.
>>>>>>
>>>>>
>>>>> I hear you but I'm also somewhat shying away from defining this at this point.
>>>>
>>>> Would you define them before the new UAPI is released though?
>>>>
>>>> I agree there is no need to have them in this patch series, but I think
>>>> we'd hit the below problems if the UAPI is released without them.
>>>>
>>>>> There are already too many things that need to happen and I will focus on the
>>>>> actual color blocks (LUTs, matrices) first. We'll always be able to add a new
>>>>> (read-only) colorop type to define scaling and tap behavior at any point and
>>>>> a client is free to ignore a color pipeline if it doesn't find any tap/scale
>>>>> info in it.
>>>>
>>>> How would userspace know to look for tap/scale info, if there is no
>>>> upstream definition even on paper?
>>>>
>>>
>>> So far OSes did not care about this. Whether that's good or bad is
>>> something everyone can answer for themselves.
>>>
>>> If you write a compositor and really need this you can just ignore
>>> color pipelines that don't have this, i.e., you'll probably want
>>> to wait with implementing color pipeline support until you have what
>>> you need from DRM/KMS.
>>>
>>> This is not to say I don't want to have support for Weston. But I'm
>>> wondering if we place too much importance on getting every little
>>> thing figured out whereas we could be making forward progress and
>>> address more aspects of a color pipeline in the future. There is a
>>> reason gamescope has made a huge difference in driving the color
>>> management work forward.
>>>
>>>> And the opposite case, if someone writes userspace without tap/scale
>>>> colorops, and then drivers add those, and there is no pipeline without
>>>> them, because they always exist. Would that userspace disregard all
>>>> those pipelines because it does not understand tap/scale colorops,
>>>> leaving no usable pipelines? Would that not be kernel regressing
>>>> userspace?
>>>>
>>>
>>> The simple solution is to leave previously advertised pipelines
>>> untouched and add a new version that does include scaling information.
>>>
>>>> If the kernel keeps on exposing pipelines without the colorops, it
>>>> fails the basic promise of the whole design: that all pixel value
>>>> affecting operations are at least listed if not controllable.
>>>>
>>>> How will we avoid painting ourselves in a corner?
>>>>
>>>> Maybe we need a colorop for "here be dragons" documented as having
>>>> unknown and unreliable effects, until driver authors are sure that
>>>> everything has been modelled in the pipeline and there are no unknowns?
>>>> Or a flag on the pipelines, if we can have that. Then we can at least
>>>> tell when the pipeline does not fulfil the basic promise.
>>>>
>>>
>>> The will always be dragons at some level.
>>
>> Do I understand right that the goal of fully understood color pipelines
>> is a lost cause?
>>
>> That every pipeline might always have something unknown and there is no
>> way for userspace to know if it does? Maybe because driver developers
>> don't know either?
>>
>> By something unknown I refer to anything outside of basic precision
>> issues. Doing interpolation or mixing of inputs on the wrong side of a
>> known non-linear colorop, for example.
> 
> I don't think that's the case. Hardware vendors should understand the
> hardware and exposing everything that affects the values is the goal
> here. There will be a transitional period where the pipelines might
> not expose every detail yet but that's fine. It's better than what we
> have now and should become even better with time. It would maybe be
> helpful in the future to have a cap, or property, or whatever, to
> indicate that the pipelines are "complete" descriptions of what
> happens to the values but we can discuss it when it becomes relevant.
> 

I agree, for the most part. But how do you then define "complete" if
you exclude precision issues?

>> An incremental UAPI development approach is fine by me, meaning that
>> pipelines might not be complete at first, but I believe that requires
>> telling userspace whether the driver developers consider the pipeline
>> complete (no undescribed operations that would significantly change
>> results from the expected results given the UAPI exposed pipeline).
>>
>> The prime example of what I would like to know is that if a FB
>> contains PQ-encoded image and I use a color pipeline to scale that
>> image up, will the interpolation happen before or after the non-linear
>> colorop that decodes PQ. That is a significant difference as pointed
>> out by Joshua.
>>

That's fair and I want to give that to you. My concern stems from
the sentiment that I hear that any pipeline that doesn't explicitly
advertise this is useless. I don't agree there. Let's not let perfect
be the enemy of good.

Harry

>>
>> Thanks,
>> pq
> 

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 9627 bytes --]

On Wed, 8 Nov 2023 09:31:17 -0500
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-11-08 06:40, Sebastian Wick wrote:
> > On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:  
> >>
> >> On Tue, 7 Nov 2023 11:58:26 -0500
> >> Harry Wentland <harry.wentland@amd.com> wrote:
> >>  
> >>> On 2023-11-07 04:55, Pekka Paalanen wrote:  
> >>>> On Mon, 6 Nov 2023 11:19:27 -0500
> >>>> Harry Wentland <harry.wentland@amd.com> wrote:
> >>>>  
> >>>>> On 2023-10-20 06:36, Pekka Paalanen wrote:  
> >>>>>> On Thu, 19 Oct 2023 10:56:40 -0400
> >>>>>> Harry Wentland <harry.wentland@amd.com> wrote:
> >>>>>>  
> >>>>>>> On 2023-10-10 12:13, Melissa Wen wrote:  
> >>>>>>>> O 09/08, Harry Wentland wrote:  
> >>>>>>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>  
> >>>>>>
> >>>>>> ...
> >>>>>>  
> >>>>>>>> Also, with this new plane API in place, I understand that we will
> >>>>>>>> already need think on how to deal with the mixing between old drm color
> >>>>>>>> properties (color encoding and color range) and these new way of setting
> >>>>>>>> plane color properties. IIUC, Pekka asked a related question about it
> >>>>>>>> when talking about CRTC automatic RGB->YUV (?)
> >>>>>>>>  
> >>>>>>>
> >>>>>>> We'll still need to confirm whether we'll want to deprecate these
> >>>>>>> existing properties. If we do that we'd want a client prop. Things
> >>>>>>> should still work without deprecating them, if drivers just pick up
> >>>>>>> after the initial encoding and range CSC.
> >>>>>>>
> >>>>>>> But realistically it might be better to deprecate them and turn them
> >>>>>>> into explicit colorops.  
> >>>>>>
> >>>>>> The existing properties would need to be explicitly reflected in the
> >>>>>> new pipelines anyway, otherwise there would always be doubt at which
> >>>>>> point of a pipeline the old properties apply, and they might even
> >>>>>> need to change positions between pipelines.
> >>>>>>
> >>>>>> I think it is simply easier to just hide all old color related
> >>>>>> properties when userspace sets the client-cap to enable pipelines. The
> >>>>>> problem is to make sure to hide all old properties on all drivers that
> >>>>>> support the client-cap.
> >>>>>>
> >>>>>> As a pipeline must be complete (describe everything that happens to
> >>>>>> pixel values), it's going to be a flag day per driver.
> >>>>>>
> >>>>>> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
> >>>>>> as well. Maybe it's purely informative and non-configurable, keyed by
> >>>>>> FB pixel format, but still.
> >>>>>>
> >>>>>> We also need a colorop to represent sample filtering, e.g. bilinear,
> >>>>>> like I think Sebastian may have mentioned in the past. Everything
> >>>>>> before the sample filter happens "per tap" as Joshua Ashton put it, and
> >>>>>> everything after it happens on the sample that was computed as a
> >>>>>> weighted average of the filter tap inputs (texels).
> >>>>>>
> >>>>>> There could be colorops other than sample filtering that operate on
> >>>>>> more than one sample at a time, like blur or sharpness. There could
> >>>>>> even be colorops that change the image size like adding padding that
> >>>>>> the following colorop hardware requires, and then yet another colorop
> >>>>>> that clips that padding away. For an example, see
> >>>>>> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
> >>>>>>
> >>>>>> If that padding and its color can affect the pipeline results of the
> >>>>>> pixels near the padding (e.g. some convolution is applied with them,
> >>>>>> which may be the reason why padding is necessary to begin with), then
> >>>>>> it would be best to model it.
> >>>>>>  
> >>>>>
> >>>>> I hear you but I'm also somewhat shying away from defining this at this point.  
> >>>>
> >>>> Would you define them before the new UAPI is released though?
> >>>>
> >>>> I agree there is no need to have them in this patch series, but I think
> >>>> we'd hit the below problems if the UAPI is released without them.
> >>>>  
> >>>>> There are already too many things that need to happen and I will focus on the
> >>>>> actual color blocks (LUTs, matrices) first. We'll always be able to add a new
> >>>>> (read-only) colorop type to define scaling and tap behavior at any point and
> >>>>> a client is free to ignore a color pipeline if it doesn't find any tap/scale
> >>>>> info in it.  
> >>>>
> >>>> How would userspace know to look for tap/scale info, if there is no
> >>>> upstream definition even on paper?
> >>>>  
> >>>
> >>> So far OSes did not care about this. Whether that's good or bad is
> >>> something everyone can answer for themselves.
> >>>
> >>> If you write a compositor and really need this you can just ignore
> >>> color pipelines that don't have this, i.e., you'll probably want
> >>> to wait with implementing color pipeline support until you have what
> >>> you need from DRM/KMS.
> >>>
> >>> This is not to say I don't want to have support for Weston. But I'm
> >>> wondering if we place too much importance on getting every little
> >>> thing figured out whereas we could be making forward progress and
> >>> address more aspects of a color pipeline in the future. There is a
> >>> reason gamescope has made a huge difference in driving the color
> >>> management work forward.
> >>>  
> >>>> And the opposite case, if someone writes userspace without tap/scale
> >>>> colorops, and then drivers add those, and there is no pipeline without
> >>>> them, because they always exist. Would that userspace disregard all
> >>>> those pipelines because it does not understand tap/scale colorops,
> >>>> leaving no usable pipelines? Would that not be kernel regressing
> >>>> userspace?
> >>>>  
> >>>
> >>> The simple solution is to leave previously advertised pipelines
> >>> untouched and add a new version that does include scaling information.
> >>>  
> >>>> If the kernel keeps on exposing pipelines without the colorops, it
> >>>> fails the basic promise of the whole design: that all pixel value
> >>>> affecting operations are at least listed if not controllable.
> >>>>
> >>>> How will we avoid painting ourselves in a corner?
> >>>>
> >>>> Maybe we need a colorop for "here be dragons" documented as having
> >>>> unknown and unreliable effects, until driver authors are sure that
> >>>> everything has been modelled in the pipeline and there are no unknowns?
> >>>> Or a flag on the pipelines, if we can have that. Then we can at least
> >>>> tell when the pipeline does not fulfil the basic promise.
> >>>>  
> >>>
> >>> The will always be dragons at some level.  
> >>
> >> Do I understand right that the goal of fully understood color pipelines
> >> is a lost cause?
> >>
> >> That every pipeline might always have something unknown and there is no
> >> way for userspace to know if it does? Maybe because driver developers
> >> don't know either?
> >>
> >> By something unknown I refer to anything outside of basic precision
> >> issues. Doing interpolation or mixing of inputs on the wrong side of a
> >> known non-linear colorop, for example.  
> > 
> > I don't think that's the case. Hardware vendors should understand the
> > hardware and exposing everything that affects the values is the goal
> > here. There will be a transitional period where the pipelines might
> > not expose every detail yet but that's fine. It's better than what we
> > have now and should become even better with time. It would maybe be
> > helpful in the future to have a cap, or property, or whatever, to
> > indicate that the pipelines are "complete" descriptions of what
> > happens to the values but we can discuss it when it becomes relevant.
> >   
> 
> I agree, for the most part. But how do you then define "complete" if
> you exclude precision issues?

If someone can develop some kind of precision indication in the UAPI,
we might be able to answer that question then.

> >> An incremental UAPI development approach is fine by me, meaning that
> >> pipelines might not be complete at first, but I believe that requires
> >> telling userspace whether the driver developers consider the pipeline
> >> complete (no undescribed operations that would significantly change
> >> results from the expected results given the UAPI exposed pipeline).
> >>
> >> The prime example of what I would like to know is that if a FB
> >> contains PQ-encoded image and I use a color pipeline to scale that
> >> image up, will the interpolation happen before or after the non-linear
> >> colorop that decodes PQ. That is a significant difference as pointed
> >> out by Joshua.
> >>  
> 
> That's fair and I want to give that to you. My concern stems from
> the sentiment that I hear that any pipeline that doesn't explicitly
> advertise this is useless. I don't agree there. Let's not let perfect
> be the enemy of good.

It's up to the use case. The policy of what is sufficient should reside
in userspace.

What about matching compositor shader composition with KMS?

Can we use that as a rough precision threshold? If userspace implements
the exact same color pipeline as the KMS UAPI describes, then that and
the KMS composited result should be indistinguishable in side-by-side
or alternating visual inspection on any monitor in isolation.

Did this whole effort not start from wanting to off-load things to
display hardware but still maintain visual equivalence to software/GPU
composition?


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-11-08 11:19, Pekka Paalanen wrote:
> On Wed, 8 Nov 2023 09:31:17 -0500
> Harry Wentland <harry.wentland@amd.com> wrote:
> 
>> On 2023-11-08 06:40, Sebastian Wick wrote:
>>> On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:  
>>>>
>>>> On Tue, 7 Nov 2023 11:58:26 -0500
>>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>>  
>>>>> On 2023-11-07 04:55, Pekka Paalanen wrote:  
>>>>>> On Mon, 6 Nov 2023 11:19:27 -0500
>>>>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>>>>  
>>>>>>> On 2023-10-20 06:36, Pekka Paalanen wrote:  
>>>>>>>> On Thu, 19 Oct 2023 10:56:40 -0400
>>>>>>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>>>>>>  
>>>>>>>>> On 2023-10-10 12:13, Melissa Wen wrote:  
>>>>>>>>>> O 09/08, Harry Wentland wrote:  
>>>>>>>>>>> Signed-off-by: Harry Wentland <harry.wentland@amd.com>  
>>>>>>>>
>>>>>>>> ...
>>>>>>>>  
>>>>>>>>>> Also, with this new plane API in place, I understand that we will
>>>>>>>>>> already need think on how to deal with the mixing between old drm color
>>>>>>>>>> properties (color encoding and color range) and these new way of setting
>>>>>>>>>> plane color properties. IIUC, Pekka asked a related question about it
>>>>>>>>>> when talking about CRTC automatic RGB->YUV (?)
>>>>>>>>>>  
>>>>>>>>>
>>>>>>>>> We'll still need to confirm whether we'll want to deprecate these
>>>>>>>>> existing properties. If we do that we'd want a client prop. Things
>>>>>>>>> should still work without deprecating them, if drivers just pick up
>>>>>>>>> after the initial encoding and range CSC.
>>>>>>>>>
>>>>>>>>> But realistically it might be better to deprecate them and turn them
>>>>>>>>> into explicit colorops.  
>>>>>>>>
>>>>>>>> The existing properties would need to be explicitly reflected in the
>>>>>>>> new pipelines anyway, otherwise there would always be doubt at which
>>>>>>>> point of a pipeline the old properties apply, and they might even
>>>>>>>> need to change positions between pipelines.
>>>>>>>>
>>>>>>>> I think it is simply easier to just hide all old color related
>>>>>>>> properties when userspace sets the client-cap to enable pipelines. The
>>>>>>>> problem is to make sure to hide all old properties on all drivers that
>>>>>>>> support the client-cap.
>>>>>>>>
>>>>>>>> As a pipeline must be complete (describe everything that happens to
>>>>>>>> pixel values), it's going to be a flag day per driver.
>>>>>>>>
>>>>>>>> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline
>>>>>>>> as well. Maybe it's purely informative and non-configurable, keyed by
>>>>>>>> FB pixel format, but still.
>>>>>>>>
>>>>>>>> We also need a colorop to represent sample filtering, e.g. bilinear,
>>>>>>>> like I think Sebastian may have mentioned in the past. Everything
>>>>>>>> before the sample filter happens "per tap" as Joshua Ashton put it, and
>>>>>>>> everything after it happens on the sample that was computed as a
>>>>>>>> weighted average of the filter tap inputs (texels).
>>>>>>>>
>>>>>>>> There could be colorops other than sample filtering that operate on
>>>>>>>> more than one sample at a time, like blur or sharpness. There could
>>>>>>>> even be colorops that change the image size like adding padding that
>>>>>>>> the following colorop hardware requires, and then yet another colorop
>>>>>>>> that clips that padding away. For an example, see
>>>>>>>> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html
>>>>>>>>
>>>>>>>> If that padding and its color can affect the pipeline results of the
>>>>>>>> pixels near the padding (e.g. some convolution is applied with them,
>>>>>>>> which may be the reason why padding is necessary to begin with), then
>>>>>>>> it would be best to model it.
>>>>>>>>  
>>>>>>>
>>>>>>> I hear you but I'm also somewhat shying away from defining this at this point.  
>>>>>>
>>>>>> Would you define them before the new UAPI is released though?
>>>>>>
>>>>>> I agree there is no need to have them in this patch series, but I think
>>>>>> we'd hit the below problems if the UAPI is released without them.
>>>>>>  
>>>>>>> There are already too many things that need to happen and I will focus on the
>>>>>>> actual color blocks (LUTs, matrices) first. We'll always be able to add a new
>>>>>>> (read-only) colorop type to define scaling and tap behavior at any point and
>>>>>>> a client is free to ignore a color pipeline if it doesn't find any tap/scale
>>>>>>> info in it.  
>>>>>>
>>>>>> How would userspace know to look for tap/scale info, if there is no
>>>>>> upstream definition even on paper?
>>>>>>  
>>>>>
>>>>> So far OSes did not care about this. Whether that's good or bad is
>>>>> something everyone can answer for themselves.
>>>>>
>>>>> If you write a compositor and really need this you can just ignore
>>>>> color pipelines that don't have this, i.e., you'll probably want
>>>>> to wait with implementing color pipeline support until you have what
>>>>> you need from DRM/KMS.
>>>>>
>>>>> This is not to say I don't want to have support for Weston. But I'm
>>>>> wondering if we place too much importance on getting every little
>>>>> thing figured out whereas we could be making forward progress and
>>>>> address more aspects of a color pipeline in the future. There is a
>>>>> reason gamescope has made a huge difference in driving the color
>>>>> management work forward.
>>>>>  
>>>>>> And the opposite case, if someone writes userspace without tap/scale
>>>>>> colorops, and then drivers add those, and there is no pipeline without
>>>>>> them, because they always exist. Would that userspace disregard all
>>>>>> those pipelines because it does not understand tap/scale colorops,
>>>>>> leaving no usable pipelines? Would that not be kernel regressing
>>>>>> userspace?
>>>>>>  
>>>>>
>>>>> The simple solution is to leave previously advertised pipelines
>>>>> untouched and add a new version that does include scaling information.
>>>>>  
>>>>>> If the kernel keeps on exposing pipelines without the colorops, it
>>>>>> fails the basic promise of the whole design: that all pixel value
>>>>>> affecting operations are at least listed if not controllable.
>>>>>>
>>>>>> How will we avoid painting ourselves in a corner?
>>>>>>
>>>>>> Maybe we need a colorop for "here be dragons" documented as having
>>>>>> unknown and unreliable effects, until driver authors are sure that
>>>>>> everything has been modelled in the pipeline and there are no unknowns?
>>>>>> Or a flag on the pipelines, if we can have that. Then we can at least
>>>>>> tell when the pipeline does not fulfil the basic promise.
>>>>>>  
>>>>>
>>>>> The will always be dragons at some level.  
>>>>
>>>> Do I understand right that the goal of fully understood color pipelines
>>>> is a lost cause?
>>>>
>>>> That every pipeline might always have something unknown and there is no
>>>> way for userspace to know if it does? Maybe because driver developers
>>>> don't know either?
>>>>
>>>> By something unknown I refer to anything outside of basic precision
>>>> issues. Doing interpolation or mixing of inputs on the wrong side of a
>>>> known non-linear colorop, for example.  
>>>
>>> I don't think that's the case. Hardware vendors should understand the
>>> hardware and exposing everything that affects the values is the goal
>>> here. There will be a transitional period where the pipelines might
>>> not expose every detail yet but that's fine. It's better than what we
>>> have now and should become even better with time. It would maybe be
>>> helpful in the future to have a cap, or property, or whatever, to
>>> indicate that the pipelines are "complete" descriptions of what
>>> happens to the values but we can discuss it when it becomes relevant.
>>>   
>>
>> I agree, for the most part. But how do you then define "complete" if
>> you exclude precision issues?
> 
> If someone can develop some kind of precision indication in the UAPI,
> we might be able to answer that question then.
> 
>>>> An incremental UAPI development approach is fine by me, meaning that
>>>> pipelines might not be complete at first, but I believe that requires
>>>> telling userspace whether the driver developers consider the pipeline
>>>> complete (no undescribed operations that would significantly change
>>>> results from the expected results given the UAPI exposed pipeline).
>>>>
>>>> The prime example of what I would like to know is that if a FB
>>>> contains PQ-encoded image and I use a color pipeline to scale that
>>>> image up, will the interpolation happen before or after the non-linear
>>>> colorop that decodes PQ. That is a significant difference as pointed
>>>> out by Joshua.
>>>>  
>>
>> That's fair and I want to give that to you. My concern stems from
>> the sentiment that I hear that any pipeline that doesn't explicitly
>> advertise this is useless. I don't agree there. Let's not let perfect
>> be the enemy of good.
> 
> It's up to the use case. The policy of what is sufficient should reside
> in userspace.
> 
> What about matching compositor shader composition with KMS?
> 
> Can we use that as a rough precision threshold? If userspace implements
> the exact same color pipeline as the KMS UAPI describes, then that and
> the KMS composited result should be indistinguishable in side-by-side
> or alternating visual inspection on any monitor in isolation.
> 
> Did this whole effort not start from wanting to off-load things to
> display hardware but still maintain visual equivalence to software/GPU
> composition?
> 

I agree with you and I want all that as well.

All I'm saying is that every userspace won't have the same policy of
what is sufficient. Just because Weston has a very high threshold
doesn't mean we can't move forward with a workable solution for other
userspace, as long as we don't do something that prevents us from
doing the perfect solution eventually.

And yes, I do want a solution that works for Weston and hear your
comments on what that requires.

Harry

> 
> Thanks,
> pq

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 5029 bytes --]

On Wed, 8 Nov 2023 11:27:35 -0500
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-11-08 11:19, Pekka Paalanen wrote:
> > On Wed, 8 Nov 2023 09:31:17 -0500
> > Harry Wentland <harry.wentland@amd.com> wrote:
> >   
> >> On 2023-11-08 06:40, Sebastian Wick wrote:  
> >>> On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:    

...

> >>>> An incremental UAPI development approach is fine by me, meaning that
> >>>> pipelines might not be complete at first, but I believe that requires
> >>>> telling userspace whether the driver developers consider the pipeline
> >>>> complete (no undescribed operations that would significantly change
> >>>> results from the expected results given the UAPI exposed pipeline).
> >>>>
> >>>> The prime example of what I would like to know is that if a FB
> >>>> contains PQ-encoded image and I use a color pipeline to scale that
> >>>> image up, will the interpolation happen before or after the non-linear
> >>>> colorop that decodes PQ. That is a significant difference as pointed
> >>>> out by Joshua.
> >>>>    
> >>
> >> That's fair and I want to give that to you. My concern stems from
> >> the sentiment that I hear that any pipeline that doesn't explicitly
> >> advertise this is useless. I don't agree there. Let's not let perfect
> >> be the enemy of good.  
> > 
> > It's up to the use case. The policy of what is sufficient should reside
> > in userspace.
> > 
> > What about matching compositor shader composition with KMS?
> > 
> > Can we use that as a rough precision threshold? If userspace implements
> > the exact same color pipeline as the KMS UAPI describes, then that and
> > the KMS composited result should be indistinguishable in side-by-side
> > or alternating visual inspection on any monitor in isolation.
> > 
> > Did this whole effort not start from wanting to off-load things to
> > display hardware but still maintain visual equivalence to software/GPU
> > composition?
> >   
> 
> I agree with you and I want all that as well.
> 
> All I'm saying is that every userspace won't have the same policy of
> what is sufficient. Just because Weston has a very high threshold
> doesn't mean we can't move forward with a workable solution for other
> userspace, as long as we don't do something that prevents us from
> doing the perfect solution eventually.
> 
> And yes, I do want a solution that works for Weston and hear your
> comments on what that requires.

I totally agree.

How will that be reflected in the UAPI? If some pipelines are different
from others in correctness/strictness perspective, how will userspace
tell them apart?

Is the current proposal along the lines of: userspace creates a
software pipeline first, and if it cannot map all operations on it to
KMS color pipeline colorops, then the KMS pipeline is not sufficient?

The gist being, if the software pipeline is scaling the image for
example, then it must find a scaling colorop in the KMS pipeline if it
cares about the scaling correctness.

Another example is YUV pixel format on an FB that magically turns into
some kind of RGB when sampled, but there is no colorop to tell what
happens. I suppose userspace cannot assume that the lack of colorop
there means an identity YUV->RGB matrix either? How to model
that? I guess the already mentioned pixel format requirements on a
pipeline would help, making it impossible to use a pipeline without a
YUV->RGB colorop on a YUV FB unless the lack of colorop does indeed
mean an identity matrix.

The same with sub-sampled YUV which probably needs to always(?) be
expanded into fully sampled at the beginning of a pipeline? Chroma
siting.

This is in addition to the previously discussed userspace policy that
if a KMS color pipeline contains colorops the userspace does not
recognise, userspace probably should not pick that pipeline under any
conditions, because it might do something completely unexpected.

I think the above could work, but I feel it requires documenting several
examples like scaling that might not exist in the colorop definitions
at first. Otherwise particularly userspace developers might not come to
think of them, whether they care about those operations. I haven't read
the latest doc yet, so I'm not sure if it's already there.

There is still a gap though: what if the hardware does something
significant that is not modelled in the KMS pipeline with colorops? For
example, always using a hard-wired sRGB curve to decode before blending
and encode after blending. Or that cursor plane always uses the color
pipeline set on the primary plane. How to stop userspace from being
surprised?

Your comments sounded to me like you are letting go of the original
design goals. I'm happy to hear that's not the case. Even if you were,
that is a decision you can make since you are doing the work, and if I
knew you're doing that intentionally I would stop complaining.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Harry Wentland]

On 2023-11-09 04:20, Pekka Paalanen wrote:
> On Wed, 8 Nov 2023 11:27:35 -0500
> Harry Wentland <harry.wentland@amd.com> wrote:
> 
>> On 2023-11-08 11:19, Pekka Paalanen wrote:
>>> On Wed, 8 Nov 2023 09:31:17 -0500
>>> Harry Wentland <harry.wentland@amd.com> wrote:
>>>    
>>>> On 2023-11-08 06:40, Sebastian Wick wrote:
>>>>> On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:
> 
> ...
> 
>>>>>> An incremental UAPI development approach is fine by me, meaning that
>>>>>> pipelines might not be complete at first, but I believe that requires
>>>>>> telling userspace whether the driver developers consider the pipeline
>>>>>> complete (no undescribed operations that would significantly change
>>>>>> results from the expected results given the UAPI exposed pipeline).
>>>>>>
>>>>>> The prime example of what I would like to know is that if a FB
>>>>>> contains PQ-encoded image and I use a color pipeline to scale that
>>>>>> image up, will the interpolation happen before or after the non-linear
>>>>>> colorop that decodes PQ. That is a significant difference as pointed
>>>>>> out by Joshua.
>>>>>>     
>>>>
>>>> That's fair and I want to give that to you. My concern stems from
>>>> the sentiment that I hear that any pipeline that doesn't explicitly
>>>> advertise this is useless. I don't agree there. Let's not let perfect
>>>> be the enemy of good.
>>>
>>> It's up to the use case. The policy of what is sufficient should reside
>>> in userspace.
>>>
>>> What about matching compositor shader composition with KMS?
>>>
>>> Can we use that as a rough precision threshold? If userspace implements
>>> the exact same color pipeline as the KMS UAPI describes, then that and
>>> the KMS composited result should be indistinguishable in side-by-side
>>> or alternating visual inspection on any monitor in isolation.
>>>
>>> Did this whole effort not start from wanting to off-load things to
>>> display hardware but still maintain visual equivalence to software/GPU
>>> composition?
>>>    
>>
>> I agree with you and I want all that as well.
>>
>> All I'm saying is that every userspace won't have the same policy of
>> what is sufficient. Just because Weston has a very high threshold
>> doesn't mean we can't move forward with a workable solution for other
>> userspace, as long as we don't do something that prevents us from
>> doing the perfect solution eventually.
>>
>> And yes, I do want a solution that works for Weston and hear your
>> comments on what that requires.
> 
> I totally agree.
> 
> How will that be reflected in the UAPI? If some pipelines are different
> from others in correctness/strictness perspective, how will userspace
> tell them apart?
> 
> Is the current proposal along the lines of: userspace creates a
> software pipeline first, and if it cannot map all operations on it to
> KMS color pipeline colorops, then the KMS pipeline is not sufficient?
> 
> The gist being, if the software pipeline is scaling the image for
> example, then it must find a scaling colorop in the KMS pipeline if it
> cares about the scaling correctness.
> 

With a simplified model of an imaginary color pipeline I expect this
to look like this:

Color Pipeline 1:
   EOTF Curve - CTM

Color Pipeline 2:
   EOTF Curve - scale - CTM

Realistically both would most likely map to the same HW blocks.

Assuming userspace A and B do the following:
   EOTF Curve - scale - CTM

Userspace A doesn't care about scaling and would only look for:
   EOTF Curve - CTM

and find a match with Color Pipeline 1.

Userspace B cares about scaling and would look for
   EOTF Curve - scale - CTM

and find a match with Color Pipeline 2.

If Color Pipeline 2 is not exposed in the first iteration of the
driver's implementation userspace A would still be able to make
use of it, but userspace B would not, since it requires a defined
scale operation. If the driver then exposes Color Pipeline 2 in a
second iteration userspace B can find a match for what it needs
and make use of it.

Realistically userspace B would not attempt to implement a DRM/KMS
color pipeline implementation unless it knows that there's a driver
that can do what it needs.

> Another example is YUV pixel format on an FB that magically turns into
> some kind of RGB when sampled, but there is no colorop to tell what
> happens. I suppose userspace cannot assume that the lack of colorop
> there means an identity YUV->RGB matrix either? How to model
> that? I guess the already mentioned pixel format requirements on a
> pipeline would help, making it impossible to use a pipeline without a
> YUV->RGB colorop on a YUV FB unless the lack of colorop does indeed
> mean an identity matrix.
> 

I agree.

> The same with sub-sampled YUV which probably needs to always(?) be
> expanded into fully sampled at the beginning of a pipeline? Chroma
> siting.
> 
> This is in addition to the previously discussed userspace policy that
> if a KMS color pipeline contains colorops the userspace does not
> recognise, userspace probably should not pick that pipeline under any
> conditions, because it might do something completely unexpected.
> 

Unless those colorops can be put into bypass.

> I think the above could work, but I feel it requires documenting several
> examples like scaling that might not exist in the colorop definitions
> at first. Otherwise particularly userspace developers might not come to
> think of them, whether they care about those operations. I haven't read
> the latest doc yet, so I'm not sure if it's already there.
> 

True.

But I'm somewhat reluctant to define things that don't have an
implementation by a driver and an associated IGT test. I've seen
too many definitions (like the drm_connector Colorspace property)
that define a bunch of things but it's unclear whether they are
actually used. Once you have those you can't change their definition
either, even if they are wrong. And you might not find out they are
wrong until you try to implement support end-to-end.

The age-old chicken-and-egg dilemma. It's really problematic to
define things that haven't been validated end-to-end.

> There is still a gap though: what if the hardware does something
> significant that is not modelled in the KMS pipeline with colorops? For
> example, always using a hard-wired sRGB curve to decode before blending
> and encode after blending. Or that cursor plane always uses the color
> pipeline set on the primary plane. How to stop userspace from being
> surprised?
> 

Yeah, it shouldn't. Anything extra that's done should be modelled with
a colorop. But I might be somewhat contradicting myself here because
this would mean that we'd need to model scaling.

Cursors are funky on AMD and I need to think about them more (though
I've been saying that for years :D ). Maybe on AMD we might want a
custom colorop for cursors that basically says "this plane will inherit
colorops from the underlying plane".

> Your comments sounded to me like you are letting go of the original
> design goals. I'm happy to hear that's not the case. Even if you were,
> that is a decision you can make since you are doing the work, and if I
> knew you're doing that intentionally I would stop complaining.
> 

Apologies for the misunderstanding. I agree with the original design
goals but I'm also trying to find a minimal workable solution that
allows us to iterate and improve on it going forward.

Harry

> 
> Thanks,
> pq

Re: [RFC PATCH 01/10] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 9189 bytes --]

On Fri, 10 Nov 2023 15:27:03 -0500
Harry Wentland <harry.wentland@amd.com> wrote:

> On 2023-11-09 04:20, Pekka Paalanen wrote:
> > On Wed, 8 Nov 2023 11:27:35 -0500
> > Harry Wentland <harry.wentland@amd.com> wrote:
> >   
> >> On 2023-11-08 11:19, Pekka Paalanen wrote:  
> >>> On Wed, 8 Nov 2023 09:31:17 -0500
> >>> Harry Wentland <harry.wentland@amd.com> wrote:
> >>>      
> >>>> On 2023-11-08 06:40, Sebastian Wick wrote:  
> >>>>> On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:  
> > 
> > ...
> >   
> >>>>>> An incremental UAPI development approach is fine by me, meaning that
> >>>>>> pipelines might not be complete at first, but I believe that requires
> >>>>>> telling userspace whether the driver developers consider the pipeline
> >>>>>> complete (no undescribed operations that would significantly change
> >>>>>> results from the expected results given the UAPI exposed pipeline).
> >>>>>>
> >>>>>> The prime example of what I would like to know is that if a FB
> >>>>>> contains PQ-encoded image and I use a color pipeline to scale that
> >>>>>> image up, will the interpolation happen before or after the non-linear
> >>>>>> colorop that decodes PQ. That is a significant difference as pointed
> >>>>>> out by Joshua.
> >>>>>>       
> >>>>
> >>>> That's fair and I want to give that to you. My concern stems from
> >>>> the sentiment that I hear that any pipeline that doesn't explicitly
> >>>> advertise this is useless. I don't agree there. Let's not let perfect
> >>>> be the enemy of good.  
> >>>
> >>> It's up to the use case. The policy of what is sufficient should reside
> >>> in userspace.
> >>>
> >>> What about matching compositor shader composition with KMS?
> >>>
> >>> Can we use that as a rough precision threshold? If userspace implements
> >>> the exact same color pipeline as the KMS UAPI describes, then that and
> >>> the KMS composited result should be indistinguishable in side-by-side
> >>> or alternating visual inspection on any monitor in isolation.
> >>>
> >>> Did this whole effort not start from wanting to off-load things to
> >>> display hardware but still maintain visual equivalence to software/GPU
> >>> composition?
> >>>      
> >>
> >> I agree with you and I want all that as well.
> >>
> >> All I'm saying is that every userspace won't have the same policy of
> >> what is sufficient. Just because Weston has a very high threshold
> >> doesn't mean we can't move forward with a workable solution for other
> >> userspace, as long as we don't do something that prevents us from
> >> doing the perfect solution eventually.
> >>
> >> And yes, I do want a solution that works for Weston and hear your
> >> comments on what that requires.  
> > 
> > I totally agree.
> > 
> > How will that be reflected in the UAPI? If some pipelines are different
> > from others in correctness/strictness perspective, how will userspace
> > tell them apart?
> > 
> > Is the current proposal along the lines of: userspace creates a
> > software pipeline first, and if it cannot map all operations on it to
> > KMS color pipeline colorops, then the KMS pipeline is not sufficient?
> > 
> > The gist being, if the software pipeline is scaling the image for
> > example, then it must find a scaling colorop in the KMS pipeline if it
> > cares about the scaling correctness.
> >   
> 
> With a simplified model of an imaginary color pipeline I expect this
> to look like this:
> 
> Color Pipeline 1:
>    EOTF Curve - CTM
> 
> Color Pipeline 2:
>    EOTF Curve - scale - CTM
> 
> Realistically both would most likely map to the same HW blocks.
> 
> Assuming userspace A and B do the following:
>    EOTF Curve - scale - CTM
> 
> Userspace A doesn't care about scaling and would only look for:
>    EOTF Curve - CTM
> 
> and find a match with Color Pipeline 1.
> 
> Userspace B cares about scaling and would look for
>    EOTF Curve - scale - CTM
> 
> and find a match with Color Pipeline 2.
> 
> If Color Pipeline 2 is not exposed in the first iteration of the
> driver's implementation userspace A would still be able to make
> use of it, but userspace B would not, since it requires a defined
> scale operation. If the driver then exposes Color Pipeline 2 in a
> second iteration userspace B can find a match for what it needs
> and make use of it.
> 
> Realistically userspace B would not attempt to implement a DRM/KMS
> color pipeline implementation unless it knows that there's a driver
> that can do what it needs.
> 
> > Another example is YUV pixel format on an FB that magically turns into
> > some kind of RGB when sampled, but there is no colorop to tell what
> > happens. I suppose userspace cannot assume that the lack of colorop
> > there means an identity YUV->RGB matrix either? How to model
> > that? I guess the already mentioned pixel format requirements on a
> > pipeline would help, making it impossible to use a pipeline without a
> > YUV->RGB colorop on a YUV FB unless the lack of colorop does indeed
> > mean an identity matrix.
> >   
> 
> I agree.
> 
> > The same with sub-sampled YUV which probably needs to always(?) be
> > expanded into fully sampled at the beginning of a pipeline? Chroma
> > siting.
> > 
> > This is in addition to the previously discussed userspace policy that
> > if a KMS color pipeline contains colorops the userspace does not
> > recognise, userspace probably should not pick that pipeline under any
> > conditions, because it might do something completely unexpected.
> >   
> 
> Unless those colorops can be put into bypass.
> 
> > I think the above could work, but I feel it requires documenting several
> > examples like scaling that might not exist in the colorop definitions
> > at first. Otherwise particularly userspace developers might not come to
> > think of them, whether they care about those operations. I haven't read
> > the latest doc yet, so I'm not sure if it's already there.
> >   
> 
> True.
> 
> But I'm somewhat reluctant to define things that don't have an
> implementation by a driver and an associated IGT test.

Hi Harry,

you do not need to define those colorops.

All I'm asking for is that the documentation includes examples, e.g. if
you do scaling and you care about how and in which color encoding space
it is done, make sure the pipeline you pick explicitly describes that.
If one cannot find even definitions for such colorops, then obviously
they are not explicit.

People might not realise that there are more than one way to do each
thing. That goes for anything that is possible to program or even
automatically programmed without being explicit in a color pipeline,
like YUV<->RGB conversions.

Or is all this so obvious it does not need pointing out in docs?

> I've seen
> too many definitions (like the drm_connector Colorspace property)
> that define a bunch of things but it's unclear whether they are
> actually used. Once you have those you can't change their definition
> either, even if they are wrong. And you might not find out they are
> wrong until you try to implement support end-to-end.
> 
> The age-old chicken-and-egg dilemma. It's really problematic to
> define things that haven't been validated end-to-end.
> 
> > There is still a gap though: what if the hardware does something
> > significant that is not modelled in the KMS pipeline with colorops? For
> > example, always using a hard-wired sRGB curve to decode before blending
> > and encode after blending. Or that cursor plane always uses the color
> > pipeline set on the primary plane. How to stop userspace from being
> > surprised?
> >   
> 
> Yeah, it shouldn't. Anything extra that's done should be modelled with
> a colorop. But I might be somewhat contradicting myself here because
> this would mean that we'd need to model scaling.

Exactly. If unknowns need to be allowed at first, there should be some
indication to userspace when all unknowns in a pipeline have been
eliminated (either explicitly modelled, or guaranteed to bypass). Most
userspace won't care, but those who would need the new color pipeline
UAPI the most will.

> Cursors are funky on AMD and I need to think about them more (though
> I've been saying that for years :D ). Maybe on AMD we might want a
> custom colorop for cursors that basically says "this plane will inherit
> colorops from the underlying plane".

Sounds good to me.

Is it really any underlying plane at that point (pixel by pixel?) and
not the primary plane?

> > Your comments sounded to me like you are letting go of the original
> > design goals. I'm happy to hear that's not the case. Even if you were,
> > that is a decision you can make since you are doing the work, and if I
> > knew you're doing that intentionally I would stop complaining.
> >   
> 
> Apologies for the misunderstanding. I agree with the original design
> goals but I'm also trying to find a minimal workable solution that
> allows us to iterate and improve on it going forward.

That's all fine.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]