* [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel
Cc: Sebastian Wick, Pekka Paalanen, Shashank Sharma, wayland-devel,
Melissa Wen, Jonas Ådahl, Uma Shankar
Introduction
============
Modern hardwares have various color processing capabilities both
at pre-blending and post-blending phases in the color pipeline.
The current drm implementation exposes only the post-blending
color hardware blocks. Support for pre-blending hardware is missing.
There are multiple use cases where pre-blending color hardware will
be useful:
a) Linearization of input buffers encoded in various transfer
functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Luts
and other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware
to user-space. This will help userspace/middleware to use display
hardware for color processing and blending instead of doing it through
GPU shaders.
Work done so far and relevant references
========================================
Some implementation is done by Intel and AMD/Igalia to address the same.
Broad consensus is there that we need a generic API at drm core to suffice
the use case of various HW vendors. Below are the links capturing the
discussion so far.
Intel's Plane Color Implementation: https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation: https://patchwork.freedesktop.org/series/116862/
Hackfest conclusions
====================
HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders
together and converge on a common uapi expectations. Participants from Intel, AMD,
Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and
maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple
hardware vendors, challenges in generalizing the same and the path forward. Consensus
was made that drm core should implement descriptive APIs and not go with prescriptive
APIs. DRM core should just expose the hardware capabilities; enabling, customizing and
programming the same should be done by the user-space. Driver should just honor
the user space request without doing any operations internally.
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC
which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
Design considerations
=====================
Following are the important aspects taken into account while designing the current RFC
proposal:
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HW blocks.
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive implementation if required.
10.Scope to handle any modification in hardware as technology evolves
The current proposal takes into account the above considerations while keeping the implementation
as generic as possible leaving scope for future additions or modifications.
This proposal is also in line to the details mentioned by Simon's RFC covering all
the aspects discussed in hackfest.
Outline of the implementation
============================
Each Color Hardware block will be represented by a data structure drm_color_op.
These color operations will form the building blocks of a color pipeline which
best represents the underlying Hardware. Color operations can be re-arranged,
substracted or added to create distinct color pipelines to accurately describe
the Hardware blocks present in the display engine.
In this proposal, a color pipeline is represented as an array of
struct drm_color_op. For individual color operation, we add blobs to advertise
the capability of the particular Hardware block.
This color pipeline is then packaged within a blob for the user space to
retrieve it.
To advertise the available color pipelines, an immutable ENUM property
"GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
With each blob id representing a distinct color pipeline based on underlying HW
capabilities and their respective combinations.
Once the user space decides on a color pipeline, it can set the pipeline and
the corresponding data for the hardware blocks within the pipeline with
the BLOB property "SET_COLOR_PIPELINE".
Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the patch
IGT and test details
====================
A set of IGT tests is written to demonstrate the usage of the proposed UAPIs
along with some sanity validation.
Details of the IGT test can be found here:
https://patchwork.freedesktop.org/series/123018/
Opens
=====
a. To come up with a list of common HW blocks which can be defined generically by the DRM
core in agreement with all the stakeholders
b. To enhance/finalize the data structure to define segmented LUTs generically.
Out of Scope
============
a. The coefficients for CTM and LUT value calculations are out of scope of the proposal.
b. The onus of programming the HW blocks and their values is on user-space. Driver will
just provide the interface for the same.
c. In order to compute LUT values and coefficients, a helper library can be created in
user-space. However, it is out of scope for the current proposal.
Acknowledgements and credits
============================
There are multiple contributors who have helped us to reach to this proposal. Special mention
to Ville Syrjala<ville.syrjala@linux.intel.com>, Pekka Paalanen<pekka.paalanen@collabora.com>,
Simon Ser<contact@emersion.fr>, Harry Wentland<harry.wentland@amd.com>,
Melissa Wen<mwen@igalia.com>, Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
Bhanu<bhanuprakash.modem@intel.com> and Shashank<shashank.sharma@amd.com>.
Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for organizing the HDR hackfest.
UAPI dependency and Usermode development
========================================
The current KMS implementation requires a user space consumer for it to be accepted upstream.
Work is ongoing in weston and mutter community to get color management and HDR support implemented
in the respective stacks.
=================================================================================
We have tried to take care of all the scenarios and use-cases which possibly could exists in the
current proposal. Thanks to everyone who has contributed in all color management discussions so
far. Let's work together to improve the current proposal and get this thing implemented in
upstream linux. All the feedback and suggestions to enhance the design are welcome.
Regards,
Uma Shankar
Chaitanya Kumar Borah
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Chaitanya Kumar Borah (14):
drm: Add color operation structure
drm: Add plane get color pipeline property
drm: Add helper to add color pipeline
drm: Manage color blob states
drm: Replace individual color blobs
drm: Reset pipeline when user sends NULL blob
drm: Reset plane color state on pipeline switch request
drm/i915/color: Add HDR plane LUT range data to color pipeline
drm/i915/color: Add SDR plane LUT range data to color pipeline
drm/i915/color: Add color pipelines to plane
drm/i915/color: Create and attach set color pipeline property
drm/i915/color: Enable plane color features
drm/i915/color: Add a dummy pipeline with 3D LUT
drm/i915/color: Add example implementation for vendor specific color
operation
Uma Shankar (19):
drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
drm: Add structures for setting color pipeline
drm: Add set colorpipeline property
drm: Add Enhanced Gamma LUT precision structure
drm: Add color lut range structure
drm: Add color information to plane state
drm/i915/color: Add lut range for SDR planes
drm/i915/color: Add lut range for HDR planes
drm/i915/color: Add color pipeline for HDR planes
drm/i915/color: Add color pipeline for SDR planes
drm/i915/color: Add plane color callbacks
drm/i915/color: Load plane color luts from atomic flip
drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
drm/i915/xelpd: Add register definitions for Plane Degamma
drm/i915/color: Add color functions for ADL
drm/i915/color: Program Plane Pre-CSC Registers
drm/i915/xelpd: Add register definitions for Plane Post CSC
drm/i915/xelpd: Program Plane Post CSC Registers
drm/i915/color: Enable Plane CSC
.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
drivers/gpu/drm/i915/display/intel_color.h | 7 +-
.../drm/i915/display/skl_universal_plane.c | 21 +-
drivers/gpu/drm/i915/i915_reg.h | 124 ++++
include/drm/drm_plane.h | 82 +++
include/uapi/drm/drm_mode.h | 196 +++++
include/uapi/drm/i915_drm.h | 25 +
11 files changed, 1899 insertions(+), 3 deletions(-)
create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
--
2.38.1
^ permalink raw reply [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel
Cc: Sebastian Wick, Pekka Paalanen, Simon Ser, Alexander Goins,
wayland-devel, Melissa Wen, Jonas Ådahl, Harry Wentland
Introduction
============
Modern hardwares have various color processing capabilities both
at pre-blending and post-blending phases in the color pipeline.
The current drm implementation exposes only the post-blending
color hardware blocks. Support for pre-blending hardware is missing.
There are multiple use cases where pre-blending color hardware will
be useful:
a) Linearization of input buffers encoded in various transfer
functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Luts
and other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware
to user-space. This will help userspace/middleware to use display
hardware for color processing and blending instead of doing it through
GPU shaders.
Work done so far and relevant references
========================================
Some implementation is done by Intel and AMD/Igalia to address the same.
Broad consensus is there that we need a generic API at drm core to suffice
the use case of various HW vendors. Below are the links capturing the
discussion so far.
Intel's Plane Color Implementation: https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation: https://patchwork.freedesktop.org/series/116862/
Hackfest conclusions
====================
HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders
together and converge on a common uapi expectations. Participants from Intel, AMD,
Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and
maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple
hardware vendors, challenges in generalizing the same and the path forward. Consensus
was made that drm core should implement descriptive APIs and not go with prescriptive
APIs. DRM core should just expose the hardware capabilities; enabling, customizing and
programming the same should be done by the user-space. Driver should just honor
the user space request without doing any operations internally.
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC
which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
Design considerations
=====================
Following are the important aspects taken into account while designing the current RFC
proposal:
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HW blocks.
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive implementation if required.
10.Scope to handle any modification in hardware as technology evolves
The current proposal takes into account the above considerations while keeping the implementation
as generic as possible leaving scope for future additions or modifications.
This proposal is also in line to the details mentioned by Simon's RFC covering all
the aspects discussed in hackfest.
Outline of the implementation
============================
Each Color Hardware block will be represented by a data structure drm_color_op.
These color operations will form the building blocks of a color pipeline which
best represents the underlying Hardware. Color operations can be re-arranged,
substracted or added to create distinct color pipelines to accurately describe
the Hardware blocks present in the display engine.
In this proposal, a color pipeline is represented as an array of
struct drm_color_op. For individual color operation, we add blobs to advertise
the capability of the particular Hardware block.
This color pipeline is then packaged within a blob for the user space to
retrieve it.
To advertise the available color pipelines, an immutable ENUM property
"GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
With each blob id representing a distinct color pipeline based on underlying HW
capabilities and their respective combinations.
Once the user space decides on a color pipeline, it can set the pipeline and
the corresponding data for the hardware blocks within the pipeline with
the BLOB property "SET_COLOR_PIPELINE".
Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the patch
IGT and test details
====================
A set of IGT tests is written to demonstrate the usage of the proposed UAPIs
along with some sanity validation.
Details of the IGT test can be found here:
https://patchwork.freedesktop.org/series/123018/
Opens
=====
a. To come up with a list of common HW blocks which can be defined generically by the DRM
core in agreement with all the stakeholders
b. To enhance/finalize the data structure to define segmented LUTs generically.
Out of Scope
============
a. The coefficients for CTM and LUT value calculations are out of scope of the proposal.
b. The onus of programming the HW blocks and their values is on user-space. Driver will
just provide the interface for the same.
c. In order to compute LUT values and coefficients, a helper library can be created in
user-space. However, it is out of scope for the current proposal.
Acknowledgements and credits
============================
There are multiple contributors who have helped us to reach to this proposal. Special mention
to Ville Syrjala<ville.syrjala@linux.intel.com>, Pekka Paalanen<pekka.paalanen@collabora.com>,
Simon Ser<contact@emersion.fr>, Harry Wentland<harry.wentland@amd.com>,
Melissa Wen<mwen@igalia.com>, Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
Bhanu<bhanuprakash.modem@intel.com> and Shashank<shashank.sharma@amd.com>.
Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for organizing the HDR hackfest.
UAPI dependency and Usermode development
========================================
The current KMS implementation requires a user space consumer for it to be accepted upstream.
Work is ongoing in weston and mutter community to get color management and HDR support implemented
in the respective stacks.
=================================================================================
We have tried to take care of all the scenarios and use-cases which possibly could exists in the
current proposal. Thanks to everyone who has contributed in all color management discussions so
far. Let's work together to improve the current proposal and get this thing implemented in
upstream linux. All the feedback and suggestions to enhance the design are welcome.
Regards,
Uma Shankar
Chaitanya Kumar Borah
Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Melissa Wen <mwen@igalia.com>
Cc: Jonas Ådahl <jadahl@redhat.com>
Cc: Sebastian Wick <sebastian.wick@redhat.com>
Cc: Shashank Sharma <shashank.sharma@amd.com>
Cc: Alexander Goins <agoins@nvidia.com>
Chaitanya Kumar Borah (14):
drm: Add color operation structure
drm: Add plane get color pipeline property
drm: Add helper to add color pipeline
drm: Manage color blob states
drm: Replace individual color blobs
drm: Reset pipeline when user sends NULL blob
drm: Reset plane color state on pipeline switch request
drm/i915/color: Add HDR plane LUT range data to color pipeline
drm/i915/color: Add SDR plane LUT range data to color pipeline
drm/i915/color: Add color pipelines to plane
drm/i915/color: Create and attach set color pipeline property
drm/i915/color: Enable plane color features
drm/i915/color: Add a dummy pipeline with 3D LUT
drm/i915/color: Add example implementation for vendor specific color
operation
Uma Shankar (19):
drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
drm: Add structures for setting color pipeline
drm: Add set colorpipeline property
drm: Add Enhanced Gamma LUT precision structure
drm: Add color lut range structure
drm: Add color information to plane state
drm/i915/color: Add lut range for SDR planes
drm/i915/color: Add lut range for HDR planes
drm/i915/color: Add color pipeline for HDR planes
drm/i915/color: Add color pipeline for SDR planes
drm/i915/color: Add plane color callbacks
drm/i915/color: Load plane color luts from atomic flip
drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
drm/i915/xelpd: Add register definitions for Plane Degamma
drm/i915/color: Add color functions for ADL
drm/i915/color: Program Plane Pre-CSC Registers
drm/i915/xelpd: Add register definitions for Plane Post CSC
drm/i915/xelpd: Program Plane Post CSC Registers
drm/i915/color: Enable Plane CSC
.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
drivers/gpu/drm/i915/display/intel_color.h | 7 +-
.../drm/i915/display/skl_universal_plane.c | 21 +-
drivers/gpu/drm/i915/i915_reg.h | 124 ++++
include/drm/drm_plane.h | 82 +++
include/uapi/drm/drm_mode.h | 196 +++++
include/uapi/drm/i915_drm.h | 25 +
11 files changed, 1899 insertions(+), 3 deletions(-)
create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
--
2.38.1
^ permalink raw reply [flat|nested] 109+ messages in thread
* [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
1 file changed, 394 insertions(+)
create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst
new file mode 100644
index 000000000000..60ce515b6ea7
--- /dev/null
+++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
@@ -0,0 +1,394 @@
+=======================================
+ Plane Color Pipeline: A UAPI proposal
+=======================================
+
+To build the proposal on, lets take the premise of a color pipeline as shown
+below.
+
+ +-------------------------------------------+
+ | RAM |
+ | +------+ +---------+ +---------+ |
+ | | FB 1 | | FB 2 | | FB N | |
+ | +------+ +---------+ +---------+ |
+ +-------------------------------------------+
+ | Plane Color Hardware Block |
+ +--------------------------------------------+
+ | +---v-----+ +---v-------+ +---v------+ |
+ | | Plane A | | Plane B | | Plane N | |
+ | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
+ | +---+-----+ +---+-------+ +---+------+ |
+ | | | | |
+ | +---v-----+ +---v-------+ +---v------+ |
+ | |Plane A | | Plane B | | Plane N | |
+ | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
+ | +---+-----+ +----+------+ +----+-----+ |
+ | | | | |
+ | +---v-----+ +----v------+ +----v-----+ |
+ | | Plane A | | Plane B | | Plane N | |
+ | |Post-CSC | | Post-CSC | | Post-CSC | |
+ | +---+-----+ +----+------+ +----+-----+ |
+ | | | | |
+ +--------------------------------------------+
++------v--------------v---------------v-------|
+|| ||
+|| Pipe Blender ||
++--------------------+------------------------+
+| | |
+| +-----------v----------+ |
+| | Pipe Pre-CSC | |
+| | | |
+| +-----------+----------+ |
+| | Pipe Color |
+| +-----------v----------+ Hardware |
+| | Pipe CSC/CTM | |
+| | | |
+| +-----------+----------+ |
+| | |
+| +-----------v----------+ |
+| | Pipe Post-CSC | |
+| | | |
+| +-----------+----------+ |
+| | |
++---------------------------------------------+
+ |
+ v
+ Pipe Output
+
+Each plane consists of the following color blocks
+ * Pre-CSC : This block can used to linearize the input frame buffer data.
+ The linear data then can be further acted on by the following
+ color hardware blocks in the display hardware pipeline
+
+ * CSC/CTM: Used to program color transformation matrix, this block is used
+ to perform color space conversions like BT2020 to BT709 or BT601
+ etc. This block acts on the linearized data coming from the
+ Pre-CSC HW block.
+
+ * Post-CSC: This HW block can be used to non-linearize frame buffer data to
+ match the sink. Another use case of it could be to perform Tone
+ mapping for HDR use-cases.
+
+Data from multiple planes will then be fed to pipe/crtc where it will get blended.
+There is a similar set of HW blocks available at pipe/crtc level which acts on
+this blended data.
+
+Below is a sample usecase fo video playback with sub-titles and playback
+controls
+
+┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
+│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
+│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
+│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
+└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
+(subtitles) │
+ │
+┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
+│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
+│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
+│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
+└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
+(Playback controls UI) │ │
+ │ │
+┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
+│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
+│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
+│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
+└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │
+(video frame) │ │ │
+ │ │ │
+┌────────────────────────────────────────────────────────────────────────┴─┴─┘
+│
+│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
+│ │ CRTC │ │ CRTC │ │ CRTC │
+└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port
+ │ │ │ │ │ │
+ └─────────────┘ └─────────────┘ └───────────────┘
+
+This RFC is intended to propose an uAPI for the pre-blending color pipeline
+(however, can be also extended to post blending pipeline).
+
+Below are the design considerations while designing the uAPI.
+
+1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
+2. Position of the HW block can be programmable
+3. LUTs can be one dimentional or three dimentional
+4. Number of LUT entries can vary across platforms
+5. Precision of LUT entries can vary across platforms
+6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
+ Piece-Wise Linear(PWL) etc
+7. There can be parameterized/non-parameterized fixed function HW blocks.
+ e.g. Just a hardware bit, to convert from one color space to another.
+
+Plane Color Pipeline: Design Proposal
+=====================================
+Each Color Hardware block will be represented by the structure drm_color_op.
+
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ u32 blob_id;
+ u32 private_flags;
+};
+
+The members of which will constitute:
+1. name: A standardised enum for the color hardware block
+2. type: The type of mathematical operation done by the hardware block.
+ e.g. 1D Curve, 3D Curve, Matrix etc.
+3. blob id: Id pointing to a blob containing information about the hardware
+ block advertising the respective capabilities to the userspace.
+ It can be an optional field depending on the members "name" and "type".
+4. private_flags: This can be used to provide vendor specific hints
+ to user space
+
+
+ For example to represent LUTs, we introduce the drm_color_lut_range
+ structure. It can represent LUTs with varied number of entries and
+ distributions (Multi segmented, Logarithmic etc).
+
+ struct drm_color_lut_range {
+ /* DRM_MODE_LUT_* */
+ __u32 flags;
+ /* number of points on the curve */
+ __u16 count;
+ /* input/output bits per component */
+ __u8 input_bpc, output_bpc;
+ /* input start/end values */
+ __s32 start, end;
+ /* output min/max values */
+ __s32 min, max;
+ };
+
+Note: More details on exact usage and implementation of this structure can be
+ found in the RFC. This structure is taken as is from the series.
+ https://patchwork.freedesktop.org/series/90825/
+ However, we can add more members to it to encompass all use-cases.
+ For example. we can add a precision field to advertise the
+ bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
+ field for vendor specific use cases.
+
+ At the same time, we don't need to pass any additional information for the
+ CSC block as userspace and driver already agrees struct drm_color_ctm as
+ a uAPI to pass on data.
+
+These color operations will form the building blocks of a color pipeline which
+best represents the underlying Hardware. Color operations can be re-arranged,
+substracted or added to create distinct color pipelines to accurately describe
+the Hardware blocks present in the display engine.
+
+In this proposal, a color pipeline is represented as an array of
+struct drm_color_op.
+
+struct drm_color_op color_pipeline_1[]
+
+For example to represent the pre-blending color pipeline as described above
+
+We can create a color pipeline like this.
+
+struct drm_color_op color_pipeline_1[] = {
+ {
+ .name = DRM_CB_PRE_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0; /* actual value to be populated during plane
+ initialization*/
+ },
+ {
+ .name = DRM_CB_CSC,
+ .type = MATRIX,
+ .blob_id = 0;
+ },
+ {
+ .name = DRM_CB_POST_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0;
+ },
+};
+
+Then, for individual color operation, we add blobs to advertise the capability
+of the particular Hardware block. In case of the example pipeline, we add
+blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
+For the "csc" block we pass no blob id to user space as it is known to both
+user space and driver that drm_color_ctm structure is to be used for such
+operation.
+
+To represent, this in a diagram.
+
+ struct drm_color_op color_pipeline_1[]
+ +---------------------------+
+ | | drm_color_op
+ | +---------------------+--+-----------+---------------------+
+ | | | | | |
+ | | | | | +-----------------+ |
+ | | | | | | name | |
+ | | | | | +-----------------+ |
+ | | | | | | type | |
+ | | color_op_1 | | | +-----------------+ |
+ | | | | | | blob id | +--------+
+ | | | | | +-----------------+ | |
+ | | | | | | private | | |
+ | | | | | | flags | | |
+ | | | | | +-----------------+ | |
+ | | | | | | |
+ | +---------------------+--+-----------+---------------------+ |
+ | | |
+ | | |
+ | +---------------------+ | |
+ | | | | drm_color_lut_range |
+ | | color_op_2 | | +-------------------------+ |
+ | | | | | | |
+ | | | | | +---------------------+ | |
+ | +---------------------+ | | | segment 1 { | |<---+
+ | | | | ... | |
+ | +---------------------+ | | | .input_bpc = 16, | |
+ | | | | | | .output_bpc = 16, | |
+ | | color_op_3 | | | | ... | |
+ | | | | | | } | |
+ | | | | | +---------------------+ |
+ | +---------------------+ | | |
+ | . | | +---------------------+ |
+ | . | | | segment 2 { | |
+ | . | | | ... | |
+ +---------------------------+ | | } | |
+ | | | |
+ | | | |
+ | | | |
+ | +---------------------+ |
+ | . |
+ | . |
+ | . |
+ +-------------------------+
+
+
+
+This color pipeline is then packaged within a blob for the user space to
+retrieve it. Details can be found in the next section
+
+Exposing a color pipeline to user space
+=======================================
+
+To advertise the available color pipelines, an immutable ENUM property
+"GET_COLOR_PIPELINE" is introduced.
+This enum property has blob id's as values. With each blob id representing
+a distinct color pipeline based on underlying HW capabilities and their
+respective combinations.
+
+The following output of drm_info [1], shows how color pipelines are visible
+to userspace.
+
+├───Plane 0
+ │ ├───Object ID: 31
+ │ ├───CRTCs: {0}
+ │ ├───Legacy info
+ ...
+ │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
+ color pipeline 1, color pipeline 2}= no color pipeline
+
+To understand the capabilities of individual pipelines, first the userspace
+need to retrieve the pipeline blob with the blob ids retrieved by reading the
+enum property.
+
+Once the color pipeline is retrieved, user can then parse through
+individual drm_color_op blocks to understand the capabilities of each
+hardware block.
+
+Check IGT series to see how user space can parse through color pipelines.
+Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
+
+Setting up a color pipeline
+===========================
+
+Once the user space decides on a color pipeline, it can set the pipeline and
+the corresponding data for the hardware blocks within the pipeline with
+the BLOB property "SET_COLOR_PIPELINE".
+
+To achieve this two structures are introduced
+
+1. struct drm_color_op_data: It represents data to be passed onto individual
+ color hardware blocks. It contains three members
+ a) name: to identify the color operation block
+ b) blob_id: pointing to the blob with data for the
+ corresponding hardware block
+
+ struct drm_color_op_data {
+ enum color_op_block name;
+ u32 blob_id;
+ };
+
+2. struct drm_color_pipeline: This structure represents the aggregate
+ pipeline to be set. it contains the following members
+ a) num: pipeline number to be selected
+ b) size: size of the data to be passed onto the driver
+ c) data: array of struct drm_color_op_data with data for
+ the hardware block/s that userspace wants to
+ set values for.
+
+ struct drm_color_pipeline {
+ int num;
+ int size;
+ struct drm_color_op_data *data;
+ };
+
+ User can either:
+ 1. send data for all the color operations in a pipeline as shown in [2].
+ The color operation data need not be in order that the pipeline advertises
+ however, it should not contain data for any
+ color operation that is not present in the pipeline.
+
+ Note: This check for pipeline is not yet implemented but if the
+ wider proposal is accepted we have few solutions in mind.
+
+ 2. send data for a subset of color operations as shown in [3].For the
+ color operation that userspace does not send data will retain their
+ older state.
+
+ 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
+ property to NULL as shown in both [2] and [3]
+
+ 4. change the color pipeline as demonstrated in [3].
+ On the new pipeline, the user is expected to setup all the color hardware block
+ Once the user requests a pipeline change, the driver will provide it a clean slate
+ which means that all the data previously set by the user will be discarded even if
+ there are common hardware blocks between the two(previous and current) pipelines.
+
+IGT implementation can be found here [4]
+
+Representing Fixed function hardware
+====================================
+
+To provide support for fixed function hardware, the driver could expose vendor
+specific struct drm_color_op with parameters that both the userspace and
+driver agrees on. To demonstrate, let's consider a hyphothetical fixed
+function hardware block that converts BT601 to BT2020.
+The driver can choose to advertise the block as such.
+
+struct drm_color_op color_pipeline_X[] = {
+ {
+ .name = DRM_CB_PRIVATE,
+ .type = FIXED_FUNCTION,
+ .blob_id = 45;
+ },
+}
+
+Where the blob represents some vendor specific enums, strings or any other
+appropriate data types which both the user-space and drivers are aligned on.
+
+blob:45 {
+ VENDORXXX_BT602_TO_BT2020,
+};
+
+For enabling or passing parameters to such blocks, the user can send data
+to the driver wrapped within a blob like any other color operation block.
+
+ struct drm_color_op_data {
+ .name = DRM_CB_PRIVATE;
+ .blob_id = 46;
+ } ;
+
+where blob with id 46 contains data to enable the fixed function hardware(s).
+
+References
+==========
+
+[1] https://gitlab.freedesktop.org/emersion/drm_info
+[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
+[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
+[4] https://patchwork.freedesktop.org/series/123018/
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
1 file changed, 394 insertions(+)
create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst
new file mode 100644
index 000000000000..60ce515b6ea7
--- /dev/null
+++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
@@ -0,0 +1,394 @@
+=======================================
+ Plane Color Pipeline: A UAPI proposal
+=======================================
+
+To build the proposal on, lets take the premise of a color pipeline as shown
+below.
+
+ +-------------------------------------------+
+ | RAM |
+ | +------+ +---------+ +---------+ |
+ | | FB 1 | | FB 2 | | FB N | |
+ | +------+ +---------+ +---------+ |
+ +-------------------------------------------+
+ | Plane Color Hardware Block |
+ +--------------------------------------------+
+ | +---v-----+ +---v-------+ +---v------+ |
+ | | Plane A | | Plane B | | Plane N | |
+ | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
+ | +---+-----+ +---+-------+ +---+------+ |
+ | | | | |
+ | +---v-----+ +---v-------+ +---v------+ |
+ | |Plane A | | Plane B | | Plane N | |
+ | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
+ | +---+-----+ +----+------+ +----+-----+ |
+ | | | | |
+ | +---v-----+ +----v------+ +----v-----+ |
+ | | Plane A | | Plane B | | Plane N | |
+ | |Post-CSC | | Post-CSC | | Post-CSC | |
+ | +---+-----+ +----+------+ +----+-----+ |
+ | | | | |
+ +--------------------------------------------+
++------v--------------v---------------v-------|
+|| ||
+|| Pipe Blender ||
++--------------------+------------------------+
+| | |
+| +-----------v----------+ |
+| | Pipe Pre-CSC | |
+| | | |
+| +-----------+----------+ |
+| | Pipe Color |
+| +-----------v----------+ Hardware |
+| | Pipe CSC/CTM | |
+| | | |
+| +-----------+----------+ |
+| | |
+| +-----------v----------+ |
+| | Pipe Post-CSC | |
+| | | |
+| +-----------+----------+ |
+| | |
++---------------------------------------------+
+ |
+ v
+ Pipe Output
+
+Each plane consists of the following color blocks
+ * Pre-CSC : This block can used to linearize the input frame buffer data.
+ The linear data then can be further acted on by the following
+ color hardware blocks in the display hardware pipeline
+
+ * CSC/CTM: Used to program color transformation matrix, this block is used
+ to perform color space conversions like BT2020 to BT709 or BT601
+ etc. This block acts on the linearized data coming from the
+ Pre-CSC HW block.
+
+ * Post-CSC: This HW block can be used to non-linearize frame buffer data to
+ match the sink. Another use case of it could be to perform Tone
+ mapping for HDR use-cases.
+
+Data from multiple planes will then be fed to pipe/crtc where it will get blended.
+There is a similar set of HW blocks available at pipe/crtc level which acts on
+this blended data.
+
+Below is a sample usecase fo video playback with sub-titles and playback
+controls
+
+┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
+│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
+│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
+│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
+└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
+(subtitles) │
+ │
+┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
+│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
+│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
+│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
+└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
+(Playback controls UI) │ │
+ │ │
+┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
+│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
+│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
+│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
+└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │
+(video frame) │ │ │
+ │ │ │
+┌────────────────────────────────────────────────────────────────────────┴─┴─┘
+│
+│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
+│ │ CRTC │ │ CRTC │ │ CRTC │
+└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port
+ │ │ │ │ │ │
+ └─────────────┘ └─────────────┘ └───────────────┘
+
+This RFC is intended to propose an uAPI for the pre-blending color pipeline
+(however, can be also extended to post blending pipeline).
+
+Below are the design considerations while designing the uAPI.
+
+1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
+2. Position of the HW block can be programmable
+3. LUTs can be one dimentional or three dimentional
+4. Number of LUT entries can vary across platforms
+5. Precision of LUT entries can vary across platforms
+6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
+ Piece-Wise Linear(PWL) etc
+7. There can be parameterized/non-parameterized fixed function HW blocks.
+ e.g. Just a hardware bit, to convert from one color space to another.
+
+Plane Color Pipeline: Design Proposal
+=====================================
+Each Color Hardware block will be represented by the structure drm_color_op.
+
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ u32 blob_id;
+ u32 private_flags;
+};
+
+The members of which will constitute:
+1. name: A standardised enum for the color hardware block
+2. type: The type of mathematical operation done by the hardware block.
+ e.g. 1D Curve, 3D Curve, Matrix etc.
+3. blob id: Id pointing to a blob containing information about the hardware
+ block advertising the respective capabilities to the userspace.
+ It can be an optional field depending on the members "name" and "type".
+4. private_flags: This can be used to provide vendor specific hints
+ to user space
+
+
+ For example to represent LUTs, we introduce the drm_color_lut_range
+ structure. It can represent LUTs with varied number of entries and
+ distributions (Multi segmented, Logarithmic etc).
+
+ struct drm_color_lut_range {
+ /* DRM_MODE_LUT_* */
+ __u32 flags;
+ /* number of points on the curve */
+ __u16 count;
+ /* input/output bits per component */
+ __u8 input_bpc, output_bpc;
+ /* input start/end values */
+ __s32 start, end;
+ /* output min/max values */
+ __s32 min, max;
+ };
+
+Note: More details on exact usage and implementation of this structure can be
+ found in the RFC. This structure is taken as is from the series.
+ https://patchwork.freedesktop.org/series/90825/
+ However, we can add more members to it to encompass all use-cases.
+ For example. we can add a precision field to advertise the
+ bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
+ field for vendor specific use cases.
+
+ At the same time, we don't need to pass any additional information for the
+ CSC block as userspace and driver already agrees struct drm_color_ctm as
+ a uAPI to pass on data.
+
+These color operations will form the building blocks of a color pipeline which
+best represents the underlying Hardware. Color operations can be re-arranged,
+substracted or added to create distinct color pipelines to accurately describe
+the Hardware blocks present in the display engine.
+
+In this proposal, a color pipeline is represented as an array of
+struct drm_color_op.
+
+struct drm_color_op color_pipeline_1[]
+
+For example to represent the pre-blending color pipeline as described above
+
+We can create a color pipeline like this.
+
+struct drm_color_op color_pipeline_1[] = {
+ {
+ .name = DRM_CB_PRE_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0; /* actual value to be populated during plane
+ initialization*/
+ },
+ {
+ .name = DRM_CB_CSC,
+ .type = MATRIX,
+ .blob_id = 0;
+ },
+ {
+ .name = DRM_CB_POST_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0;
+ },
+};
+
+Then, for individual color operation, we add blobs to advertise the capability
+of the particular Hardware block. In case of the example pipeline, we add
+blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
+For the "csc" block we pass no blob id to user space as it is known to both
+user space and driver that drm_color_ctm structure is to be used for such
+operation.
+
+To represent, this in a diagram.
+
+ struct drm_color_op color_pipeline_1[]
+ +---------------------------+
+ | | drm_color_op
+ | +---------------------+--+-----------+---------------------+
+ | | | | | |
+ | | | | | +-----------------+ |
+ | | | | | | name | |
+ | | | | | +-----------------+ |
+ | | | | | | type | |
+ | | color_op_1 | | | +-----------------+ |
+ | | | | | | blob id | +--------+
+ | | | | | +-----------------+ | |
+ | | | | | | private | | |
+ | | | | | | flags | | |
+ | | | | | +-----------------+ | |
+ | | | | | | |
+ | +---------------------+--+-----------+---------------------+ |
+ | | |
+ | | |
+ | +---------------------+ | |
+ | | | | drm_color_lut_range |
+ | | color_op_2 | | +-------------------------+ |
+ | | | | | | |
+ | | | | | +---------------------+ | |
+ | +---------------------+ | | | segment 1 { | |<---+
+ | | | | ... | |
+ | +---------------------+ | | | .input_bpc = 16, | |
+ | | | | | | .output_bpc = 16, | |
+ | | color_op_3 | | | | ... | |
+ | | | | | | } | |
+ | | | | | +---------------------+ |
+ | +---------------------+ | | |
+ | . | | +---------------------+ |
+ | . | | | segment 2 { | |
+ | . | | | ... | |
+ +---------------------------+ | | } | |
+ | | | |
+ | | | |
+ | | | |
+ | +---------------------+ |
+ | . |
+ | . |
+ | . |
+ +-------------------------+
+
+
+
+This color pipeline is then packaged within a blob for the user space to
+retrieve it. Details can be found in the next section
+
+Exposing a color pipeline to user space
+=======================================
+
+To advertise the available color pipelines, an immutable ENUM property
+"GET_COLOR_PIPELINE" is introduced.
+This enum property has blob id's as values. With each blob id representing
+a distinct color pipeline based on underlying HW capabilities and their
+respective combinations.
+
+The following output of drm_info [1], shows how color pipelines are visible
+to userspace.
+
+├───Plane 0
+ │ ├───Object ID: 31
+ │ ├───CRTCs: {0}
+ │ ├───Legacy info
+ ...
+ │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
+ color pipeline 1, color pipeline 2}= no color pipeline
+
+To understand the capabilities of individual pipelines, first the userspace
+need to retrieve the pipeline blob with the blob ids retrieved by reading the
+enum property.
+
+Once the color pipeline is retrieved, user can then parse through
+individual drm_color_op blocks to understand the capabilities of each
+hardware block.
+
+Check IGT series to see how user space can parse through color pipelines.
+Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
+
+Setting up a color pipeline
+===========================
+
+Once the user space decides on a color pipeline, it can set the pipeline and
+the corresponding data for the hardware blocks within the pipeline with
+the BLOB property "SET_COLOR_PIPELINE".
+
+To achieve this two structures are introduced
+
+1. struct drm_color_op_data: It represents data to be passed onto individual
+ color hardware blocks. It contains three members
+ a) name: to identify the color operation block
+ b) blob_id: pointing to the blob with data for the
+ corresponding hardware block
+
+ struct drm_color_op_data {
+ enum color_op_block name;
+ u32 blob_id;
+ };
+
+2. struct drm_color_pipeline: This structure represents the aggregate
+ pipeline to be set. it contains the following members
+ a) num: pipeline number to be selected
+ b) size: size of the data to be passed onto the driver
+ c) data: array of struct drm_color_op_data with data for
+ the hardware block/s that userspace wants to
+ set values for.
+
+ struct drm_color_pipeline {
+ int num;
+ int size;
+ struct drm_color_op_data *data;
+ };
+
+ User can either:
+ 1. send data for all the color operations in a pipeline as shown in [2].
+ The color operation data need not be in order that the pipeline advertises
+ however, it should not contain data for any
+ color operation that is not present in the pipeline.
+
+ Note: This check for pipeline is not yet implemented but if the
+ wider proposal is accepted we have few solutions in mind.
+
+ 2. send data for a subset of color operations as shown in [3].For the
+ color operation that userspace does not send data will retain their
+ older state.
+
+ 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
+ property to NULL as shown in both [2] and [3]
+
+ 4. change the color pipeline as demonstrated in [3].
+ On the new pipeline, the user is expected to setup all the color hardware block
+ Once the user requests a pipeline change, the driver will provide it a clean slate
+ which means that all the data previously set by the user will be discarded even if
+ there are common hardware blocks between the two(previous and current) pipelines.
+
+IGT implementation can be found here [4]
+
+Representing Fixed function hardware
+====================================
+
+To provide support for fixed function hardware, the driver could expose vendor
+specific struct drm_color_op with parameters that both the userspace and
+driver agrees on. To demonstrate, let's consider a hyphothetical fixed
+function hardware block that converts BT601 to BT2020.
+The driver can choose to advertise the block as such.
+
+struct drm_color_op color_pipeline_X[] = {
+ {
+ .name = DRM_CB_PRIVATE,
+ .type = FIXED_FUNCTION,
+ .blob_id = 45;
+ },
+}
+
+Where the blob represents some vendor specific enums, strings or any other
+appropriate data types which both the user-space and drivers are aligned on.
+
+blob:45 {
+ VENDORXXX_BT602_TO_BT2020,
+};
+
+For enabling or passing parameters to such blocks, the user can send data
+to the driver wrapped within a blob like any other color operation block.
+
+ struct drm_color_op_data {
+ .name = DRM_CB_PRIVATE;
+ .blob_id = 46;
+ } ;
+
+where blob with id 46 contains data to enable the fixed function hardware(s).
+
+References
+==========
+
+[1] https://gitlab.freedesktop.org/emersion/drm_info
+[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
+[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
+[4] https://patchwork.freedesktop.org/series/123018/
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 02/33] drm: Add color operation structure
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Each Color Hardware block will be represented uniquely
in the color pipeline. Define the structure to represent
the same.
These color operations will form the building blocks of
a color pipeline which best represents the underlying
Hardware. Color operations can be re-arranged, substracted
or added to create distinct color pipelines to accurately
describe the Hardware blocks present in the display engine.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index ea1b639bcb28..882479f41745 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -943,6 +943,78 @@ struct hdr_output_metadata {
};
};
+/**
+ * enum color_op_block
+ *
+ * Enums to identify hardware color blocks.
+ *
+ * @DRM_CB_PRE_CSC: LUT before the CTM unit
+ * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
+ * @DRM_CB_POST_CSC: LUT after the CTM unit
+ * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
+ * color components
+ * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
+ * can expose a custom hardware by defining a
+ * color operation block with this name as
+ * identifier
+ */
+enum color_op_block {
+ DRM_CB_INVAL = -1,
+
+ DRM_CB_PRE_CSC = 0,
+ DRM_CB_CSC,
+ DRM_CB_POST_CSC,
+ DRM_CB_3D_LUT,
+
+ /* Any new generic hardware block can be updated here */
+
+ /*
+ * PRIVATE is kept at 255 to make it future proof and leave
+ * scope for any new addition
+ */
+ DRM_CB_PRIVATE = 255,
+ DRM_CB_MAX = DRM_CB_PRIVATE,
+};
+
+/**
+ * enum color_op_type
+ *
+ * These enums are to identify the mathematical operation that
+ * a hardware block is capable of.
+ * @CURVE_1D: It represents a one dimensional lookup table
+ * @CURVE_3D: Represents lut value for each color component for 3d lut capable hardware
+ * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
+ * @FIXED_FUNCTION: To enable and program any custom fixed function hardware unit
+ */
+enum color_op_type {
+ CURVE_1D,
+ CURVE_3D,
+ MATRIX,
+ FIXED_FUNCTION,
+};
+
+/**
+ * @struct drm_color_op
+ *
+ * This structure is used to represent the capability of
+ * individual color hardware blocks.
+ *
+ * @name: a standardized enum to identify the color hardware block
+ * @type: The type of mathematical operation it can perform
+ * @blob_id: Id pointing to a blob containing information about
+ * the hardware block which advertizes its capabilities
+ * to the userspace. It can be an optional field depending
+ * on the members "name" and "type".
+ * @private_flags: This can be used to provide vendor specific hints
+ * to user space
+ */
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ __u32 blob_id;
+ __u32 private_flags;
+};
+
/**
* DRM_MODE_PAGE_FLIP_EVENT
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 02/33] drm: Add color operation structure
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Each Color Hardware block will be represented uniquely
in the color pipeline. Define the structure to represent
the same.
These color operations will form the building blocks of
a color pipeline which best represents the underlying
Hardware. Color operations can be re-arranged, substracted
or added to create distinct color pipelines to accurately
describe the Hardware blocks present in the display engine.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index ea1b639bcb28..882479f41745 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -943,6 +943,78 @@ struct hdr_output_metadata {
};
};
+/**
+ * enum color_op_block
+ *
+ * Enums to identify hardware color blocks.
+ *
+ * @DRM_CB_PRE_CSC: LUT before the CTM unit
+ * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
+ * @DRM_CB_POST_CSC: LUT after the CTM unit
+ * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
+ * color components
+ * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
+ * can expose a custom hardware by defining a
+ * color operation block with this name as
+ * identifier
+ */
+enum color_op_block {
+ DRM_CB_INVAL = -1,
+
+ DRM_CB_PRE_CSC = 0,
+ DRM_CB_CSC,
+ DRM_CB_POST_CSC,
+ DRM_CB_3D_LUT,
+
+ /* Any new generic hardware block can be updated here */
+
+ /*
+ * PRIVATE is kept at 255 to make it future proof and leave
+ * scope for any new addition
+ */
+ DRM_CB_PRIVATE = 255,
+ DRM_CB_MAX = DRM_CB_PRIVATE,
+};
+
+/**
+ * enum color_op_type
+ *
+ * These enums are to identify the mathematical operation that
+ * a hardware block is capable of.
+ * @CURVE_1D: It represents a one dimensional lookup table
+ * @CURVE_3D: Represents lut value for each color component for 3d lut capable hardware
+ * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
+ * @FIXED_FUNCTION: To enable and program any custom fixed function hardware unit
+ */
+enum color_op_type {
+ CURVE_1D,
+ CURVE_3D,
+ MATRIX,
+ FIXED_FUNCTION,
+};
+
+/**
+ * @struct drm_color_op
+ *
+ * This structure is used to represent the capability of
+ * individual color hardware blocks.
+ *
+ * @name: a standardized enum to identify the color hardware block
+ * @type: The type of mathematical operation it can perform
+ * @blob_id: Id pointing to a blob containing information about
+ * the hardware block which advertizes its capabilities
+ * to the userspace. It can be an optional field depending
+ * on the members "name" and "type".
+ * @private_flags: This can be used to provide vendor specific hints
+ * to user space
+ */
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ __u32 blob_id;
+ __u32 private_flags;
+};
+
/**
* DRM_MODE_PAGE_FLIP_EVENT
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 03/33] drm: Add plane get color pipeline property
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Each hardware plane can consist of multiple color hardware blocks.
These hardware blocks are defined by a color pipeline. In case,
hardware blocks can be re-arranged/muxed a distinct pipeline can
be defined to represent the same.
Introduce a new enum plane property "GET_COLOR_PIPELINE" to expose
the color pipelines that a particular plane supports. This enum
property has blob id's as values. With each blob id representing
a distinct color pipeline based on underlying HW capabilities and
their respective combinations. Add helpers to create and attach
the property to a plane.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_color_mgmt.c | 46 ++++++++++++++++++++++++++++++++
include/drm/drm_plane.h | 10 +++++++
2 files changed, 56 insertions(+)
diff --git a/drivers/gpu/drm/drm_color_mgmt.c b/drivers/gpu/drm/drm_color_mgmt.c
index d021497841b8..026d057d1f1f 100644
--- a/drivers/gpu/drm/drm_color_mgmt.c
+++ b/drivers/gpu/drm/drm_color_mgmt.c
@@ -588,6 +588,52 @@ int drm_plane_create_color_properties(struct drm_plane *plane,
}
EXPORT_SYMBOL(drm_plane_create_color_properties);
+/**
+ * drm_plane_create_get_color_pipeline_property - create property to expose color pipelines
+ * @dev: DRM device
+ * @plane: plane object
+ * @num_val: number of color pipelines supported
+ *
+ * create enum property to expose color pipelines to userspace. This enum
+ * property has blob id's as values. With each blob id representing
+ * a distinct color pipeline based on underlying HW capabilities and
+ * their respective combinations.
+ */
+int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane,
+ int num_val)
+{
+ struct drm_property *prop;
+
+ prop = drm_property_create(dev, DRM_MODE_PROP_ENUM |
+ DRM_MODE_PROP_IMMUTABLE,
+ "GET_COLOR_PIPELINE", num_val);
+ if (!prop)
+ return -ENOMEM;
+
+ plane->get_color_pipeline_prop = prop;
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_plane_create_get_color_pipeline_property);
+
+/**
+ * drm_plane_attach_get_color_pipeline_property - attach get color pipeline property to a plane
+ * @plane: plane object
+ *
+ * Attach "GET_COLOR_PIPELINE" property to a plane. The property will be visible to
+ * the userspace once we attach the property.
+ */
+void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane)
+{
+ if (!plane->get_color_pipeline_prop)
+ return;
+
+ drm_object_attach_property(&plane->base,
+ plane->get_color_pipeline_prop, 0);
+}
+EXPORT_SYMBOL(drm_plane_attach_get_color_pipeline_property);
+
/**
* drm_color_lut_check - check validity of lookup table
* @lut: property blob containing LUT to check
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index 79d62856defb..256c97ead698 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -748,6 +748,12 @@ struct drm_plane {
* scaling.
*/
struct drm_property *scaling_filter_property;
+
+ /**
+ * @get_color_pipeline_prop: Optional Plane property to get the color pipelines
+ * that the plane supports
+ */
+ struct drm_property *get_color_pipeline_prop;
};
#define obj_to_plane(x) container_of(x, struct drm_plane, base)
@@ -945,5 +951,9 @@ drm_plane_get_damage_clips(const struct drm_plane_state *state);
int drm_plane_create_scaling_filter_property(struct drm_plane *plane,
unsigned int supported_filters);
+int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane,
+ int num_val);
+void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane);
#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 03/33] drm: Add plane get color pipeline property
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Each hardware plane can consist of multiple color hardware blocks.
These hardware blocks are defined by a color pipeline. In case,
hardware blocks can be re-arranged/muxed a distinct pipeline can
be defined to represent the same.
Introduce a new enum plane property "GET_COLOR_PIPELINE" to expose
the color pipelines that a particular plane supports. This enum
property has blob id's as values. With each blob id representing
a distinct color pipeline based on underlying HW capabilities and
their respective combinations. Add helpers to create and attach
the property to a plane.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_color_mgmt.c | 46 ++++++++++++++++++++++++++++++++
include/drm/drm_plane.h | 10 +++++++
2 files changed, 56 insertions(+)
diff --git a/drivers/gpu/drm/drm_color_mgmt.c b/drivers/gpu/drm/drm_color_mgmt.c
index d021497841b8..026d057d1f1f 100644
--- a/drivers/gpu/drm/drm_color_mgmt.c
+++ b/drivers/gpu/drm/drm_color_mgmt.c
@@ -588,6 +588,52 @@ int drm_plane_create_color_properties(struct drm_plane *plane,
}
EXPORT_SYMBOL(drm_plane_create_color_properties);
+/**
+ * drm_plane_create_get_color_pipeline_property - create property to expose color pipelines
+ * @dev: DRM device
+ * @plane: plane object
+ * @num_val: number of color pipelines supported
+ *
+ * create enum property to expose color pipelines to userspace. This enum
+ * property has blob id's as values. With each blob id representing
+ * a distinct color pipeline based on underlying HW capabilities and
+ * their respective combinations.
+ */
+int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane,
+ int num_val)
+{
+ struct drm_property *prop;
+
+ prop = drm_property_create(dev, DRM_MODE_PROP_ENUM |
+ DRM_MODE_PROP_IMMUTABLE,
+ "GET_COLOR_PIPELINE", num_val);
+ if (!prop)
+ return -ENOMEM;
+
+ plane->get_color_pipeline_prop = prop;
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_plane_create_get_color_pipeline_property);
+
+/**
+ * drm_plane_attach_get_color_pipeline_property - attach get color pipeline property to a plane
+ * @plane: plane object
+ *
+ * Attach "GET_COLOR_PIPELINE" property to a plane. The property will be visible to
+ * the userspace once we attach the property.
+ */
+void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane)
+{
+ if (!plane->get_color_pipeline_prop)
+ return;
+
+ drm_object_attach_property(&plane->base,
+ plane->get_color_pipeline_prop, 0);
+}
+EXPORT_SYMBOL(drm_plane_attach_get_color_pipeline_property);
+
/**
* drm_color_lut_check - check validity of lookup table
* @lut: property blob containing LUT to check
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index 79d62856defb..256c97ead698 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -748,6 +748,12 @@ struct drm_plane {
* scaling.
*/
struct drm_property *scaling_filter_property;
+
+ /**
+ * @get_color_pipeline_prop: Optional Plane property to get the color pipelines
+ * that the plane supports
+ */
+ struct drm_property *get_color_pipeline_prop;
};
#define obj_to_plane(x) container_of(x, struct drm_plane, base)
@@ -945,5 +951,9 @@ drm_plane_get_damage_clips(const struct drm_plane_state *state);
int drm_plane_create_scaling_filter_property(struct drm_plane *plane,
unsigned int supported_filters);
+int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane,
+ int num_val);
+void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane);
#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 04/33] drm: Add helper to add color pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Create a helper function to add a color pipeline for a plane.
Color pipeline is an array of struct drm_color_op which represent
a possible logical combination of color operations. Color
operations can be re-arranged, substracted or added to create
distinct color pipelines to accurately describe the Hardware blocks
present in the display engine.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_color_mgmt.c | 42 ++++++++++++++++++++++++++++++++
include/drm/drm_plane.h | 3 +++
2 files changed, 45 insertions(+)
diff --git a/drivers/gpu/drm/drm_color_mgmt.c b/drivers/gpu/drm/drm_color_mgmt.c
index 026d057d1f1f..43d0187faa98 100644
--- a/drivers/gpu/drm/drm_color_mgmt.c
+++ b/drivers/gpu/drm/drm_color_mgmt.c
@@ -634,6 +634,48 @@ void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane)
}
EXPORT_SYMBOL(drm_plane_attach_get_color_pipeline_property);
+/**
+ * drm_plane_add_color_pipeline - helper to add a color pipeline
+ * @plane: plane object
+ * @name: name of the color pipeline
+ * @color_pipeline: an array of 'struct drm_color_op' to represent a color pipeline
+ * @len: size of the color pipeline
+ *
+ * This helper can be used to add a distinct color pipeline to a plane. A driver
+ * can provide a meaningful name to the pipeline as it desires.
+ */
+int drm_plane_add_color_pipeline(struct drm_plane *plane, char *name,
+ struct drm_color_op *color_pipeline,
+ size_t len)
+{
+ int ret;
+ struct drm_property *prop;
+ struct drm_property_blob *blob;
+ u32 id = 0;
+
+ prop = plane->get_color_pipeline_prop;
+
+ if (color_pipeline && !len)
+ return -EINVAL;
+
+ if (color_pipeline) {
+ blob = drm_property_create_blob(plane->dev, len, color_pipeline);
+ if (IS_ERR(blob))
+ return PTR_ERR(blob);
+
+ id = blob->base.id;
+ };
+
+ ret = drm_property_add_enum(prop, id, name);
+ if (ret) {
+ if (blob)
+ drm_property_blob_put(blob);
+ return ret;
+ }
+ return 0;
+}
+EXPORT_SYMBOL(drm_plane_add_color_pipeline);
+
/**
* drm_color_lut_check - check validity of lookup table
* @lut: property blob containing LUT to check
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index 256c97ead698..ffd7887c2acf 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -955,5 +955,8 @@ int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
struct drm_plane *plane,
int num_val);
void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane);
+int drm_plane_add_color_pipeline(struct drm_plane *plane, char *name,
+ struct drm_color_op *color_pipeline,
+ size_t len);
#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 04/33] drm: Add helper to add color pipeline
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Create a helper function to add a color pipeline for a plane.
Color pipeline is an array of struct drm_color_op which represent
a possible logical combination of color operations. Color
operations can be re-arranged, substracted or added to create
distinct color pipelines to accurately describe the Hardware blocks
present in the display engine.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_color_mgmt.c | 42 ++++++++++++++++++++++++++++++++
include/drm/drm_plane.h | 3 +++
2 files changed, 45 insertions(+)
diff --git a/drivers/gpu/drm/drm_color_mgmt.c b/drivers/gpu/drm/drm_color_mgmt.c
index 026d057d1f1f..43d0187faa98 100644
--- a/drivers/gpu/drm/drm_color_mgmt.c
+++ b/drivers/gpu/drm/drm_color_mgmt.c
@@ -634,6 +634,48 @@ void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane)
}
EXPORT_SYMBOL(drm_plane_attach_get_color_pipeline_property);
+/**
+ * drm_plane_add_color_pipeline - helper to add a color pipeline
+ * @plane: plane object
+ * @name: name of the color pipeline
+ * @color_pipeline: an array of 'struct drm_color_op' to represent a color pipeline
+ * @len: size of the color pipeline
+ *
+ * This helper can be used to add a distinct color pipeline to a plane. A driver
+ * can provide a meaningful name to the pipeline as it desires.
+ */
+int drm_plane_add_color_pipeline(struct drm_plane *plane, char *name,
+ struct drm_color_op *color_pipeline,
+ size_t len)
+{
+ int ret;
+ struct drm_property *prop;
+ struct drm_property_blob *blob;
+ u32 id = 0;
+
+ prop = plane->get_color_pipeline_prop;
+
+ if (color_pipeline && !len)
+ return -EINVAL;
+
+ if (color_pipeline) {
+ blob = drm_property_create_blob(plane->dev, len, color_pipeline);
+ if (IS_ERR(blob))
+ return PTR_ERR(blob);
+
+ id = blob->base.id;
+ };
+
+ ret = drm_property_add_enum(prop, id, name);
+ if (ret) {
+ if (blob)
+ drm_property_blob_put(blob);
+ return ret;
+ }
+ return 0;
+}
+EXPORT_SYMBOL(drm_plane_add_color_pipeline);
+
/**
* drm_color_lut_check - check validity of lookup table
* @lut: property blob containing LUT to check
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index 256c97ead698..ffd7887c2acf 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -955,5 +955,8 @@ int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
struct drm_plane *plane,
int num_val);
void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane);
+int drm_plane_add_color_pipeline(struct drm_plane *plane, char *name,
+ struct drm_color_op *color_pipeline,
+ size_t len);
#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 05/33] drm: Add structures for setting color pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add structures using which user space can set a color pipeline it
desires. The patch introduces two structures
struct drm_color_op_data represents data to be passed
onto individual color hardware blocks.
struct drm_color_pipeline represents the aggregate of drm_color_op_data
structures to program the respective color hardware blocks of the
pipeline. It also contains the pipeline number to be set.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/uapi/drm/drm_mode.h | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index 882479f41745..a21825ee93e2 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -1015,6 +1015,36 @@ struct drm_color_op {
__u32 private_flags;
};
+/**
+ * struct drm_color_op_data
+ *
+ * Structure for userspace to send data to a particular color operation
+ * block.
+ *
+ * @name: Name of the color block for which the data is being sent
+ * @blob_id: Id pointing to blob with data for the color operation block
+ */
+struct drm_color_op_data {
+ enum color_op_block name;
+ __u32 blob_id;
+};
+
+/**
+ * struct drm_color_pipeline
+ *
+ * This structure represents the aggregate pipeline to be set
+ *
+ * @num: pipeline number to be selected
+ * @size: size of the data to be passed onto the driver
+ * @data: array of struct drm_color_op_data with data for the
+ * hardware block/s that user space wants to set values for
+ */
+struct drm_color_pipeline {
+ int num;
+ int size;
+ struct drm_color_op_data *data;
+};
+
/**
* DRM_MODE_PAGE_FLIP_EVENT
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 05/33] drm: Add structures for setting color pipeline
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add structures using which user space can set a color pipeline it
desires. The patch introduces two structures
struct drm_color_op_data represents data to be passed
onto individual color hardware blocks.
struct drm_color_pipeline represents the aggregate of drm_color_op_data
structures to program the respective color hardware blocks of the
pipeline. It also contains the pipeline number to be set.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/uapi/drm/drm_mode.h | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index 882479f41745..a21825ee93e2 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -1015,6 +1015,36 @@ struct drm_color_op {
__u32 private_flags;
};
+/**
+ * struct drm_color_op_data
+ *
+ * Structure for userspace to send data to a particular color operation
+ * block.
+ *
+ * @name: Name of the color block for which the data is being sent
+ * @blob_id: Id pointing to blob with data for the color operation block
+ */
+struct drm_color_op_data {
+ enum color_op_block name;
+ __u32 blob_id;
+};
+
+/**
+ * struct drm_color_pipeline
+ *
+ * This structure represents the aggregate pipeline to be set
+ *
+ * @num: pipeline number to be selected
+ * @size: size of the data to be passed onto the driver
+ * @data: array of struct drm_color_op_data with data for the
+ * hardware block/s that user space wants to set values for
+ */
+struct drm_color_pipeline {
+ int num;
+ int size;
+ struct drm_color_op_data *data;
+};
+
/**
* DRM_MODE_PAGE_FLIP_EVENT
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 06/33] drm: Add set colorpipeline property
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add a new plane blob property "SET_COLOR_PIPELINE" using
which the user can select a color pipeline and send data
for corresponding hardware blocks.
Once the user space decides on a color pipeline, it can
set the pipeline and corresponding data for the hardware
blocks within the pipeline.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 12 +++++++++
drivers/gpu/drm/drm_color_mgmt.c | 42 +++++++++++++++++++++++++++++++
include/drm/drm_plane.h | 22 ++++++++++++++++
3 files changed, 76 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 98d3b10c08ae..a2d3393d21a2 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -590,6 +590,15 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
return ret;
} else if (property == plane->scaling_filter_property) {
state->scaling_filter = val;
+ } else if (property == plane->set_color_pipeline_prop) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->set_color_pipeline_data,
+ val,
+ -1,
+ sizeof(struct drm_color_pipeline),
+ &replaced);
+ state->color_mgmt_changed |= replaced;
+ return ret;
} else if (plane->funcs->atomic_set_property) {
return plane->funcs->atomic_set_property(plane, state,
property, val);
@@ -651,6 +660,9 @@ drm_atomic_plane_get_property(struct drm_plane *plane,
state->fb_damage_clips->base.id : 0;
} else if (property == plane->scaling_filter_property) {
*val = state->scaling_filter;
+ } else if (property == plane->set_color_pipeline_prop) {
+ *val = (state->set_color_pipeline_data) ?
+ state->set_color_pipeline_data->base.id : 0;
} else if (plane->funcs->atomic_get_property) {
return plane->funcs->atomic_get_property(plane, state, property, val);
} else {
diff --git a/drivers/gpu/drm/drm_color_mgmt.c b/drivers/gpu/drm/drm_color_mgmt.c
index 43d0187faa98..3ef58da94556 100644
--- a/drivers/gpu/drm/drm_color_mgmt.c
+++ b/drivers/gpu/drm/drm_color_mgmt.c
@@ -634,6 +634,48 @@ void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane)
}
EXPORT_SYMBOL(drm_plane_attach_get_color_pipeline_property);
+/**
+ * drm_plane_create_set_color_pipeline_property - create property to set color pipeline
+ * @dev: DRM device
+ * @plane: plane object
+ *
+ * create blob property using which the user space can set up a plane color pipeline.
+ * Userspace can send data for one or multiple hardware blocks in the pipeline.
+ */
+int drm_plane_create_set_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane)
+{
+ struct drm_property *prop;
+
+ prop = drm_property_create(dev, DRM_MODE_PROP_BLOB,
+ "SET_COLOR_PIPELINE", 0);
+ if (!prop)
+ return -ENOMEM;
+
+ plane->set_color_pipeline_prop = prop;
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_plane_create_set_color_pipeline_property);
+
+/**
+ * drm_plane_attach_set_color_pipeline_property - attach set color pipeline property to a plane
+ * @plane: plane object
+ *
+ * Attach "SET_COLOR_PIPELINE" property to a plane. The property will be visible to
+ * the userspace once we attach the property. The default value is set to 0 indicating
+ * no colorpipeline which essentially disables all the color HW blocks in the pipeline.
+ */
+void drm_plane_attach_set_color_pipeline_property(struct drm_plane *plane)
+{
+ if (!plane->set_color_pipeline_prop)
+ return;
+
+ drm_object_attach_property(&plane->base,
+ plane->set_color_pipeline_prop, 0);
+}
+EXPORT_SYMBOL(drm_plane_attach_set_color_pipeline_property);
+
/**
* drm_plane_add_color_pipeline - helper to add a color pipeline
* @plane: plane object
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index ffd7887c2acf..fcd589cb38f2 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -237,6 +237,20 @@ struct drm_plane_state {
/** @state: backpointer to global drm_atomic_state */
struct drm_atomic_state *state;
+
+ /**
+ * @set_color_pipeline_data:
+ *
+ * Stores information about the current selected color pipeline
+ */
+ struct drm_property_blob *set_color_pipeline_data;
+
+ /**
+ * @color_mgmt_changed: Plane color pipeline state has changed
+ * Used by the atomic helpers and
+ * drivers to steer the atomic commit control flow.
+ */
+ u8 color_mgmt_changed : 1;
};
static inline struct drm_rect
@@ -754,6 +768,11 @@ struct drm_plane {
* that the plane supports
*/
struct drm_property *get_color_pipeline_prop;
+
+ /**
+ * @set_color_pipeline_prop: Optional Plane property to set the color pipeline
+ */
+ struct drm_property *set_color_pipeline_prop;
};
#define obj_to_plane(x) container_of(x, struct drm_plane, base)
@@ -955,6 +974,9 @@ int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
struct drm_plane *plane,
int num_val);
void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane);
+int drm_plane_create_set_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane);
+void drm_plane_attach_set_color_pipeline_property(struct drm_plane *plane);
int drm_plane_add_color_pipeline(struct drm_plane *plane, char *name,
struct drm_color_op *color_pipeline,
size_t len);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 06/33] drm: Add set colorpipeline property
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add a new plane blob property "SET_COLOR_PIPELINE" using
which the user can select a color pipeline and send data
for corresponding hardware blocks.
Once the user space decides on a color pipeline, it can
set the pipeline and corresponding data for the hardware
blocks within the pipeline.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 12 +++++++++
drivers/gpu/drm/drm_color_mgmt.c | 42 +++++++++++++++++++++++++++++++
include/drm/drm_plane.h | 22 ++++++++++++++++
3 files changed, 76 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 98d3b10c08ae..a2d3393d21a2 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -590,6 +590,15 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
return ret;
} else if (property == plane->scaling_filter_property) {
state->scaling_filter = val;
+ } else if (property == plane->set_color_pipeline_prop) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->set_color_pipeline_data,
+ val,
+ -1,
+ sizeof(struct drm_color_pipeline),
+ &replaced);
+ state->color_mgmt_changed |= replaced;
+ return ret;
} else if (plane->funcs->atomic_set_property) {
return plane->funcs->atomic_set_property(plane, state,
property, val);
@@ -651,6 +660,9 @@ drm_atomic_plane_get_property(struct drm_plane *plane,
state->fb_damage_clips->base.id : 0;
} else if (property == plane->scaling_filter_property) {
*val = state->scaling_filter;
+ } else if (property == plane->set_color_pipeline_prop) {
+ *val = (state->set_color_pipeline_data) ?
+ state->set_color_pipeline_data->base.id : 0;
} else if (plane->funcs->atomic_get_property) {
return plane->funcs->atomic_get_property(plane, state, property, val);
} else {
diff --git a/drivers/gpu/drm/drm_color_mgmt.c b/drivers/gpu/drm/drm_color_mgmt.c
index 43d0187faa98..3ef58da94556 100644
--- a/drivers/gpu/drm/drm_color_mgmt.c
+++ b/drivers/gpu/drm/drm_color_mgmt.c
@@ -634,6 +634,48 @@ void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane)
}
EXPORT_SYMBOL(drm_plane_attach_get_color_pipeline_property);
+/**
+ * drm_plane_create_set_color_pipeline_property - create property to set color pipeline
+ * @dev: DRM device
+ * @plane: plane object
+ *
+ * create blob property using which the user space can set up a plane color pipeline.
+ * Userspace can send data for one or multiple hardware blocks in the pipeline.
+ */
+int drm_plane_create_set_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane)
+{
+ struct drm_property *prop;
+
+ prop = drm_property_create(dev, DRM_MODE_PROP_BLOB,
+ "SET_COLOR_PIPELINE", 0);
+ if (!prop)
+ return -ENOMEM;
+
+ plane->set_color_pipeline_prop = prop;
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_plane_create_set_color_pipeline_property);
+
+/**
+ * drm_plane_attach_set_color_pipeline_property - attach set color pipeline property to a plane
+ * @plane: plane object
+ *
+ * Attach "SET_COLOR_PIPELINE" property to a plane. The property will be visible to
+ * the userspace once we attach the property. The default value is set to 0 indicating
+ * no colorpipeline which essentially disables all the color HW blocks in the pipeline.
+ */
+void drm_plane_attach_set_color_pipeline_property(struct drm_plane *plane)
+{
+ if (!plane->set_color_pipeline_prop)
+ return;
+
+ drm_object_attach_property(&plane->base,
+ plane->set_color_pipeline_prop, 0);
+}
+EXPORT_SYMBOL(drm_plane_attach_set_color_pipeline_property);
+
/**
* drm_plane_add_color_pipeline - helper to add a color pipeline
* @plane: plane object
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index ffd7887c2acf..fcd589cb38f2 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -237,6 +237,20 @@ struct drm_plane_state {
/** @state: backpointer to global drm_atomic_state */
struct drm_atomic_state *state;
+
+ /**
+ * @set_color_pipeline_data:
+ *
+ * Stores information about the current selected color pipeline
+ */
+ struct drm_property_blob *set_color_pipeline_data;
+
+ /**
+ * @color_mgmt_changed: Plane color pipeline state has changed
+ * Used by the atomic helpers and
+ * drivers to steer the atomic commit control flow.
+ */
+ u8 color_mgmt_changed : 1;
};
static inline struct drm_rect
@@ -754,6 +768,11 @@ struct drm_plane {
* that the plane supports
*/
struct drm_property *get_color_pipeline_prop;
+
+ /**
+ * @set_color_pipeline_prop: Optional Plane property to set the color pipeline
+ */
+ struct drm_property *set_color_pipeline_prop;
};
#define obj_to_plane(x) container_of(x, struct drm_plane, base)
@@ -955,6 +974,9 @@ int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,
struct drm_plane *plane,
int num_val);
void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane);
+int drm_plane_create_set_color_pipeline_property(struct drm_device *dev,
+ struct drm_plane *plane);
+void drm_plane_attach_set_color_pipeline_property(struct drm_plane *plane);
int drm_plane_add_color_pipeline(struct drm_plane *plane, char *name,
struct drm_color_op *color_pipeline,
size_t len);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 07/33] drm: Add Enhanced Gamma LUT precision structure
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Existing LUT precision structure is having only 16 bit
precision. This is not enough for upcoming enhanced hardwares
and advance usecases like HDR processing. Hence added a new
structure with 32 bit precision values.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/uapi/drm/drm_mode.h | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index a21825ee93e2..1cd656b0e994 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -943,6 +943,23 @@ struct hdr_output_metadata {
};
};
+/**
+ * struct drm_color_lut_ext - Represents high precision lut values
+ *
+ * Creating 64 bit palette entries for better data
+ * precision. This will be required for HDR and
+ * similar color processing usecases.
+ */
+struct drm_color_lut_ext {
+ /*
+ * Data is U32.32 fixed point format.
+ */
+ __u64 red;
+ __u64 green;
+ __u64 blue;
+ __u64 reserved;
+};
+
/**
* enum color_op_block
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 07/33] drm: Add Enhanced Gamma LUT precision structure
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Existing LUT precision structure is having only 16 bit
precision. This is not enough for upcoming enhanced hardwares
and advance usecases like HDR processing. Hence added a new
structure with 32 bit precision values.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/uapi/drm/drm_mode.h | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index a21825ee93e2..1cd656b0e994 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -943,6 +943,23 @@ struct hdr_output_metadata {
};
};
+/**
+ * struct drm_color_lut_ext - Represents high precision lut values
+ *
+ * Creating 64 bit palette entries for better data
+ * precision. This will be required for HDR and
+ * similar color processing usecases.
+ */
+struct drm_color_lut_ext {
+ /*
+ * Data is U32.32 fixed point format.
+ */
+ __u64 red;
+ __u64 green;
+ __u64 blue;
+ __u64 reserved;
+};
+
/**
* enum color_op_block
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 08/33] drm: Add color lut range structure
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add color lut range structure which is to be used to advertize
the capabilities of pre-csc/post-csc color operation blocks.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/uapi/drm/drm_mode.h | 77 +++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index 1cd656b0e994..6ce7bd0926e0 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -1091,6 +1091,83 @@ struct drm_color_pipeline {
DRM_MODE_PAGE_FLIP_ASYNC | \
DRM_MODE_PAGE_FLIP_TARGET)
+/**
+ * DRM_MODE_LUT_POST_CSC
+ *
+ * LUT is for post csc (after CTM)
+ */
+#define DRM_MODE_LUT_POST_CSC BIT(0)
+
+/**
+ * DRM_MODE_LUT_PRE_CSC
+ *
+ * LUT is for pre csc (before CTM)
+ */
+#define DRM_MODE_LUT_PRE_CSC BIT(1)
+
+/**
+ * DRM_MODE_LUT_INTERPOLATE
+ *
+ * linearly interpolate between the points
+ */
+#define DRM_MODE_LUT_INTERPOLATE BIT(2)
+
+/**
+ * DRM_MODE_LUT_REUSE_LAST
+ *
+ * the last value of the previous range is the
+ * first value of the current range.
+ */
+#define DRM_MODE_LUT_REUSE_LAST BIT(3)
+
+/**
+ * DRM_MODE_LUT_NON_DECREASING
+ *
+ * the curve must be non-decreasing
+ */
+#define DRM_MODE_LUT_NON_DECREASING BIT(4)
+
+/**
+ * DRM_MODE_LUT_REFLECT_NEGATIVE
+ *
+ * the curve is reflected across origin for negative inputs
+ */
+#define DRM_MODE_LUT_REFLECT_NEGATIVE BIT(5)
+
+/**
+ * DRM_MODE_LUT_SINGLE_CHANNEL
+ *
+ * the same curve (red) is used for blue and green channels as well
+ */
+#define DRM_MODE_LUT_SINGLE_CHANNEL BIT(6)
+
+/**
+ * struct drm_color_lut_range
+ *
+ * structure to advertise capability of a color hardware
+ * block that accepts LUT values. It can represent LUTs with
+ * varied number of entries and distributions
+ * (Multi segmented, Logarithmic etc).
+ */
+
+struct drm_color_lut_range {
+ /* DRM_MODE_LUT_* */
+ __u32 flags;
+ /* number of points on the curve */
+ __u16 count;
+ /* input/output bits per component */
+ __u8 input_bpc, output_bpc;
+ /* input start/end values */
+ __s32 start, end;
+ /* output min/max values */
+ __s32 min, max;
+};
+
+enum lut_type {
+ LUT_TYPE_PRE_CSC = 0,
+ LUT_TYPE_POST_CSC = 1,
+};
+
/*
* Request a page flip on the specified crtc.
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 08/33] drm: Add color lut range structure
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add color lut range structure which is to be used to advertize
the capabilities of pre-csc/post-csc color operation blocks.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/uapi/drm/drm_mode.h | 77 +++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index 1cd656b0e994..6ce7bd0926e0 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -1091,6 +1091,83 @@ struct drm_color_pipeline {
DRM_MODE_PAGE_FLIP_ASYNC | \
DRM_MODE_PAGE_FLIP_TARGET)
+/**
+ * DRM_MODE_LUT_POST_CSC
+ *
+ * LUT is for post csc (after CTM)
+ */
+#define DRM_MODE_LUT_POST_CSC BIT(0)
+
+/**
+ * DRM_MODE_LUT_PRE_CSC
+ *
+ * LUT is for pre csc (before CTM)
+ */
+#define DRM_MODE_LUT_PRE_CSC BIT(1)
+
+/**
+ * DRM_MODE_LUT_INTERPOLATE
+ *
+ * linearly interpolate between the points
+ */
+#define DRM_MODE_LUT_INTERPOLATE BIT(2)
+
+/**
+ * DRM_MODE_LUT_REUSE_LAST
+ *
+ * the last value of the previous range is the
+ * first value of the current range.
+ */
+#define DRM_MODE_LUT_REUSE_LAST BIT(3)
+
+/**
+ * DRM_MODE_LUT_NON_DECREASING
+ *
+ * the curve must be non-decreasing
+ */
+#define DRM_MODE_LUT_NON_DECREASING BIT(4)
+
+/**
+ * DRM_MODE_LUT_REFLECT_NEGATIVE
+ *
+ * the curve is reflected across origin for negative inputs
+ */
+#define DRM_MODE_LUT_REFLECT_NEGATIVE BIT(5)
+
+/**
+ * DRM_MODE_LUT_SINGLE_CHANNEL
+ *
+ * the same curve (red) is used for blue and green channels as well
+ */
+#define DRM_MODE_LUT_SINGLE_CHANNEL BIT(6)
+
+/**
+ * struct drm_color_lut_range
+ *
+ * structure to advertise capability of a color hardware
+ * block that accepts LUT values. It can represent LUTs with
+ * varied number of entries and distributions
+ * (Multi segmented, Logarithmic etc).
+ */
+
+struct drm_color_lut_range {
+ /* DRM_MODE_LUT_* */
+ __u32 flags;
+ /* number of points on the curve */
+ __u16 count;
+ /* input/output bits per component */
+ __u8 input_bpc, output_bpc;
+ /* input start/end values */
+ __s32 start, end;
+ /* output min/max values */
+ __s32 min, max;
+};
+
+enum lut_type {
+ LUT_TYPE_PRE_CSC = 0,
+ LUT_TYPE_POST_CSC = 1,
+};
+
/*
* Request a page flip on the specified crtc.
*
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 09/33] drm: Add color information to plane state
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add a new structure drm_plane_color to plane state. It consists
of blobs with data needed for respective color HW blocks.
Currently defining below blobs
pre-csc: can be used to linearize the input frame buffer data.
csc: used for color space conversion.
post-csc: can be used non-linearize frame buffer data or
to perform Tone mapping for HDR use-cases
private: can be used for vendor specific fixed function operations
This can be extended to include other color operations as well.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/drm/drm_plane.h | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index fcd589cb38f2..601b01e47a93 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -245,6 +245,47 @@ struct drm_plane_state {
*/
struct drm_property_blob *set_color_pipeline_data;
+ /**
+ * @drm_plane_color:
+ *
+ * Encapsulates all color states.
+ */
+ struct drm_plane_color {
+ /**
+ * @pre_csc_lut:
+ *
+ * Lookup table for converting framebuffer pixel data before apply the
+ * color conversion matrix @ctm. See drm_plane_enable_color_mgmt(). The
+ * blob (if not NULL) is an array of &struct drm_color_lut_ext.
+ */
+ struct drm_property_blob *pre_csc_lut;
+
+ /**
+ * @ctm:
+ *
+ * Color transformation matrix. See drm_plane_enable_color_mgmt(). The
+ * blob (if not NULL) is a &struct drm_color_ctm.
+ */
+ struct drm_property_blob *ctm;
+
+ /**
+ * @post_csc_lut:
+ *
+ * Lookup table for converting framebuffer pixel data after applying the
+ * color conversion matrix @ctm. See drm_plane_enable_color_mgmt(). The
+ * blob (if not NULL) is an array of &struct drm_color_lut_ext.
+ */
+ struct drm_property_blob *post_csc_lut;
+
+ /**
+ * @private_color_op_data:
+ *
+ * This blob is intended for drivers to implement driver private color operations.
+ * For example: Parameterized/non-parameterized fixed function operations
+ */
+ struct drm_property_blob *private_color_op_data;
+ } color;
+
/**
* @color_mgmt_changed: Plane color pipeline state has changed
* Used by the atomic helpers and
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 09/33] drm: Add color information to plane state
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add a new structure drm_plane_color to plane state. It consists
of blobs with data needed for respective color HW blocks.
Currently defining below blobs
pre-csc: can be used to linearize the input frame buffer data.
csc: used for color space conversion.
post-csc: can be used non-linearize frame buffer data or
to perform Tone mapping for HDR use-cases
private: can be used for vendor specific fixed function operations
This can be extended to include other color operations as well.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
include/drm/drm_plane.h | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index fcd589cb38f2..601b01e47a93 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -245,6 +245,47 @@ struct drm_plane_state {
*/
struct drm_property_blob *set_color_pipeline_data;
+ /**
+ * @drm_plane_color:
+ *
+ * Encapsulates all color states.
+ */
+ struct drm_plane_color {
+ /**
+ * @pre_csc_lut:
+ *
+ * Lookup table for converting framebuffer pixel data before apply the
+ * color conversion matrix @ctm. See drm_plane_enable_color_mgmt(). The
+ * blob (if not NULL) is an array of &struct drm_color_lut_ext.
+ */
+ struct drm_property_blob *pre_csc_lut;
+
+ /**
+ * @ctm:
+ *
+ * Color transformation matrix. See drm_plane_enable_color_mgmt(). The
+ * blob (if not NULL) is a &struct drm_color_ctm.
+ */
+ struct drm_property_blob *ctm;
+
+ /**
+ * @post_csc_lut:
+ *
+ * Lookup table for converting framebuffer pixel data after applying the
+ * color conversion matrix @ctm. See drm_plane_enable_color_mgmt(). The
+ * blob (if not NULL) is an array of &struct drm_color_lut_ext.
+ */
+ struct drm_property_blob *post_csc_lut;
+
+ /**
+ * @private_color_op_data:
+ *
+ * This blob is intended for drivers to implement driver private color operations.
+ * For example: Parameterized/non-parameterized fixed function operations
+ */
+ struct drm_property_blob *private_color_op_data;
+ } color;
+
/**
* @color_mgmt_changed: Plane color pipeline state has changed
* Used by the atomic helpers and
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 10/33] drm: Manage color blob states
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:03 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
This patch manages the references for color blobs.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_state_helper.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_state_helper.c b/drivers/gpu/drm/drm_atomic_state_helper.c
index 784e63d70a42..a554e04c2ce3 100644
--- a/drivers/gpu/drm/drm_atomic_state_helper.c
+++ b/drivers/gpu/drm/drm_atomic_state_helper.c
@@ -338,6 +338,19 @@ void __drm_atomic_helper_plane_duplicate_state(struct drm_plane *plane,
state->fence = NULL;
state->commit = NULL;
state->fb_damage_clips = NULL;
+
+ if (state->set_color_pipeline_data)
+ drm_property_blob_get(state->set_color_pipeline_data);
+ if (state->color.pre_csc_lut)
+ drm_property_blob_get(state->color.pre_csc_lut);
+ if (state->color.ctm)
+ drm_property_blob_get(state->color.ctm);
+ if (state->color.post_csc_lut)
+ drm_property_blob_get(state->color.post_csc_lut);
+ if (state->color.private_color_op_data)
+ drm_property_blob_get(state->color.private_color_op_data);
+
+ state->color_mgmt_changed = false;
}
EXPORT_SYMBOL(__drm_atomic_helper_plane_duplicate_state);
@@ -384,6 +397,11 @@ void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)
drm_crtc_commit_put(state->commit);
drm_property_blob_put(state->fb_damage_clips);
+ drm_property_blob_put(state->set_color_pipeline_data);
+ drm_property_blob_put(state->color.pre_csc_lut);
+ drm_property_blob_put(state->color.ctm);
+ drm_property_blob_put(state->color.post_csc_lut);
+ drm_property_blob_put(state->color.private_color_op_data);
}
EXPORT_SYMBOL(__drm_atomic_helper_plane_destroy_state);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 10/33] drm: Manage color blob states
@ 2023-08-29 16:03 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:03 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
This patch manages the references for color blobs.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_state_helper.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_state_helper.c b/drivers/gpu/drm/drm_atomic_state_helper.c
index 784e63d70a42..a554e04c2ce3 100644
--- a/drivers/gpu/drm/drm_atomic_state_helper.c
+++ b/drivers/gpu/drm/drm_atomic_state_helper.c
@@ -338,6 +338,19 @@ void __drm_atomic_helper_plane_duplicate_state(struct drm_plane *plane,
state->fence = NULL;
state->commit = NULL;
state->fb_damage_clips = NULL;
+
+ if (state->set_color_pipeline_data)
+ drm_property_blob_get(state->set_color_pipeline_data);
+ if (state->color.pre_csc_lut)
+ drm_property_blob_get(state->color.pre_csc_lut);
+ if (state->color.ctm)
+ drm_property_blob_get(state->color.ctm);
+ if (state->color.post_csc_lut)
+ drm_property_blob_get(state->color.post_csc_lut);
+ if (state->color.private_color_op_data)
+ drm_property_blob_get(state->color.private_color_op_data);
+
+ state->color_mgmt_changed = false;
}
EXPORT_SYMBOL(__drm_atomic_helper_plane_duplicate_state);
@@ -384,6 +397,11 @@ void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)
drm_crtc_commit_put(state->commit);
drm_property_blob_put(state->fb_damage_clips);
+ drm_property_blob_put(state->set_color_pipeline_data);
+ drm_property_blob_put(state->color.pre_csc_lut);
+ drm_property_blob_put(state->color.ctm);
+ drm_property_blob_put(state->color.post_csc_lut);
+ drm_property_blob_put(state->color.private_color_op_data);
}
EXPORT_SYMBOL(__drm_atomic_helper_plane_destroy_state);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 11/33] drm: Replace individual color blobs
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Replace the color operation blobs depending on the values sent by
userspace.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 97 +++++++++++++++++++++++++++++++
1 file changed, 97 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index a2d3393d21a2..20f9366865ca 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -404,6 +404,92 @@ drm_atomic_replace_property_blob_from_id(struct drm_device *dev,
return 0;
}
+/*
+ * Helper to replace individual color blobs for a plane. The function
+ * changes all the color blobs sent by userspace agnostic of the color
+ * pipeline chosen. Since, the information about color pipeline is
+ * available at driver level, the driver should check for
+ * the sanity of the userspace data.
+ */
+static
+int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
+ struct drm_plane_state *state,
+ uint64_t color_pipeline_blob_id,
+ bool *replaced)
+{
+ struct drm_device *dev = plane->dev;
+ struct drm_property_blob *new_blob;
+ struct drm_color_pipeline *color_pipeline;
+ struct drm_color_op_data *color_op;
+ int ret = 0, i;
+ bool blob_replaced = false;
+ bool temp_replaced = false;
+
+ new_blob = drm_property_lookup_blob(dev, color_pipeline_blob_id);
+
+ if (!new_blob) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ color_pipeline = new_blob->data;
+ color_op = kzalloc(color_pipeline->size, GFP_KERNEL);
+ if (!color_op) {
+ ret = -ENOMEM;
+ goto mem_fail;
+ }
+
+ if (copy_from_user(color_op, color_pipeline->data, color_pipeline->size)) {
+ ret = -EFAULT;
+ goto copy_fail;
+ }
+
+ for (i = 0; i < color_pipeline->size / sizeof(struct drm_color_op_data); i++) {
+ if (color_op[i].name == DRM_CB_CSC) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.ctm,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_ctm),
+ &blob_replaced);
+ } else if (color_op[i].name == DRM_CB_PRE_CSC) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.pre_csc_lut,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_lut_ext),
+ &blob_replaced);
+ } else if (color_op[i].name == DRM_CB_POST_CSC) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.post_csc_lut,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_lut_ext),
+ &blob_replaced);
+ } else if (color_op[i].name == DRM_CB_PRIVATE) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.private_color_op_data,
+ color_op[i].blob_id,
+ -1, -1,
+ &blob_replaced);
+ } else {
+ ret = -EINVAL;
+ goto copy_fail;
+ }
+
+ if (ret)
+ goto copy_fail;
+
+ temp_replaced |= blob_replaced;
+ }
+
+copy_fail:
+ kfree(color_op);
+mem_fail:
+ drm_property_blob_put(new_blob);
+out:
+ if (!ret)
+ *replaced |= temp_replaced;
+ return ret;
+}
+
static int drm_atomic_crtc_set_property(struct drm_crtc *crtc,
struct drm_crtc_state *state, struct drm_property *property,
uint64_t val)
@@ -597,6 +683,17 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
-1,
sizeof(struct drm_color_pipeline),
&replaced);
+ if (replaced) {
+ /* Consider actual color parameter change only when
+ * individual color blobs are replaced. Hence, reset
+ * the replaced boolean.
+ */
+ replaced = false;
+ ret = drm_plane_replace_color_op_blobs(plane, state,
+ val,
+ &replaced);
+ }
+
state->color_mgmt_changed |= replaced;
return ret;
} else if (plane->funcs->atomic_set_property) {
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 11/33] drm: Replace individual color blobs
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Replace the color operation blobs depending on the values sent by
userspace.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 97 +++++++++++++++++++++++++++++++
1 file changed, 97 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index a2d3393d21a2..20f9366865ca 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -404,6 +404,92 @@ drm_atomic_replace_property_blob_from_id(struct drm_device *dev,
return 0;
}
+/*
+ * Helper to replace individual color blobs for a plane. The function
+ * changes all the color blobs sent by userspace agnostic of the color
+ * pipeline chosen. Since, the information about color pipeline is
+ * available at driver level, the driver should check for
+ * the sanity of the userspace data.
+ */
+static
+int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
+ struct drm_plane_state *state,
+ uint64_t color_pipeline_blob_id,
+ bool *replaced)
+{
+ struct drm_device *dev = plane->dev;
+ struct drm_property_blob *new_blob;
+ struct drm_color_pipeline *color_pipeline;
+ struct drm_color_op_data *color_op;
+ int ret = 0, i;
+ bool blob_replaced = false;
+ bool temp_replaced = false;
+
+ new_blob = drm_property_lookup_blob(dev, color_pipeline_blob_id);
+
+ if (!new_blob) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ color_pipeline = new_blob->data;
+ color_op = kzalloc(color_pipeline->size, GFP_KERNEL);
+ if (!color_op) {
+ ret = -ENOMEM;
+ goto mem_fail;
+ }
+
+ if (copy_from_user(color_op, color_pipeline->data, color_pipeline->size)) {
+ ret = -EFAULT;
+ goto copy_fail;
+ }
+
+ for (i = 0; i < color_pipeline->size / sizeof(struct drm_color_op_data); i++) {
+ if (color_op[i].name == DRM_CB_CSC) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.ctm,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_ctm),
+ &blob_replaced);
+ } else if (color_op[i].name == DRM_CB_PRE_CSC) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.pre_csc_lut,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_lut_ext),
+ &blob_replaced);
+ } else if (color_op[i].name == DRM_CB_POST_CSC) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.post_csc_lut,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_lut_ext),
+ &blob_replaced);
+ } else if (color_op[i].name == DRM_CB_PRIVATE) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.private_color_op_data,
+ color_op[i].blob_id,
+ -1, -1,
+ &blob_replaced);
+ } else {
+ ret = -EINVAL;
+ goto copy_fail;
+ }
+
+ if (ret)
+ goto copy_fail;
+
+ temp_replaced |= blob_replaced;
+ }
+
+copy_fail:
+ kfree(color_op);
+mem_fail:
+ drm_property_blob_put(new_blob);
+out:
+ if (!ret)
+ *replaced |= temp_replaced;
+ return ret;
+}
+
static int drm_atomic_crtc_set_property(struct drm_crtc *crtc,
struct drm_crtc_state *state, struct drm_property *property,
uint64_t val)
@@ -597,6 +683,17 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
-1,
sizeof(struct drm_color_pipeline),
&replaced);
+ if (replaced) {
+ /* Consider actual color parameter change only when
+ * individual color blobs are replaced. Hence, reset
+ * the replaced boolean.
+ */
+ replaced = false;
+ ret = drm_plane_replace_color_op_blobs(plane, state,
+ val,
+ &replaced);
+ }
+
state->color_mgmt_changed |= replaced;
return ret;
} else if (plane->funcs->atomic_set_property) {
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 12/33] drm: Reset pipeline when user sends NULL blob
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
User can disable the color pipeline entirely, thereby
disabling all the color hardware blocks in the pipeline.
User should set NULL as the blob id and invoke SET_COLOR_PIPELINE
property. Driver will disable all the color hardware blocks by
updating respective blob id's as NULL.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 50 +++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 20f9366865ca..259cd4f5f520 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -411,6 +411,53 @@ drm_atomic_replace_property_blob_from_id(struct drm_device *dev,
* available at driver level, the driver should check for
* the sanity of the userspace data.
*/
+static
+int drm_plane_reset_color_op_blobs(struct drm_plane *plane,
+ struct drm_plane_state *state,
+ bool *replaced)
+{
+ struct drm_device *dev = plane->dev;
+ int ret;
+ bool blob_replaced = false;
+ bool temp_replaced = false;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.ctm,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+ if (ret)
+ goto out;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.pre_csc_lut,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
+ if (ret)
+ goto out;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.post_csc_lut,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.private_color_op_data,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
+ if (ret)
+ goto out;
+out:
+ if (!ret)
+ *replaced |= temp_replaced;
+ return ret;
+}
+
static
int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
struct drm_plane_state *state,
@@ -425,6 +472,9 @@ int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
bool blob_replaced = false;
bool temp_replaced = false;
+ if (!color_pipeline_blob_id)
+ return drm_plane_reset_color_op_blobs(plane, state, replaced);
+
new_blob = drm_property_lookup_blob(dev, color_pipeline_blob_id);
if (!new_blob) {
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 12/33] drm: Reset pipeline when user sends NULL blob
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
User can disable the color pipeline entirely, thereby
disabling all the color hardware blocks in the pipeline.
User should set NULL as the blob id and invoke SET_COLOR_PIPELINE
property. Driver will disable all the color hardware blocks by
updating respective blob id's as NULL.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 50 +++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 20f9366865ca..259cd4f5f520 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -411,6 +411,53 @@ drm_atomic_replace_property_blob_from_id(struct drm_device *dev,
* available at driver level, the driver should check for
* the sanity of the userspace data.
*/
+static
+int drm_plane_reset_color_op_blobs(struct drm_plane *plane,
+ struct drm_plane_state *state,
+ bool *replaced)
+{
+ struct drm_device *dev = plane->dev;
+ int ret;
+ bool blob_replaced = false;
+ bool temp_replaced = false;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.ctm,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+ if (ret)
+ goto out;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.pre_csc_lut,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
+ if (ret)
+ goto out;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.post_csc_lut,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.private_color_op_data,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
+ if (ret)
+ goto out;
+out:
+ if (!ret)
+ *replaced |= temp_replaced;
+ return ret;
+}
+
static
int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
struct drm_plane_state *state,
@@ -425,6 +472,9 @@ int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
bool blob_replaced = false;
bool temp_replaced = false;
+ if (!color_pipeline_blob_id)
+ return drm_plane_reset_color_op_blobs(plane, state, replaced);
+
new_blob = drm_property_lookup_blob(dev, color_pipeline_blob_id);
if (!new_blob) {
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 13/33] drm: Reset plane color state on pipeline switch request
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
When a pipeline switch is requested by user, driver resets
blobs for all the hardware blocks to get to clean state. These
are then populated with the new blob id's as programmed by user.
For the already enabled hardware blocks, if the user does not
add entry in the new switch request, the blob id's will remain
NULL eventually resulting in disabling of that hardware block.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 52 ++++++++++++++++++++++++++++---
1 file changed, 48 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 259cd4f5f520..9e0fb36d1f47 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -362,6 +362,38 @@ static s32 __user *get_out_fence_for_connector(struct drm_atomic_state *state,
return fence_ptr;
}
+static
+bool color_pipeline_change_requested(struct drm_device *dev,
+ struct drm_property_blob *plane_cp_set_blob,
+ uint64_t blob_id)
+{
+ bool is_change_requested = false;
+ struct drm_property_blob *new_blob = NULL;
+ struct drm_color_pipeline *old_cp, *new_cp;
+
+ /*
+ * User is setting the pipeline for the first time
+ */
+ if (!plane_cp_set_blob)
+ goto out;
+
+ old_cp = plane_cp_set_blob->data;
+
+ if (blob_id != 0) {
+ new_blob = drm_property_lookup_blob(dev, blob_id);
+ if (!new_blob)
+ goto out;
+
+ new_cp = new_blob->data;
+
+ if (old_cp->num != new_cp->num)
+ is_change_requested = true;
+ }
+ drm_property_blob_put(new_blob);
+out:
+ return is_change_requested;
+}
+
static int
drm_atomic_replace_property_blob_from_id(struct drm_device *dev,
struct drm_property_blob **blob,
@@ -727,6 +759,12 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
} else if (property == plane->scaling_filter_property) {
state->scaling_filter = val;
} else if (property == plane->set_color_pipeline_prop) {
+ bool cp_change_requested;
+
+ cp_change_requested = color_pipeline_change_requested(dev,
+ state->set_color_pipeline_data,
+ val);
+
ret = drm_atomic_replace_property_blob_from_id(dev,
&state->set_color_pipeline_data,
val,
@@ -736,12 +774,18 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
if (replaced) {
/* Consider actual color parameter change only when
* individual color blobs are replaced. Hence, reset
- * the replaced boolean.
+ * the replaced boolean but first reset all color
+ * blobs if color pipeline change is requested.
*/
+ if (val && cp_change_requested)
+ ret = drm_plane_reset_color_op_blobs(plane,
+ state, &replaced);
replaced = false;
- ret = drm_plane_replace_color_op_blobs(plane, state,
- val,
- &replaced);
+ if (!ret) {
+ ret = drm_plane_replace_color_op_blobs(plane, state,
+ val,
+ &replaced);
+ }
}
state->color_mgmt_changed |= replaced;
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 13/33] drm: Reset plane color state on pipeline switch request
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
When a pipeline switch is requested by user, driver resets
blobs for all the hardware blocks to get to clean state. These
are then populated with the new blob id's as programmed by user.
For the already enabled hardware blocks, if the user does not
add entry in the new switch request, the blob id's will remain
NULL eventually resulting in disabling of that hardware block.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_uapi.c | 52 ++++++++++++++++++++++++++++---
1 file changed, 48 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 259cd4f5f520..9e0fb36d1f47 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -362,6 +362,38 @@ static s32 __user *get_out_fence_for_connector(struct drm_atomic_state *state,
return fence_ptr;
}
+static
+bool color_pipeline_change_requested(struct drm_device *dev,
+ struct drm_property_blob *plane_cp_set_blob,
+ uint64_t blob_id)
+{
+ bool is_change_requested = false;
+ struct drm_property_blob *new_blob = NULL;
+ struct drm_color_pipeline *old_cp, *new_cp;
+
+ /*
+ * User is setting the pipeline for the first time
+ */
+ if (!plane_cp_set_blob)
+ goto out;
+
+ old_cp = plane_cp_set_blob->data;
+
+ if (blob_id != 0) {
+ new_blob = drm_property_lookup_blob(dev, blob_id);
+ if (!new_blob)
+ goto out;
+
+ new_cp = new_blob->data;
+
+ if (old_cp->num != new_cp->num)
+ is_change_requested = true;
+ }
+ drm_property_blob_put(new_blob);
+out:
+ return is_change_requested;
+}
+
static int
drm_atomic_replace_property_blob_from_id(struct drm_device *dev,
struct drm_property_blob **blob,
@@ -727,6 +759,12 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
} else if (property == plane->scaling_filter_property) {
state->scaling_filter = val;
} else if (property == plane->set_color_pipeline_prop) {
+ bool cp_change_requested;
+
+ cp_change_requested = color_pipeline_change_requested(dev,
+ state->set_color_pipeline_data,
+ val);
+
ret = drm_atomic_replace_property_blob_from_id(dev,
&state->set_color_pipeline_data,
val,
@@ -736,12 +774,18 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,
if (replaced) {
/* Consider actual color parameter change only when
* individual color blobs are replaced. Hence, reset
- * the replaced boolean.
+ * the replaced boolean but first reset all color
+ * blobs if color pipeline change is requested.
*/
+ if (val && cp_change_requested)
+ ret = drm_plane_reset_color_op_blobs(plane,
+ state, &replaced);
replaced = false;
- ret = drm_plane_replace_color_op_blobs(plane, state,
- val,
- &replaced);
+ if (!ret) {
+ ret = drm_plane_replace_color_op_blobs(plane, state,
+ val,
+ &replaced);
+ }
}
state->color_mgmt_changed |= replaced;
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 14/33] drm/i915/color: Add lut range for SDR planes
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add lut range information for SDR planes. This is used to
hint the userspace what kind of LUT values are needed by
the hardware block. Pre-CSC and Post-CSC blocks have similar
lut range for HDR planes.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 55 ++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 5918e2e9bcdd..3900e3748a0e 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3778,6 +3778,61 @@ static const struct intel_color_funcs ilk_color_funcs = {
.get_config = ilk_get_config,
};
+/* FIXME input bpc? */
+static const struct drm_color_lut_range xelpd_pre_post_csc_sdr[] = {
+ /* segment 1 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 32,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = 0, .end = (1 << 16) - (1 << 16) / 33,
+ .min = 0, .max = (1 << 16) - 1,
+ },
+ /* segment 2 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = (1 << 16) - (1 << 16) / 33, .end = 1 << 16,
+ .min = 0, .max = 1 << 16,
+ },
+ /* Segment 3 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = 1 << 16, .end = 3 << 16,
+ .min = 0, .max = (8 << 16) - 1,
+ },
+ /* Segment 4 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = 3 << 16, .end = 7 << 16,
+ .min = 0, .max = (8 << 16) - 1,
+ },
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 14/33] drm/i915/color: Add lut range for SDR planes
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add lut range information for SDR planes. This is used to
hint the userspace what kind of LUT values are needed by
the hardware block. Pre-CSC and Post-CSC blocks have similar
lut range for HDR planes.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 55 ++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 5918e2e9bcdd..3900e3748a0e 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3778,6 +3778,61 @@ static const struct intel_color_funcs ilk_color_funcs = {
.get_config = ilk_get_config,
};
+/* FIXME input bpc? */
+static const struct drm_color_lut_range xelpd_pre_post_csc_sdr[] = {
+ /* segment 1 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 32,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = 0, .end = (1 << 16) - (1 << 16) / 33,
+ .min = 0, .max = (1 << 16) - 1,
+ },
+ /* segment 2 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = (1 << 16) - (1 << 16) / 33, .end = 1 << 16,
+ .min = 0, .max = 1 << 16,
+ },
+ /* Segment 3 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = 1 << 16, .end = 3 << 16,
+ .min = 0, .max = (8 << 16) - 1,
+ },
+ /* Segment 4 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 16, .output_bpc = 16,
+ .start = 3 << 16, .end = 7 << 16,
+ .min = 0, .max = (8 << 16) - 1,
+ },
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 15/33] drm/i915/color: Add lut range for HDR planes
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add lut range information for HDR planes. This is used to
hint the userspace what kind of LUT values are needed by
the hardware block. Pre-CSC and Post-CSC blocks have
different lut ranges for HDR planes.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 108 +++++++++++++++++++++
1 file changed, 108 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 3900e3748a0e..58b6d70043ca 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3833,6 +3833,114 @@ static const struct drm_color_lut_range xelpd_pre_post_csc_sdr[] = {
},
};
+/* FIXME input bpc? */
+static const struct drm_color_lut_range xelpd_pre_csc_hdr[] = {
+ /* segment 1 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 128,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 0, .end = (1 << 24) - 1,
+ .min = 0, .max = (1 << 24) - 1,
+ },
+ /* segment 2 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = (1 << 24) - 1, .end = 1 << 24,
+ .min = 0, .max = (1 << 27) - 1,
+ },
+ /* Segment 3 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 1 << 24, .end = 3 << 24,
+ .min = 0, .max = (1 << 27) - 1,
+ },
+ /* Segment 4 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 3 << 24, .end = 7 << 24,
+ .min = 0, .max = (1 << 27) - 1,
+ },
+};
+
+/* FIXME input bpc? */
+static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
+ /*
+ * ToDo: Add Segment 1
+ * There is an optional fine segment added with 9 lut values
+ * Will be added later
+ */
+
+ /* segment 2 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 32,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 0, .end = (1 << 24) - 1,
+ .min = 0, .max = (1 << 24) - 1,
+ },
+ /* segment 3 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = (1 << 24) - 1, .end = 1 << 24,
+ .min = 0, .max = 1 << 24,
+ },
+ /* Segment 4 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 1 << 24, .end = 3 << 24,
+ .min = 0, .max = (3 << 24),
+ },
+ /* Segment 5 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 3 << 24, .end = 7 << 24,
+ .min = 0, .max = (7 << 24),
+ },
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 15/33] drm/i915/color: Add lut range for HDR planes
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add lut range information for HDR planes. This is used to
hint the userspace what kind of LUT values are needed by
the hardware block. Pre-CSC and Post-CSC blocks have
different lut ranges for HDR planes.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 108 +++++++++++++++++++++
1 file changed, 108 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 3900e3748a0e..58b6d70043ca 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3833,6 +3833,114 @@ static const struct drm_color_lut_range xelpd_pre_post_csc_sdr[] = {
},
};
+/* FIXME input bpc? */
+static const struct drm_color_lut_range xelpd_pre_csc_hdr[] = {
+ /* segment 1 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 128,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 0, .end = (1 << 24) - 1,
+ .min = 0, .max = (1 << 24) - 1,
+ },
+ /* segment 2 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = (1 << 24) - 1, .end = 1 << 24,
+ .min = 0, .max = (1 << 27) - 1,
+ },
+ /* Segment 3 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 1 << 24, .end = 3 << 24,
+ .min = 0, .max = (1 << 27) - 1,
+ },
+ /* Segment 4 */
+ {
+ .flags = (DRM_MODE_LUT_PRE_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 3 << 24, .end = 7 << 24,
+ .min = 0, .max = (1 << 27) - 1,
+ },
+};
+
+/* FIXME input bpc? */
+static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
+ /*
+ * ToDo: Add Segment 1
+ * There is an optional fine segment added with 9 lut values
+ * Will be added later
+ */
+
+ /* segment 2 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 32,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 0, .end = (1 << 24) - 1,
+ .min = 0, .max = (1 << 24) - 1,
+ },
+ /* segment 3 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = (1 << 24) - 1, .end = 1 << 24,
+ .min = 0, .max = 1 << 24,
+ },
+ /* Segment 4 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 1 << 24, .end = 3 << 24,
+ .min = 0, .max = (3 << 24),
+ },
+ /* Segment 5 */
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_REUSE_LAST |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 1,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 3 << 24, .end = 7 << 24,
+ .min = 0, .max = (7 << 24),
+ },
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 16/33] drm/i915/color: Add color pipeline for HDR planes
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add color pipeline for HDR planes. It consists of the following
hardware blocks.
* Pre-CSC : This block can used to linearize the input frame buffer data.
The linear data then can be further acted on by the following
color hardware blocks in the display hardware pipeline
* CSC/CTM: Used to program color transformation matrix, this block is used
to perform color space conversions like BT2020 to BT709 or BT601
etc. This block acts on the linearized data coming from the
Pre-CSC HW block.
* Post-CSC: This HW block can be used to non-linearize frame buffer data to
match the sink. Another use case of it could be to perform Tone
mapping for HDR use-cases.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 58b6d70043ca..8c2a858fc452 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3941,6 +3941,24 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
},
};
+struct drm_color_op color_pipeline_hdr[] = {
+ {
+ .name = DRM_CB_PRE_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0, /* To be updated during plane initialization */
+ },
+ {
+ .name = DRM_CB_CSC,
+ .type = MATRIX,
+ .blob_id = 0,
+ },
+ {
+ .name = DRM_CB_POST_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0,
+ },
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 16/33] drm/i915/color: Add color pipeline for HDR planes
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add color pipeline for HDR planes. It consists of the following
hardware blocks.
* Pre-CSC : This block can used to linearize the input frame buffer data.
The linear data then can be further acted on by the following
color hardware blocks in the display hardware pipeline
* CSC/CTM: Used to program color transformation matrix, this block is used
to perform color space conversions like BT2020 to BT709 or BT601
etc. This block acts on the linearized data coming from the
Pre-CSC HW block.
* Post-CSC: This HW block can be used to non-linearize frame buffer data to
match the sink. Another use case of it could be to perform Tone
mapping for HDR use-cases.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 58b6d70043ca..8c2a858fc452 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3941,6 +3941,24 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
},
};
+struct drm_color_op color_pipeline_hdr[] = {
+ {
+ .name = DRM_CB_PRE_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0, /* To be updated during plane initialization */
+ },
+ {
+ .name = DRM_CB_CSC,
+ .type = MATRIX,
+ .blob_id = 0,
+ },
+ {
+ .name = DRM_CB_POST_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0,
+ },
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 17/33] drm/i915/color: Add color pipeline for SDR planes
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
SDR planes provides programmable color hardware blocks for
Pre-CSC and Post-CSC operations. Add a color pipeline to
expose these capabilities.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 8c2a858fc452..09e50659befd 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3941,6 +3941,23 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
},
};
+struct drm_color_op color_pipeline_sdr[] = {
+ {
+ .name = DRM_CB_PRE_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0, /* To be updated during plane initialization */
+ },
+ /*
+ * SDR planes have fixed function CSC capabilities.
+ * TODO: Add support for it
+ */
+ {
+ .name = DRM_CB_POST_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0,
+ },
+};
+
struct drm_color_op color_pipeline_hdr[] = {
{
.name = DRM_CB_PRE_CSC,
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 17/33] drm/i915/color: Add color pipeline for SDR planes
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
SDR planes provides programmable color hardware blocks for
Pre-CSC and Post-CSC operations. Add a color pipeline to
expose these capabilities.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 8c2a858fc452..09e50659befd 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3941,6 +3941,23 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
},
};
+struct drm_color_op color_pipeline_sdr[] = {
+ {
+ .name = DRM_CB_PRE_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0, /* To be updated during plane initialization */
+ },
+ /*
+ * SDR planes have fixed function CSC capabilities.
+ * TODO: Add support for it
+ */
+ {
+ .name = DRM_CB_POST_CSC,
+ .type = CURVE_1D,
+ .blob_id = 0,
+ },
+};
+
struct drm_color_op color_pipeline_hdr[] = {
{
.name = DRM_CB_PRE_CSC,
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 18/33] drm/i915/color: Add HDR plane LUT range data to color pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Create a helper function to add details about LUT ranges that HDR
planes can support. Userspace can parse through this information
to generate proper LUT data for respective hardware blocks. It will
be exposed to the user space by the color pipeline.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 47 ++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 09e50659befd..99ae3f4fca05 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -28,6 +28,7 @@
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dsb.h"
+#include "skl_universal_plane.h"
struct intel_color_funcs {
int (*color_check)(struct intel_crtc_state *crtc_state);
@@ -3976,6 +3977,52 @@ struct drm_color_op color_pipeline_hdr[] = {
},
};
+__maybe_unused
+static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
+{
+ struct drm_i915_private *i915 = to_i915(plane->dev);
+ struct drm_property_blob *blob[2] = {NULL};
+ int ret = 0, i = 0;
+
+ if (icl_is_hdr_plane(i915, to_intel_plane(plane)->id)) {
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(xelpd_pre_csc_hdr),
+ xelpd_pre_csc_hdr);
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ /*
+ * In HDR color pipeline PRE-CSC and POST-CSC are positioned
+ * at 0th and 2nd index/position
+ */
+ color_pipeline_hdr[0].blob_id =
+ blob[i++]->base.id;
+
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(xelpd_post_csc_hdr),
+ xelpd_post_csc_hdr);
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ color_pipeline_hdr[2].blob_id =
+ blob[i++]->base.id;
+ }
+
+out:
+ if (ret) {
+ for (int j = 0; j < i; j++) {
+ if (blob[j])
+ drm_property_blob_put(blob[j]);
+ }
+ }
+
+ return ret;
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 18/33] drm/i915/color: Add HDR plane LUT range data to color pipeline
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Create a helper function to add details about LUT ranges that HDR
planes can support. Userspace can parse through this information
to generate proper LUT data for respective hardware blocks. It will
be exposed to the user space by the color pipeline.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 47 ++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 09e50659befd..99ae3f4fca05 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -28,6 +28,7 @@
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dsb.h"
+#include "skl_universal_plane.h"
struct intel_color_funcs {
int (*color_check)(struct intel_crtc_state *crtc_state);
@@ -3976,6 +3977,52 @@ struct drm_color_op color_pipeline_hdr[] = {
},
};
+__maybe_unused
+static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
+{
+ struct drm_i915_private *i915 = to_i915(plane->dev);
+ struct drm_property_blob *blob[2] = {NULL};
+ int ret = 0, i = 0;
+
+ if (icl_is_hdr_plane(i915, to_intel_plane(plane)->id)) {
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(xelpd_pre_csc_hdr),
+ xelpd_pre_csc_hdr);
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ /*
+ * In HDR color pipeline PRE-CSC and POST-CSC are positioned
+ * at 0th and 2nd index/position
+ */
+ color_pipeline_hdr[0].blob_id =
+ blob[i++]->base.id;
+
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(xelpd_post_csc_hdr),
+ xelpd_post_csc_hdr);
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ color_pipeline_hdr[2].blob_id =
+ blob[i++]->base.id;
+ }
+
+out:
+ if (ret) {
+ for (int j = 0; j < i; j++) {
+ if (blob[j])
+ drm_property_blob_put(blob[j]);
+ }
+ }
+
+ return ret;
+};
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 19/33] drm/i915/color: Add SDR plane LUT range data to color pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Add LUT ranges for color blocks in SDR planes. Userspace can
parse through this information to generate proper LUT data for
respective hardware blocks. It will be exposed to the user space
by the color pipeline.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 99ae3f4fca05..a8c6be70c859 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4010,6 +4010,22 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
color_pipeline_hdr[2].blob_id =
blob[i++]->base.id;
+ } else {
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(xelpd_pre_post_csc_sdr),
+ xelpd_pre_post_csc_sdr);
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ /*
+ * In SDR color pipeline PRE-CSC and POST-CSC blocks are positioned
+ * at 0th and 1st index/postion.
+ * LUT ranges for SDR planes are similar for pre and post-csc blocks
+ */
+ color_pipeline_sdr[0].blob_id =
+ color_pipeline_sdr[1].blob_id = blob[i++]->base.id;
}
out:
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 19/33] drm/i915/color: Add SDR plane LUT range data to color pipeline
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Add LUT ranges for color blocks in SDR planes. Userspace can
parse through this information to generate proper LUT data for
respective hardware blocks. It will be exposed to the user space
by the color pipeline.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 99ae3f4fca05..a8c6be70c859 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4010,6 +4010,22 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
color_pipeline_hdr[2].blob_id =
blob[i++]->base.id;
+ } else {
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(xelpd_pre_post_csc_sdr),
+ xelpd_pre_post_csc_sdr);
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ /*
+ * In SDR color pipeline PRE-CSC and POST-CSC blocks are positioned
+ * at 0th and 1st index/postion.
+ * LUT ranges for SDR planes are similar for pre and post-csc blocks
+ */
+ color_pipeline_sdr[0].blob_id =
+ color_pipeline_sdr[1].blob_id = blob[i++]->base.id;
}
out:
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 20/33] drm/i915/color: Add color pipelines to plane
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Add supported plane color pipelines. To represent all
hardware blocks in their inactive state, we introduce a pipeline
called "no color pipeline" which is the default pipeline.
Add respective color pipelines for SDR and HDR planes. Create and
attach plane enum property "GET_COLOR_PIPELINE" to expose these
pipelines to userspace.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 31 +++++++++++++++++++++-
drivers/gpu/drm/i915/display/intel_color.h | 3 ++-
2 files changed, 32 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index a8c6be70c859..9f5d2cd0f97a 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3977,7 +3977,6 @@ struct drm_color_op color_pipeline_hdr[] = {
},
};
-__maybe_unused
static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
{
struct drm_i915_private *i915 = to_i915(plane->dev);
@@ -4039,6 +4038,36 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
return ret;
};
+__maybe_unused
+void intel_color_plane_init(struct drm_plane *plane)
+{
+ struct drm_i915_private *i915 = to_i915(plane->dev);
+
+ if (DISPLAY_VER(i915) < 13)
+ return;
+
+ drm_plane_create_get_color_pipeline_property(plane->dev, plane, 2);
+
+ intel_prepare_plane_color_pipeline(plane);
+
+ /*
+ * default pipeline is set as 0 or "no color pipeline". All color h/w
+ * blocks are disabled at this stage.
+ */
+ drm_plane_add_color_pipeline(plane, "no color pipeline", NULL, 0);
+
+ if (icl_is_hdr_plane(i915, to_intel_plane(plane)->id))
+ drm_plane_add_color_pipeline(plane, "color pipeline hdr",
+ color_pipeline_hdr,
+ sizeof(color_pipeline_hdr));
+ else
+ drm_plane_add_color_pipeline(plane, "color pipeline sdr",
+ color_pipeline_sdr,
+ sizeof(color_pipeline_sdr));
+
+ drm_plane_attach_get_color_pipeline_property(plane);
+}
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index 8002492be709..aa649d13c6fa 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -10,6 +10,7 @@
struct intel_crtc_state;
struct intel_crtc;
+struct drm_plane;
struct drm_i915_private;
struct drm_property_blob;
@@ -29,5 +30,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,
const struct drm_property_blob *blob2,
bool is_pre_csc_lut);
void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
-
+void intel_color_plane_init(struct drm_plane *plane);
#endif /* __INTEL_COLOR_H__ */
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 20/33] drm/i915/color: Add color pipelines to plane
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Add supported plane color pipelines. To represent all
hardware blocks in their inactive state, we introduce a pipeline
called "no color pipeline" which is the default pipeline.
Add respective color pipelines for SDR and HDR planes. Create and
attach plane enum property "GET_COLOR_PIPELINE" to expose these
pipelines to userspace.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 31 +++++++++++++++++++++-
drivers/gpu/drm/i915/display/intel_color.h | 3 ++-
2 files changed, 32 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index a8c6be70c859..9f5d2cd0f97a 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3977,7 +3977,6 @@ struct drm_color_op color_pipeline_hdr[] = {
},
};
-__maybe_unused
static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
{
struct drm_i915_private *i915 = to_i915(plane->dev);
@@ -4039,6 +4038,36 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
return ret;
};
+__maybe_unused
+void intel_color_plane_init(struct drm_plane *plane)
+{
+ struct drm_i915_private *i915 = to_i915(plane->dev);
+
+ if (DISPLAY_VER(i915) < 13)
+ return;
+
+ drm_plane_create_get_color_pipeline_property(plane->dev, plane, 2);
+
+ intel_prepare_plane_color_pipeline(plane);
+
+ /*
+ * default pipeline is set as 0 or "no color pipeline". All color h/w
+ * blocks are disabled at this stage.
+ */
+ drm_plane_add_color_pipeline(plane, "no color pipeline", NULL, 0);
+
+ if (icl_is_hdr_plane(i915, to_intel_plane(plane)->id))
+ drm_plane_add_color_pipeline(plane, "color pipeline hdr",
+ color_pipeline_hdr,
+ sizeof(color_pipeline_hdr));
+ else
+ drm_plane_add_color_pipeline(plane, "color pipeline sdr",
+ color_pipeline_sdr,
+ sizeof(color_pipeline_sdr));
+
+ drm_plane_attach_get_color_pipeline_property(plane);
+}
+
void intel_color_crtc_init(struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(crtc->base.dev);
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index 8002492be709..aa649d13c6fa 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -10,6 +10,7 @@
struct intel_crtc_state;
struct intel_crtc;
+struct drm_plane;
struct drm_i915_private;
struct drm_property_blob;
@@ -29,5 +30,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,
const struct drm_property_blob *blob2,
bool is_pre_csc_lut);
void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
-
+void intel_color_plane_init(struct drm_plane *plane);
#endif /* __INTEL_COLOR_H__ */
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 21/33] drm/i915/color: Create and attach set color pipeline property
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Create and attach "SET_COLOR_PIPELINE" property to planes.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 9f5d2cd0f97a..feff8ac45f47 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4066,6 +4066,9 @@ void intel_color_plane_init(struct drm_plane *plane)
sizeof(color_pipeline_sdr));
drm_plane_attach_get_color_pipeline_property(plane);
+
+ drm_plane_create_set_color_pipeline_property(plane->dev, plane);
+ drm_plane_attach_set_color_pipeline_property(plane);
}
void intel_color_crtc_init(struct intel_crtc *crtc)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 21/33] drm/i915/color: Create and attach set color pipeline property
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Create and attach "SET_COLOR_PIPELINE" property to planes.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 9f5d2cd0f97a..feff8ac45f47 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4066,6 +4066,9 @@ void intel_color_plane_init(struct drm_plane *plane)
sizeof(color_pipeline_sdr));
drm_plane_attach_get_color_pipeline_property(plane);
+
+ drm_plane_create_set_color_pipeline_property(plane->dev, plane);
+ drm_plane_attach_set_color_pipeline_property(plane);
}
void intel_color_crtc_init(struct intel_crtc *crtc)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 22/33] drm/i915/color: Add plane color callbacks
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add callbacks for color plane operations.
load_plane_luts: used to load pre/post csc luts
load_plane_csc_matrix: used to load csc matrix
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index feff8ac45f47..faf16107d339 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -81,6 +81,12 @@ struct intel_color_funcs {
* Read config other than LUTs and CSCs, before them. Optional.
*/
void (*get_config)(struct intel_crtc_state *crtc_state);
+
+ /*
+ * Plane color callbacks
+ */
+ void (*load_plane_csc_matrix)(const struct drm_plane_state *plane_state);
+ void (*load_plane_luts)(const struct drm_plane_state *plane_state);
};
#define CTM_COEFF_SIGN (1ULL << 63)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 22/33] drm/i915/color: Add plane color callbacks
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add callbacks for color plane operations.
load_plane_luts: used to load pre/post csc luts
load_plane_csc_matrix: used to load csc matrix
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index feff8ac45f47..faf16107d339 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -81,6 +81,12 @@ struct intel_color_funcs {
* Read config other than LUTs and CSCs, before them. Optional.
*/
void (*get_config)(struct intel_crtc_state *crtc_state);
+
+ /*
+ * Plane color callbacks
+ */
+ void (*load_plane_csc_matrix)(const struct drm_plane_state *plane_state);
+ void (*load_plane_luts)(const struct drm_plane_state *plane_state);
};
#define CTM_COEFF_SIGN (1ULL << 63)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 23/33] drm/i915/color: Load plane color luts from atomic flip
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Load plane color luts as part of atomic plane updates.
This will be done only if the plane color luts are changed.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 8 ++++++++
drivers/gpu/drm/i915/display/intel_color.h | 2 ++
drivers/gpu/drm/i915/display/skl_universal_plane.c | 4 ++++
3 files changed, 14 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index faf16107d339..df2fc8f98dc9 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -1863,6 +1863,14 @@ void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
i915->display.funcs.color->load_luts(crtc_state);
}
+void intel_color_load_plane_luts(const struct drm_plane_state *plane_state)
+{
+ struct drm_i915_private *i915 = to_i915(plane_state->plane->dev);
+
+ if (i915->display.funcs.color->load_plane_luts)
+ i915->display.funcs.color->load_plane_luts(plane_state);
+}
+
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index aa649d13c6fa..93382df101d9 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -13,6 +13,7 @@ struct intel_crtc;
struct drm_plane;
struct drm_i915_private;
struct drm_property_blob;
+struct drm_plane_state;
void intel_color_init_hooks(struct drm_i915_private *i915);
int intel_color_init(struct drm_i915_private *i915);
@@ -31,4 +32,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,
bool is_pre_csc_lut);
void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
void intel_color_plane_init(struct drm_plane *plane);
+void intel_color_load_plane_luts(const struct drm_plane_state *plane_state);
#endif /* __INTEL_COLOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index 4566c95da1ca..ce7c367fe2da 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -11,6 +11,7 @@
#include "i915_drv.h"
#include "i915_reg.h"
#include "intel_atomic_plane.h"
+#include "intel_color.h"
#include "intel_de.h"
#include "intel_display_irq.h"
#include "intel_display_types.h"
@@ -1268,6 +1269,9 @@ icl_plane_update_noarm(struct intel_plane *plane,
if (plane_state->force_black)
icl_plane_csc_load_black(plane);
+ if (plane_state->uapi.color_mgmt_changed)
+ intel_color_load_plane_luts(&plane_state->uapi);
+
intel_psr2_program_plane_sel_fetch_noarm(plane, crtc_state, plane_state, color_plane);
}
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 23/33] drm/i915/color: Load plane color luts from atomic flip
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Load plane color luts as part of atomic plane updates.
This will be done only if the plane color luts are changed.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 8 ++++++++
drivers/gpu/drm/i915/display/intel_color.h | 2 ++
drivers/gpu/drm/i915/display/skl_universal_plane.c | 4 ++++
3 files changed, 14 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index faf16107d339..df2fc8f98dc9 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -1863,6 +1863,14 @@ void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
i915->display.funcs.color->load_luts(crtc_state);
}
+void intel_color_load_plane_luts(const struct drm_plane_state *plane_state)
+{
+ struct drm_i915_private *i915 = to_i915(plane_state->plane->dev);
+
+ if (i915->display.funcs.color->load_plane_luts)
+ i915->display.funcs.color->load_plane_luts(plane_state);
+}
+
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index aa649d13c6fa..93382df101d9 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -13,6 +13,7 @@ struct intel_crtc;
struct drm_plane;
struct drm_i915_private;
struct drm_property_blob;
+struct drm_plane_state;
void intel_color_init_hooks(struct drm_i915_private *i915);
int intel_color_init(struct drm_i915_private *i915);
@@ -31,4 +32,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,
bool is_pre_csc_lut);
void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
void intel_color_plane_init(struct drm_plane *plane);
+void intel_color_load_plane_luts(const struct drm_plane_state *plane_state);
#endif /* __INTEL_COLOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index 4566c95da1ca..ce7c367fe2da 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -11,6 +11,7 @@
#include "i915_drv.h"
#include "i915_reg.h"
#include "intel_atomic_plane.h"
+#include "intel_color.h"
#include "intel_de.h"
#include "intel_display_irq.h"
#include "intel_display_types.h"
@@ -1268,6 +1269,9 @@ icl_plane_update_noarm(struct intel_plane *plane,
if (plane_state->force_black)
icl_plane_csc_load_black(plane);
+ if (plane_state->uapi.color_mgmt_changed)
+ intel_color_load_plane_luts(&plane_state->uapi);
+
intel_psr2_program_plane_sel_fetch_noarm(plane, crtc_state, plane_state, color_plane);
}
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 24/33] drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Extended glk_plane_color_ctl to have plane color checks. This helps
enabling the csc, degamma or gamma block based on user inputs.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/skl_universal_plane.c | 13 ++++++++++++-
drivers/gpu/drm/i915/i915_reg.h | 1 +
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index ce7c367fe2da..c08875fa965e 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -965,7 +965,18 @@ static u32 glk_plane_color_ctl(const struct intel_crtc_state *crtc_state,
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
u32 plane_color_ctl = 0;
- plane_color_ctl |= PLANE_COLOR_PLANE_GAMMA_DISABLE;
+ /* FIXME needs hw.gamma_lut */
+ if (!plane_state->uapi.color.pre_csc_lut)
+ plane_color_ctl |= PLANE_COLOR_PLANE_GAMMA_DISABLE;
+
+ /* FIXME needs hw.degamma_lut */
+ if (plane_state->uapi.color.post_csc_lut)
+ plane_color_ctl |= PLANE_COLOR_PRE_CSC_GAMMA_ENABLE;
+
+ /* FIXME needs hw.ctm */
+ if (plane_state->uapi.color.ctm)
+ plane_color_ctl |= PLANE_COLOR_PLANE_CSC_ENABLE;
+
plane_color_ctl |= glk_plane_color_ctl_alpha(plane_state);
if (fb->format->is_yuv && !icl_is_hdr_plane(dev_priv, plane->id)) {
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index e00e4d569ba9..d50bfe1000d3 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -3730,6 +3730,7 @@
#define PLANE_COLOR_CSC_MODE_YUV709_TO_RGB709 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 2)
#define PLANE_COLOR_CSC_MODE_YUV2020_TO_RGB2020 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 3)
#define PLANE_COLOR_CSC_MODE_RGB709_TO_RGB2020 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 4)
+#define PLANE_COLOR_PRE_CSC_GAMMA_ENABLE REG_BIT(14)
#define PLANE_COLOR_PLANE_GAMMA_DISABLE REG_BIT(13)
#define PLANE_COLOR_ALPHA_MASK REG_GENMASK(5, 4)
#define PLANE_COLOR_ALPHA_DISABLE REG_FIELD_PREP(PLANE_COLOR_ALPHA_MASK, 0)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 24/33] drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Extended glk_plane_color_ctl to have plane color checks. This helps
enabling the csc, degamma or gamma block based on user inputs.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/skl_universal_plane.c | 13 ++++++++++++-
drivers/gpu/drm/i915/i915_reg.h | 1 +
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index ce7c367fe2da..c08875fa965e 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -965,7 +965,18 @@ static u32 glk_plane_color_ctl(const struct intel_crtc_state *crtc_state,
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
u32 plane_color_ctl = 0;
- plane_color_ctl |= PLANE_COLOR_PLANE_GAMMA_DISABLE;
+ /* FIXME needs hw.gamma_lut */
+ if (!plane_state->uapi.color.pre_csc_lut)
+ plane_color_ctl |= PLANE_COLOR_PLANE_GAMMA_DISABLE;
+
+ /* FIXME needs hw.degamma_lut */
+ if (plane_state->uapi.color.post_csc_lut)
+ plane_color_ctl |= PLANE_COLOR_PRE_CSC_GAMMA_ENABLE;
+
+ /* FIXME needs hw.ctm */
+ if (plane_state->uapi.color.ctm)
+ plane_color_ctl |= PLANE_COLOR_PLANE_CSC_ENABLE;
+
plane_color_ctl |= glk_plane_color_ctl_alpha(plane_state);
if (fb->format->is_yuv && !icl_is_hdr_plane(dev_priv, plane->id)) {
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index e00e4d569ba9..d50bfe1000d3 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -3730,6 +3730,7 @@
#define PLANE_COLOR_CSC_MODE_YUV709_TO_RGB709 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 2)
#define PLANE_COLOR_CSC_MODE_YUV2020_TO_RGB2020 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 3)
#define PLANE_COLOR_CSC_MODE_RGB709_TO_RGB2020 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 4)
+#define PLANE_COLOR_PRE_CSC_GAMMA_ENABLE REG_BIT(14)
#define PLANE_COLOR_PLANE_GAMMA_DISABLE REG_BIT(13)
#define PLANE_COLOR_ALPHA_MASK REG_GENMASK(5, 4)
#define PLANE_COLOR_ALPHA_DISABLE REG_FIELD_PREP(PLANE_COLOR_ALPHA_MASK, 0)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 25/33] drm/i915/xelpd: Add register definitions for Plane Degamma
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add macros to define Plane Degamma registers
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/i915_reg.h | 49 +++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index d50bfe1000d3..5fa7461066ab 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -6226,6 +6226,55 @@ enum skl_power_gate {
#define _MMIO_PLANE_GAMC(plane, i, a, b) _MMIO(_PIPE(plane, a, b) + (i) * 4)
+/* Display13 Plane Degmma Reg */
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A 0x701d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B 0x711d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_A 0x702d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_B 0x712d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B)
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_B)
+#define PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe))
+
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A 0x701d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B 0x711d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2_A 0x702d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2_B 0x712d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B)
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_ENH_2_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_2_B)
+#define PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_2(pipe))
+
+#define _PLANE_PRE_CSC_GAMC_INDEX_1_A 0x704d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_1_B 0x714d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_2_A 0x705d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_2_B 0x715d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_1_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_1_B)
+#define _PLANE_PRE_CSC_GAMC_INDEX_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_2_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_2_B)
+#define PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_2(pipe))
+
+#define _PLANE_PRE_CSC_GAMC_DATA_1_A 0x704d4
+#define _PLANE_PRE_CSC_GAMC_DATA_1_B 0x714d4
+#define _PLANE_PRE_CSC_GAMC_DATA_2_A 0x705d4
+#define _PLANE_PRE_CSC_GAMC_DATA_2_B 0x715d4
+#define _PLANE_PRE_CSC_GAMC_DATA_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_1_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_1_B)
+#define _PLANE_PRE_CSC_GAMC_DATA_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_2_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_2_B)
+#define PLANE_PRE_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_2(pipe))
+
/* Plane CSC Registers */
#define _PLANE_CSC_RY_GY_1_A 0x70210
#define _PLANE_CSC_RY_GY_2_A 0x70310
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 25/33] drm/i915/xelpd: Add register definitions for Plane Degamma
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add macros to define Plane Degamma registers
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/i915_reg.h | 49 +++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index d50bfe1000d3..5fa7461066ab 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -6226,6 +6226,55 @@ enum skl_power_gate {
#define _MMIO_PLANE_GAMC(plane, i, a, b) _MMIO(_PIPE(plane, a, b) + (i) * 4)
+/* Display13 Plane Degmma Reg */
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A 0x701d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B 0x711d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_A 0x702d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_B 0x712d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B)
+#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_B)
+#define PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe))
+
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A 0x701d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B 0x711d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2_A 0x702d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2_B 0x712d4
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B)
+#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_ENH_2_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_2_B)
+#define PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_2(pipe))
+
+#define _PLANE_PRE_CSC_GAMC_INDEX_1_A 0x704d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_1_B 0x714d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_2_A 0x705d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_2_B 0x715d0
+#define _PLANE_PRE_CSC_GAMC_INDEX_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_1_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_1_B)
+#define _PLANE_PRE_CSC_GAMC_INDEX_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_2_A, \
+ _PLANE_PRE_CSC_GAMC_INDEX_2_B)
+#define PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_2(pipe))
+
+#define _PLANE_PRE_CSC_GAMC_DATA_1_A 0x704d4
+#define _PLANE_PRE_CSC_GAMC_DATA_1_B 0x714d4
+#define _PLANE_PRE_CSC_GAMC_DATA_2_A 0x705d4
+#define _PLANE_PRE_CSC_GAMC_DATA_2_B 0x715d4
+#define _PLANE_PRE_CSC_GAMC_DATA_1(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_1_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_1_B)
+#define _PLANE_PRE_CSC_GAMC_DATA_2(pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_2_A, \
+ _PLANE_PRE_CSC_GAMC_DATA_2_B)
+#define PLANE_PRE_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_2(pipe))
+
/* Plane CSC Registers */
#define _PLANE_CSC_RY_GY_1_A 0x70210
#define _PLANE_CSC_RY_GY_2_A 0x70310
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 26/33] drm/i915/color: Add color functions for ADL
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Register color callbacks for ADL and beyond. While we have to register
new callbacks for pre-blending color operations, re-use callbacks for
post-blend operations.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index df2fc8f98dc9..3f3c1ac10330 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3704,6 +3704,16 @@ static const struct intel_color_funcs i9xx_color_funcs = {
.get_config = i9xx_get_config,
};
+static const struct intel_color_funcs xelpd_color_funcs = {
+ .color_check = icl_color_check,
+ .color_commit_noarm = icl_color_commit_noarm,
+ .color_commit_arm = icl_color_commit_arm,
+ .load_luts = icl_load_luts,
+ .read_luts = icl_read_luts,
+ .lut_equal = icl_lut_equal,
+ .read_csc = icl_read_csc,
+};
+
static const struct intel_color_funcs tgl_color_funcs = {
.color_check = icl_color_check,
.color_commit_noarm = icl_color_commit_noarm,
@@ -4141,7 +4151,9 @@ void intel_color_init_hooks(struct drm_i915_private *i915)
else
i915->display.funcs.color = &i9xx_color_funcs;
} else {
- if (DISPLAY_VER(i915) >= 12)
+ if (DISPLAY_VER(i915) >= 13)
+ i915->display.funcs.color = &xelpd_color_funcs;
+ else if (DISPLAY_VER(i915) == 12)
i915->display.funcs.color = &tgl_color_funcs;
else if (DISPLAY_VER(i915) == 11)
i915->display.funcs.color = &icl_color_funcs;
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 26/33] drm/i915/color: Add color functions for ADL
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Register color callbacks for ADL and beyond. While we have to register
new callbacks for pre-blending color operations, re-use callbacks for
post-blend operations.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index df2fc8f98dc9..3f3c1ac10330 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -3704,6 +3704,16 @@ static const struct intel_color_funcs i9xx_color_funcs = {
.get_config = i9xx_get_config,
};
+static const struct intel_color_funcs xelpd_color_funcs = {
+ .color_check = icl_color_check,
+ .color_commit_noarm = icl_color_commit_noarm,
+ .color_commit_arm = icl_color_commit_arm,
+ .load_luts = icl_load_luts,
+ .read_luts = icl_read_luts,
+ .lut_equal = icl_lut_equal,
+ .read_csc = icl_read_csc,
+};
+
static const struct intel_color_funcs tgl_color_funcs = {
.color_check = icl_color_check,
.color_commit_noarm = icl_color_commit_noarm,
@@ -4141,7 +4151,9 @@ void intel_color_init_hooks(struct drm_i915_private *i915)
else
i915->display.funcs.color = &i9xx_color_funcs;
} else {
- if (DISPLAY_VER(i915) >= 12)
+ if (DISPLAY_VER(i915) >= 13)
+ i915->display.funcs.color = &xelpd_color_funcs;
+ else if (DISPLAY_VER(i915) == 12)
i915->display.funcs.color = &tgl_color_funcs;
else if (DISPLAY_VER(i915) == 11)
i915->display.funcs.color = &icl_color_funcs;
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 27/33] drm/i915/color: Program Plane Pre-CSC Registers
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Extract the LUT and program plane pre-csc registers.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 120 +++++++++++++++++++++
drivers/gpu/drm/i915/i915_reg.h | 1 +
2 files changed, 121 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 3f3c1ac10330..56bcf750b047 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -185,6 +185,29 @@ static bool lut_is_legacy(const struct drm_property_blob *lut)
return lut && drm_color_lut_size(lut) == LEGACY_LUT_LENGTH;
}
+/*
+ * Added to accommodate enhanced LUT precision.
+ * Max LUT precision is 32 bits.
+ */
+static u64 drm_color_lut_extract_ext(u64 user_input, u32 bit_precision)
+{
+ u64 val = user_input & 0xffffffff;
+ u32 max;
+
+ if (bit_precision > 32)
+ return 0;
+
+ max = 0xffffffff >> (32 - bit_precision);
+ /* Round only if we're not using full precision. */
+ if (bit_precision < 32) {
+ val += 1UL << (32 - bit_precision - 1);
+ val >>= 32 - bit_precision;
+ }
+
+ return ((user_input & 0xffffffff00000000) |
+ clamp_val(val, 0, max));
+}
+
/*
* When using limited range, multiply the matrix given by userspace by
* the matrix that we would use for the limited range.
@@ -1856,6 +1879,102 @@ static void chv_load_luts(const struct intel_crtc_state *crtc_state)
crtc_state->cgm_mode);
}
+static void xelpd_program_plane_pre_csc_lut(const struct drm_plane_state *state,
+ struct drm_color_lut_ext *pre_csc_lut,
+ u32 offset)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->plane->dev);
+ enum pipe pipe = to_intel_plane(state->plane)->pipe;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ u32 i, lut_size;
+
+ if (icl_is_hdr_plane(dev_priv, plane)) {
+ lut_size = 128;
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, 0),
+ PLANE_PAL_PREC_AUTO_INCREMENT);
+
+ if (pre_csc_lut) {
+ for (i = 0; i < lut_size; i++) {
+ u64 word = drm_color_lut_extract_ext(pre_csc_lut[i].green, 24);
+ u32 lut_val = (word & 0xffffff);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ lut_val);
+ }
+
+ /* Program the max register to clamp values > 1.0. */
+ while (i < 131)
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ pre_csc_lut[i++].green);
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 24) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ 1 << 24);
+ } while (i++ < 130);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, 0), 0);
+ } else {
+ lut_size = 32;
+
+ /*
+ * First 3 planes are HDR, so reduce by 3 to get to the right
+ * SDR plane offset
+ */
+ plane = plane - 3;
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, 0),
+ PLANE_PAL_PREC_AUTO_INCREMENT);
+
+ if (pre_csc_lut) {
+ for (i = 0; i < lut_size; i++)
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0),
+ pre_csc_lut[i].green);
+ /* Program the max register to clamp values > 1.0. */
+ while (i < 35)
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0),
+ pre_csc_lut[i++].green);
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 16) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0),
+ 1 << 16);
+ } while (i++ < 34);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, 0), 0);
+ }
+}
+
+static void xelpd_plane_load_luts(const struct drm_plane_state *plane_state)
+{
+ const struct drm_property_blob *pre_csc_lut_blob =
+ plane_state->color.pre_csc_lut;
+ struct drm_color_lut_ext *pre_csc_lut = NULL;
+
+ if (pre_csc_lut_blob) {
+ pre_csc_lut = pre_csc_lut_blob->data;
+ xelpd_program_plane_pre_csc_lut(plane_state, pre_csc_lut, 0);
+ }
+}
+
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -3712,6 +3831,7 @@ static const struct intel_color_funcs xelpd_color_funcs = {
.read_luts = icl_read_luts,
.lut_equal = icl_lut_equal,
.read_csc = icl_read_csc,
+ .load_plane_luts = xelpd_plane_load_luts,
};
static const struct intel_color_funcs tgl_color_funcs = {
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index 5fa7461066ab..d26d6294d231 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -6238,6 +6238,7 @@ enum skl_power_gate {
#define PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
_MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \
_PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe))
+#define PLANE_PAL_PREC_AUTO_INCREMENT REG_BIT(10)
#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A 0x701d4
#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B 0x711d4
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 27/33] drm/i915/color: Program Plane Pre-CSC Registers
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Extract the LUT and program plane pre-csc registers.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 120 +++++++++++++++++++++
drivers/gpu/drm/i915/i915_reg.h | 1 +
2 files changed, 121 insertions(+)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 3f3c1ac10330..56bcf750b047 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -185,6 +185,29 @@ static bool lut_is_legacy(const struct drm_property_blob *lut)
return lut && drm_color_lut_size(lut) == LEGACY_LUT_LENGTH;
}
+/*
+ * Added to accommodate enhanced LUT precision.
+ * Max LUT precision is 32 bits.
+ */
+static u64 drm_color_lut_extract_ext(u64 user_input, u32 bit_precision)
+{
+ u64 val = user_input & 0xffffffff;
+ u32 max;
+
+ if (bit_precision > 32)
+ return 0;
+
+ max = 0xffffffff >> (32 - bit_precision);
+ /* Round only if we're not using full precision. */
+ if (bit_precision < 32) {
+ val += 1UL << (32 - bit_precision - 1);
+ val >>= 32 - bit_precision;
+ }
+
+ return ((user_input & 0xffffffff00000000) |
+ clamp_val(val, 0, max));
+}
+
/*
* When using limited range, multiply the matrix given by userspace by
* the matrix that we would use for the limited range.
@@ -1856,6 +1879,102 @@ static void chv_load_luts(const struct intel_crtc_state *crtc_state)
crtc_state->cgm_mode);
}
+static void xelpd_program_plane_pre_csc_lut(const struct drm_plane_state *state,
+ struct drm_color_lut_ext *pre_csc_lut,
+ u32 offset)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->plane->dev);
+ enum pipe pipe = to_intel_plane(state->plane)->pipe;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ u32 i, lut_size;
+
+ if (icl_is_hdr_plane(dev_priv, plane)) {
+ lut_size = 128;
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, 0),
+ PLANE_PAL_PREC_AUTO_INCREMENT);
+
+ if (pre_csc_lut) {
+ for (i = 0; i < lut_size; i++) {
+ u64 word = drm_color_lut_extract_ext(pre_csc_lut[i].green, 24);
+ u32 lut_val = (word & 0xffffff);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ lut_val);
+ }
+
+ /* Program the max register to clamp values > 1.0. */
+ while (i < 131)
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ pre_csc_lut[i++].green);
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 24) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ 1 << 24);
+ } while (i++ < 130);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, 0), 0);
+ } else {
+ lut_size = 32;
+
+ /*
+ * First 3 planes are HDR, so reduce by 3 to get to the right
+ * SDR plane offset
+ */
+ plane = plane - 3;
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, 0),
+ PLANE_PAL_PREC_AUTO_INCREMENT);
+
+ if (pre_csc_lut) {
+ for (i = 0; i < lut_size; i++)
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0),
+ pre_csc_lut[i].green);
+ /* Program the max register to clamp values > 1.0. */
+ while (i < 35)
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0),
+ pre_csc_lut[i++].green);
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 16) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_DATA(pipe, plane, 0),
+ 1 << 16);
+ } while (i++ < 34);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, 0), 0);
+ }
+}
+
+static void xelpd_plane_load_luts(const struct drm_plane_state *plane_state)
+{
+ const struct drm_property_blob *pre_csc_lut_blob =
+ plane_state->color.pre_csc_lut;
+ struct drm_color_lut_ext *pre_csc_lut = NULL;
+
+ if (pre_csc_lut_blob) {
+ pre_csc_lut = pre_csc_lut_blob->data;
+ xelpd_program_plane_pre_csc_lut(plane_state, pre_csc_lut, 0);
+ }
+}
+
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -3712,6 +3831,7 @@ static const struct intel_color_funcs xelpd_color_funcs = {
.read_luts = icl_read_luts,
.lut_equal = icl_lut_equal,
.read_csc = icl_read_csc,
+ .load_plane_luts = xelpd_plane_load_luts,
};
static const struct intel_color_funcs tgl_color_funcs = {
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index 5fa7461066ab..d26d6294d231 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -6238,6 +6238,7 @@ enum skl_power_gate {
#define PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
_MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \
_PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe))
+#define PLANE_PAL_PREC_AUTO_INCREMENT REG_BIT(10)
#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A 0x701d4
#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B 0x711d4
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 28/33] drm/i915/xelpd: Add register definitions for Plane Post CSC
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Add macros to define Plane Post CSC registers
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/i915_reg.h | 73 +++++++++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index d26d6294d231..5e4271e7b735 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -6226,6 +6226,79 @@ enum skl_power_gate {
#define _MMIO_PLANE_GAMC(plane, i, a, b) _MMIO(_PIPE(plane, a, b) + (i) * 4)
+/* Display13 Plane Gamma Reg */
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A 0x70160
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_B 0x71160
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A 0x70260
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_B 0x71260
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A 0x70164
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_B 0x71164
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A 0x70264
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_B 0x71264
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_SEG0_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1_A 0x701d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1_B 0x711d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2_A 0x702d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2_B 0x712d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_1_A 0x701dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_1_B 0x711dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_2_A 0x702dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_2_B 0x712dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_INDEX_1_A 0x704d8
+#define _PLANE_POST_CSC_GAMC_INDEX_1_B 0x714d8
+#define _PLANE_POST_CSC_GAMC_INDEX_2_A 0x705d8
+#define _PLANE_POST_CSC_GAMC_INDEX_2_B 0x715d8
+#define _PLANE_POST_CSC_GAMC_INDEX_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_1_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_1_B)
+#define _PLANE_POST_CSC_GAMC_INDEX_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_2_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_2_B)
+#define PLANE_POST_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_DATA_1_A 0x704dc
+#define _PLANE_POST_CSC_GAMC_DATA_1_B 0x714dc
+#define _PLANE_POST_CSC_GAMC_DATA_2_A 0x705dc
+#define _PLANE_POST_CSC_GAMC_DATA_2_B 0x715dc
+#define _PLANE_POST_CSC_GAMC_DATA_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_1_A, \
+ _PLANE_POST_CSC_GAMC_DATA_1_B)
+#define _PLANE_POST_CSC_GAMC_DATA_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_2_A, \
+ _PLANE_POST_CSC_GAMC_DATA_2_B)
+#define PLANE_POST_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_2(pipe))
+
/* Display13 Plane Degmma Reg */
#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A 0x701d0
#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B 0x711d0
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 28/33] drm/i915/xelpd: Add register definitions for Plane Post CSC
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Add macros to define Plane Post CSC registers
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/i915_reg.h | 73 +++++++++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index d26d6294d231..5e4271e7b735 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -6226,6 +6226,79 @@ enum skl_power_gate {
#define _MMIO_PLANE_GAMC(plane, i, a, b) _MMIO(_PIPE(plane, a, b) + (i) * 4)
+/* Display13 Plane Gamma Reg */
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A 0x70160
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_B 0x71160
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A 0x70260
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_B 0x71260
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A 0x70164
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_B 0x71164
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A 0x70264
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_B 0x71264
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_SEG0_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1_A 0x701d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1_B 0x711d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2_A 0x702d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2_B 0x712d8
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_1_A 0x701dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_1_B 0x711dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_2_A 0x702dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_2_B 0x712dc
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_ENH_1_A, \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_1_B)
+#define _PLANE_POST_CSC_GAMC_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_ENH_2_A, \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_2_B)
+#define PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_INDEX_1_A 0x704d8
+#define _PLANE_POST_CSC_GAMC_INDEX_1_B 0x714d8
+#define _PLANE_POST_CSC_GAMC_INDEX_2_A 0x705d8
+#define _PLANE_POST_CSC_GAMC_INDEX_2_B 0x715d8
+#define _PLANE_POST_CSC_GAMC_INDEX_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_1_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_1_B)
+#define _PLANE_POST_CSC_GAMC_INDEX_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_2_A, \
+ _PLANE_POST_CSC_GAMC_INDEX_2_B)
+#define PLANE_POST_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_2(pipe))
+
+#define _PLANE_POST_CSC_GAMC_DATA_1_A 0x704dc
+#define _PLANE_POST_CSC_GAMC_DATA_1_B 0x714dc
+#define _PLANE_POST_CSC_GAMC_DATA_2_A 0x705dc
+#define _PLANE_POST_CSC_GAMC_DATA_2_B 0x715dc
+#define _PLANE_POST_CSC_GAMC_DATA_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_1_A, \
+ _PLANE_POST_CSC_GAMC_DATA_1_B)
+#define _PLANE_POST_CSC_GAMC_DATA_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_2_A, \
+ _PLANE_POST_CSC_GAMC_DATA_2_B)
+#define PLANE_POST_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_2(pipe))
+
/* Display13 Plane Degmma Reg */
#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A 0x701d0
#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B 0x711d0
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 29/33] drm/i915/xelpd: Program Plane Post CSC Registers
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Extract the LUT and program plane post csc registers.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 95 +++++++++++++++++++++-
1 file changed, 94 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 56bcf750b047..ff996b9ee77d 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -1963,16 +1963,109 @@ static void xelpd_program_plane_pre_csc_lut(const struct drm_plane_state *state,
}
}
+static void xelpd_program_plane_post_csc_lut(const struct drm_plane_state *state,
+ struct drm_color_lut_ext *post_csc_lut,
+ u32 offset)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->plane->dev);
+ enum pipe pipe = to_intel_plane(state->plane)->pipe;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ u32 i, lut_size;
+
+ if (icl_is_hdr_plane(dev_priv, plane)) {
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, 0),
+ offset | PLANE_PAL_PREC_AUTO_INCREMENT);
+ if (post_csc_lut) {
+ lut_size = 32;
+ for (i = 0; i < lut_size; i++) {
+ u64 word = drm_color_lut_extract_ext(post_csc_lut[i].green, 24);
+ u32 lut_val = (word & 0xffffff);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ lut_val);
+ }
+
+ do {
+ /* Program the max register to clamp values > 1.0. */
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ post_csc_lut[i].green);
+ } while (i++ < 34);
+ } else {
+ lut_size = 32;
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 24) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ 1 << 24);
+ } while (i++ < 34);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, 0), 0);
+ } else {
+ lut_size = 32;
+ /*
+ * First 3 planes are HDR, so reduce by 3 to get to the right
+ * SDR plane offset
+ */
+ plane = plane - 3;
+
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX(pipe, plane, 0),
+ offset | PLANE_PAL_PREC_AUTO_INCREMENT);
+
+ if (post_csc_lut) {
+ for (i = 0; i < lut_size; i++)
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0),
+ post_csc_lut[i].green & 0xffff);
+ /* Program the max register to clamp values > 1.0. */
+ while (i < 35)
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0),
+ post_csc_lut[i++].green & 0x3ffff);
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 16) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0),
+ (1 << 16));
+ } while (i++ < 34);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX(pipe, plane, 0), 0);
+ }
+}
+
static void xelpd_plane_load_luts(const struct drm_plane_state *plane_state)
{
const struct drm_property_blob *pre_csc_lut_blob =
plane_state->color.pre_csc_lut;
- struct drm_color_lut_ext *pre_csc_lut = NULL;
+ const struct drm_property_blob *post_csc_lut_blob =
+ plane_state->color.post_csc_lut;
+ struct drm_color_lut_ext *pre_csc_lut, *post_csc_lut;
if (pre_csc_lut_blob) {
pre_csc_lut = pre_csc_lut_blob->data;
xelpd_program_plane_pre_csc_lut(plane_state, pre_csc_lut, 0);
}
+
+ if (post_csc_lut_blob) {
+ post_csc_lut = post_csc_lut_blob->data;
+ xelpd_program_plane_post_csc_lut(plane_state, post_csc_lut, 0);
+ }
}
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 29/33] drm/i915/xelpd: Program Plane Post CSC Registers
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Extract the LUT and program plane post csc registers.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 95 +++++++++++++++++++++-
1 file changed, 94 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 56bcf750b047..ff996b9ee77d 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -1963,16 +1963,109 @@ static void xelpd_program_plane_pre_csc_lut(const struct drm_plane_state *state,
}
}
+static void xelpd_program_plane_post_csc_lut(const struct drm_plane_state *state,
+ struct drm_color_lut_ext *post_csc_lut,
+ u32 offset)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->plane->dev);
+ enum pipe pipe = to_intel_plane(state->plane)->pipe;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ u32 i, lut_size;
+
+ if (icl_is_hdr_plane(dev_priv, plane)) {
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, 0),
+ offset | PLANE_PAL_PREC_AUTO_INCREMENT);
+ if (post_csc_lut) {
+ lut_size = 32;
+ for (i = 0; i < lut_size; i++) {
+ u64 word = drm_color_lut_extract_ext(post_csc_lut[i].green, 24);
+ u32 lut_val = (word & 0xffffff);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ lut_val);
+ }
+
+ do {
+ /* Program the max register to clamp values > 1.0. */
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ post_csc_lut[i].green);
+ } while (i++ < 34);
+ } else {
+ lut_size = 32;
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 24) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, 0),
+ 1 << 24);
+ } while (i++ < 34);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, 0), 0);
+ } else {
+ lut_size = 32;
+ /*
+ * First 3 planes are HDR, so reduce by 3 to get to the right
+ * SDR plane offset
+ */
+ plane = plane - 3;
+
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX(pipe, plane, 0),
+ offset | PLANE_PAL_PREC_AUTO_INCREMENT);
+
+ if (post_csc_lut) {
+ for (i = 0; i < lut_size; i++)
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0),
+ post_csc_lut[i].green & 0xffff);
+ /* Program the max register to clamp values > 1.0. */
+ while (i < 35)
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0),
+ post_csc_lut[i++].green & 0x3ffff);
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ u32 v = (i * ((1 << 16) - 1)) / (lut_size - 1);
+
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0), v);
+ }
+
+ do {
+ intel_de_write_fw(dev_priv,
+ PLANE_POST_CSC_GAMC_DATA(pipe, plane, 0),
+ (1 << 16));
+ } while (i++ < 34);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_POST_CSC_GAMC_INDEX(pipe, plane, 0), 0);
+ }
+}
+
static void xelpd_plane_load_luts(const struct drm_plane_state *plane_state)
{
const struct drm_property_blob *pre_csc_lut_blob =
plane_state->color.pre_csc_lut;
- struct drm_color_lut_ext *pre_csc_lut = NULL;
+ const struct drm_property_blob *post_csc_lut_blob =
+ plane_state->color.post_csc_lut;
+ struct drm_color_lut_ext *pre_csc_lut, *post_csc_lut;
if (pre_csc_lut_blob) {
pre_csc_lut = pre_csc_lut_blob->data;
xelpd_program_plane_pre_csc_lut(plane_state, pre_csc_lut, 0);
}
+
+ if (post_csc_lut_blob) {
+ post_csc_lut = post_csc_lut_blob->data;
+ xelpd_program_plane_post_csc_lut(plane_state, post_csc_lut, 0);
+ }
}
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 30/33] drm/i915/color: Enable Plane CSC
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
Implement plane CSC for Xe_LPD.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 86 +++++++++++++++++++
drivers/gpu/drm/i915/display/intel_color.h | 1 +
.../drm/i915/display/skl_universal_plane.c | 4 +-
3 files changed, 90 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index ff996b9ee77d..956080fb7fcd 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -2068,6 +2068,83 @@ static void xelpd_plane_load_luts(const struct drm_plane_state *plane_state)
}
}
+static void xelpd_load_plane_csc_matrix(const struct drm_plane_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->plane->dev);
+ enum pipe pipe = to_intel_plane(state->plane)->pipe;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ struct drm_color_ctm *ctm;
+ const u64 *input;
+ u16 coeffs[9] = {};
+ u16 postoff = 0;
+ int i;
+
+ if (!icl_is_hdr_plane(dev_priv, plane) || !state->color.ctm)
+ return;
+
+ ctm = state->color.ctm->data;
+ input = ctm->matrix;
+
+ /*
+ * Convert fixed point S31.32 input to format supported by the
+ * hardware.
+ */
+ for (i = 0; i < ARRAY_SIZE(coeffs); i++) {
+ u64 abs_coeff = ((1ULL << 63) - 1) & input[i];
+
+ /*
+ * Clamp input value to min/max supported by
+ * hardware.
+ */
+ abs_coeff = clamp_val(abs_coeff, 0, CTM_COEFF_4_0 - 1);
+
+ /* sign bit */
+ if (CTM_COEFF_NEGATIVE(input[i]))
+ coeffs[i] |= 1 << 15;
+
+ if (abs_coeff < CTM_COEFF_0_125)
+ coeffs[i] |= (3 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 12);
+ else if (abs_coeff < CTM_COEFF_0_25)
+ coeffs[i] |= (2 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 11);
+ else if (abs_coeff < CTM_COEFF_0_5)
+ coeffs[i] |= (1 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 10);
+ else if (abs_coeff < CTM_COEFF_1_0)
+ coeffs[i] |= ILK_CSC_COEFF_FP(abs_coeff, 9);
+ else if (abs_coeff < CTM_COEFF_2_0)
+ coeffs[i] |= (7 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 8);
+ else
+ coeffs[i] |= (6 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 7);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 0),
+ coeffs[0] << 16 | coeffs[1]);
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 1),
+ coeffs[2] << 16);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 2),
+ coeffs[3] << 16 | coeffs[4]);
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 3),
+ coeffs[5] << 16);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 4),
+ coeffs[6] << 16 | coeffs[7]);
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 5),
+ coeffs[8] << 16);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 0), 0);
+ intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 1), 0);
+ intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 2), 0);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 0), postoff);
+ intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 1), postoff);
+ intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 2), postoff);
+}
+
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -2083,6 +2160,14 @@ void intel_color_load_plane_luts(const struct drm_plane_state *plane_state)
i915->display.funcs.color->load_plane_luts(plane_state);
}
+void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state)
+{
+ struct drm_i915_private *i915 = to_i915(plane_state->plane->dev);
+
+ if (i915->display.funcs.color->load_plane_csc_matrix)
+ i915->display.funcs.color->load_plane_csc_matrix(plane_state);
+}
+
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -3925,6 +4010,7 @@ static const struct intel_color_funcs xelpd_color_funcs = {
.lut_equal = icl_lut_equal,
.read_csc = icl_read_csc,
.load_plane_luts = xelpd_plane_load_luts,
+ .load_plane_csc_matrix = xelpd_load_plane_csc_matrix,
};
static const struct intel_color_funcs tgl_color_funcs = {
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index 93382df101d9..a513c88d3bfc 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -33,4 +33,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,
void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
void intel_color_plane_init(struct drm_plane *plane);
void intel_color_load_plane_luts(const struct drm_plane_state *plane_state);
+void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state);
#endif /* __INTEL_COLOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index c08875fa965e..c85548d3210a 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -1280,8 +1280,10 @@ icl_plane_update_noarm(struct intel_plane *plane,
if (plane_state->force_black)
icl_plane_csc_load_black(plane);
- if (plane_state->uapi.color_mgmt_changed)
+ if (plane_state->uapi.color_mgmt_changed) {
intel_color_load_plane_luts(&plane_state->uapi);
+ intel_color_load_plane_csc_matrix(&plane_state->uapi);
+ }
intel_psr2_program_plane_sel_fetch_noarm(plane, crtc_state, plane_state, color_plane);
}
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 30/33] drm/i915/color: Enable Plane CSC
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
Implement plane CSC for Xe_LPD.
Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 86 +++++++++++++++++++
drivers/gpu/drm/i915/display/intel_color.h | 1 +
.../drm/i915/display/skl_universal_plane.c | 4 +-
3 files changed, 90 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index ff996b9ee77d..956080fb7fcd 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -2068,6 +2068,83 @@ static void xelpd_plane_load_luts(const struct drm_plane_state *plane_state)
}
}
+static void xelpd_load_plane_csc_matrix(const struct drm_plane_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->plane->dev);
+ enum pipe pipe = to_intel_plane(state->plane)->pipe;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ struct drm_color_ctm *ctm;
+ const u64 *input;
+ u16 coeffs[9] = {};
+ u16 postoff = 0;
+ int i;
+
+ if (!icl_is_hdr_plane(dev_priv, plane) || !state->color.ctm)
+ return;
+
+ ctm = state->color.ctm->data;
+ input = ctm->matrix;
+
+ /*
+ * Convert fixed point S31.32 input to format supported by the
+ * hardware.
+ */
+ for (i = 0; i < ARRAY_SIZE(coeffs); i++) {
+ u64 abs_coeff = ((1ULL << 63) - 1) & input[i];
+
+ /*
+ * Clamp input value to min/max supported by
+ * hardware.
+ */
+ abs_coeff = clamp_val(abs_coeff, 0, CTM_COEFF_4_0 - 1);
+
+ /* sign bit */
+ if (CTM_COEFF_NEGATIVE(input[i]))
+ coeffs[i] |= 1 << 15;
+
+ if (abs_coeff < CTM_COEFF_0_125)
+ coeffs[i] |= (3 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 12);
+ else if (abs_coeff < CTM_COEFF_0_25)
+ coeffs[i] |= (2 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 11);
+ else if (abs_coeff < CTM_COEFF_0_5)
+ coeffs[i] |= (1 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 10);
+ else if (abs_coeff < CTM_COEFF_1_0)
+ coeffs[i] |= ILK_CSC_COEFF_FP(abs_coeff, 9);
+ else if (abs_coeff < CTM_COEFF_2_0)
+ coeffs[i] |= (7 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 8);
+ else
+ coeffs[i] |= (6 << 12) |
+ ILK_CSC_COEFF_FP(abs_coeff, 7);
+ }
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 0),
+ coeffs[0] << 16 | coeffs[1]);
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 1),
+ coeffs[2] << 16);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 2),
+ coeffs[3] << 16 | coeffs[4]);
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 3),
+ coeffs[5] << 16);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 4),
+ coeffs[6] << 16 | coeffs[7]);
+ intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 5),
+ coeffs[8] << 16);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 0), 0);
+ intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 1), 0);
+ intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 2), 0);
+
+ intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 0), postoff);
+ intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 1), postoff);
+ intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 2), postoff);
+}
+
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -2083,6 +2160,14 @@ void intel_color_load_plane_luts(const struct drm_plane_state *plane_state)
i915->display.funcs.color->load_plane_luts(plane_state);
}
+void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state)
+{
+ struct drm_i915_private *i915 = to_i915(plane_state->plane->dev);
+
+ if (i915->display.funcs.color->load_plane_csc_matrix)
+ i915->display.funcs.color->load_plane_csc_matrix(plane_state);
+}
+
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -3925,6 +4010,7 @@ static const struct intel_color_funcs xelpd_color_funcs = {
.lut_equal = icl_lut_equal,
.read_csc = icl_read_csc,
.load_plane_luts = xelpd_plane_load_luts,
+ .load_plane_csc_matrix = xelpd_load_plane_csc_matrix,
};
static const struct intel_color_funcs tgl_color_funcs = {
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index 93382df101d9..a513c88d3bfc 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -33,4 +33,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,
void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
void intel_color_plane_init(struct drm_plane *plane);
void intel_color_load_plane_luts(const struct drm_plane_state *plane_state);
+void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state);
#endif /* __INTEL_COLOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index c08875fa965e..c85548d3210a 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -1280,8 +1280,10 @@ icl_plane_update_noarm(struct intel_plane *plane,
if (plane_state->force_black)
icl_plane_csc_load_black(plane);
- if (plane_state->uapi.color_mgmt_changed)
+ if (plane_state->uapi.color_mgmt_changed) {
intel_color_load_plane_luts(&plane_state->uapi);
+ intel_color_load_plane_csc_matrix(&plane_state->uapi);
+ }
intel_psr2_program_plane_sel_fetch_noarm(plane, crtc_state, plane_state, color_plane);
}
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 31/33] drm/i915/color: Enable plane color features
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Initialize and expose all plane color features.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 1 -
drivers/gpu/drm/i915/display/skl_universal_plane.c | 1 +
2 files changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 956080fb7fcd..4e5c82c88bd4 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4361,7 +4361,6 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
return ret;
};
-__maybe_unused
void intel_color_plane_init(struct drm_plane *plane)
{
struct drm_i915_private *i915 = to_i915(plane->dev);
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index c85548d3210a..2e4ca55fdbb2 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -2389,6 +2389,7 @@ skl_universal_plane_create(struct drm_i915_private *dev_priv,
BIT(DRM_SCALING_FILTER_NEAREST_NEIGHBOR));
intel_plane_helper_add(plane);
+ intel_color_plane_init(&plane->base);
return plane;
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 31/33] drm/i915/color: Enable plane color features
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Initialize and expose all plane color features.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 1 -
drivers/gpu/drm/i915/display/skl_universal_plane.c | 1 +
2 files changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 956080fb7fcd..4e5c82c88bd4 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4361,7 +4361,6 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
return ret;
};
-__maybe_unused
void intel_color_plane_init(struct drm_plane *plane)
{
struct drm_i915_private *i915 = to_i915(plane->dev);
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index c85548d3210a..2e4ca55fdbb2 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -2389,6 +2389,7 @@ skl_universal_plane_create(struct drm_i915_private *dev_priv,
BIT(DRM_SCALING_FILTER_NEAREST_NEIGHBOR));
intel_plane_helper_add(plane);
+ intel_color_plane_init(&plane->base);
return plane;
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 32/33] drm/i915/color: Add a dummy pipeline with 3D LUT
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
This patch is to demonstrate how a pipeline can be added.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_state_helper.c | 3 ++
drivers/gpu/drm/drm_atomic_uapi.c | 15 +++++++++
drivers/gpu/drm/i915/display/intel_color.c | 37 ++++++++++++++++++++--
include/drm/drm_plane.h | 6 ++++
4 files changed, 59 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/drm_atomic_state_helper.c b/drivers/gpu/drm/drm_atomic_state_helper.c
index a554e04c2ce3..9c389d97b344 100644
--- a/drivers/gpu/drm/drm_atomic_state_helper.c
+++ b/drivers/gpu/drm/drm_atomic_state_helper.c
@@ -349,6 +349,8 @@ void __drm_atomic_helper_plane_duplicate_state(struct drm_plane *plane,
drm_property_blob_get(state->color.post_csc_lut);
if (state->color.private_color_op_data)
drm_property_blob_get(state->color.private_color_op_data);
+ if (state->color.lut_3d)
+ drm_property_blob_get(state->color.lut_3d);
state->color_mgmt_changed = false;
}
@@ -402,6 +404,7 @@ void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)
drm_property_blob_put(state->color.ctm);
drm_property_blob_put(state->color.post_csc_lut);
drm_property_blob_put(state->color.private_color_op_data);
+ drm_property_blob_put(state->color.lut_3d);
}
EXPORT_SYMBOL(__drm_atomic_helper_plane_destroy_state);
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 9e0fb36d1f47..5629db763fd1 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -482,6 +482,15 @@ int drm_plane_reset_color_op_blobs(struct drm_plane *plane,
&blob_replaced);
temp_replaced |= blob_replaced;
+ if (ret)
+ goto out;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.lut_3d,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
if (ret)
goto out;
out:
@@ -551,6 +560,12 @@ int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
color_op[i].blob_id,
-1, -1,
&blob_replaced);
+ } else if (color_op[i].name == DRM_CB_3D_LUT) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.lut_3d,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_lut_ext),
+ &blob_replaced);
} else {
ret = -EINVAL;
goto copy_fail;
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 4e5c82c88bd4..2352ddb4a96a 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4265,6 +4265,19 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
},
};
+static const struct drm_color_lut_range dummy_3d_lut_range[] = {
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 32,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 0, .end = (1 << 24) - 1,
+ .min = 0, .max = (1 << 24) - 1,
+ },
+};
+
struct drm_color_op color_pipeline_sdr[] = {
{
.name = DRM_CB_PRE_CSC,
@@ -4300,10 +4313,17 @@ struct drm_color_op color_pipeline_hdr[] = {
},
};
+struct drm_color_op color_pipeline_3dlut[] = {
+ {
+ .name = DRM_CB_3D_LUT,
+ .type = CURVE_3D,
+ },
+};
+
static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
{
struct drm_i915_private *i915 = to_i915(plane->dev);
- struct drm_property_blob *blob[2] = {NULL};
+ struct drm_property_blob *blob[3] = {NULL};
int ret = 0, i = 0;
if (icl_is_hdr_plane(i915, to_intel_plane(plane)->id)) {
@@ -4350,6 +4370,17 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
color_pipeline_sdr[1].blob_id = blob[i++]->base.id;
}
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(dummy_3d_lut_range),
+ dummy_3d_lut_range);
+
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ color_pipeline_3dlut[0].blob_id = blob[i++]->base.id;
+
out:
if (ret) {
for (int j = 0; j < i; j++) {
@@ -4368,7 +4399,7 @@ void intel_color_plane_init(struct drm_plane *plane)
if (DISPLAY_VER(i915) < 13)
return;
- drm_plane_create_get_color_pipeline_property(plane->dev, plane, 2);
+ drm_plane_create_get_color_pipeline_property(plane->dev, plane, 3);
intel_prepare_plane_color_pipeline(plane);
@@ -4387,6 +4418,8 @@ void intel_color_plane_init(struct drm_plane *plane)
color_pipeline_sdr,
sizeof(color_pipeline_sdr));
+ drm_plane_add_color_pipeline(plane, "color pipeline 3dlut", color_pipeline_3dlut,
+ sizeof(color_pipeline_3dlut));
drm_plane_attach_get_color_pipeline_property(plane);
drm_plane_create_set_color_pipeline_property(plane->dev, plane);
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index 601b01e47a93..5cb84fa32dd5 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -284,6 +284,12 @@ struct drm_plane_state {
* For example: Parameterized/non-parameterized fixed function operations
*/
struct drm_property_blob *private_color_op_data;
+
+ /* @lut_3d:
+ *
+ * Three dimensional luts
+ */
+ struct drm_property_blob *lut_3d;
} color;
/**
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 32/33] drm/i915/color: Add a dummy pipeline with 3D LUT
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
This patch is to demonstrate how a pipeline can be added.
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/drm_atomic_state_helper.c | 3 ++
drivers/gpu/drm/drm_atomic_uapi.c | 15 +++++++++
drivers/gpu/drm/i915/display/intel_color.c | 37 ++++++++++++++++++++--
include/drm/drm_plane.h | 6 ++++
4 files changed, 59 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/drm_atomic_state_helper.c b/drivers/gpu/drm/drm_atomic_state_helper.c
index a554e04c2ce3..9c389d97b344 100644
--- a/drivers/gpu/drm/drm_atomic_state_helper.c
+++ b/drivers/gpu/drm/drm_atomic_state_helper.c
@@ -349,6 +349,8 @@ void __drm_atomic_helper_plane_duplicate_state(struct drm_plane *plane,
drm_property_blob_get(state->color.post_csc_lut);
if (state->color.private_color_op_data)
drm_property_blob_get(state->color.private_color_op_data);
+ if (state->color.lut_3d)
+ drm_property_blob_get(state->color.lut_3d);
state->color_mgmt_changed = false;
}
@@ -402,6 +404,7 @@ void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)
drm_property_blob_put(state->color.ctm);
drm_property_blob_put(state->color.post_csc_lut);
drm_property_blob_put(state->color.private_color_op_data);
+ drm_property_blob_put(state->color.lut_3d);
}
EXPORT_SYMBOL(__drm_atomic_helper_plane_destroy_state);
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 9e0fb36d1f47..5629db763fd1 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -482,6 +482,15 @@ int drm_plane_reset_color_op_blobs(struct drm_plane *plane,
&blob_replaced);
temp_replaced |= blob_replaced;
+ if (ret)
+ goto out;
+
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.lut_3d,
+ 0, -1, -1,
+ &blob_replaced);
+ temp_replaced |= blob_replaced;
+
if (ret)
goto out;
out:
@@ -551,6 +560,12 @@ int drm_plane_replace_color_op_blobs(struct drm_plane *plane,
color_op[i].blob_id,
-1, -1,
&blob_replaced);
+ } else if (color_op[i].name == DRM_CB_3D_LUT) {
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.lut_3d,
+ color_op[i].blob_id,
+ -1, sizeof(struct drm_color_lut_ext),
+ &blob_replaced);
} else {
ret = -EINVAL;
goto copy_fail;
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 4e5c82c88bd4..2352ddb4a96a 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -4265,6 +4265,19 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = {
},
};
+static const struct drm_color_lut_range dummy_3d_lut_range[] = {
+ {
+ .flags = (DRM_MODE_LUT_POST_CSC |
+ DRM_MODE_LUT_REFLECT_NEGATIVE |
+ DRM_MODE_LUT_INTERPOLATE |
+ DRM_MODE_LUT_NON_DECREASING),
+ .count = 32,
+ .input_bpc = 24, .output_bpc = 16,
+ .start = 0, .end = (1 << 24) - 1,
+ .min = 0, .max = (1 << 24) - 1,
+ },
+};
+
struct drm_color_op color_pipeline_sdr[] = {
{
.name = DRM_CB_PRE_CSC,
@@ -4300,10 +4313,17 @@ struct drm_color_op color_pipeline_hdr[] = {
},
};
+struct drm_color_op color_pipeline_3dlut[] = {
+ {
+ .name = DRM_CB_3D_LUT,
+ .type = CURVE_3D,
+ },
+};
+
static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
{
struct drm_i915_private *i915 = to_i915(plane->dev);
- struct drm_property_blob *blob[2] = {NULL};
+ struct drm_property_blob *blob[3] = {NULL};
int ret = 0, i = 0;
if (icl_is_hdr_plane(i915, to_intel_plane(plane)->id)) {
@@ -4350,6 +4370,17 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
color_pipeline_sdr[1].blob_id = blob[i++]->base.id;
}
+ blob[i] = drm_property_create_blob(plane->dev,
+ sizeof(dummy_3d_lut_range),
+ dummy_3d_lut_range);
+
+ if (IS_ERR(blob[i])) {
+ ret = PTR_ERR(blob[i]);
+ goto out;
+ }
+
+ color_pipeline_3dlut[0].blob_id = blob[i++]->base.id;
+
out:
if (ret) {
for (int j = 0; j < i; j++) {
@@ -4368,7 +4399,7 @@ void intel_color_plane_init(struct drm_plane *plane)
if (DISPLAY_VER(i915) < 13)
return;
- drm_plane_create_get_color_pipeline_property(plane->dev, plane, 2);
+ drm_plane_create_get_color_pipeline_property(plane->dev, plane, 3);
intel_prepare_plane_color_pipeline(plane);
@@ -4387,6 +4418,8 @@ void intel_color_plane_init(struct drm_plane *plane)
color_pipeline_sdr,
sizeof(color_pipeline_sdr));
+ drm_plane_add_color_pipeline(plane, "color pipeline 3dlut", color_pipeline_3dlut,
+ sizeof(color_pipeline_3dlut));
drm_plane_attach_get_color_pipeline_property(plane);
drm_plane_create_set_color_pipeline_property(plane->dev, plane);
diff --git a/include/drm/drm_plane.h b/include/drm/drm_plane.h
index 601b01e47a93..5cb84fa32dd5 100644
--- a/include/drm/drm_plane.h
+++ b/include/drm/drm_plane.h
@@ -284,6 +284,12 @@ struct drm_plane_state {
* For example: Parameterized/non-parameterized fixed function operations
*/
struct drm_property_blob *private_color_op_data;
+
+ /* @lut_3d:
+ *
+ * Three dimensional luts
+ */
+ struct drm_property_blob *lut_3d;
} color;
/**
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [RFC 33/33] drm/i915/color: Add example implementation for vendor specific color operation
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 16:04 ` Uma Shankar
-1 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: Uma Shankar, Chaitanya Kumar Borah, wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
This is an example of how vendor specific color operation could be
supported by the uapi
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 42 ++++++++++++++++---
drivers/gpu/drm/i915/display/intel_color.h | 1 +
.../drm/i915/display/skl_universal_plane.c | 1 +
include/uapi/drm/i915_drm.h | 25 +++++++++++
4 files changed, 64 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 2352ddb4a96a..5acc89b0cbf7 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -21,6 +21,7 @@
* DEALINGS IN THE SOFTWARE.
*
*/
+#include <uapi/drm/i915_drm.h>
#include "i915_reg.h"
#include "intel_color.h"
@@ -87,6 +88,7 @@ struct intel_color_funcs {
*/
void (*load_plane_csc_matrix)(const struct drm_plane_state *plane_state);
void (*load_plane_luts)(const struct drm_plane_state *plane_state);
+ void (*load_private)(const struct drm_plane_state *plane_state);
};
#define CTM_COEFF_SIGN (1ULL << 63)
@@ -2145,6 +2147,25 @@ static void xelpd_load_plane_csc_matrix(const struct drm_plane_state *state)
intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 2), postoff);
}
+static void xelpd_load_private(const struct drm_plane_state *state)
+{
+ struct drm_i915_private *i915 = to_i915(state->plane->dev);
+ struct i915_color_op_data *op_data;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ int i, num;
+
+ if (icl_is_hdr_plane(i915, plane) || !state->color.private_color_op_data)
+ return;
+
+ op_data = state->color.private_color_op_data->data;
+ num = state->color.private_color_op_data->length / sizeof(struct i915_color_op_data);
+
+ for (i = 0; i < num; i++) {
+ if (op_data[i].flag == I915_COLOR_OP_FIXED_FUNC_CSC)
+ DRM_DEBUG_KMS("CSC OP [%d]", op_data[i].csc_type);
+ }
+}
+
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -2168,6 +2189,14 @@ void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state
i915->display.funcs.color->load_plane_csc_matrix(plane_state);
}
+void intel_color_load_private(const struct drm_plane_state *plane_state)
+{
+ struct drm_i915_private *i915 = to_i915(plane_state->plane->dev);
+
+ if (i915->display.funcs.color->load_private)
+ i915->display.funcs.color->load_private(plane_state);
+}
+
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -4011,6 +4040,7 @@ static const struct intel_color_funcs xelpd_color_funcs = {
.read_csc = icl_read_csc,
.load_plane_luts = xelpd_plane_load_luts,
.load_plane_csc_matrix = xelpd_load_plane_csc_matrix,
+ .load_private = xelpd_load_private,
};
static const struct intel_color_funcs tgl_color_funcs = {
@@ -4284,10 +4314,12 @@ struct drm_color_op color_pipeline_sdr[] = {
.type = CURVE_1D,
.blob_id = 0, /* To be updated during plane initialization */
},
- /*
- * SDR planes have fixed function CSC capabilities.
- * TODO: Add support for it
- */
+ {
+ .name = DRM_CB_PRIVATE,
+ .type = FIXED_FUNCTION,
+ .blob_id = 0,
+ .private_flags = I915_COLOR_OP_FIXED_FUNC_CSC,
+ },
{
.name = DRM_CB_POST_CSC,
.type = CURVE_1D,
@@ -4367,7 +4399,7 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
* LUT ranges for SDR planes are similar for pre and post-csc blocks
*/
color_pipeline_sdr[0].blob_id =
- color_pipeline_sdr[1].blob_id = blob[i++]->base.id;
+ color_pipeline_sdr[2].blob_id = blob[i++]->base.id;
}
blob[i] = drm_property_create_blob(plane->dev,
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index a513c88d3bfc..aa8841f1d1ef 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -34,4 +34,5 @@ void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
void intel_color_plane_init(struct drm_plane *plane);
void intel_color_load_plane_luts(const struct drm_plane_state *plane_state);
void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state);
+void intel_color_load_private(const struct drm_plane_state *plane_state);
#endif /* __INTEL_COLOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index 2e4ca55fdbb2..e7228da3358d 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -1283,6 +1283,7 @@ icl_plane_update_noarm(struct intel_plane *plane,
if (plane_state->uapi.color_mgmt_changed) {
intel_color_load_plane_luts(&plane_state->uapi);
intel_color_load_plane_csc_matrix(&plane_state->uapi);
+ intel_color_load_private(&plane_state->uapi);
}
intel_psr2_program_plane_sel_fetch_noarm(plane, crtc_state, plane_state, color_plane);
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index 7000e5910a1d..e7f87ad0645c 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -3841,6 +3841,31 @@ struct drm_i915_gem_create_ext_set_pat {
/* ID of the protected content session managed by i915 when PXP is active */
#define I915_PROTECTED_CONTENT_DEFAULT_SESSION 0xf
+/* I915 specific color operation */
+#define I915_COLOR_OP_FIXED_FUNC_CSC (1 << 0)
+
+/**
+ * enum i915_csc_operation
+ *
+ * Color conversion operations which can be performed by a fixed function h/w
+ * of type I915_COLOR_OP_FIXED_FUNC_CSC
+ */
+enum i915_csc_operation {
+ I915_CSC_YUV601_TO_RGB601,
+ I915_CSC_YUV709_TO_RGB709,
+ I915_CSC_YUV2020_TO_RGB2020,
+ I915_CSC_RGB709_TO_RGB2020,
+ I915_CSC_MAX,
+};
+
+struct i915_color_op_data {
+ __u32 flag; /* to identify i915 specific color operation */
+ union {
+ enum i915_csc_operation csc_type;
+ /* Add more structures here */
+ };
+};
+
#if defined(__cplusplus)
}
#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Intel-gfx] [RFC 33/33] drm/i915/color: Add example implementation for vendor specific color operation
@ 2023-08-29 16:04 ` Uma Shankar
0 siblings, 0 replies; 109+ messages in thread
From: Uma Shankar @ 2023-08-29 16:04 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: wayland-devel
From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
This is an example of how vendor specific color operation could be
supported by the uapi
Co-developed-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Uma Shankar <uma.shankar@intel.com>
Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
---
drivers/gpu/drm/i915/display/intel_color.c | 42 ++++++++++++++++---
drivers/gpu/drm/i915/display/intel_color.h | 1 +
.../drm/i915/display/skl_universal_plane.c | 1 +
include/uapi/drm/i915_drm.h | 25 +++++++++++
4 files changed, 64 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 2352ddb4a96a..5acc89b0cbf7 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -21,6 +21,7 @@
* DEALINGS IN THE SOFTWARE.
*
*/
+#include <uapi/drm/i915_drm.h>
#include "i915_reg.h"
#include "intel_color.h"
@@ -87,6 +88,7 @@ struct intel_color_funcs {
*/
void (*load_plane_csc_matrix)(const struct drm_plane_state *plane_state);
void (*load_plane_luts)(const struct drm_plane_state *plane_state);
+ void (*load_private)(const struct drm_plane_state *plane_state);
};
#define CTM_COEFF_SIGN (1ULL << 63)
@@ -2145,6 +2147,25 @@ static void xelpd_load_plane_csc_matrix(const struct drm_plane_state *state)
intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 2), postoff);
}
+static void xelpd_load_private(const struct drm_plane_state *state)
+{
+ struct drm_i915_private *i915 = to_i915(state->plane->dev);
+ struct i915_color_op_data *op_data;
+ enum plane_id plane = to_intel_plane(state->plane)->id;
+ int i, num;
+
+ if (icl_is_hdr_plane(i915, plane) || !state->color.private_color_op_data)
+ return;
+
+ op_data = state->color.private_color_op_data->data;
+ num = state->color.private_color_op_data->length / sizeof(struct i915_color_op_data);
+
+ for (i = 0; i < num; i++) {
+ if (op_data[i].flag == I915_COLOR_OP_FIXED_FUNC_CSC)
+ DRM_DEBUG_KMS("CSC OP [%d]", op_data[i].csc_type);
+ }
+}
+
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -2168,6 +2189,14 @@ void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state
i915->display.funcs.color->load_plane_csc_matrix(plane_state);
}
+void intel_color_load_private(const struct drm_plane_state *plane_state)
+{
+ struct drm_i915_private *i915 = to_i915(plane_state->plane->dev);
+
+ if (i915->display.funcs.color->load_private)
+ i915->display.funcs.color->load_private(plane_state);
+}
+
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
@@ -4011,6 +4040,7 @@ static const struct intel_color_funcs xelpd_color_funcs = {
.read_csc = icl_read_csc,
.load_plane_luts = xelpd_plane_load_luts,
.load_plane_csc_matrix = xelpd_load_plane_csc_matrix,
+ .load_private = xelpd_load_private,
};
static const struct intel_color_funcs tgl_color_funcs = {
@@ -4284,10 +4314,12 @@ struct drm_color_op color_pipeline_sdr[] = {
.type = CURVE_1D,
.blob_id = 0, /* To be updated during plane initialization */
},
- /*
- * SDR planes have fixed function CSC capabilities.
- * TODO: Add support for it
- */
+ {
+ .name = DRM_CB_PRIVATE,
+ .type = FIXED_FUNCTION,
+ .blob_id = 0,
+ .private_flags = I915_COLOR_OP_FIXED_FUNC_CSC,
+ },
{
.name = DRM_CB_POST_CSC,
.type = CURVE_1D,
@@ -4367,7 +4399,7 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)
* LUT ranges for SDR planes are similar for pre and post-csc blocks
*/
color_pipeline_sdr[0].blob_id =
- color_pipeline_sdr[1].blob_id = blob[i++]->base.id;
+ color_pipeline_sdr[2].blob_id = blob[i++]->base.id;
}
blob[i] = drm_property_create_blob(plane->dev,
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index a513c88d3bfc..aa8841f1d1ef 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -34,4 +34,5 @@ void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
void intel_color_plane_init(struct drm_plane *plane);
void intel_color_load_plane_luts(const struct drm_plane_state *plane_state);
void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state);
+void intel_color_load_private(const struct drm_plane_state *plane_state);
#endif /* __INTEL_COLOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index 2e4ca55fdbb2..e7228da3358d 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -1283,6 +1283,7 @@ icl_plane_update_noarm(struct intel_plane *plane,
if (plane_state->uapi.color_mgmt_changed) {
intel_color_load_plane_luts(&plane_state->uapi);
intel_color_load_plane_csc_matrix(&plane_state->uapi);
+ intel_color_load_private(&plane_state->uapi);
}
intel_psr2_program_plane_sel_fetch_noarm(plane, crtc_state, plane_state, color_plane);
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index 7000e5910a1d..e7f87ad0645c 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -3841,6 +3841,31 @@ struct drm_i915_gem_create_ext_set_pat {
/* ID of the protected content session managed by i915 when PXP is active */
#define I915_PROTECTED_CONTENT_DEFAULT_SESSION 0xf
+/* I915 specific color operation */
+#define I915_COLOR_OP_FIXED_FUNC_CSC (1 << 0)
+
+/**
+ * enum i915_csc_operation
+ *
+ * Color conversion operations which can be performed by a fixed function h/w
+ * of type I915_COLOR_OP_FIXED_FUNC_CSC
+ */
+enum i915_csc_operation {
+ I915_CSC_YUV601_TO_RGB601,
+ I915_CSC_YUV709_TO_RGB709,
+ I915_CSC_YUV2020_TO_RGB2020,
+ I915_CSC_RGB709_TO_RGB2020,
+ I915_CSC_MAX,
+};
+
+struct i915_color_op_data {
+ __u32 flag; /* to identify i915 specific color operation */
+ union {
+ enum i915_csc_operation csc_type;
+ /* Add more structures here */
+ };
+};
+
#if defined(__cplusplus)
}
#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [RFC 00/33] Add Support for Plane Color Pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 19:26 ` Harry Wentland
-1 siblings, 0 replies; 109+ messages in thread
From: Harry Wentland @ 2023-08-29 19:26 UTC (permalink / raw)
To: Uma Shankar, intel-gfx, dri-devel
Cc: Sebastian Wick, Pekka Paalanen, Shashank Sharma, wayland-devel,
Melissa Wen, Jonas Ådahl, Naseer Ahmed, Christopher Braga
+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
On 2023-08-29 12:03, Uma Shankar wrote:
> Introduction
> ============
>
> Modern hardwares have various color processing capabilities both
> at pre-blending and post-blending phases in the color pipeline.
> The current drm implementation exposes only the post-blending
> color hardware blocks. Support for pre-blending hardware is missing.
> There are multiple use cases where pre-blending color hardware will
> be useful:
> a) Linearization of input buffers encoded in various transfer
> functions.
> b) Color Space conversion
> c) Tone mapping
> d) Frame buffer format conversion
> e) Non-linearization of buffer(apply transfer function)
> f) 3D Luts
>
> and other miscellaneous color operations.
>
> Hence, there is a need to expose the color capabilities of the hardware
> to user-space. This will help userspace/middleware to use display
> hardware for color processing and blending instead of doing it through
> GPU shaders.
>
Thanks, Uma, for sending this. I've been working on something similar
but you beat me to it. :)
>
> Work done so far and relevant references
> ========================================
>
> Some implementation is done by Intel and AMD/Igalia to address the same.
> Broad consensus is there that we need a generic API at drm core to suffice
> the use case of various HW vendors. Below are the links capturing the
> discussion so far.
>
> Intel's Plane Color Implementation: https://patchwork.freedesktop.org/series/90825/
> AMD's Plane Color Implementation: https://patchwork.freedesktop.org/series/116862/
>
>
> Hackfest conclusions
> ====================
>
> HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders
> together and converge on a common uapi expectations. Participants from Intel, AMD,
> Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and
> maintainers.
>
> Discussions happened on the uapi expectations, opens, nature of hardware of multiple
> hardware vendors, challenges in generalizing the same and the path forward. Consensus
> was made that drm core should implement descriptive APIs and not go with prescriptive
> APIs. DRM core should just expose the hardware capabilities; enabling, customizing and
> programming the same should be done by the user-space. Driver should just honor
> the user space request without doing any operations internally.
>
> Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC
> which can be referred to here:
>
> https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>
>
> Design considerations
> =====================
>
> Following are the important aspects taken into account while designing the current RFC
> proposal:
>
> 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
> 2. Position of the HW block in the pipeline can be programmable
> 3. LUTs can be one dimentional or three dimentional
> 4. Number of LUT entries can vary across platforms
> 5. Precision of LUT entries can vary across platforms
> 6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> Piece-Wise Linear(PWL) etc
> 7. There can be parameterized/non-parameterized fixed function HW blocks.
> e.g. Just a hardware bit, to convert from one color space to another.
> 8. Custom non-standard HW implementation.
> 9. Leaving scope for some vendor defined pescriptive implementation if required.
> 10.Scope to handle any modification in hardware as technology evolves
>
> The current proposal takes into account the above considerations while keeping the implementation
> as generic as possible leaving scope for future additions or modifications.
>
> This proposal is also in line to the details mentioned by Simon's RFC covering all
> the aspects discussed in hackfest.
>
>
> Outline of the implementation
> ============================
>
> Each Color Hardware block will be represented by a data structure drm_color_op.
> These color operations will form the building blocks of a color pipeline which
> best represents the underlying Hardware. Color operations can be re-arranged,
> substracted or added to create distinct color pipelines to accurately describe
> the Hardware blocks present in the display engine.
Who is doing the arranging of color operations? IMO a driver should
define one or more respective pipelines that can be selected by
userspace. This seems to be what you're talking about after (I haven't
reviewed the whole thing yet). Might be best to drop this sentence or to
add clarifications in order to avoid confusion.
>
> In this proposal, a color pipeline is represented as an array of
> struct drm_color_op. For individual color operation, we add blobs to advertise
> the capability of the particular Hardware block.
>
> This color pipeline is then packaged within a blob for the user space to
> retrieve it.
>
Would it be better to expose the drm_color_ops directly, instead of
packing a array of drm_color_ops into a blob and then giving that to
userspace.
> To advertise the available color pipelines, an immutable ENUM property
> "GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
> With each blob id representing a distinct color pipeline based on underlying HW
> capabilities and their respective combinations.
>
> Once the user space decides on a color pipeline, it can set the pipeline and
> the corresponding data for the hardware blocks within the pipeline with
> the BLOB property "SET_COLOR_PIPELINE".
>
When I discussed this at the hackfest with Simon he proposed a new DRM
object, (I've called it a drm_colorop) to represent a color operation.
Each drm_colorop has a "NEXT" pointer to another drm_colorop, or NULL if
its the last in the pipeline. You can then have a mutable enum property
on the plane to discover and select a color pipeline.
This seems a bit more transparent than a blob. You can see my changes
(unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs
libdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffs
I'll take time to review your whole series and will see whether we can
somehow keep the best parts of each.
Curious to hear other opinions on the blob vs new DRM object question as
well.
> Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the patch
>
> IGT and test details
> ====================
>
> A set of IGT tests is written to demonstrate the usage of the proposed UAPIs
> along with some sanity validation.
>
> Details of the IGT test can be found here:
> https://patchwork.freedesktop.org/series/123018/
>
> Opens
> =====
>
> a. To come up with a list of common HW blocks which can be defined generically by the DRM
> core in agreement with all the stakeholders
> b. To enhance/finalize the data structure to define segmented LUTs generically.
>
It would be good to add some basic support in VKMS. My work has been
based on VKMS. Once we kinda settle on an approach I'll look at exposing
the AMD private properties from Melissa through the API.
>
> Out of Scope
> ============
>
> a. The coefficients for CTM and LUT value calculations are out of scope of the proposal.
> b. The onus of programming the HW blocks and their values is on user-space. Driver will
> just provide the interface for the same.
> c. In order to compute LUT values and coefficients, a helper library can be created in
> user-space. However, it is out of scope for the current proposal.
>
> Acknowledgements and credits
> ============================
>
> There are multiple contributors who have helped us to reach to this proposal. Special mention
> to Ville Syrjala<ville.syrjala@linux.intel.com>, Pekka Paalanen<pekka.paalanen@collabora.com>,
> Simon Ser<contact@emersion.fr>, Harry Wentland<harry.wentland@amd.com>,
> Melissa Wen<mwen@igalia.com>, Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
> Bhanu<bhanuprakash.modem@intel.com> and Shashank<shashank.sharma@amd.com>.
>
> Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for organizing the HDR hackfest.
>
>
> UAPI dependency and Usermode development
> ========================================
>
> The current KMS implementation requires a user space consumer for it to be accepted upstream.
> Work is ongoing in weston and mutter community to get color management and HDR support implemented
> in the respective stacks.
>
If we can get AMD properties encoded using a Color Pipeline API we can
probably use gamescope as the userspace vehicle.
I'm reviewing this in sequence, so there's a chance I'm missing context.
Please bear with me if some of my comments are answered later in the series.
Again, thanks for sending this.
Harry
> =================================================================================
>
> We have tried to take care of all the scenarios and use-cases which possibly could exists in the
> current proposal. Thanks to everyone who has contributed in all color management discussions so
> far. Let's work together to improve the current proposal and get this thing implemented in
> upstream linux. All the feedback and suggestions to enhance the design are welcome.
>
> Regards,
> Uma Shankar
> Chaitanya Kumar Borah
>
> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: Jonas Ådahl <jadahl@redhat.com>
> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> Cc: Shashank Sharma <shashank.sharma@amd.com>
> Cc: Alexander Goins <agoins@nvidia.com>
>
> Chaitanya Kumar Borah (14):
> drm: Add color operation structure
> drm: Add plane get color pipeline property
> drm: Add helper to add color pipeline
> drm: Manage color blob states
> drm: Replace individual color blobs
> drm: Reset pipeline when user sends NULL blob
> drm: Reset plane color state on pipeline switch request
> drm/i915/color: Add HDR plane LUT range data to color pipeline
> drm/i915/color: Add SDR plane LUT range data to color pipeline
> drm/i915/color: Add color pipelines to plane
> drm/i915/color: Create and attach set color pipeline property
> drm/i915/color: Enable plane color features
> drm/i915/color: Add a dummy pipeline with 3D LUT
> drm/i915/color: Add example implementation for vendor specific color
> operation
>
> Uma Shankar (19):
> drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> drm: Add structures for setting color pipeline
> drm: Add set colorpipeline property
> drm: Add Enhanced Gamma LUT precision structure
> drm: Add color lut range structure
> drm: Add color information to plane state
> drm/i915/color: Add lut range for SDR planes
> drm/i915/color: Add lut range for HDR planes
> drm/i915/color: Add color pipeline for HDR planes
> drm/i915/color: Add color pipeline for SDR planes
> drm/i915/color: Add plane color callbacks
> drm/i915/color: Load plane color luts from atomic flip
> drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> drm/i915/xelpd: Add register definitions for Plane Degamma
> drm/i915/color: Add color functions for ADL
> drm/i915/color: Program Plane Pre-CSC Registers
> drm/i915/xelpd: Add register definitions for Plane Post CSC
> drm/i915/xelpd: Program Plane Post CSC Registers
> drm/i915/color: Enable Plane CSC
>
> .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> .../drm/i915/display/skl_universal_plane.c | 21 +-
> drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> include/drm/drm_plane.h | 82 +++
> include/uapi/drm/drm_mode.h | 196 +++++
> include/uapi/drm/i915_drm.h | 25 +
> 11 files changed, 1899 insertions(+), 3 deletions(-)
> create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-08-29 19:26 ` Harry Wentland
0 siblings, 0 replies; 109+ messages in thread
From: Harry Wentland @ 2023-08-29 19:26 UTC (permalink / raw)
To: Uma Shankar, intel-gfx, dri-devel
Cc: Sebastian Wick, Pekka Paalanen, Simon Ser, Alexander Goins,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Christopher Braga
+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
On 2023-08-29 12:03, Uma Shankar wrote:
> Introduction
> ============
>
> Modern hardwares have various color processing capabilities both
> at pre-blending and post-blending phases in the color pipeline.
> The current drm implementation exposes only the post-blending
> color hardware blocks. Support for pre-blending hardware is missing.
> There are multiple use cases where pre-blending color hardware will
> be useful:
> a) Linearization of input buffers encoded in various transfer
> functions.
> b) Color Space conversion
> c) Tone mapping
> d) Frame buffer format conversion
> e) Non-linearization of buffer(apply transfer function)
> f) 3D Luts
>
> and other miscellaneous color operations.
>
> Hence, there is a need to expose the color capabilities of the hardware
> to user-space. This will help userspace/middleware to use display
> hardware for color processing and blending instead of doing it through
> GPU shaders.
>
Thanks, Uma, for sending this. I've been working on something similar
but you beat me to it. :)
>
> Work done so far and relevant references
> ========================================
>
> Some implementation is done by Intel and AMD/Igalia to address the same.
> Broad consensus is there that we need a generic API at drm core to suffice
> the use case of various HW vendors. Below are the links capturing the
> discussion so far.
>
> Intel's Plane Color Implementation: https://patchwork.freedesktop.org/series/90825/
> AMD's Plane Color Implementation: https://patchwork.freedesktop.org/series/116862/
>
>
> Hackfest conclusions
> ====================
>
> HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders
> together and converge on a common uapi expectations. Participants from Intel, AMD,
> Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and
> maintainers.
>
> Discussions happened on the uapi expectations, opens, nature of hardware of multiple
> hardware vendors, challenges in generalizing the same and the path forward. Consensus
> was made that drm core should implement descriptive APIs and not go with prescriptive
> APIs. DRM core should just expose the hardware capabilities; enabling, customizing and
> programming the same should be done by the user-space. Driver should just honor
> the user space request without doing any operations internally.
>
> Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC
> which can be referred to here:
>
> https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>
>
> Design considerations
> =====================
>
> Following are the important aspects taken into account while designing the current RFC
> proposal:
>
> 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
> 2. Position of the HW block in the pipeline can be programmable
> 3. LUTs can be one dimentional or three dimentional
> 4. Number of LUT entries can vary across platforms
> 5. Precision of LUT entries can vary across platforms
> 6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> Piece-Wise Linear(PWL) etc
> 7. There can be parameterized/non-parameterized fixed function HW blocks.
> e.g. Just a hardware bit, to convert from one color space to another.
> 8. Custom non-standard HW implementation.
> 9. Leaving scope for some vendor defined pescriptive implementation if required.
> 10.Scope to handle any modification in hardware as technology evolves
>
> The current proposal takes into account the above considerations while keeping the implementation
> as generic as possible leaving scope for future additions or modifications.
>
> This proposal is also in line to the details mentioned by Simon's RFC covering all
> the aspects discussed in hackfest.
>
>
> Outline of the implementation
> ============================
>
> Each Color Hardware block will be represented by a data structure drm_color_op.
> These color operations will form the building blocks of a color pipeline which
> best represents the underlying Hardware. Color operations can be re-arranged,
> substracted or added to create distinct color pipelines to accurately describe
> the Hardware blocks present in the display engine.
Who is doing the arranging of color operations? IMO a driver should
define one or more respective pipelines that can be selected by
userspace. This seems to be what you're talking about after (I haven't
reviewed the whole thing yet). Might be best to drop this sentence or to
add clarifications in order to avoid confusion.
>
> In this proposal, a color pipeline is represented as an array of
> struct drm_color_op. For individual color operation, we add blobs to advertise
> the capability of the particular Hardware block.
>
> This color pipeline is then packaged within a blob for the user space to
> retrieve it.
>
Would it be better to expose the drm_color_ops directly, instead of
packing a array of drm_color_ops into a blob and then giving that to
userspace.
> To advertise the available color pipelines, an immutable ENUM property
> "GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
> With each blob id representing a distinct color pipeline based on underlying HW
> capabilities and their respective combinations.
>
> Once the user space decides on a color pipeline, it can set the pipeline and
> the corresponding data for the hardware blocks within the pipeline with
> the BLOB property "SET_COLOR_PIPELINE".
>
When I discussed this at the hackfest with Simon he proposed a new DRM
object, (I've called it a drm_colorop) to represent a color operation.
Each drm_colorop has a "NEXT" pointer to another drm_colorop, or NULL if
its the last in the pipeline. You can then have a mutable enum property
on the plane to discover and select a color pipeline.
This seems a bit more transparent than a blob. You can see my changes
(unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs
libdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffs
I'll take time to review your whole series and will see whether we can
somehow keep the best parts of each.
Curious to hear other opinions on the blob vs new DRM object question as
well.
> Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the patch
>
> IGT and test details
> ====================
>
> A set of IGT tests is written to demonstrate the usage of the proposed UAPIs
> along with some sanity validation.
>
> Details of the IGT test can be found here:
> https://patchwork.freedesktop.org/series/123018/
>
> Opens
> =====
>
> a. To come up with a list of common HW blocks which can be defined generically by the DRM
> core in agreement with all the stakeholders
> b. To enhance/finalize the data structure to define segmented LUTs generically.
>
It would be good to add some basic support in VKMS. My work has been
based on VKMS. Once we kinda settle on an approach I'll look at exposing
the AMD private properties from Melissa through the API.
>
> Out of Scope
> ============
>
> a. The coefficients for CTM and LUT value calculations are out of scope of the proposal.
> b. The onus of programming the HW blocks and their values is on user-space. Driver will
> just provide the interface for the same.
> c. In order to compute LUT values and coefficients, a helper library can be created in
> user-space. However, it is out of scope for the current proposal.
>
> Acknowledgements and credits
> ============================
>
> There are multiple contributors who have helped us to reach to this proposal. Special mention
> to Ville Syrjala<ville.syrjala@linux.intel.com>, Pekka Paalanen<pekka.paalanen@collabora.com>,
> Simon Ser<contact@emersion.fr>, Harry Wentland<harry.wentland@amd.com>,
> Melissa Wen<mwen@igalia.com>, Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
> Bhanu<bhanuprakash.modem@intel.com> and Shashank<shashank.sharma@amd.com>.
>
> Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for organizing the HDR hackfest.
>
>
> UAPI dependency and Usermode development
> ========================================
>
> The current KMS implementation requires a user space consumer for it to be accepted upstream.
> Work is ongoing in weston and mutter community to get color management and HDR support implemented
> in the respective stacks.
>
If we can get AMD properties encoded using a Color Pipeline API we can
probably use gamescope as the userspace vehicle.
I'm reviewing this in sequence, so there's a chance I'm missing context.
Please bear with me if some of my comments are answered later in the series.
Again, thanks for sending this.
Harry
> =================================================================================
>
> We have tried to take care of all the scenarios and use-cases which possibly could exists in the
> current proposal. Thanks to everyone who has contributed in all color management discussions so
> far. Let's work together to improve the current proposal and get this thing implemented in
> upstream linux. All the feedback and suggestions to enhance the design are welcome.
>
> Regards,
> Uma Shankar
> Chaitanya Kumar Borah
>
> Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: Jonas Ådahl <jadahl@redhat.com>
> Cc: Sebastian Wick <sebastian.wick@redhat.com>
> Cc: Shashank Sharma <shashank.sharma@amd.com>
> Cc: Alexander Goins <agoins@nvidia.com>
>
> Chaitanya Kumar Borah (14):
> drm: Add color operation structure
> drm: Add plane get color pipeline property
> drm: Add helper to add color pipeline
> drm: Manage color blob states
> drm: Replace individual color blobs
> drm: Reset pipeline when user sends NULL blob
> drm: Reset plane color state on pipeline switch request
> drm/i915/color: Add HDR plane LUT range data to color pipeline
> drm/i915/color: Add SDR plane LUT range data to color pipeline
> drm/i915/color: Add color pipelines to plane
> drm/i915/color: Create and attach set color pipeline property
> drm/i915/color: Enable plane color features
> drm/i915/color: Add a dummy pipeline with 3D LUT
> drm/i915/color: Add example implementation for vendor specific color
> operation
>
> Uma Shankar (19):
> drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> drm: Add structures for setting color pipeline
> drm: Add set colorpipeline property
> drm: Add Enhanced Gamma LUT precision structure
> drm: Add color lut range structure
> drm: Add color information to plane state
> drm/i915/color: Add lut range for SDR planes
> drm/i915/color: Add lut range for HDR planes
> drm/i915/color: Add color pipeline for HDR planes
> drm/i915/color: Add color pipeline for SDR planes
> drm/i915/color: Add plane color callbacks
> drm/i915/color: Load plane color luts from atomic flip
> drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> drm/i915/xelpd: Add register definitions for Plane Degamma
> drm/i915/color: Add color functions for ADL
> drm/i915/color: Program Plane Pre-CSC Registers
> drm/i915/xelpd: Add register definitions for Plane Post CSC
> drm/i915/xelpd: Program Plane Post CSC Registers
> drm/i915/color: Enable Plane CSC
>
> .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> .../drm/i915/display/skl_universal_plane.c | 21 +-
> drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> include/drm/drm_plane.h | 82 +++
> include/uapi/drm/drm_mode.h | 196 +++++
> include/uapi/drm/i915_drm.h | 25 +
> 11 files changed, 1899 insertions(+), 3 deletions(-)
> create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-29 19:40 ` Harry Wentland
-1 siblings, 0 replies; 109+ messages in thread
From: Harry Wentland @ 2023-08-29 19:40 UTC (permalink / raw)
To: Uma Shankar, intel-gfx, dri-devel; +Cc: Chaitanya Kumar Borah, wayland-devel
On 2023-08-29 12:03, Uma Shankar wrote:
> Add the documentation for the new proposed Plane Color Pipeline.
>
> Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> ---
> .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> 1 file changed, 394 insertions(+)
> create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
>
> diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst
> new file mode 100644
> index 000000000000..60ce515b6ea7
> --- /dev/null
> +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> @@ -0,0 +1,394 @@
> +=======================================
> + Plane Color Pipeline: A UAPI proposal
> +=======================================
> +
> +To build the proposal on, lets take the premise of a color pipeline as shown
> +below.
> +
> + +-------------------------------------------+
> + | RAM |
> + | +------+ +---------+ +---------+ |
> + | | FB 1 | | FB 2 | | FB N | |
> + | +------+ +---------+ +---------+ |
> + +-------------------------------------------+
> + | Plane Color Hardware Block |
> + +--------------------------------------------+
> + | +---v-----+ +---v-------+ +---v------+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> + | +---+-----+ +---+-------+ +---+------+ |
> + | | | | |
> + | +---v-----+ +---v-------+ +---v------+ |
> + | |Plane A | | Plane B | | Plane N | |
> + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + | +---v-----+ +----v------+ +----v-----+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | |Post-CSC | | Post-CSC | | Post-CSC | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + +--------------------------------------------+
> ++------v--------------v---------------v-------|
> +|| ||
> +|| Pipe Blender ||
> ++--------------------+------------------------+
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Pre-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | Pipe Color |
> +| +-----------v----------+ Hardware |
> +| | Pipe CSC/CTM | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Post-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> ++---------------------------------------------+
> + |
> + v
> + Pipe Output
> +
> +Each plane consists of the following color blocks
> + * Pre-CSC : This block can used to linearize the input frame buffer data.
> + The linear data then can be further acted on by the following
> + color hardware blocks in the display hardware pipeline
> +
> + * CSC/CTM: Used to program color transformation matrix, this block is used
> + to perform color space conversions like BT2020 to BT709 or BT601
> + etc. This block acts on the linearized data coming from the
> + Pre-CSC HW block.
> +
> + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> + match the sink. Another use case of it could be to perform Tone
> + mapping for HDR use-cases.
> +
> +Data from multiple planes will then be fed to pipe/crtc where it will get blended.
> +There is a similar set of HW blocks available at pipe/crtc level which acts on
> +this blended data.
> +
> +Below is a sample usecase fo video playback with sub-titles and playback
> +controls
> +
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
> +(subtitles) │
> + │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
> +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
> +(Playback controls UI) │ │
> + │ │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
> +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │
> +(video frame) │ │ │
> + │ │ │
> +┌────────────────────────────────────────────────────────────────────────┴─┴─┘
> +│
> +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> +│ │ CRTC │ │ CRTC │ │ CRTC │
> +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port
> + │ │ │ │ │ │
> + └─────────────┘ └─────────────┘ └───────────────┘
> +
> +This RFC is intended to propose an uAPI for the pre-blending color pipeline
> +(however, can be also extended to post blending pipeline).
> +
> +Below are the design considerations while designing the uAPI.
> +
> +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
> +2. Position of the HW block can be programmable
> +3. LUTs can be one dimentional or three dimentional
> +4. Number of LUT entries can vary across platforms
> +5. Precision of LUT entries can vary across platforms
> +6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> + Piece-Wise Linear(PWL) etc
> +7. There can be parameterized/non-parameterized fixed function HW blocks.
> + e.g. Just a hardware bit, to convert from one color space to another.
> +
> +Plane Color Pipeline: Design Proposal
> +=====================================
> +Each Color Hardware block will be represented by the structure drm_color_op.
> +
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type;
> + u32 blob_id;
> + u32 private_flags;
> +};
> +
> +The members of which will constitute:
> +1. name: A standardised enum for the color hardware block
> +2. type: The type of mathematical operation done by the hardware block.
> + e.g. 1D Curve, 3D Curve, Matrix etc.
> +3. blob id: Id pointing to a blob containing information about the hardware
> + block advertising the respective capabilities to the userspace.
> + It can be an optional field depending on the members "name" and "type".
> +4. private_flags: This can be used to provide vendor specific hints
> + to user space
> +
> +
> + For example to represent LUTs, we introduce the drm_color_lut_range
> + structure. It can represent LUTs with varied number of entries and
> + distributions (Multi segmented, Logarithmic etc).
> +
> + struct drm_color_lut_range {
> + /* DRM_MODE_LUT_* */
> + __u32 flags;
> + /* number of points on the curve */
> + __u16 count;
> + /* input/output bits per component */
> + __u8 input_bpc, output_bpc;
> + /* input start/end values */
> + __s32 start, end;
> + /* output min/max values */
> + __s32 min, max;
> + };
> +
> +Note: More details on exact usage and implementation of this structure can be
> + found in the RFC. This structure is taken as is from the series.
> + https://patchwork.freedesktop.org/series/90825/
> + However, we can add more members to it to encompass all use-cases.
> + For example. we can add a precision field to advertise the
> + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> + field for vendor specific use cases.
> +
> + At the same time, we don't need to pass any additional information for the
> + CSC block as userspace and driver already agrees struct drm_color_ctm as
> + a uAPI to pass on data.
> +
> +These color operations will form the building blocks of a color pipeline which
> +best represents the underlying Hardware. Color operations can be re-arranged,
> +substracted or added to create distinct color pipelines to accurately describe
> +the Hardware blocks present in the display engine.
> +
What do you mean when you're talking of re-arranging, or subtracting
color operations?
> +In this proposal, a color pipeline is represented as an array of
> +struct drm_color_op.
> +
> +struct drm_color_op color_pipeline_1[]
> +
> +For example to represent the pre-blending color pipeline as described above
> +
> +We can create a color pipeline like this.
> +
> +struct drm_color_op color_pipeline_1[] = {
> + {
> + .name = DRM_CB_PRE_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0; /* actual value to be populated during plane
> + initialization*/
> + },
> + {
> + .name = DRM_CB_CSC,
> + .type = MATRIX,
> + .blob_id = 0;
> + },
> + {
> + .name = DRM_CB_POST_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0;
> + },
> +};
> +
> +Then, for individual color operation, we add blobs to advertise the capability
> +of the particular Hardware block. In case of the example pipeline, we add
> +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
> +For the "csc" block we pass no blob id to user space as it is known to both
> +user space and driver that drm_color_ctm structure is to be used for such
> +operation.
> +
> +To represent, this in a diagram.
> +
> + struct drm_color_op color_pipeline_1[]
> + +---------------------------+
> + | | drm_color_op
> + | +---------------------+--+-----------+---------------------+
> + | | | | | |
> + | | | | | +-----------------+ |
> + | | | | | | name | |
> + | | | | | +-----------------+ |
> + | | | | | | type | |
> + | | color_op_1 | | | +-----------------+ |
> + | | | | | | blob id | +--------+
> + | | | | | +-----------------+ | |
> + | | | | | | private | | |
> + | | | | | | flags | | |
> + | | | | | +-----------------+ | |
> + | | | | | | |
> + | +---------------------+--+-----------+---------------------+ |
> + | | |
> + | | |
> + | +---------------------+ | |
> + | | | | drm_color_lut_range |
> + | | color_op_2 | | +-------------------------+ |
> + | | | | | | |
> + | | | | | +---------------------+ | |
> + | +---------------------+ | | | segment 1 { | |<---+
> + | | | | ... | |
> + | +---------------------+ | | | .input_bpc = 16, | |
> + | | | | | | .output_bpc = 16, | |
> + | | color_op_3 | | | | ... | |
> + | | | | | | } | |
> + | | | | | +---------------------+ |
> + | +---------------------+ | | |
> + | . | | +---------------------+ |
> + | . | | | segment 2 { | |
> + | . | | | ... | |
> + +---------------------------+ | | } | |
> + | | | |
> + | | | |
> + | | | |
> + | +---------------------+ |
> + | . |
> + | . |
> + | . |
> + +-------------------------+
> +
> +
> +
> +This color pipeline is then packaged within a blob for the user space to
> +retrieve it. Details can be found in the next section
> +
Not sure I like blobs that contain other blob ids.
> +Exposing a color pipeline to user space
> +=======================================
> +
> +To advertise the available color pipelines, an immutable ENUM property
> +"GET_COLOR_PIPELINE" is introduced.
> +This enum property has blob id's as values. With each blob id representing
> +a distinct color pipeline based on underlying HW capabilities and their
> +respective combinations.
> +
> +The following output of drm_info [1], shows how color pipelines are visible
> +to userspace.
> +
> +├───Plane 0
> + │ ├───Object ID: 31
> + │ ├───CRTCs: {0}
> + │ ├───Legacy info
> + ...
> + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> + color pipeline 1, color pipeline 2}= no color pipeline
> +
> +To understand the capabilities of individual pipelines, first the userspace
> +need to retrieve the pipeline blob with the blob ids retrieved by reading the
> +enum property.
> +
> +Once the color pipeline is retrieved, user can then parse through
> +individual drm_color_op blocks to understand the capabilities of each
> +hardware block.
> +
> +Check IGT series to see how user space can parse through color pipelines.
> +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
> +
> +Setting up a color pipeline
> +===========================
> +
> +Once the user space decides on a color pipeline, it can set the pipeline and
> +the corresponding data for the hardware blocks within the pipeline with
> +the BLOB property "SET_COLOR_PIPELINE".
> +
> +To achieve this two structures are introduced
> +
> +1. struct drm_color_op_data: It represents data to be passed onto individual
> + color hardware blocks. It contains three members
> + a) name: to identify the color operation block
> + b) blob_id: pointing to the blob with data for the
> + corresponding hardware block
> +
> + struct drm_color_op_data {
> + enum color_op_block name;
> + u32 blob_id;
> + };
> +
> +2. struct drm_color_pipeline: This structure represents the aggregate
> + pipeline to be set. it contains the following members
> + a) num: pipeline number to be selected
> + b) size: size of the data to be passed onto the driver
> + c) data: array of struct drm_color_op_data with data for
> + the hardware block/s that userspace wants to
> + set values for.
> +
> + struct drm_color_pipeline {
> + int num;
> + int size;
> + struct drm_color_op_data *data;
> + };
> +
> + User can either:
> + 1. send data for all the color operations in a pipeline as shown in [2].
> + The color operation data need not be in order that the pipeline advertises
> + however, it should not contain data for any
> + color operation that is not present in the pipeline.
> +
> + Note: This check for pipeline is not yet implemented but if the
> + wider proposal is accepted we have few solutions in mind.
> +
> + 2. send data for a subset of color operations as shown in [3].For the
> + color operation that userspace does not send data will retain their
> + older state.
> +
> + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> + property to NULL as shown in both [2] and [3]
> +
> + 4. change the color pipeline as demonstrated in [3].
> + On the new pipeline, the user is expected to setup all the color hardware block
> + Once the user requests a pipeline change, the driver will provide it a clean slate
> + which means that all the data previously set by the user will be discarded even if
> + there are common hardware blocks between the two(previous and current) pipelines.
> +
> +IGT implementation can be found here [4]
> +
> +Representing Fixed function hardware
> +====================================
> +
> +To provide support for fixed function hardware, the driver could expose vendor
> +specific struct drm_color_op with parameters that both the userspace and
> +driver agrees on. To demonstrate, let's consider a hyphothetical fixed
> +function hardware block that converts BT601 to BT2020.
> +The driver can choose to advertise the block as such.
> +
> +struct drm_color_op color_pipeline_X[] = {
> + {
> + .name = DRM_CB_PRIVATE,
> + .type = FIXED_FUNCTION,
> + .blob_id = 45;
> + },
> +}
> +
> +Where the blob represents some vendor specific enums, strings or any other
> +appropriate data types which both the user-space and drivers are aligned on.
> +
> +blob:45 {
> + VENDORXXX_BT602_TO_BT2020,
> +};
> +
> +For enabling or passing parameters to such blocks, the user can send data
> +to the driver wrapped within a blob like any other color operation block.
> +
> + struct drm_color_op_data {
> + .name = DRM_CB_PRIVATE;
> + .blob_id = 46;
> + } ;
> +
> +where blob with id 46 contains data to enable the fixed function hardware(s).
> +
One major thing missing from the RFC is an explanation on why we want to
go with a prescriptive model, rather than a descriptive model. This
question came up in Dave's response to Simon's RFC:
https://lore.kernel.org/dri-devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.com/
This is a rough attempt at such an explanation:
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d8d
Harry
> +References
> +==========
> +
> +[1] https://gitlab.freedesktop.org/emersion/drm_info
> +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-08-29 19:40 ` Harry Wentland
0 siblings, 0 replies; 109+ messages in thread
From: Harry Wentland @ 2023-08-29 19:40 UTC (permalink / raw)
To: Uma Shankar, intel-gfx, dri-devel; +Cc: wayland-devel
On 2023-08-29 12:03, Uma Shankar wrote:
> Add the documentation for the new proposed Plane Color Pipeline.
>
> Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> ---
> .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> 1 file changed, 394 insertions(+)
> create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
>
> diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst
> new file mode 100644
> index 000000000000..60ce515b6ea7
> --- /dev/null
> +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> @@ -0,0 +1,394 @@
> +=======================================
> + Plane Color Pipeline: A UAPI proposal
> +=======================================
> +
> +To build the proposal on, lets take the premise of a color pipeline as shown
> +below.
> +
> + +-------------------------------------------+
> + | RAM |
> + | +------+ +---------+ +---------+ |
> + | | FB 1 | | FB 2 | | FB N | |
> + | +------+ +---------+ +---------+ |
> + +-------------------------------------------+
> + | Plane Color Hardware Block |
> + +--------------------------------------------+
> + | +---v-----+ +---v-------+ +---v------+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> + | +---+-----+ +---+-------+ +---+------+ |
> + | | | | |
> + | +---v-----+ +---v-------+ +---v------+ |
> + | |Plane A | | Plane B | | Plane N | |
> + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + | +---v-----+ +----v------+ +----v-----+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | |Post-CSC | | Post-CSC | | Post-CSC | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + +--------------------------------------------+
> ++------v--------------v---------------v-------|
> +|| ||
> +|| Pipe Blender ||
> ++--------------------+------------------------+
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Pre-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | Pipe Color |
> +| +-----------v----------+ Hardware |
> +| | Pipe CSC/CTM | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Post-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> ++---------------------------------------------+
> + |
> + v
> + Pipe Output
> +
> +Each plane consists of the following color blocks
> + * Pre-CSC : This block can used to linearize the input frame buffer data.
> + The linear data then can be further acted on by the following
> + color hardware blocks in the display hardware pipeline
> +
> + * CSC/CTM: Used to program color transformation matrix, this block is used
> + to perform color space conversions like BT2020 to BT709 or BT601
> + etc. This block acts on the linearized data coming from the
> + Pre-CSC HW block.
> +
> + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> + match the sink. Another use case of it could be to perform Tone
> + mapping for HDR use-cases.
> +
> +Data from multiple planes will then be fed to pipe/crtc where it will get blended.
> +There is a similar set of HW blocks available at pipe/crtc level which acts on
> +this blended data.
> +
> +Below is a sample usecase fo video playback with sub-titles and playback
> +controls
> +
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
> +(subtitles) │
> + │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
> +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
> +(Playback controls UI) │ │
> + │ │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
> +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │
> +(video frame) │ │ │
> + │ │ │
> +┌────────────────────────────────────────────────────────────────────────┴─┴─┘
> +│
> +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> +│ │ CRTC │ │ CRTC │ │ CRTC │
> +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port
> + │ │ │ │ │ │
> + └─────────────┘ └─────────────┘ └───────────────┘
> +
> +This RFC is intended to propose an uAPI for the pre-blending color pipeline
> +(however, can be also extended to post blending pipeline).
> +
> +Below are the design considerations while designing the uAPI.
> +
> +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
> +2. Position of the HW block can be programmable
> +3. LUTs can be one dimentional or three dimentional
> +4. Number of LUT entries can vary across platforms
> +5. Precision of LUT entries can vary across platforms
> +6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> + Piece-Wise Linear(PWL) etc
> +7. There can be parameterized/non-parameterized fixed function HW blocks.
> + e.g. Just a hardware bit, to convert from one color space to another.
> +
> +Plane Color Pipeline: Design Proposal
> +=====================================
> +Each Color Hardware block will be represented by the structure drm_color_op.
> +
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type;
> + u32 blob_id;
> + u32 private_flags;
> +};
> +
> +The members of which will constitute:
> +1. name: A standardised enum for the color hardware block
> +2. type: The type of mathematical operation done by the hardware block.
> + e.g. 1D Curve, 3D Curve, Matrix etc.
> +3. blob id: Id pointing to a blob containing information about the hardware
> + block advertising the respective capabilities to the userspace.
> + It can be an optional field depending on the members "name" and "type".
> +4. private_flags: This can be used to provide vendor specific hints
> + to user space
> +
> +
> + For example to represent LUTs, we introduce the drm_color_lut_range
> + structure. It can represent LUTs with varied number of entries and
> + distributions (Multi segmented, Logarithmic etc).
> +
> + struct drm_color_lut_range {
> + /* DRM_MODE_LUT_* */
> + __u32 flags;
> + /* number of points on the curve */
> + __u16 count;
> + /* input/output bits per component */
> + __u8 input_bpc, output_bpc;
> + /* input start/end values */
> + __s32 start, end;
> + /* output min/max values */
> + __s32 min, max;
> + };
> +
> +Note: More details on exact usage and implementation of this structure can be
> + found in the RFC. This structure is taken as is from the series.
> + https://patchwork.freedesktop.org/series/90825/
> + However, we can add more members to it to encompass all use-cases.
> + For example. we can add a precision field to advertise the
> + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> + field for vendor specific use cases.
> +
> + At the same time, we don't need to pass any additional information for the
> + CSC block as userspace and driver already agrees struct drm_color_ctm as
> + a uAPI to pass on data.
> +
> +These color operations will form the building blocks of a color pipeline which
> +best represents the underlying Hardware. Color operations can be re-arranged,
> +substracted or added to create distinct color pipelines to accurately describe
> +the Hardware blocks present in the display engine.
> +
What do you mean when you're talking of re-arranging, or subtracting
color operations?
> +In this proposal, a color pipeline is represented as an array of
> +struct drm_color_op.
> +
> +struct drm_color_op color_pipeline_1[]
> +
> +For example to represent the pre-blending color pipeline as described above
> +
> +We can create a color pipeline like this.
> +
> +struct drm_color_op color_pipeline_1[] = {
> + {
> + .name = DRM_CB_PRE_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0; /* actual value to be populated during plane
> + initialization*/
> + },
> + {
> + .name = DRM_CB_CSC,
> + .type = MATRIX,
> + .blob_id = 0;
> + },
> + {
> + .name = DRM_CB_POST_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0;
> + },
> +};
> +
> +Then, for individual color operation, we add blobs to advertise the capability
> +of the particular Hardware block. In case of the example pipeline, we add
> +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
> +For the "csc" block we pass no blob id to user space as it is known to both
> +user space and driver that drm_color_ctm structure is to be used for such
> +operation.
> +
> +To represent, this in a diagram.
> +
> + struct drm_color_op color_pipeline_1[]
> + +---------------------------+
> + | | drm_color_op
> + | +---------------------+--+-----------+---------------------+
> + | | | | | |
> + | | | | | +-----------------+ |
> + | | | | | | name | |
> + | | | | | +-----------------+ |
> + | | | | | | type | |
> + | | color_op_1 | | | +-----------------+ |
> + | | | | | | blob id | +--------+
> + | | | | | +-----------------+ | |
> + | | | | | | private | | |
> + | | | | | | flags | | |
> + | | | | | +-----------------+ | |
> + | | | | | | |
> + | +---------------------+--+-----------+---------------------+ |
> + | | |
> + | | |
> + | +---------------------+ | |
> + | | | | drm_color_lut_range |
> + | | color_op_2 | | +-------------------------+ |
> + | | | | | | |
> + | | | | | +---------------------+ | |
> + | +---------------------+ | | | segment 1 { | |<---+
> + | | | | ... | |
> + | +---------------------+ | | | .input_bpc = 16, | |
> + | | | | | | .output_bpc = 16, | |
> + | | color_op_3 | | | | ... | |
> + | | | | | | } | |
> + | | | | | +---------------------+ |
> + | +---------------------+ | | |
> + | . | | +---------------------+ |
> + | . | | | segment 2 { | |
> + | . | | | ... | |
> + +---------------------------+ | | } | |
> + | | | |
> + | | | |
> + | | | |
> + | +---------------------+ |
> + | . |
> + | . |
> + | . |
> + +-------------------------+
> +
> +
> +
> +This color pipeline is then packaged within a blob for the user space to
> +retrieve it. Details can be found in the next section
> +
Not sure I like blobs that contain other blob ids.
> +Exposing a color pipeline to user space
> +=======================================
> +
> +To advertise the available color pipelines, an immutable ENUM property
> +"GET_COLOR_PIPELINE" is introduced.
> +This enum property has blob id's as values. With each blob id representing
> +a distinct color pipeline based on underlying HW capabilities and their
> +respective combinations.
> +
> +The following output of drm_info [1], shows how color pipelines are visible
> +to userspace.
> +
> +├───Plane 0
> + │ ├───Object ID: 31
> + │ ├───CRTCs: {0}
> + │ ├───Legacy info
> + ...
> + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> + color pipeline 1, color pipeline 2}= no color pipeline
> +
> +To understand the capabilities of individual pipelines, first the userspace
> +need to retrieve the pipeline blob with the blob ids retrieved by reading the
> +enum property.
> +
> +Once the color pipeline is retrieved, user can then parse through
> +individual drm_color_op blocks to understand the capabilities of each
> +hardware block.
> +
> +Check IGT series to see how user space can parse through color pipelines.
> +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
> +
> +Setting up a color pipeline
> +===========================
> +
> +Once the user space decides on a color pipeline, it can set the pipeline and
> +the corresponding data for the hardware blocks within the pipeline with
> +the BLOB property "SET_COLOR_PIPELINE".
> +
> +To achieve this two structures are introduced
> +
> +1. struct drm_color_op_data: It represents data to be passed onto individual
> + color hardware blocks. It contains three members
> + a) name: to identify the color operation block
> + b) blob_id: pointing to the blob with data for the
> + corresponding hardware block
> +
> + struct drm_color_op_data {
> + enum color_op_block name;
> + u32 blob_id;
> + };
> +
> +2. struct drm_color_pipeline: This structure represents the aggregate
> + pipeline to be set. it contains the following members
> + a) num: pipeline number to be selected
> + b) size: size of the data to be passed onto the driver
> + c) data: array of struct drm_color_op_data with data for
> + the hardware block/s that userspace wants to
> + set values for.
> +
> + struct drm_color_pipeline {
> + int num;
> + int size;
> + struct drm_color_op_data *data;
> + };
> +
> + User can either:
> + 1. send data for all the color operations in a pipeline as shown in [2].
> + The color operation data need not be in order that the pipeline advertises
> + however, it should not contain data for any
> + color operation that is not present in the pipeline.
> +
> + Note: This check for pipeline is not yet implemented but if the
> + wider proposal is accepted we have few solutions in mind.
> +
> + 2. send data for a subset of color operations as shown in [3].For the
> + color operation that userspace does not send data will retain their
> + older state.
> +
> + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> + property to NULL as shown in both [2] and [3]
> +
> + 4. change the color pipeline as demonstrated in [3].
> + On the new pipeline, the user is expected to setup all the color hardware block
> + Once the user requests a pipeline change, the driver will provide it a clean slate
> + which means that all the data previously set by the user will be discarded even if
> + there are common hardware blocks between the two(previous and current) pipelines.
> +
> +IGT implementation can be found here [4]
> +
> +Representing Fixed function hardware
> +====================================
> +
> +To provide support for fixed function hardware, the driver could expose vendor
> +specific struct drm_color_op with parameters that both the userspace and
> +driver agrees on. To demonstrate, let's consider a hyphothetical fixed
> +function hardware block that converts BT601 to BT2020.
> +The driver can choose to advertise the block as such.
> +
> +struct drm_color_op color_pipeline_X[] = {
> + {
> + .name = DRM_CB_PRIVATE,
> + .type = FIXED_FUNCTION,
> + .blob_id = 45;
> + },
> +}
> +
> +Where the blob represents some vendor specific enums, strings or any other
> +appropriate data types which both the user-space and drivers are aligned on.
> +
> +blob:45 {
> + VENDORXXX_BT602_TO_BT2020,
> +};
> +
> +For enabling or passing parameters to such blocks, the user can send data
> +to the driver wrapped within a blob like any other color operation block.
> +
> + struct drm_color_op_data {
> + .name = DRM_CB_PRIVATE;
> + .blob_id = 46;
> + } ;
> +
> +where blob with id 46 contains data to enable the fixed function hardware(s).
> +
One major thing missing from the RFC is an explanation on why we want to
go with a prescriptive model, rather than a descriptive model. This
question came up in Dave's response to Simon's RFC:
https://lore.kernel.org/dri-devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.com/
This is a rough attempt at such an explanation:
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d8d
Harry
> +References
> +==========
> +
> +[1] https://gitlab.freedesktop.org/emersion/drm_info
> +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Add Support for Plane Color Pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
` (34 preceding siblings ...)
(?)
@ 2023-08-29 20:02 ` Patchwork
-1 siblings, 0 replies; 109+ messages in thread
From: Patchwork @ 2023-08-29 20:02 UTC (permalink / raw)
To: Uma Shankar; +Cc: intel-gfx
== Series Details ==
Series: Add Support for Plane Color Pipeline
URL : https://patchwork.freedesktop.org/series/123023/
State : warning
== Summary ==
Error: dim checkpatch failed
5b6e9790b630 drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
-:14: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#14:
new file mode 100644
-:19: WARNING:SPDX_LICENSE_TAG: Missing or malformed SPDX-License-Identifier tag in line 1
#19: FILE: Documentation/gpu/rfc/plane_color_pipeline.rst:1:
+=======================================
total: 0 errors, 2 warnings, 0 checks, 394 lines checked
4aeebe895139 drm: Add color operation structure
970dacdffa01 drm: Add plane get color pipeline property
7b46fd8d80a1 drm: Add helper to add color pipeline
97ea5d644a12 drm: Add structures for setting color pipeline
1e6046947815 drm: Add set colorpipeline property
-:28: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#28: FILE: drivers/gpu/drm/drm_atomic_uapi.c:595:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->set_color_pipeline_data,
total: 0 errors, 0 warnings, 1 checks, 112 lines checked
63ce75dc0166 drm: Add Enhanced Gamma LUT precision structure
bef188076fbe drm: Add color lut range structure
ae5fe78f0d56 drm: Add color information to plane state
19343ed5ea63 drm: Manage color blob states
862b423779e4 drm: Replace individual color blobs
-:31: CHECK:PREFER_KERNEL_TYPES: Prefer kernel type 'u64' over 'uint64_t'
#31: FILE: drivers/gpu/drm/drm_atomic_uapi.c:417:
+ uint64_t color_pipeline_blob_id,
-:64: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#64: FILE: drivers/gpu/drm/drm_atomic_uapi.c:450:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.ctm,
-:70: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#70: FILE: drivers/gpu/drm/drm_atomic_uapi.c:456:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.pre_csc_lut,
-:76: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#76: FILE: drivers/gpu/drm/drm_atomic_uapi.c:462:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.post_csc_lut,
-:82: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#82: FILE: drivers/gpu/drm/drm_atomic_uapi.c:468:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.private_color_op_data,
total: 0 errors, 0 warnings, 5 checks, 109 lines checked
ea3a48a26b45 drm: Reset pipeline when user sends NULL blob
b75c6a76b293 drm: Reset plane color state on pipeline switch request
-:67: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#67: FILE: drivers/gpu/drm/drm_atomic_uapi.c:765:
+ cp_change_requested = color_pipeline_change_requested(dev,
+ state->set_color_pipeline_data,
-:83: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#83: FILE: drivers/gpu/drm/drm_atomic_uapi.c:782:
+ ret = drm_plane_reset_color_op_blobs(plane,
+ state, &replaced);
total: 0 errors, 0 warnings, 2 checks, 72 lines checked
a018bab5b7c5 drm/i915/color: Add lut range for SDR planes
11ce867cfea0 drm/i915/color: Add lut range for HDR planes
a4e18914ef30 drm/i915/color: Add color pipeline for HDR planes
cfc3198574b2 drm/i915/color: Add color pipeline for SDR planes
35769022d6c9 drm/i915/color: Add HDR plane LUT range data to color pipeline
4e6de9ba7fbd drm/i915/color: Add SDR plane LUT range data to color pipeline
1ca1fc5c040f drm/i915/color: Add color pipelines to plane
9b1b69f14b09 drm/i915/color: Create and attach set color pipeline property
d9b196b1c211 drm/i915/color: Add plane color callbacks
5eba0edf5e53 drm/i915/color: Load plane color luts from atomic flip
30d22691d27a drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
7b6f6ba37ab3 drm/i915/xelpd: Add register definitions for Plane Degamma
-:29: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#29: FILE: drivers/gpu/drm/i915/i915_reg.h:6238:
+#define PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe))
-:41: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#41: FILE: drivers/gpu/drm/i915/i915_reg.h:6250:
+#define PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_2(pipe))
-:53: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#53: FILE: drivers/gpu/drm/i915/i915_reg.h:6262:
+#define PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_2(pipe))
-:65: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#65: FILE: drivers/gpu/drm/i915/i915_reg.h:6274:
+#define PLANE_PRE_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_2(pipe))
total: 0 errors, 0 warnings, 4 checks, 55 lines checked
e4902513d633 drm/i915/color: Add color functions for ADL
b7087ab3594d drm/i915/color: Program Plane Pre-CSC Registers
af571e45b3e7 drm/i915/xelpd: Add register definitions for Plane Post CSC
-:25: WARNING:LONG_LINE: line length of 110 exceeds 100 columns
#25: FILE: drivers/gpu/drm/i915/i915_reg.h:6234:
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A, \
-:27: WARNING:LONG_LINE: line length of 110 exceeds 100 columns
#27: FILE: drivers/gpu/drm/i915/i915_reg.h:6236:
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A, \
-:29: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#29: FILE: drivers/gpu/drm/i915/i915_reg.h:6238:
+#define PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe))
-:37: WARNING:LONG_LINE: line length of 109 exceeds 100 columns
#37: FILE: drivers/gpu/drm/i915/i915_reg.h:6246:
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A, \
-:39: WARNING:LONG_LINE: line length of 109 exceeds 100 columns
#39: FILE: drivers/gpu/drm/i915/i915_reg.h:6248:
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A, \
-:41: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#41: FILE: drivers/gpu/drm/i915/i915_reg.h:6250:
+#define PLANE_POST_CSC_GAMC_SEG0_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe))
-:53: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#53: FILE: drivers/gpu/drm/i915/i915_reg.h:6262:
+#define PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_2(pipe))
-:65: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#65: FILE: drivers/gpu/drm/i915/i915_reg.h:6274:
+#define PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_2(pipe))
-:77: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#77: FILE: drivers/gpu/drm/i915/i915_reg.h:6286:
+#define PLANE_POST_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_2(pipe))
-:89: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects?
#89: FILE: drivers/gpu/drm/i915/i915_reg.h:6298:
+#define PLANE_POST_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_2(pipe))
total: 0 errors, 4 warnings, 6 checks, 79 lines checked
7746bc9e6186 drm/i915/xelpd: Program Plane Post CSC Registers
47f636cfee5f drm/i915/color: Enable Plane CSC
ef86823dca18 drm/i915/color: Enable plane color features
138b81f07c65 drm/i915/color: Add a dummy pipeline with 3D LUT
-:59: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#59: FILE: drivers/gpu/drm/drm_atomic_uapi.c:565:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.lut_3d,
total: 0 errors, 0 warnings, 1 checks, 124 lines checked
d1d9a14565f2 drm/i915/color: Add example implementation for vendor specific color operation
^ permalink raw reply [flat|nested] 109+ messages in thread
* [Intel-gfx] ✗ Fi.CI.SPARSE: warning for Add Support for Plane Color Pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
` (35 preceding siblings ...)
(?)
@ 2023-08-29 20:02 ` Patchwork
-1 siblings, 0 replies; 109+ messages in thread
From: Patchwork @ 2023-08-29 20:02 UTC (permalink / raw)
To: Uma Shankar; +Cc: intel-gfx
== Series Details ==
Series: Add Support for Plane Color Pipeline
URL : https://patchwork.freedesktop.org/series/123023/
State : warning
== Summary ==
Error: dim sparse failed
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.
^ permalink raw reply [flat|nested] 109+ messages in thread
* [Intel-gfx] ✗ Fi.CI.BAT: failure for Add Support for Plane Color Pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
` (36 preceding siblings ...)
(?)
@ 2023-08-29 20:17 ` Patchwork
-1 siblings, 0 replies; 109+ messages in thread
From: Patchwork @ 2023-08-29 20:17 UTC (permalink / raw)
To: Uma Shankar; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 6892 bytes --]
== Series Details ==
Series: Add Support for Plane Color Pipeline
URL : https://patchwork.freedesktop.org/series/123023/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_13574 -> Patchwork_123023v1
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_123023v1 absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in Patchwork_123023v1, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/index.html
Participating hosts (40 -> 38)
------------------------------
Missing (2): fi-kbl-soraka fi-snb-2520m
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_123023v1:
### IGT changes ###
#### Possible regressions ####
* igt@debugfs_test@read_all_entries:
- bat-adlp-11: [PASS][1] -> [ABORT][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-adlp-11/igt@debugfs_test@read_all_entries.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-adlp-11/igt@debugfs_test@read_all_entries.html
* igt@i915_module_load@load:
- bat-dg2-11: [PASS][3] -> [ABORT][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-11/igt@i915_module_load@load.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-11/igt@i915_module_load@load.html
- bat-dg2-9: [PASS][5] -> [ABORT][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-9/igt@i915_module_load@load.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-9/igt@i915_module_load@load.html
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses.
They do not affect the overall result.
* igt@i915_module_load@load:
- {bat-dg2-13}: [DMESG-WARN][7] ([i915#8879]) -> [ABORT][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-13/igt@i915_module_load@load.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-13/igt@i915_module_load@load.html
- {bat-dg2-14}: [DMESG-WARN][9] ([i915#8879]) -> [ABORT][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-14/igt@i915_module_load@load.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-14/igt@i915_module_load@load.html
Known issues
------------
Here are the changes found in Patchwork_123023v1 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_suspend@basic-s3-without-i915:
- bat-rpls-2: [PASS][11] -> [ABORT][12] ([i915#7978] / [i915#8668])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-rpls-2/igt@i915_suspend@basic-s3-without-i915.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-rpls-2/igt@i915_suspend@basic-s3-without-i915.html
* igt@kms_psr@sprite_plane_onoff:
- bat-rplp-1: NOTRUN -> [ABORT][13] ([i915#8442] / [i915#8668] / [i915#8712])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-rplp-1/igt@kms_psr@sprite_plane_onoff.html
#### Warnings ####
* igt@kms_psr@cursor_plane_move:
- bat-rplp-1: [ABORT][14] ([i915#8469] / [i915#8668]) -> [SKIP][15] ([i915#1072])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-rplp-1/igt@kms_psr@cursor_plane_move.html
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-rplp-1/igt@kms_psr@cursor_plane_move.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
[i915#7978]: https://gitlab.freedesktop.org/drm/intel/issues/7978
[i915#8442]: https://gitlab.freedesktop.org/drm/intel/issues/8442
[i915#8469]: https://gitlab.freedesktop.org/drm/intel/issues/8469
[i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668
[i915#8712]: https://gitlab.freedesktop.org/drm/intel/issues/8712
[i915#8879]: https://gitlab.freedesktop.org/drm/intel/issues/8879
Build changes
-------------
* Linux: CI_DRM_13574 -> Patchwork_123023v1
CI-20190529: 20190529
CI_DRM_13574: 2a7ec376910bf7ab4ae8a2594e73d9b5c1570046 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_7459: 7459
Patchwork_123023v1: 2a7ec376910bf7ab4ae8a2594e73d9b5c1570046 @ git://anongit.freedesktop.org/gfx-ci/linux
### Linux commits
ecc6154eb822 drm/i915/color: Add example implementation for vendor specific color operation
c10cba0b8b80 drm/i915/color: Add a dummy pipeline with 3D LUT
6e9e83d26c3d drm/i915/color: Enable plane color features
cd9576deea7b drm/i915/color: Enable Plane CSC
5074441371bb drm/i915/xelpd: Program Plane Post CSC Registers
89decd8c4135 drm/i915/xelpd: Add register definitions for Plane Post CSC
a8a81ddbe737 drm/i915/color: Program Plane Pre-CSC Registers
57032feab0b9 drm/i915/color: Add color functions for ADL
b3d1021ff46e drm/i915/xelpd: Add register definitions for Plane Degamma
98d568bd1d0a drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
34b5a55fbbab drm/i915/color: Load plane color luts from atomic flip
d8d431692734 drm/i915/color: Add plane color callbacks
22ea46c5f4db drm/i915/color: Create and attach set color pipeline property
13852a46688b drm/i915/color: Add color pipelines to plane
5666e64f3220 drm/i915/color: Add SDR plane LUT range data to color pipeline
95df87f4d41d drm/i915/color: Add HDR plane LUT range data to color pipeline
d4bc2af110fe drm/i915/color: Add color pipeline for SDR planes
5ba690dfd71c drm/i915/color: Add color pipeline for HDR planes
6e555e736ce4 drm/i915/color: Add lut range for HDR planes
f08d49d53577 drm/i915/color: Add lut range for SDR planes
bdd8237bbb19 drm: Reset plane color state on pipeline switch request
835faea2c02e drm: Reset pipeline when user sends NULL blob
f207dc394c37 drm: Replace individual color blobs
9191449a2b6d drm: Manage color blob states
3c033443a4d7 drm: Add color information to plane state
c4886eb7d358 drm: Add color lut range structure
f86cbf894428 drm: Add Enhanced Gamma LUT precision structure
fcca79be7769 drm: Add set colorpipeline property
66caa50d297f drm: Add structures for setting color pipeline
49719816cfa5 drm: Add helper to add color pipeline
a0c8dce28b55 drm: Add plane get color pipeline property
9d16aa3dfca1 drm: Add color operation structure
761ad4cedd79 drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/index.html
[-- Attachment #2: Type: text/html, Size: 8105 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [RFC 00/33] Add Support for Plane Color Pipeline
2023-08-29 19:26 ` [Intel-gfx] " Harry Wentland
@ 2023-08-30 8:47 ` Shankar, Uma
-1 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-08-30 8:47 UTC (permalink / raw)
To: Harry Wentland, intel-gfx, dri-devel
Cc: Sebastian Wick, Pekka Paalanen, Shashank Sharma, wayland-devel,
Melissa Wen, Jonas Ådahl, Naseer Ahmed, Christopher Braga
> -----Original Message-----
> From: Harry Wentland <harry.wentland@amd.com>
> Sent: Wednesday, August 30, 2023 12:56 AM
> To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> devel@lists.freedesktop.org
> Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> <ville.syrjala@linux.intel.com>; Pekka Paalanen <pekka.paalanen@collabora.com>;
> Simon Ser <contact@emersion.fr>; Melissa Wen <mwen@igalia.com>; Jonas Ådahl
> <jadahl@redhat.com>; Sebastian Wick <sebastian.wick@redhat.com>; Shashank
> Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> <quic_cbraga@quicinc.com>
> Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
>
> +CC Naseer and Chris, FYI
>
> See https://patchwork.freedesktop.org/series/123024/ for whole series.
>
> On 2023-08-29 12:03, Uma Shankar wrote:
> > Introduction
> > ============
> >
> > Modern hardwares have various color processing capabilities both at
> > pre-blending and post-blending phases in the color pipeline.
> > The current drm implementation exposes only the post-blending color
> > hardware blocks. Support for pre-blending hardware is missing.
> > There are multiple use cases where pre-blending color hardware will be
> > useful:
> > a) Linearization of input buffers encoded in various transfer
> > functions.
> > b) Color Space conversion
> > c) Tone mapping
> > d) Frame buffer format conversion
> > e) Non-linearization of buffer(apply transfer function)
> > f) 3D Luts
> >
> > and other miscellaneous color operations.
> >
> > Hence, there is a need to expose the color capabilities of the
> > hardware to user-space. This will help userspace/middleware to use
> > display hardware for color processing and blending instead of doing it
> > through GPU shaders.
> >
>
> Thanks, Uma, for sending this. I've been working on something similar but you beat
> me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
> >
> > Work done so far and relevant references
> > ========================================
> >
> > Some implementation is done by Intel and AMD/Igalia to address the same.
> > Broad consensus is there that we need a generic API at drm core to
> > suffice the use case of various HW vendors. Below are the links
> > capturing the discussion so far.
> >
> > Intel's Plane Color Implementation:
> > https://patchwork.freedesktop.org/series/90825/
> > AMD's Plane Color Implementation:
> > https://patchwork.freedesktop.org/series/116862/
> >
> >
> > Hackfest conclusions
> > ====================
> >
> > HDR/Color Hackfest was organised by Redhat to bring all the industry
> > stakeholders together and converge on a common uapi expectations.
> > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and
> > other prominent user-space developers and maintainers.
> >
> > Discussions happened on the uapi expectations, opens, nature of
> > hardware of multiple hardware vendors, challenges in generalizing the
> > same and the path forward. Consensus was made that drm core should
> > implement descriptive APIs and not go with prescriptive APIs. DRM core
> > should just expose the hardware capabilities; enabling, customizing
> > and programming the same should be done by the user-space. Driver should just
> honor the user space request without doing any operations internally.
> >
> > Thanks to Simon Ser, for nicely documenting the design consensus and
> > an UAPI RFC which can be referred to here:
> >
> > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5
> >
> nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1Q
> Wn48
> > 8=@emersion.fr/
> >
> >
> > Design considerations
> > =====================
> >
> > Following are the important aspects taken into account while designing
> > the current RFC
> > proposal:
> >
> > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one
> can be used)
> > 2. Position of the HW block in the pipeline can be programmable
> > 3. LUTs can be one dimentional or three dimentional
> > 4. Number of LUT entries can vary across platforms
> > 5. Precision of LUT entries can vary across platforms
> > 6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> > Piece-Wise Linear(PWL) etc
> > 7. There can be parameterized/non-parameterized fixed function HW
> blocks.
> > e.g. Just a hardware bit, to convert from one color space to another.
> > 8. Custom non-standard HW implementation.
> > 9. Leaving scope for some vendor defined pescriptive implementation if
> required.
> > 10.Scope to handle any modification in hardware as technology evolves
> >
> > The current proposal takes into account the above considerations while
> > keeping the implementation as generic as possible leaving scope for future
> additions or modifications.
> >
> > This proposal is also in line to the details mentioned by Simon's RFC
> > covering all the aspects discussed in hackfest.
> >
> >
> > Outline of the implementation
> > ============================
> >
> > Each Color Hardware block will be represented by a data structure drm_color_op.
> > These color operations will form the building blocks of a color
> > pipeline which best represents the underlying Hardware. Color
> > operations can be re-arranged, substracted or added to create distinct
> > color pipelines to accurately describe the Hardware blocks present in the display
> engine.
>
> Who is doing the arranging of color operations? IMO a driver should define one or
> more respective pipelines that can be selected by userspace. This seems to be what
> you're talking about after (I haven't reviewed the whole thing yet). Might be best to
> drop this sentence or to add clarifications in order to avoid confusion.
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement
and possible combinations. There can be multiple pipelines possible if hardware can be
muxed or order can be re-arranged (all viable combinations should be defined as a pipeline in driver).
Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> >
> > In this proposal, a color pipeline is represented as an array of
> > struct drm_color_op. For individual color operation, we add blobs to
> > advertise the capability of the particular Hardware block.
> >
> > This color pipeline is then packaged within a blob for the user space
> > to retrieve it.
> >
>
> Would it be better to expose the drm_color_ops directly, instead of packing a array
> of drm_color_ops into a blob and then giving that to userspace.
Advantage we see in packing in blobs is that interface will be cleaner. There will be just
one GET_COLOR_PIPELINE property to invoke by user and then its just parsing the data.
This way the entire underlying hardware blocks with pipeline will be available to user.
For a particular hardware block in pipeline, user can get the relevant details from blob
representing that particular block. We have created IGT tests (links mentioned in cover-letter)
to demonstrate how it can be done. This is just to clarify the idea.
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL
hardware blocks. Also we have left scope for custom private hardware blocks as well which driver
can work with respective HAL and get that implemented.
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL
agree.
> > To advertise the available color pipelines, an immutable ENUM property
> > "GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
> > With each blob id representing a distinct color pipeline based on
> > underlying HW capabilities and their respective combinations.
> >
> > Once the user space decides on a color pipeline, it can set the
> > pipeline and the corresponding data for the hardware blocks within the
> > pipeline with the BLOB property "SET_COLOR_PIPELINE".
> >
>
> When I discussed this at the hackfest with Simon he proposed a new DRM object,
> (I've called it a drm_colorop) to represent a color operation.
> Each drm_colorop has a "NEXT" pointer to another drm_colorop, or NULL if its the
> last in the pipeline. You can then have a mutable enum property on the plane to
> discover and select a color pipeline.
Yes, the proposal is inspired by this idea. Sure, we can work together to enhance the design.
Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon
and you Harry. Management of the pipeline, addition, deletion and flexibility to represent
hardware is more with blobs with the relevant structures agreed in UAPI.
> This seems a bit more transparent than a blob. You can see my changes
> (unfortunately very WIP, don't look too closely at individual patches) at
> https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs
>
> libdrm changes:
> https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
Sure, will check the same.
> IGT changes:
> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffs
>
> I'll take time to review your whole series and will see whether we can somehow
> keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
> Curious to hear other opinions on the blob vs new DRM object question as well.
Yeah, request community and stakeholders to share feedback and suggestions.
We will work on the feedback to improve the design.
> > Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the
> > patch
> >
> > IGT and test details
> > ====================
> >
> > A set of IGT tests is written to demonstrate the usage of the proposed
> > UAPIs along with some sanity validation.
> >
> > Details of the IGT test can be found here:
> > https://patchwork.freedesktop.org/series/123018/
> >
> > Opens
> > =====
> >
> > a. To come up with a list of common HW blocks which can be defined generically
> by the DRM
> > core in agreement with all the stakeholders b. To enhance/finalize
> > the data structure to define segmented LUTs generically.
> >
>
> It would be good to add some basic support in VKMS. My work has been based on
> VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private
> properties from Melissa through the API.
Yeah sure Harry.
> >
> > Out of Scope
> > ============
> >
> > a. The coefficients for CTM and LUT value calculations are out of scope of the
> proposal.
> > b. The onus of programming the HW blocks and their values is on user-space.
> Driver will
> > just provide the interface for the same.
> > c. In order to compute LUT values and coefficients, a helper library can be created
> in
> > user-space. However, it is out of scope for the current proposal.
> >
> > Acknowledgements and credits
> > ============================
> >
> > There are multiple contributors who have helped us to reach to this
> > proposal. Special mention to Ville
> > Syrjala<ville.syrjala@linux.intel.com>, Pekka
> > Paalanen<pekka.paalanen@collabora.com>,
> > Simon Ser<contact@emersion.fr>, Harry
> > Wentland<harry.wentland@amd.com>, Melissa Wen<mwen@igalia.com>,
> > Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
> Bhanu<bhanuprakash.modem@intel.com> and
> Shashank<shashank.sharma@amd.com>.
> >
> > Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for
> organizing the HDR hackfest.
> >
> >
> > UAPI dependency and Usermode development
> > ========================================
> >
> > The current KMS implementation requires a user space consumer for it to be
> accepted upstream.
> > Work is ongoing in weston and mutter community to get color management
> > and HDR support implemented in the respective stacks.
> >
>
> If we can get AMD properties encoded using a Color Pipeline API we can probably
> use gamescope as the userspace vehicle.
Yeah, nice.
> I'm reviewing this in sequence, so there's a chance I'm missing context.
> Please bear with me if some of my comments are answered later in the series.
No worries, really appreciate the feedback and support.
> Again, thanks for sending this.
Always welcome.
Regards,
Uma Shankar
> Harry
>
> >
> ====================================================================
> ==
> > ===========
> >
> > We have tried to take care of all the scenarios and use-cases which
> > possibly could exists in the current proposal. Thanks to everyone who
> > has contributed in all color management discussions so far. Let's work
> > together to improve the current proposal and get this thing implemented in
> upstream linux. All the feedback and suggestions to enhance the design are
> welcome.
> >
> > Regards,
> > Uma Shankar
> > Chaitanya Kumar Borah
> >
> > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > Cc: Simon Ser <contact@emersion.fr>
> > Cc: Harry Wentland <harry.wentland@amd.com>
> > Cc: Melissa Wen <mwen@igalia.com>
> > Cc: Jonas Ådahl <jadahl@redhat.com>
> > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > Cc: Alexander Goins <agoins@nvidia.com>
> >
> > Chaitanya Kumar Borah (14):
> > drm: Add color operation structure
> > drm: Add plane get color pipeline property
> > drm: Add helper to add color pipeline
> > drm: Manage color blob states
> > drm: Replace individual color blobs
> > drm: Reset pipeline when user sends NULL blob
> > drm: Reset plane color state on pipeline switch request
> > drm/i915/color: Add HDR plane LUT range data to color pipeline
> > drm/i915/color: Add SDR plane LUT range data to color pipeline
> > drm/i915/color: Add color pipelines to plane
> > drm/i915/color: Create and attach set color pipeline property
> > drm/i915/color: Enable plane color features
> > drm/i915/color: Add a dummy pipeline with 3D LUT
> > drm/i915/color: Add example implementation for vendor specific color
> > operation
> >
> > Uma Shankar (19):
> > drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> > drm: Add structures for setting color pipeline
> > drm: Add set colorpipeline property
> > drm: Add Enhanced Gamma LUT precision structure
> > drm: Add color lut range structure
> > drm: Add color information to plane state
> > drm/i915/color: Add lut range for SDR planes
> > drm/i915/color: Add lut range for HDR planes
> > drm/i915/color: Add color pipeline for HDR planes
> > drm/i915/color: Add color pipeline for SDR planes
> > drm/i915/color: Add plane color callbacks
> > drm/i915/color: Load plane color luts from atomic flip
> > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> > drm/i915/xelpd: Add register definitions for Plane Degamma
> > drm/i915/color: Add color functions for ADL
> > drm/i915/color: Program Plane Pre-CSC Registers
> > drm/i915/xelpd: Add register definitions for Plane Post CSC
> > drm/i915/xelpd: Program Plane Post CSC Registers
> > drm/i915/color: Enable Plane CSC
> >
> > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> > drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> > drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> > drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> > drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> > drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> > .../drm/i915/display/skl_universal_plane.c | 21 +-
> > drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> > include/drm/drm_plane.h | 82 +++
> > include/uapi/drm/drm_mode.h | 196 +++++
> > include/uapi/drm/i915_drm.h | 25 +
> > 11 files changed, 1899 insertions(+), 3 deletions(-)
> > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> >
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-08-30 8:47 ` Shankar, Uma
0 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-08-30 8:47 UTC (permalink / raw)
To: Harry Wentland, intel-gfx, dri-devel
Cc: Sebastian Wick, Pekka Paalanen, Simon Ser, Alexander Goins,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Christopher Braga
> -----Original Message-----
> From: Harry Wentland <harry.wentland@amd.com>
> Sent: Wednesday, August 30, 2023 12:56 AM
> To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> devel@lists.freedesktop.org
> Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> <ville.syrjala@linux.intel.com>; Pekka Paalanen <pekka.paalanen@collabora.com>;
> Simon Ser <contact@emersion.fr>; Melissa Wen <mwen@igalia.com>; Jonas Ådahl
> <jadahl@redhat.com>; Sebastian Wick <sebastian.wick@redhat.com>; Shashank
> Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> <quic_cbraga@quicinc.com>
> Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
>
> +CC Naseer and Chris, FYI
>
> See https://patchwork.freedesktop.org/series/123024/ for whole series.
>
> On 2023-08-29 12:03, Uma Shankar wrote:
> > Introduction
> > ============
> >
> > Modern hardwares have various color processing capabilities both at
> > pre-blending and post-blending phases in the color pipeline.
> > The current drm implementation exposes only the post-blending color
> > hardware blocks. Support for pre-blending hardware is missing.
> > There are multiple use cases where pre-blending color hardware will be
> > useful:
> > a) Linearization of input buffers encoded in various transfer
> > functions.
> > b) Color Space conversion
> > c) Tone mapping
> > d) Frame buffer format conversion
> > e) Non-linearization of buffer(apply transfer function)
> > f) 3D Luts
> >
> > and other miscellaneous color operations.
> >
> > Hence, there is a need to expose the color capabilities of the
> > hardware to user-space. This will help userspace/middleware to use
> > display hardware for color processing and blending instead of doing it
> > through GPU shaders.
> >
>
> Thanks, Uma, for sending this. I've been working on something similar but you beat
> me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
> >
> > Work done so far and relevant references
> > ========================================
> >
> > Some implementation is done by Intel and AMD/Igalia to address the same.
> > Broad consensus is there that we need a generic API at drm core to
> > suffice the use case of various HW vendors. Below are the links
> > capturing the discussion so far.
> >
> > Intel's Plane Color Implementation:
> > https://patchwork.freedesktop.org/series/90825/
> > AMD's Plane Color Implementation:
> > https://patchwork.freedesktop.org/series/116862/
> >
> >
> > Hackfest conclusions
> > ====================
> >
> > HDR/Color Hackfest was organised by Redhat to bring all the industry
> > stakeholders together and converge on a common uapi expectations.
> > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and
> > other prominent user-space developers and maintainers.
> >
> > Discussions happened on the uapi expectations, opens, nature of
> > hardware of multiple hardware vendors, challenges in generalizing the
> > same and the path forward. Consensus was made that drm core should
> > implement descriptive APIs and not go with prescriptive APIs. DRM core
> > should just expose the hardware capabilities; enabling, customizing
> > and programming the same should be done by the user-space. Driver should just
> honor the user space request without doing any operations internally.
> >
> > Thanks to Simon Ser, for nicely documenting the design consensus and
> > an UAPI RFC which can be referred to here:
> >
> > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5
> >
> nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1Q
> Wn48
> > 8=@emersion.fr/
> >
> >
> > Design considerations
> > =====================
> >
> > Following are the important aspects taken into account while designing
> > the current RFC
> > proposal:
> >
> > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one
> can be used)
> > 2. Position of the HW block in the pipeline can be programmable
> > 3. LUTs can be one dimentional or three dimentional
> > 4. Number of LUT entries can vary across platforms
> > 5. Precision of LUT entries can vary across platforms
> > 6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> > Piece-Wise Linear(PWL) etc
> > 7. There can be parameterized/non-parameterized fixed function HW
> blocks.
> > e.g. Just a hardware bit, to convert from one color space to another.
> > 8. Custom non-standard HW implementation.
> > 9. Leaving scope for some vendor defined pescriptive implementation if
> required.
> > 10.Scope to handle any modification in hardware as technology evolves
> >
> > The current proposal takes into account the above considerations while
> > keeping the implementation as generic as possible leaving scope for future
> additions or modifications.
> >
> > This proposal is also in line to the details mentioned by Simon's RFC
> > covering all the aspects discussed in hackfest.
> >
> >
> > Outline of the implementation
> > ============================
> >
> > Each Color Hardware block will be represented by a data structure drm_color_op.
> > These color operations will form the building blocks of a color
> > pipeline which best represents the underlying Hardware. Color
> > operations can be re-arranged, substracted or added to create distinct
> > color pipelines to accurately describe the Hardware blocks present in the display
> engine.
>
> Who is doing the arranging of color operations? IMO a driver should define one or
> more respective pipelines that can be selected by userspace. This seems to be what
> you're talking about after (I haven't reviewed the whole thing yet). Might be best to
> drop this sentence or to add clarifications in order to avoid confusion.
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement
and possible combinations. There can be multiple pipelines possible if hardware can be
muxed or order can be re-arranged (all viable combinations should be defined as a pipeline in driver).
Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> >
> > In this proposal, a color pipeline is represented as an array of
> > struct drm_color_op. For individual color operation, we add blobs to
> > advertise the capability of the particular Hardware block.
> >
> > This color pipeline is then packaged within a blob for the user space
> > to retrieve it.
> >
>
> Would it be better to expose the drm_color_ops directly, instead of packing a array
> of drm_color_ops into a blob and then giving that to userspace.
Advantage we see in packing in blobs is that interface will be cleaner. There will be just
one GET_COLOR_PIPELINE property to invoke by user and then its just parsing the data.
This way the entire underlying hardware blocks with pipeline will be available to user.
For a particular hardware block in pipeline, user can get the relevant details from blob
representing that particular block. We have created IGT tests (links mentioned in cover-letter)
to demonstrate how it can be done. This is just to clarify the idea.
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL
hardware blocks. Also we have left scope for custom private hardware blocks as well which driver
can work with respective HAL and get that implemented.
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL
agree.
> > To advertise the available color pipelines, an immutable ENUM property
> > "GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
> > With each blob id representing a distinct color pipeline based on
> > underlying HW capabilities and their respective combinations.
> >
> > Once the user space decides on a color pipeline, it can set the
> > pipeline and the corresponding data for the hardware blocks within the
> > pipeline with the BLOB property "SET_COLOR_PIPELINE".
> >
>
> When I discussed this at the hackfest with Simon he proposed a new DRM object,
> (I've called it a drm_colorop) to represent a color operation.
> Each drm_colorop has a "NEXT" pointer to another drm_colorop, or NULL if its the
> last in the pipeline. You can then have a mutable enum property on the plane to
> discover and select a color pipeline.
Yes, the proposal is inspired by this idea. Sure, we can work together to enhance the design.
Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon
and you Harry. Management of the pipeline, addition, deletion and flexibility to represent
hardware is more with blobs with the relevant structures agreed in UAPI.
> This seems a bit more transparent than a blob. You can see my changes
> (unfortunately very WIP, don't look too closely at individual patches) at
> https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs
>
> libdrm changes:
> https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
Sure, will check the same.
> IGT changes:
> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffs
>
> I'll take time to review your whole series and will see whether we can somehow
> keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
> Curious to hear other opinions on the blob vs new DRM object question as well.
Yeah, request community and stakeholders to share feedback and suggestions.
We will work on the feedback to improve the design.
> > Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the
> > patch
> >
> > IGT and test details
> > ====================
> >
> > A set of IGT tests is written to demonstrate the usage of the proposed
> > UAPIs along with some sanity validation.
> >
> > Details of the IGT test can be found here:
> > https://patchwork.freedesktop.org/series/123018/
> >
> > Opens
> > =====
> >
> > a. To come up with a list of common HW blocks which can be defined generically
> by the DRM
> > core in agreement with all the stakeholders b. To enhance/finalize
> > the data structure to define segmented LUTs generically.
> >
>
> It would be good to add some basic support in VKMS. My work has been based on
> VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private
> properties from Melissa through the API.
Yeah sure Harry.
> >
> > Out of Scope
> > ============
> >
> > a. The coefficients for CTM and LUT value calculations are out of scope of the
> proposal.
> > b. The onus of programming the HW blocks and their values is on user-space.
> Driver will
> > just provide the interface for the same.
> > c. In order to compute LUT values and coefficients, a helper library can be created
> in
> > user-space. However, it is out of scope for the current proposal.
> >
> > Acknowledgements and credits
> > ============================
> >
> > There are multiple contributors who have helped us to reach to this
> > proposal. Special mention to Ville
> > Syrjala<ville.syrjala@linux.intel.com>, Pekka
> > Paalanen<pekka.paalanen@collabora.com>,
> > Simon Ser<contact@emersion.fr>, Harry
> > Wentland<harry.wentland@amd.com>, Melissa Wen<mwen@igalia.com>,
> > Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
> Bhanu<bhanuprakash.modem@intel.com> and
> Shashank<shashank.sharma@amd.com>.
> >
> > Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for
> organizing the HDR hackfest.
> >
> >
> > UAPI dependency and Usermode development
> > ========================================
> >
> > The current KMS implementation requires a user space consumer for it to be
> accepted upstream.
> > Work is ongoing in weston and mutter community to get color management
> > and HDR support implemented in the respective stacks.
> >
>
> If we can get AMD properties encoded using a Color Pipeline API we can probably
> use gamescope as the userspace vehicle.
Yeah, nice.
> I'm reviewing this in sequence, so there's a chance I'm missing context.
> Please bear with me if some of my comments are answered later in the series.
No worries, really appreciate the feedback and support.
> Again, thanks for sending this.
Always welcome.
Regards,
Uma Shankar
> Harry
>
> >
> ====================================================================
> ==
> > ===========
> >
> > We have tried to take care of all the scenarios and use-cases which
> > possibly could exists in the current proposal. Thanks to everyone who
> > has contributed in all color management discussions so far. Let's work
> > together to improve the current proposal and get this thing implemented in
> upstream linux. All the feedback and suggestions to enhance the design are
> welcome.
> >
> > Regards,
> > Uma Shankar
> > Chaitanya Kumar Borah
> >
> > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > Cc: Simon Ser <contact@emersion.fr>
> > Cc: Harry Wentland <harry.wentland@amd.com>
> > Cc: Melissa Wen <mwen@igalia.com>
> > Cc: Jonas Ådahl <jadahl@redhat.com>
> > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > Cc: Alexander Goins <agoins@nvidia.com>
> >
> > Chaitanya Kumar Borah (14):
> > drm: Add color operation structure
> > drm: Add plane get color pipeline property
> > drm: Add helper to add color pipeline
> > drm: Manage color blob states
> > drm: Replace individual color blobs
> > drm: Reset pipeline when user sends NULL blob
> > drm: Reset plane color state on pipeline switch request
> > drm/i915/color: Add HDR plane LUT range data to color pipeline
> > drm/i915/color: Add SDR plane LUT range data to color pipeline
> > drm/i915/color: Add color pipelines to plane
> > drm/i915/color: Create and attach set color pipeline property
> > drm/i915/color: Enable plane color features
> > drm/i915/color: Add a dummy pipeline with 3D LUT
> > drm/i915/color: Add example implementation for vendor specific color
> > operation
> >
> > Uma Shankar (19):
> > drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> > drm: Add structures for setting color pipeline
> > drm: Add set colorpipeline property
> > drm: Add Enhanced Gamma LUT precision structure
> > drm: Add color lut range structure
> > drm: Add color information to plane state
> > drm/i915/color: Add lut range for SDR planes
> > drm/i915/color: Add lut range for HDR planes
> > drm/i915/color: Add color pipeline for HDR planes
> > drm/i915/color: Add color pipeline for SDR planes
> > drm/i915/color: Add plane color callbacks
> > drm/i915/color: Load plane color luts from atomic flip
> > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> > drm/i915/xelpd: Add register definitions for Plane Degamma
> > drm/i915/color: Add color functions for ADL
> > drm/i915/color: Program Plane Pre-CSC Registers
> > drm/i915/xelpd: Add register definitions for Plane Post CSC
> > drm/i915/xelpd: Program Plane Post CSC Registers
> > drm/i915/color: Enable Plane CSC
> >
> > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> > drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> > drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> > drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> > drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> > drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> > .../drm/i915/display/skl_universal_plane.c | 21 +-
> > drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> > include/drm/drm_plane.h | 82 +++
> > include/uapi/drm/drm_mode.h | 196 +++++
> > include/uapi/drm/i915_drm.h | 25 +
> > 11 files changed, 1899 insertions(+), 3 deletions(-)
> > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> >
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-08-29 19:40 ` [Intel-gfx] " Harry Wentland
@ 2023-08-30 8:59 ` Shankar, Uma
-1 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-08-30 8:59 UTC (permalink / raw)
To: Harry Wentland, intel-gfx, dri-devel
Cc: Borah, Chaitanya Kumar, wayland-devel
> -----Original Message-----
> From: Harry Wentland <harry.wentland@amd.com>
> Sent: Wednesday, August 30, 2023 1:10 AM
> To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> devel@lists.freedesktop.org
> Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color
> Pipeline
>
>
>
> On 2023-08-29 12:03, Uma Shankar wrote:
> > Add the documentation for the new proposed Plane Color Pipeline.
> >
> > Co-developed-by: Chaitanya Kumar Borah
> > <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > ---
> > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > 1 file changed, 394 insertions(+)
> > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> >
> > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > new file mode 100644
> > index 000000000000..60ce515b6ea7
> > --- /dev/null
> > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > @@ -0,0 +1,394 @@
> > +=======================================
> > + Plane Color Pipeline: A UAPI proposal
> > +=======================================
> > +
> > +To build the proposal on, lets take the premise of a color pipeline
> > +as shown below.
> > +
> > + +-------------------------------------------+
> > + | RAM |
> > + | +------+ +---------+ +---------+ |
> > + | | FB 1 | | FB 2 | | FB N | |
> > + | +------+ +---------+ +---------+ |
> > + +-------------------------------------------+
> > + | Plane Color Hardware Block |
> > + +--------------------------------------------+
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> > + | +---+-----+ +---+-------+ +---+------+ |
> > + | | | | |
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | |Plane A | | Plane B | | Plane N | |
> > + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + | +---v-----+ +----v------+ +----v-----+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | |Post-CSC | | Post-CSC | | Post-CSC | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + +--------------------------------------------+
> > ++------v--------------v---------------v-------|
> > +|| ||
> > +|| Pipe Blender ||
> > ++--------------------+------------------------+
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Pre-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | Pipe Color |
> > +| +-----------v----------+ Hardware |
> > +| | Pipe CSC/CTM | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Post-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > ++---------------------------------------------+
> > + |
> > + v
> > + Pipe Output
> > +
> > +Each plane consists of the following color blocks
> > + * Pre-CSC : This block can used to linearize the input frame buffer data.
> > + The linear data then can be further acted on by the following
> > + color hardware blocks in the display hardware pipeline
> > +
> > + * CSC/CTM: Used to program color transformation matrix, this block is used
> > + to perform color space conversions like BT2020 to BT709 or BT601
> > + etc. This block acts on the linearized data coming from the
> > + Pre-CSC HW block.
> > +
> > + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> > + match the sink. Another use case of it could be to perform Tone
> > + mapping for HDR use-cases.
> > +
> > +Data from multiple planes will then be fed to pipe/crtc where it will get blended.
> > +There is a similar set of HW blocks available at pipe/crtc level
> > +which acts on this blended data.
> > +
> > +Below is a sample usecase fo video playback with sub-titles and
> > +playback controls
> > +
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> > +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> > +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> > +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
> │
> > +(subtitles) │
> > + │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
> > +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
> │ │
> > +(Playback controls UI) │ │
> > + │ │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> │ │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
> > +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
> │ │
> > +(video frame) │ │ │
> > +
> > +│ │ │
> >
> +┌──────────────────────────────────────────────────────────────────
> ──
> > +────┴─┴─┘
> > +│
> > +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> > +│ │ CRTC │ │ CRTC │ │ CRTC │
> > +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO
> Port
> > + │ │ │ │ │ │
> > + └─────────────┘ └─────────────┘ └───────────────┘
> > +
> > +This RFC is intended to propose an uAPI for the pre-blending color
> > +pipeline (however, can be also extended to post blending pipeline).
> > +
> > +Below are the design considerations while designing the uAPI.
> > +
> > +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only
> > +one can be used) 2. Position of the HW block can be programmable 3.
> > +LUTs can be one dimentional or three dimentional 4. Number of LUT
> > +entries can vary across platforms 5. Precision of LUT entries can
> > +vary across platforms 6. Distribution of LUT entries may vary. e.g
> > +Mutli-segmented, Logarithmic,
> > + Piece-Wise Linear(PWL) etc
> > +7. There can be parameterized/non-parameterized fixed function HW blocks.
> > + e.g. Just a hardware bit, to convert from one color space to another.
> > +
> > +Plane Color Pipeline: Design Proposal
> > +=====================================
> > +Each Color Hardware block will be represented by the structure drm_color_op.
> > +
> > +struct drm_color_op {
> > + enum color_op_block name;
> > + enum color_op_type type;
> > + u32 blob_id;
> > + u32 private_flags;
> > +};
> > +
> > +The members of which will constitute:
> > +1. name: A standardised enum for the color hardware block 2. type:
> > +The type of mathematical operation done by the hardware block.
> > + e.g. 1D Curve, 3D Curve, Matrix etc.
> > +3. blob id: Id pointing to a blob containing information about the hardware
> > + block advertising the respective capabilities to the userspace.
> > + It can be an optional field depending on the members "name" and "type".
> > +4. private_flags: This can be used to provide vendor specific hints
> > + to user space
> > +
> > +
> > + For example to represent LUTs, we introduce the drm_color_lut_range
> > + structure. It can represent LUTs with varied number of entries and
> > + distributions (Multi segmented, Logarithmic etc).
> > +
> > + struct drm_color_lut_range {
> > + /* DRM_MODE_LUT_* */
> > + __u32 flags;
> > + /* number of points on the curve */
> > + __u16 count;
> > + /* input/output bits per component */
> > + __u8 input_bpc, output_bpc;
> > + /* input start/end values */
> > + __s32 start, end;
> > + /* output min/max values */
> > + __s32 min, max;
> > + };
> > +
> > +Note: More details on exact usage and implementation of this structure can be
> > + found in the RFC. This structure is taken as is from the series.
> > + https://patchwork.freedesktop.org/series/90825/
> > + However, we can add more members to it to encompass all use-cases.
> > + For example. we can add a precision field to advertise the
> > + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> > + field for vendor specific use cases.
> > +
> > + At the same time, we don't need to pass any additional information for the
> > + CSC block as userspace and driver already agrees struct drm_color_ctm as
> > + a uAPI to pass on data.
> > +
> > +These color operations will form the building blocks of a color
> > +pipeline which best represents the underlying Hardware. Color
> > +operations can be re-arranged, substracted or added to create
> > +distinct color pipelines to accurately describe the Hardware blocks present in the
> display engine.
> > +
>
> What do you mean when you're talking of re-arranging, or subtracting color
> operations?
I will re-phrase this to avoid confusion. The pipeline can be defined by driver by
Putting together the hardware blocks in some order supported by respective hardware.
If any new generation changes the behaviour, a new pipeline can be defined and exposed
for that particular platform. So pipeline definition is fully flexible but done by driver.
Userspace will just be able to use that and cannot alter the same once exposed by driver.
> > +In this proposal, a color pipeline is represented as an array of
> > +struct drm_color_op.
> > +
> > +struct drm_color_op color_pipeline_1[]
> > +
> > +For example to represent the pre-blending color pipeline as described
> > +above
> > +
> > +We can create a color pipeline like this.
> > +
> > +struct drm_color_op color_pipeline_1[] = {
> > + {
> > + .name = DRM_CB_PRE_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0; /* actual value to be populated during plane
> > + initialization*/
> > + },
> > + {
> > + .name = DRM_CB_CSC,
> > + .type = MATRIX,
> > + .blob_id = 0;
> > + },
> > + {
> > + .name = DRM_CB_POST_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0;
> > + },
> > +};
> > +
> > +Then, for individual color operation, we add blobs to advertise the
> > +capability of the particular Hardware block. In case of the example
> > +pipeline, we add blobs of type drm_color_lut_range for the "pre-csc" and "post-
> csc".
> > +For the "csc" block we pass no blob id to user space as it is known
> > +to both user space and driver that drm_color_ctm structure is to be
> > +used for such operation.
> > +
> > +To represent, this in a diagram.
> > +
> > + struct drm_color_op color_pipeline_1[]
> > + +---------------------------+
> > + | | drm_color_op
> > + | +---------------------+--+-----------+---------------------+
> > + | | | | | |
> > + | | | | | +-----------------+ |
> > + | | | | | | name | |
> > + | | | | | +-----------------+ |
> > + | | | | | | type | |
> > + | | color_op_1 | | | +-----------------+ |
> > + | | | | | | blob id | +--------+
> > + | | | | | +-----------------+ | |
> > + | | | | | | private | | |
> > + | | | | | | flags | | |
> > + | | | | | +-----------------+ | |
> > + | | | | | | |
> > + | +---------------------+--+-----------+---------------------+ |
> > + | | |
> > + | | |
> > + | +---------------------+ | |
> > + | | | | drm_color_lut_range |
> > + | | color_op_2 | | +-------------------------+ |
> > + | | | | | | |
> > + | | | | | +---------------------+ | |
> > + | +---------------------+ | | | segment 1 { | |<---+
> > + | | | | ... | |
> > + | +---------------------+ | | | .input_bpc = 16, | |
> > + | | | | | | .output_bpc = 16, | |
> > + | | color_op_3 | | | | ... | |
> > + | | | | | | } | |
> > + | | | | | +---------------------+ |
> > + | +---------------------+ | | |
> > + | . | | +---------------------+ |
> > + | . | | | segment 2 { | |
> > + | . | | | ... | |
> > + +---------------------------+ | | } | |
> > + | | | |
> > + | | | |
> > + | | | |
> > + | +---------------------+ |
> > + | . |
> > + | . |
> > + | . |
> > +
> > + +-------------------------+
> > +
> > +
> > +
> > +This color pipeline is then packaged within a blob for the user space
> > +to retrieve it. Details can be found in the next section
> > +
>
> Not sure I like blobs that contain other blob ids.
It provides flexibility and helps with just one interface to userspace. Its easy to handle and
manage once we get the hang of it 😊.
We can clearly define the steps of parsing and data structures to be used while interpreting
and parsing the blobs.
> > +Exposing a color pipeline to user space
> > +=======================================
> > +
> > +To advertise the available color pipelines, an immutable ENUM
> > +property "GET_COLOR_PIPELINE" is introduced.
> > +This enum property has blob id's as values. With each blob id
> > +representing a distinct color pipeline based on underlying HW
> > +capabilities and their respective combinations.
> > +
> > +The following output of drm_info [1], shows how color pipelines are
> > +visible to userspace.
> > +
> > +├───Plane 0
> > + │ ├───Object ID: 31
> > + │ ├───CRTCs: {0}
> > + │ ├───Legacy info
> > + ...
> > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> > + color pipeline 1, color pipeline 2}=
> no color pipeline
> > +
> > +To understand the capabilities of individual pipelines, first the
> > +userspace need to retrieve the pipeline blob with the blob ids
> > +retrieved by reading the enum property.
> > +
> > +Once the color pipeline is retrieved, user can then parse through
> > +individual drm_color_op blocks to understand the capabilities of each
> > +hardware block.
> > +
> > +Check IGT series to see how user space can parse through color pipelines.
> > +Refer the IGT series here:
> > +https://patchwork.freedesktop.org/series/123018/
> > +
> > +Setting up a color pipeline
> > +===========================
> > +
> > +Once the user space decides on a color pipeline, it can set the
> > +pipeline and the corresponding data for the hardware blocks within
> > +the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > +
> > +To achieve this two structures are introduced
> > +
> > +1. struct drm_color_op_data: It represents data to be passed onto individual
> > + color hardware blocks. It
> contains three members
> > + a) name: to identify the color operation block
> > + b) blob_id: pointing to the blob with data for the
> > + corresponding hardware block
> > +
> > + struct drm_color_op_data {
> > + enum color_op_block name;
> > + u32 blob_id;
> > + };
> > +
> > +2. struct drm_color_pipeline: This structure represents the aggregate
> > + pipeline to be set. it contains the following members
> > + a) num: pipeline number to be selected
> > + b) size: size of the data to be passed onto
> the driver
> > + c) data: array of struct
> drm_color_op_data with data for
> > + the hardware block/s that userspace wants to
> > + set values for.
> > +
> > + struct drm_color_pipeline {
> > + int num;
> > + int size;
> > + struct drm_color_op_data *data;
> > + };
> > +
> > + User can either:
> > + 1. send data for all the color operations in a pipeline as shown in [2].
> > + The color operation data need not be in order that the pipeline advertises
> > + however, it should not contain data for any
> > + color operation that is not present in the pipeline.
> > +
> > + Note: This check for pipeline is not yet implemented but if the
> > + wider proposal is accepted we have few solutions in mind.
> > +
> > + 2. send data for a subset of color operations as shown in [3].For the
> > + color operation that userspace does not send data will retain their
> > + older state.
> > +
> > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > + property to NULL as shown in both [2] and [3]
> > +
> > + 4. change the color pipeline as demonstrated in [3].
> > + On the new pipeline, the user is expected to setup all the color hardware
> block
> > + Once the user requests a pipeline change, the driver will provide it a clean
> slate
> > + which means that all the data previously set by the user will be discarded
> even if
> > + there are common hardware blocks between the two(previous and current)
> pipelines.
> > +
> > +IGT implementation can be found here [4]
> > +
> > +Representing Fixed function hardware
> > +====================================
> > +
> > +To provide support for fixed function hardware, the driver could
> > +expose vendor specific struct drm_color_op with parameters that both
> > +the userspace and driver agrees on. To demonstrate, let's consider a
> > +hyphothetical fixed function hardware block that converts BT601 to BT2020.
> > +The driver can choose to advertise the block as such.
> > +
> > +struct drm_color_op color_pipeline_X[] = {
> > + {
> > + .name = DRM_CB_PRIVATE,
> > + .type = FIXED_FUNCTION,
> > + .blob_id = 45;
> > + },
> > +}
> > +
> > +Where the blob represents some vendor specific enums, strings or any
> > +other appropriate data types which both the user-space and drivers are aligned
> on.
> > +
> > +blob:45 {
> > + VENDORXXX_BT602_TO_BT2020,
> > +};
> > +
> > +For enabling or passing parameters to such blocks, the user can send
> > +data to the driver wrapped within a blob like any other color operation block.
> > +
> > + struct drm_color_op_data {
> > + .name = DRM_CB_PRIVATE;
> > + .blob_id = 46;
> > + } ;
> > +
> > +where blob with id 46 contains data to enable the fixed function hardware(s).
> > +
>
> One major thing missing from the RFC is an explanation on why we want to go with a
> prescriptive model, rather than a descriptive model. This question came up in Dave's
> response to Simon's RFC:
> https://lore.kernel.org/dri-
> devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.co
> m/
>
> This is a rough attempt at such an explanation:
> https://gitlab.freedesktop.org/hwentland/linux/-
> /merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d
> 8d
Sure Harry, totally agree to this and will include in documentation to highlight the rationale
of going with prescriptive model.
Regards,
Uma Shankar
> Harry
>
> > +References
> > +==========
> > +
> > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > +[2]
> > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> > +[3]
> > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> > +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-08-30 8:59 ` Shankar, Uma
0 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-08-30 8:59 UTC (permalink / raw)
To: Harry Wentland, intel-gfx, dri-devel; +Cc: wayland-devel
> -----Original Message-----
> From: Harry Wentland <harry.wentland@amd.com>
> Sent: Wednesday, August 30, 2023 1:10 AM
> To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> devel@lists.freedesktop.org
> Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color
> Pipeline
>
>
>
> On 2023-08-29 12:03, Uma Shankar wrote:
> > Add the documentation for the new proposed Plane Color Pipeline.
> >
> > Co-developed-by: Chaitanya Kumar Borah
> > <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > ---
> > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > 1 file changed, 394 insertions(+)
> > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> >
> > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > new file mode 100644
> > index 000000000000..60ce515b6ea7
> > --- /dev/null
> > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > @@ -0,0 +1,394 @@
> > +=======================================
> > + Plane Color Pipeline: A UAPI proposal
> > +=======================================
> > +
> > +To build the proposal on, lets take the premise of a color pipeline
> > +as shown below.
> > +
> > + +-------------------------------------------+
> > + | RAM |
> > + | +------+ +---------+ +---------+ |
> > + | | FB 1 | | FB 2 | | FB N | |
> > + | +------+ +---------+ +---------+ |
> > + +-------------------------------------------+
> > + | Plane Color Hardware Block |
> > + +--------------------------------------------+
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> > + | +---+-----+ +---+-------+ +---+------+ |
> > + | | | | |
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | |Plane A | | Plane B | | Plane N | |
> > + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + | +---v-----+ +----v------+ +----v-----+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | |Post-CSC | | Post-CSC | | Post-CSC | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + +--------------------------------------------+
> > ++------v--------------v---------------v-------|
> > +|| ||
> > +|| Pipe Blender ||
> > ++--------------------+------------------------+
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Pre-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | Pipe Color |
> > +| +-----------v----------+ Hardware |
> > +| | Pipe CSC/CTM | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Post-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > ++---------------------------------------------+
> > + |
> > + v
> > + Pipe Output
> > +
> > +Each plane consists of the following color blocks
> > + * Pre-CSC : This block can used to linearize the input frame buffer data.
> > + The linear data then can be further acted on by the following
> > + color hardware blocks in the display hardware pipeline
> > +
> > + * CSC/CTM: Used to program color transformation matrix, this block is used
> > + to perform color space conversions like BT2020 to BT709 or BT601
> > + etc. This block acts on the linearized data coming from the
> > + Pre-CSC HW block.
> > +
> > + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> > + match the sink. Another use case of it could be to perform Tone
> > + mapping for HDR use-cases.
> > +
> > +Data from multiple planes will then be fed to pipe/crtc where it will get blended.
> > +There is a similar set of HW blocks available at pipe/crtc level
> > +which acts on this blended data.
> > +
> > +Below is a sample usecase fo video playback with sub-titles and
> > +playback controls
> > +
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> > +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> > +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> > +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
> │
> > +(subtitles) │
> > + │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
> > +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
> │ │
> > +(Playback controls UI) │ │
> > + │ │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> │ │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
> > +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
> │ │
> > +(video frame) │ │ │
> > +
> > +│ │ │
> >
> +┌──────────────────────────────────────────────────────────────────
> ──
> > +────┴─┴─┘
> > +│
> > +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> > +│ │ CRTC │ │ CRTC │ │ CRTC │
> > +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO
> Port
> > + │ │ │ │ │ │
> > + └─────────────┘ └─────────────┘ └───────────────┘
> > +
> > +This RFC is intended to propose an uAPI for the pre-blending color
> > +pipeline (however, can be also extended to post blending pipeline).
> > +
> > +Below are the design considerations while designing the uAPI.
> > +
> > +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only
> > +one can be used) 2. Position of the HW block can be programmable 3.
> > +LUTs can be one dimentional or three dimentional 4. Number of LUT
> > +entries can vary across platforms 5. Precision of LUT entries can
> > +vary across platforms 6. Distribution of LUT entries may vary. e.g
> > +Mutli-segmented, Logarithmic,
> > + Piece-Wise Linear(PWL) etc
> > +7. There can be parameterized/non-parameterized fixed function HW blocks.
> > + e.g. Just a hardware bit, to convert from one color space to another.
> > +
> > +Plane Color Pipeline: Design Proposal
> > +=====================================
> > +Each Color Hardware block will be represented by the structure drm_color_op.
> > +
> > +struct drm_color_op {
> > + enum color_op_block name;
> > + enum color_op_type type;
> > + u32 blob_id;
> > + u32 private_flags;
> > +};
> > +
> > +The members of which will constitute:
> > +1. name: A standardised enum for the color hardware block 2. type:
> > +The type of mathematical operation done by the hardware block.
> > + e.g. 1D Curve, 3D Curve, Matrix etc.
> > +3. blob id: Id pointing to a blob containing information about the hardware
> > + block advertising the respective capabilities to the userspace.
> > + It can be an optional field depending on the members "name" and "type".
> > +4. private_flags: This can be used to provide vendor specific hints
> > + to user space
> > +
> > +
> > + For example to represent LUTs, we introduce the drm_color_lut_range
> > + structure. It can represent LUTs with varied number of entries and
> > + distributions (Multi segmented, Logarithmic etc).
> > +
> > + struct drm_color_lut_range {
> > + /* DRM_MODE_LUT_* */
> > + __u32 flags;
> > + /* number of points on the curve */
> > + __u16 count;
> > + /* input/output bits per component */
> > + __u8 input_bpc, output_bpc;
> > + /* input start/end values */
> > + __s32 start, end;
> > + /* output min/max values */
> > + __s32 min, max;
> > + };
> > +
> > +Note: More details on exact usage and implementation of this structure can be
> > + found in the RFC. This structure is taken as is from the series.
> > + https://patchwork.freedesktop.org/series/90825/
> > + However, we can add more members to it to encompass all use-cases.
> > + For example. we can add a precision field to advertise the
> > + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> > + field for vendor specific use cases.
> > +
> > + At the same time, we don't need to pass any additional information for the
> > + CSC block as userspace and driver already agrees struct drm_color_ctm as
> > + a uAPI to pass on data.
> > +
> > +These color operations will form the building blocks of a color
> > +pipeline which best represents the underlying Hardware. Color
> > +operations can be re-arranged, substracted or added to create
> > +distinct color pipelines to accurately describe the Hardware blocks present in the
> display engine.
> > +
>
> What do you mean when you're talking of re-arranging, or subtracting color
> operations?
I will re-phrase this to avoid confusion. The pipeline can be defined by driver by
Putting together the hardware blocks in some order supported by respective hardware.
If any new generation changes the behaviour, a new pipeline can be defined and exposed
for that particular platform. So pipeline definition is fully flexible but done by driver.
Userspace will just be able to use that and cannot alter the same once exposed by driver.
> > +In this proposal, a color pipeline is represented as an array of
> > +struct drm_color_op.
> > +
> > +struct drm_color_op color_pipeline_1[]
> > +
> > +For example to represent the pre-blending color pipeline as described
> > +above
> > +
> > +We can create a color pipeline like this.
> > +
> > +struct drm_color_op color_pipeline_1[] = {
> > + {
> > + .name = DRM_CB_PRE_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0; /* actual value to be populated during plane
> > + initialization*/
> > + },
> > + {
> > + .name = DRM_CB_CSC,
> > + .type = MATRIX,
> > + .blob_id = 0;
> > + },
> > + {
> > + .name = DRM_CB_POST_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0;
> > + },
> > +};
> > +
> > +Then, for individual color operation, we add blobs to advertise the
> > +capability of the particular Hardware block. In case of the example
> > +pipeline, we add blobs of type drm_color_lut_range for the "pre-csc" and "post-
> csc".
> > +For the "csc" block we pass no blob id to user space as it is known
> > +to both user space and driver that drm_color_ctm structure is to be
> > +used for such operation.
> > +
> > +To represent, this in a diagram.
> > +
> > + struct drm_color_op color_pipeline_1[]
> > + +---------------------------+
> > + | | drm_color_op
> > + | +---------------------+--+-----------+---------------------+
> > + | | | | | |
> > + | | | | | +-----------------+ |
> > + | | | | | | name | |
> > + | | | | | +-----------------+ |
> > + | | | | | | type | |
> > + | | color_op_1 | | | +-----------------+ |
> > + | | | | | | blob id | +--------+
> > + | | | | | +-----------------+ | |
> > + | | | | | | private | | |
> > + | | | | | | flags | | |
> > + | | | | | +-----------------+ | |
> > + | | | | | | |
> > + | +---------------------+--+-----------+---------------------+ |
> > + | | |
> > + | | |
> > + | +---------------------+ | |
> > + | | | | drm_color_lut_range |
> > + | | color_op_2 | | +-------------------------+ |
> > + | | | | | | |
> > + | | | | | +---------------------+ | |
> > + | +---------------------+ | | | segment 1 { | |<---+
> > + | | | | ... | |
> > + | +---------------------+ | | | .input_bpc = 16, | |
> > + | | | | | | .output_bpc = 16, | |
> > + | | color_op_3 | | | | ... | |
> > + | | | | | | } | |
> > + | | | | | +---------------------+ |
> > + | +---------------------+ | | |
> > + | . | | +---------------------+ |
> > + | . | | | segment 2 { | |
> > + | . | | | ... | |
> > + +---------------------------+ | | } | |
> > + | | | |
> > + | | | |
> > + | | | |
> > + | +---------------------+ |
> > + | . |
> > + | . |
> > + | . |
> > +
> > + +-------------------------+
> > +
> > +
> > +
> > +This color pipeline is then packaged within a blob for the user space
> > +to retrieve it. Details can be found in the next section
> > +
>
> Not sure I like blobs that contain other blob ids.
It provides flexibility and helps with just one interface to userspace. Its easy to handle and
manage once we get the hang of it 😊.
We can clearly define the steps of parsing and data structures to be used while interpreting
and parsing the blobs.
> > +Exposing a color pipeline to user space
> > +=======================================
> > +
> > +To advertise the available color pipelines, an immutable ENUM
> > +property "GET_COLOR_PIPELINE" is introduced.
> > +This enum property has blob id's as values. With each blob id
> > +representing a distinct color pipeline based on underlying HW
> > +capabilities and their respective combinations.
> > +
> > +The following output of drm_info [1], shows how color pipelines are
> > +visible to userspace.
> > +
> > +├───Plane 0
> > + │ ├───Object ID: 31
> > + │ ├───CRTCs: {0}
> > + │ ├───Legacy info
> > + ...
> > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> > + color pipeline 1, color pipeline 2}=
> no color pipeline
> > +
> > +To understand the capabilities of individual pipelines, first the
> > +userspace need to retrieve the pipeline blob with the blob ids
> > +retrieved by reading the enum property.
> > +
> > +Once the color pipeline is retrieved, user can then parse through
> > +individual drm_color_op blocks to understand the capabilities of each
> > +hardware block.
> > +
> > +Check IGT series to see how user space can parse through color pipelines.
> > +Refer the IGT series here:
> > +https://patchwork.freedesktop.org/series/123018/
> > +
> > +Setting up a color pipeline
> > +===========================
> > +
> > +Once the user space decides on a color pipeline, it can set the
> > +pipeline and the corresponding data for the hardware blocks within
> > +the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > +
> > +To achieve this two structures are introduced
> > +
> > +1. struct drm_color_op_data: It represents data to be passed onto individual
> > + color hardware blocks. It
> contains three members
> > + a) name: to identify the color operation block
> > + b) blob_id: pointing to the blob with data for the
> > + corresponding hardware block
> > +
> > + struct drm_color_op_data {
> > + enum color_op_block name;
> > + u32 blob_id;
> > + };
> > +
> > +2. struct drm_color_pipeline: This structure represents the aggregate
> > + pipeline to be set. it contains the following members
> > + a) num: pipeline number to be selected
> > + b) size: size of the data to be passed onto
> the driver
> > + c) data: array of struct
> drm_color_op_data with data for
> > + the hardware block/s that userspace wants to
> > + set values for.
> > +
> > + struct drm_color_pipeline {
> > + int num;
> > + int size;
> > + struct drm_color_op_data *data;
> > + };
> > +
> > + User can either:
> > + 1. send data for all the color operations in a pipeline as shown in [2].
> > + The color operation data need not be in order that the pipeline advertises
> > + however, it should not contain data for any
> > + color operation that is not present in the pipeline.
> > +
> > + Note: This check for pipeline is not yet implemented but if the
> > + wider proposal is accepted we have few solutions in mind.
> > +
> > + 2. send data for a subset of color operations as shown in [3].For the
> > + color operation that userspace does not send data will retain their
> > + older state.
> > +
> > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > + property to NULL as shown in both [2] and [3]
> > +
> > + 4. change the color pipeline as demonstrated in [3].
> > + On the new pipeline, the user is expected to setup all the color hardware
> block
> > + Once the user requests a pipeline change, the driver will provide it a clean
> slate
> > + which means that all the data previously set by the user will be discarded
> even if
> > + there are common hardware blocks between the two(previous and current)
> pipelines.
> > +
> > +IGT implementation can be found here [4]
> > +
> > +Representing Fixed function hardware
> > +====================================
> > +
> > +To provide support for fixed function hardware, the driver could
> > +expose vendor specific struct drm_color_op with parameters that both
> > +the userspace and driver agrees on. To demonstrate, let's consider a
> > +hyphothetical fixed function hardware block that converts BT601 to BT2020.
> > +The driver can choose to advertise the block as such.
> > +
> > +struct drm_color_op color_pipeline_X[] = {
> > + {
> > + .name = DRM_CB_PRIVATE,
> > + .type = FIXED_FUNCTION,
> > + .blob_id = 45;
> > + },
> > +}
> > +
> > +Where the blob represents some vendor specific enums, strings or any
> > +other appropriate data types which both the user-space and drivers are aligned
> on.
> > +
> > +blob:45 {
> > + VENDORXXX_BT602_TO_BT2020,
> > +};
> > +
> > +For enabling or passing parameters to such blocks, the user can send
> > +data to the driver wrapped within a blob like any other color operation block.
> > +
> > + struct drm_color_op_data {
> > + .name = DRM_CB_PRIVATE;
> > + .blob_id = 46;
> > + } ;
> > +
> > +where blob with id 46 contains data to enable the fixed function hardware(s).
> > +
>
> One major thing missing from the RFC is an explanation on why we want to go with a
> prescriptive model, rather than a descriptive model. This question came up in Dave's
> response to Simon's RFC:
> https://lore.kernel.org/dri-
> devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.co
> m/
>
> This is a rough attempt at such an explanation:
> https://gitlab.freedesktop.org/hwentland/linux/-
> /merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d
> 8d
Sure Harry, totally agree to this and will include in documentation to highlight the rationale
of going with prescriptive model.
Regards,
Uma Shankar
> Harry
>
> > +References
> > +==========
> > +
> > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > +[2]
> > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> > +[3]
> > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> > +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-08-30 8:59 ` [Intel-gfx] " Shankar, Uma
@ 2023-08-30 12:28 ` Pekka Paalanen
-1 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-08-30 12:28 UTC (permalink / raw)
To: Shankar, Uma; +Cc: intel-gfx, Borah, Chaitanya Kumar, dri-devel, wayland-devel
[-- Attachment #1: Type: text/plain, Size: 13770 bytes --]
On Wed, 30 Aug 2023 08:59:36 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: Harry Wentland <harry.wentland@amd.com>
> > Sent: Wednesday, August 30, 2023 1:10 AM
> > To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> > devel@lists.freedesktop.org
> > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>; wayland-
> > devel@lists.freedesktop.org
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color
> > Pipeline
> >
> >
> >
> > On 2023-08-29 12:03, Uma Shankar wrote:
> > > Add the documentation for the new proposed Plane Color Pipeline.
> > >
> > > Co-developed-by: Chaitanya Kumar Borah
> > > <chaitanya.kumar.borah@intel.com>
> > > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > ---
> > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > 1 file changed, 394 insertions(+)
> > > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
> > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > new file mode 100644
> > > index 000000000000..60ce515b6ea7
> > > --- /dev/null
> > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
...
Hi Uma!
> > > +This color pipeline is then packaged within a blob for the user space
> > > +to retrieve it. Details can be found in the next section
> > > +
> >
> > Not sure I like blobs that contain other blob ids.
>
> It provides flexibility and helps with just one interface to userspace. Its easy to handle and
> manage once we get the hang of it 😊.
>
> We can clearly define the steps of parsing and data structures to be used while interpreting
> and parsing the blobs.
Don't forget extendability. Possibly every single struct will need some
kind of versioning, and then it's not simple to parse anymore. Add to
that new/old kernel vs. old/new userspace, and it seems a bit
nightmarish to design.
Also since it's records inside a single blob, it's like a new file
format: every record needs a standard header that allows skipping it
appropriately if userspace does not understand it, or you need a
standard index telling where everything is. Making all records the same
size would waste space, and extendability requires variable size.
I also would not assume that we can declare a standard set of blocks
and that nothing else will be needed. The existing hardware is too
diverse for that from what I have understood. I assume that some
hardware have blocks unique to them, and they want to at least expose
that functionality through a UAPI that allows at least generic
enumeration of functionality, even if it needs specialized userspace
code to actually make use of.
If we go with
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ u32 blob_id;
+ u32 private_flags;
+};
as in your proposal, I believe it can work (sorry, looking further
down, I have assumed too much of 'type'), but the enumerations will
become long, and the details blob_id is still specific to 'type'. This
is unavoidable, but we can still choose the form between blobs and
properties, integers and strings.
I have a feeling that introspection will be valuable here, to help
people understand what their hardware could do if they had the code to
use it. 'name' and 'type' being integers require a translation table to
strings before they are readable, and it would be best if the kernel
itself provided that translation.
I don't understand how 'private_flags' could be useful. There must not
be any "hidden" features. Everything a block can be programmed to do
via this UAPI must be clearly documented, there cannot be anything
private. If two hardware versions of a block differ in a meaningful or
significant way, they need to be exposed as different types of blocks.
OTOH, if one goes with a (new) DRM object with string named properties
model, all that struct versioning and file format hassle has mostly a
clear and well-understood solution. We only need to define the rules of
how userspace needs to deal with properties or values it does not
understand, so that the kernel can keep adding more.
Therefore, I'm not yet convinced with the "all blobs" design.
> > > +Exposing a color pipeline to user space
> > > +=======================================
> > > +
> > > +To advertise the available color pipelines, an immutable ENUM
> > > +property "GET_COLOR_PIPELINE" is introduced.
> > > +This enum property has blob id's as values. With each blob id
> > > +representing a distinct color pipeline based on underlying HW
> > > +capabilities and their respective combinations.
> > > +
> > > +The following output of drm_info [1], shows how color pipelines are
> > > +visible to userspace.
> > > +
> > > +├───Plane 0
> > > + │ ├───Object ID: 31
> > > + │ ├───CRTCs: {0}
> > > + │ ├───Legacy info
> > > + ...
> > > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> > > + color pipeline 1, color pipeline 2}=
> > no color pipeline
> > > +
> > > +To understand the capabilities of individual pipelines, first the
> > > +userspace need to retrieve the pipeline blob with the blob ids
> > > +retrieved by reading the enum property.
> > > +
> > > +Once the color pipeline is retrieved, user can then parse through
> > > +individual drm_color_op blocks to understand the capabilities of each
> > > +hardware block.
> > > +
> > > +Check IGT series to see how user space can parse through color pipelines.
> > > +Refer the IGT series here:
> > > +https://patchwork.freedesktop.org/series/123018/
> > > +
> > > +Setting up a color pipeline
> > > +===========================
> > > +
> > > +Once the user space decides on a color pipeline, it can set the
> > > +pipeline and the corresponding data for the hardware blocks within
> > > +the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > +
> > > +To achieve this two structures are introduced
> > > +
> > > +1. struct drm_color_op_data: It represents data to be passed onto individual
> > > + color hardware blocks. It
> > contains three members
> > > + a) name: to identify the color operation block
> > > + b) blob_id: pointing to the blob with data for the
> > > + corresponding hardware block
> > > +
> > > + struct drm_color_op_data {
> > > + enum color_op_block name;
Why is this a global fixed enum rather than a pipeline specific ordinal
or a unique-per-device ID?
There is no reason to believe that a 'name' always matches a hardware
block 1:1. When drivers accumulate multiple different alternative
pipelines due to backwards-compatibility reasons, the same 'name' could
be implemented by different hardware blocks, or the same hardware block
could implement different 'name's from different pipelines.
The names have also a problem. If you name something "pre-csc", then
how do you name the thing that the next hardware version adds between
"pre-csc" and "csc"?
> > > + u32 blob_id;
> > > + };
> > > +
> > > +2. struct drm_color_pipeline: This structure represents the aggregate
> > > + pipeline to be set. it contains the following members
> > > + a) num: pipeline number to be selected
> > > + b) size: size of the data to be passed onto
> > the driver
> > > + c) data: array of struct
> > drm_color_op_data with data for
> > > + the hardware block/s that userspace wants to
> > > + set values for.
> > > +
> > > + struct drm_color_pipeline {
> > > + int num;
> > > + int size;
> > > + struct drm_color_op_data *data;
> > > + };
> > > +
> > > + User can either:
> > > + 1. send data for all the color operations in a pipeline as shown in [2].
> > > + The color operation data need not be in order that the pipeline advertises
> > > + however, it should not contain data for any
> > > + color operation that is not present in the pipeline.
> > > +
> > > + Note: This check for pipeline is not yet implemented but if the
> > > + wider proposal is accepted we have few solutions in mind.
> > > +
> > > + 2. send data for a subset of color operations as shown in [3].For the
> > > + color operation that userspace does not send data will retain their
> > > + older state.
Retaining existing state, especially for operations that userspace does
not understand, can lead to incorrect and unexpected results. That's
why I say that userspace must understand all operations in a pipeline,
and all parameters of all used operations before it can actually use
that pipeline.
Otherwise we have the same problem as KMS properties have in general
today: when new things are added that userspace does not understand,
how will the userspace be able to maintain its old behaviour?
That question has two answers today:
- userspace learns to program everything, and accidentally
(mal)functions until then
- you do not switch between KMS clients that might leave incorrect
state in not-understood properties
Neither is a good answer, and the problem does not seem to have any
practical traction either.
For color pipelines I hope we can, and believe that we must, do better
to maintain correct behaviour while extending functionality.
> > > +
> > > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > > + property to NULL as shown in both [2] and [3]
> > > +
Is it a reset and disable, or only disable?
How is the reset state defined, if that state becomes active when the
pipeline is next enabled and data not set for the operations?
> > > + 4. change the color pipeline as demonstrated in [3].
> > > + On the new pipeline, the user is expected to setup all the color hardware
> > block
> > > + Once the user requests a pipeline change, the driver will provide it a clean
> > slate
> > > + which means that all the data previously set by the user will be discarded
> > even if
> > > + there are common hardware blocks between the two(previous and current)
> > pipelines.
> > > +
Yes, alternative pipelines need to be completely independent.
> > > +IGT implementation can be found here [4]
> > > +
> > > +Representing Fixed function hardware
> > > +====================================
> > > +
> > > +To provide support for fixed function hardware, the driver could
> > > +expose vendor specific struct drm_color_op with parameters that both
> > > +the userspace and driver agrees on. To demonstrate, let's consider a
> > > +hyphothetical fixed function hardware block that converts BT601 to BT2020.
> > > +The driver can choose to advertise the block as such.
> > > +
> > > +struct drm_color_op color_pipeline_X[] = {
> > > + {
> > > + .name = DRM_CB_PRIVATE,
What if the hardware has 5 different custom blocks like this, multiple
in the same pipeline. How do you 'name' them?
> > > + .type = FIXED_FUNCTION,
I have been assuming that 'type' uniquely defines both the operation and
the contents of the parameter blob, but this does not look like it.
What defines the operation and the parameters?
> > > + .blob_id = 45;
> > > + },
> > > +}
> > > +
> > > +Where the blob represents some vendor specific enums, strings or any
> > > +other appropriate data types which both the user-space and drivers are aligned
> > on.
We have a word for that "data ... aligned on": UAPI.
> > > +
> > > +blob:45 {
> > > + VENDORXXX_BT602_TO_BT2020,
Repeating the question from above, how will userspace know that the
contents of this blob need to be VENDORXXX_BT602_TO_BT2020 and not
something else?
> > > +};
> > > +
> > > +For enabling or passing parameters to such blocks, the user can send
> > > +data to the driver wrapped within a blob like any other color operation block.
> > > +
> > > + struct drm_color_op_data {
> > > + .name = DRM_CB_PRIVATE;
> > > + .blob_id = 46;
> > > + } ;
> > > +
> > > +where blob with id 46 contains data to enable the fixed function hardware(s).
> > > +
> >
> > One major thing missing from the RFC is an explanation on why we want to go with a
> > prescriptive model, rather than a descriptive model. This question came up in Dave's
> > response to Simon's RFC:
> > https://lore.kernel.org/dri-
> > devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.co
> > m/
> >
> > This is a rough attempt at such an explanation:
> > https://gitlab.freedesktop.org/hwentland/linux/-
> > /merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d
> > 8d
Hey Harry, that's a good piece!
>
> Sure Harry, totally agree to this and will include in documentation to highlight the rationale
> of going with prescriptive model.
Uma, the cover letter had descriptive and prescriptive mixed up.
Thanks,
pq
> > Harry
> >
> > > +References
> > > +==========
> > > +
> > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > +[2]
> > > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> > > +[3]
> > > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> > > +[4] https://patchwork.freedesktop.org/series/123018/
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-08-30 12:28 ` Pekka Paalanen
0 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-08-30 12:28 UTC (permalink / raw)
To: Shankar, Uma; +Cc: intel-gfx, Harry Wentland, dri-devel, wayland-devel
[-- Attachment #1: Type: text/plain, Size: 13770 bytes --]
On Wed, 30 Aug 2023 08:59:36 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: Harry Wentland <harry.wentland@amd.com>
> > Sent: Wednesday, August 30, 2023 1:10 AM
> > To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> > devel@lists.freedesktop.org
> > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>; wayland-
> > devel@lists.freedesktop.org
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color
> > Pipeline
> >
> >
> >
> > On 2023-08-29 12:03, Uma Shankar wrote:
> > > Add the documentation for the new proposed Plane Color Pipeline.
> > >
> > > Co-developed-by: Chaitanya Kumar Borah
> > > <chaitanya.kumar.borah@intel.com>
> > > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > ---
> > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > 1 file changed, 394 insertions(+)
> > > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
> > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > new file mode 100644
> > > index 000000000000..60ce515b6ea7
> > > --- /dev/null
> > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
...
Hi Uma!
> > > +This color pipeline is then packaged within a blob for the user space
> > > +to retrieve it. Details can be found in the next section
> > > +
> >
> > Not sure I like blobs that contain other blob ids.
>
> It provides flexibility and helps with just one interface to userspace. Its easy to handle and
> manage once we get the hang of it 😊.
>
> We can clearly define the steps of parsing and data structures to be used while interpreting
> and parsing the blobs.
Don't forget extendability. Possibly every single struct will need some
kind of versioning, and then it's not simple to parse anymore. Add to
that new/old kernel vs. old/new userspace, and it seems a bit
nightmarish to design.
Also since it's records inside a single blob, it's like a new file
format: every record needs a standard header that allows skipping it
appropriately if userspace does not understand it, or you need a
standard index telling where everything is. Making all records the same
size would waste space, and extendability requires variable size.
I also would not assume that we can declare a standard set of blocks
and that nothing else will be needed. The existing hardware is too
diverse for that from what I have understood. I assume that some
hardware have blocks unique to them, and they want to at least expose
that functionality through a UAPI that allows at least generic
enumeration of functionality, even if it needs specialized userspace
code to actually make use of.
If we go with
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ u32 blob_id;
+ u32 private_flags;
+};
as in your proposal, I believe it can work (sorry, looking further
down, I have assumed too much of 'type'), but the enumerations will
become long, and the details blob_id is still specific to 'type'. This
is unavoidable, but we can still choose the form between blobs and
properties, integers and strings.
I have a feeling that introspection will be valuable here, to help
people understand what their hardware could do if they had the code to
use it. 'name' and 'type' being integers require a translation table to
strings before they are readable, and it would be best if the kernel
itself provided that translation.
I don't understand how 'private_flags' could be useful. There must not
be any "hidden" features. Everything a block can be programmed to do
via this UAPI must be clearly documented, there cannot be anything
private. If two hardware versions of a block differ in a meaningful or
significant way, they need to be exposed as different types of blocks.
OTOH, if one goes with a (new) DRM object with string named properties
model, all that struct versioning and file format hassle has mostly a
clear and well-understood solution. We only need to define the rules of
how userspace needs to deal with properties or values it does not
understand, so that the kernel can keep adding more.
Therefore, I'm not yet convinced with the "all blobs" design.
> > > +Exposing a color pipeline to user space
> > > +=======================================
> > > +
> > > +To advertise the available color pipelines, an immutable ENUM
> > > +property "GET_COLOR_PIPELINE" is introduced.
> > > +This enum property has blob id's as values. With each blob id
> > > +representing a distinct color pipeline based on underlying HW
> > > +capabilities and their respective combinations.
> > > +
> > > +The following output of drm_info [1], shows how color pipelines are
> > > +visible to userspace.
> > > +
> > > +├───Plane 0
> > > + │ ├───Object ID: 31
> > > + │ ├───CRTCs: {0}
> > > + │ ├───Legacy info
> > > + ...
> > > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> > > + color pipeline 1, color pipeline 2}=
> > no color pipeline
> > > +
> > > +To understand the capabilities of individual pipelines, first the
> > > +userspace need to retrieve the pipeline blob with the blob ids
> > > +retrieved by reading the enum property.
> > > +
> > > +Once the color pipeline is retrieved, user can then parse through
> > > +individual drm_color_op blocks to understand the capabilities of each
> > > +hardware block.
> > > +
> > > +Check IGT series to see how user space can parse through color pipelines.
> > > +Refer the IGT series here:
> > > +https://patchwork.freedesktop.org/series/123018/
> > > +
> > > +Setting up a color pipeline
> > > +===========================
> > > +
> > > +Once the user space decides on a color pipeline, it can set the
> > > +pipeline and the corresponding data for the hardware blocks within
> > > +the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > +
> > > +To achieve this two structures are introduced
> > > +
> > > +1. struct drm_color_op_data: It represents data to be passed onto individual
> > > + color hardware blocks. It
> > contains three members
> > > + a) name: to identify the color operation block
> > > + b) blob_id: pointing to the blob with data for the
> > > + corresponding hardware block
> > > +
> > > + struct drm_color_op_data {
> > > + enum color_op_block name;
Why is this a global fixed enum rather than a pipeline specific ordinal
or a unique-per-device ID?
There is no reason to believe that a 'name' always matches a hardware
block 1:1. When drivers accumulate multiple different alternative
pipelines due to backwards-compatibility reasons, the same 'name' could
be implemented by different hardware blocks, or the same hardware block
could implement different 'name's from different pipelines.
The names have also a problem. If you name something "pre-csc", then
how do you name the thing that the next hardware version adds between
"pre-csc" and "csc"?
> > > + u32 blob_id;
> > > + };
> > > +
> > > +2. struct drm_color_pipeline: This structure represents the aggregate
> > > + pipeline to be set. it contains the following members
> > > + a) num: pipeline number to be selected
> > > + b) size: size of the data to be passed onto
> > the driver
> > > + c) data: array of struct
> > drm_color_op_data with data for
> > > + the hardware block/s that userspace wants to
> > > + set values for.
> > > +
> > > + struct drm_color_pipeline {
> > > + int num;
> > > + int size;
> > > + struct drm_color_op_data *data;
> > > + };
> > > +
> > > + User can either:
> > > + 1. send data for all the color operations in a pipeline as shown in [2].
> > > + The color operation data need not be in order that the pipeline advertises
> > > + however, it should not contain data for any
> > > + color operation that is not present in the pipeline.
> > > +
> > > + Note: This check for pipeline is not yet implemented but if the
> > > + wider proposal is accepted we have few solutions in mind.
> > > +
> > > + 2. send data for a subset of color operations as shown in [3].For the
> > > + color operation that userspace does not send data will retain their
> > > + older state.
Retaining existing state, especially for operations that userspace does
not understand, can lead to incorrect and unexpected results. That's
why I say that userspace must understand all operations in a pipeline,
and all parameters of all used operations before it can actually use
that pipeline.
Otherwise we have the same problem as KMS properties have in general
today: when new things are added that userspace does not understand,
how will the userspace be able to maintain its old behaviour?
That question has two answers today:
- userspace learns to program everything, and accidentally
(mal)functions until then
- you do not switch between KMS clients that might leave incorrect
state in not-understood properties
Neither is a good answer, and the problem does not seem to have any
practical traction either.
For color pipelines I hope we can, and believe that we must, do better
to maintain correct behaviour while extending functionality.
> > > +
> > > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > > + property to NULL as shown in both [2] and [3]
> > > +
Is it a reset and disable, or only disable?
How is the reset state defined, if that state becomes active when the
pipeline is next enabled and data not set for the operations?
> > > + 4. change the color pipeline as demonstrated in [3].
> > > + On the new pipeline, the user is expected to setup all the color hardware
> > block
> > > + Once the user requests a pipeline change, the driver will provide it a clean
> > slate
> > > + which means that all the data previously set by the user will be discarded
> > even if
> > > + there are common hardware blocks between the two(previous and current)
> > pipelines.
> > > +
Yes, alternative pipelines need to be completely independent.
> > > +IGT implementation can be found here [4]
> > > +
> > > +Representing Fixed function hardware
> > > +====================================
> > > +
> > > +To provide support for fixed function hardware, the driver could
> > > +expose vendor specific struct drm_color_op with parameters that both
> > > +the userspace and driver agrees on. To demonstrate, let's consider a
> > > +hyphothetical fixed function hardware block that converts BT601 to BT2020.
> > > +The driver can choose to advertise the block as such.
> > > +
> > > +struct drm_color_op color_pipeline_X[] = {
> > > + {
> > > + .name = DRM_CB_PRIVATE,
What if the hardware has 5 different custom blocks like this, multiple
in the same pipeline. How do you 'name' them?
> > > + .type = FIXED_FUNCTION,
I have been assuming that 'type' uniquely defines both the operation and
the contents of the parameter blob, but this does not look like it.
What defines the operation and the parameters?
> > > + .blob_id = 45;
> > > + },
> > > +}
> > > +
> > > +Where the blob represents some vendor specific enums, strings or any
> > > +other appropriate data types which both the user-space and drivers are aligned
> > on.
We have a word for that "data ... aligned on": UAPI.
> > > +
> > > +blob:45 {
> > > + VENDORXXX_BT602_TO_BT2020,
Repeating the question from above, how will userspace know that the
contents of this blob need to be VENDORXXX_BT602_TO_BT2020 and not
something else?
> > > +};
> > > +
> > > +For enabling or passing parameters to such blocks, the user can send
> > > +data to the driver wrapped within a blob like any other color operation block.
> > > +
> > > + struct drm_color_op_data {
> > > + .name = DRM_CB_PRIVATE;
> > > + .blob_id = 46;
> > > + } ;
> > > +
> > > +where blob with id 46 contains data to enable the fixed function hardware(s).
> > > +
> >
> > One major thing missing from the RFC is an explanation on why we want to go with a
> > prescriptive model, rather than a descriptive model. This question came up in Dave's
> > response to Simon's RFC:
> > https://lore.kernel.org/dri-
> > devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.co
> > m/
> >
> > This is a rough attempt at such an explanation:
> > https://gitlab.freedesktop.org/hwentland/linux/-
> > /merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d
> > 8d
Hey Harry, that's a good piece!
>
> Sure Harry, totally agree to this and will include in documentation to highlight the rationale
> of going with prescriptive model.
Uma, the cover letter had descriptive and prescriptive mixed up.
Thanks,
pq
> > Harry
> >
> > > +References
> > > +==========
> > > +
> > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > +[2]
> > > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> > > +[3]
> > > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> > > +[4] https://patchwork.freedesktop.org/series/123018/
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 02/33] drm: Add color operation structure
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-08-30 13:00 ` Pekka Paalanen
-1 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-08-30 13:00 UTC (permalink / raw)
To: Uma Shankar; +Cc: intel-gfx, Chaitanya Kumar Borah, dri-devel, wayland-devel
[-- Attachment #1: Type: text/plain, Size: 5209 bytes --]
On Tue, 29 Aug 2023 21:33:51 +0530
Uma Shankar <uma.shankar@intel.com> wrote:
> From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
>
> Each Color Hardware block will be represented uniquely
> in the color pipeline. Define the structure to represent
> the same.
>
> These color operations will form the building blocks of
> a color pipeline which best represents the underlying
> Hardware. Color operations can be re-arranged, substracted
> or added to create distinct color pipelines to accurately
> describe the Hardware blocks present in the display engine.
>
> Co-developed-by: Uma Shankar <uma.shankar@intel.com>
> Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> ---
> include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 72 insertions(+)
>
> diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> index ea1b639bcb28..882479f41745 100644
> --- a/include/uapi/drm/drm_mode.h
> +++ b/include/uapi/drm/drm_mode.h
> @@ -943,6 +943,78 @@ struct hdr_output_metadata {
> };
> };
>
> +/**
> + * enum color_op_block
> + *
> + * Enums to identify hardware color blocks.
> + *
> + * @DRM_CB_PRE_CSC: LUT before the CTM unit
> + * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
> + * @DRM_CB_POST_CSC: LUT after the CTM unit
> + * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
> + * color components
> + * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
> + * can expose a custom hardware by defining a
> + * color operation block with this name as
> + * identifier
This naming scheme does not seem to work. It assumes a far too rigid
pipeline, just like the old KMS property design. What if you have two
other operations between PRE_CSC and CSC?
What sense do PRE_CSC and POST_CSC make if you don't happen to have a
CSC operation?
What if a driver put POST_CSC before PRE_CSC in its pipeline?
What if your CSC is actually a series of three independent operations,
and in addition you have PRE_CSC and POST_CSC?
3D_LUT is an operation category, not a name. The same could be said
about private.
Given that all these are also UAPI, do we also need protect old
userspace from seeing values it does not understand?
> + */
> +enum color_op_block {
> + DRM_CB_INVAL = -1,
> +
> + DRM_CB_PRE_CSC = 0,
> + DRM_CB_CSC,
> + DRM_CB_POST_CSC,
> + DRM_CB_3D_LUT,
> +
> + /* Any new generic hardware block can be updated here */
> +
> + /*
> + * PRIVATE is kept at 255 to make it future proof and leave
> + * scope for any new addition
> + */
> + DRM_CB_PRIVATE = 255,
> + DRM_CB_MAX = DRM_CB_PRIVATE,
> +};
> +
> +/**
> + * enum color_op_type
> + *
> + * These enums are to identify the mathematical operation that
> + * a hardware block is capable of.
> + * @CURVE_1D: It represents a one dimensional lookup table
> + * @CURVE_3D: Represents lut value for each color component for 3d lut capable hardware
> + * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
> + * @FIXED_FUNCTION: To enable and program any custom fixed function hardware unit
> + */
> +enum color_op_type {
> + CURVE_1D,
> + CURVE_3D,
> + MATRIX,
> + FIXED_FUNCTION,
My assumption was that a color_op_type would clearly and uniquely
define the mathematical model of the operation and the UABI structure
of the parameter blob. That means we need different values for uniform
vs. exponentially vs. programmable distributed 1D LUT, etc.
If there is a 1D curve with pre-programmed (fixed and named) curves, we
need to enumerate all the curve types somehow. Probably each fixed
curve type should not be a different operation type, because that would
explode the number of alternative pipelines.
A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you
meant a 3D LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in
three dimensional space.
A matrix element could be with or without an offset vector I guess.
FIXED_FUNCTION would need to be replaced with e.g. your example
VENDORXXX_BT602_TO_BT2020 to work.
Have I missed something, how did you intend this to work?
Thanks,
pq
> +};
> +
> +/**
> + * @struct drm_color_op
> + *
> + * This structure is used to represent the capability of
> + * individual color hardware blocks.
> + *
> + * @name: a standardized enum to identify the color hardware block
> + * @type: The type of mathematical operation it can perform
> + * @blob_id: Id pointing to a blob containing information about
> + * the hardware block which advertizes its capabilities
> + * to the userspace. It can be an optional field depending
> + * on the members "name" and "type".
> + * @private_flags: This can be used to provide vendor specific hints
> + * to user space
> + */
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type;
> + __u32 blob_id;
> + __u32 private_flags;
> +};
> +
> /**
> * DRM_MODE_PAGE_FLIP_EVENT
> *
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 02/33] drm: Add color operation structure
@ 2023-08-30 13:00 ` Pekka Paalanen
0 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-08-30 13:00 UTC (permalink / raw)
To: Uma Shankar; +Cc: intel-gfx, dri-devel, wayland-devel
[-- Attachment #1: Type: text/plain, Size: 5209 bytes --]
On Tue, 29 Aug 2023 21:33:51 +0530
Uma Shankar <uma.shankar@intel.com> wrote:
> From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
>
> Each Color Hardware block will be represented uniquely
> in the color pipeline. Define the structure to represent
> the same.
>
> These color operations will form the building blocks of
> a color pipeline which best represents the underlying
> Hardware. Color operations can be re-arranged, substracted
> or added to create distinct color pipelines to accurately
> describe the Hardware blocks present in the display engine.
>
> Co-developed-by: Uma Shankar <uma.shankar@intel.com>
> Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> ---
> include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 72 insertions(+)
>
> diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> index ea1b639bcb28..882479f41745 100644
> --- a/include/uapi/drm/drm_mode.h
> +++ b/include/uapi/drm/drm_mode.h
> @@ -943,6 +943,78 @@ struct hdr_output_metadata {
> };
> };
>
> +/**
> + * enum color_op_block
> + *
> + * Enums to identify hardware color blocks.
> + *
> + * @DRM_CB_PRE_CSC: LUT before the CTM unit
> + * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
> + * @DRM_CB_POST_CSC: LUT after the CTM unit
> + * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
> + * color components
> + * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
> + * can expose a custom hardware by defining a
> + * color operation block with this name as
> + * identifier
This naming scheme does not seem to work. It assumes a far too rigid
pipeline, just like the old KMS property design. What if you have two
other operations between PRE_CSC and CSC?
What sense do PRE_CSC and POST_CSC make if you don't happen to have a
CSC operation?
What if a driver put POST_CSC before PRE_CSC in its pipeline?
What if your CSC is actually a series of three independent operations,
and in addition you have PRE_CSC and POST_CSC?
3D_LUT is an operation category, not a name. The same could be said
about private.
Given that all these are also UAPI, do we also need protect old
userspace from seeing values it does not understand?
> + */
> +enum color_op_block {
> + DRM_CB_INVAL = -1,
> +
> + DRM_CB_PRE_CSC = 0,
> + DRM_CB_CSC,
> + DRM_CB_POST_CSC,
> + DRM_CB_3D_LUT,
> +
> + /* Any new generic hardware block can be updated here */
> +
> + /*
> + * PRIVATE is kept at 255 to make it future proof and leave
> + * scope for any new addition
> + */
> + DRM_CB_PRIVATE = 255,
> + DRM_CB_MAX = DRM_CB_PRIVATE,
> +};
> +
> +/**
> + * enum color_op_type
> + *
> + * These enums are to identify the mathematical operation that
> + * a hardware block is capable of.
> + * @CURVE_1D: It represents a one dimensional lookup table
> + * @CURVE_3D: Represents lut value for each color component for 3d lut capable hardware
> + * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
> + * @FIXED_FUNCTION: To enable and program any custom fixed function hardware unit
> + */
> +enum color_op_type {
> + CURVE_1D,
> + CURVE_3D,
> + MATRIX,
> + FIXED_FUNCTION,
My assumption was that a color_op_type would clearly and uniquely
define the mathematical model of the operation and the UABI structure
of the parameter blob. That means we need different values for uniform
vs. exponentially vs. programmable distributed 1D LUT, etc.
If there is a 1D curve with pre-programmed (fixed and named) curves, we
need to enumerate all the curve types somehow. Probably each fixed
curve type should not be a different operation type, because that would
explode the number of alternative pipelines.
A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you
meant a 3D LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in
three dimensional space.
A matrix element could be with or without an offset vector I guess.
FIXED_FUNCTION would need to be replaced with e.g. your example
VENDORXXX_BT602_TO_BT2020 to work.
Have I missed something, how did you intend this to work?
Thanks,
pq
> +};
> +
> +/**
> + * @struct drm_color_op
> + *
> + * This structure is used to represent the capability of
> + * individual color hardware blocks.
> + *
> + * @name: a standardized enum to identify the color hardware block
> + * @type: The type of mathematical operation it can perform
> + * @blob_id: Id pointing to a blob containing information about
> + * the hardware block which advertizes its capabilities
> + * to the userspace. It can be an optional field depending
> + * on the members "name" and "type".
> + * @private_flags: This can be used to provide vendor specific hints
> + * to user space
> + */
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type;
> + __u32 blob_id;
> + __u32 private_flags;
> +};
> +
> /**
> * DRM_MODE_PAGE_FLIP_EVENT
> *
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 00/33] Add Support for Plane Color Pipeline
2023-08-30 8:47 ` [Intel-gfx] " Shankar, Uma
@ 2023-08-30 21:15 ` Sebastian Wick
-1 siblings, 0 replies; 109+ messages in thread
From: Sebastian Wick @ 2023-08-30 21:15 UTC (permalink / raw)
To: Shankar, Uma
Cc: Pekka Paalanen, Shashank Sharma, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Christopher Braga
On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
>
>
> > -----Original Message-----
> > From: Harry Wentland <harry.wentland@amd.com>
> > Sent: Wednesday, August 30, 2023 12:56 AM
> > To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> > devel@lists.freedesktop.org
> > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > <ville.syrjala@linux.intel.com>; Pekka Paalanen <pekka.paalanen@collabora.com>;
> > Simon Ser <contact@emersion.fr>; Melissa Wen <mwen@igalia.com>; Jonas Ådahl
> > <jadahl@redhat.com>; Sebastian Wick <sebastian.wick@redhat.com>; Shashank
> > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > <quic_cbraga@quicinc.com>
> > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> >
> > +CC Naseer and Chris, FYI
> >
> > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> >
> > On 2023-08-29 12:03, Uma Shankar wrote:
> > > Introduction
> > > ============
> > >
> > > Modern hardwares have various color processing capabilities both at
> > > pre-blending and post-blending phases in the color pipeline.
> > > The current drm implementation exposes only the post-blending color
> > > hardware blocks. Support for pre-blending hardware is missing.
> > > There are multiple use cases where pre-blending color hardware will be
> > > useful:
> > > a) Linearization of input buffers encoded in various transfer
> > > functions.
> > > b) Color Space conversion
> > > c) Tone mapping
> > > d) Frame buffer format conversion
> > > e) Non-linearization of buffer(apply transfer function)
> > > f) 3D Luts
> > >
> > > and other miscellaneous color operations.
> > >
> > > Hence, there is a need to expose the color capabilities of the
> > > hardware to user-space. This will help userspace/middleware to use
> > > display hardware for color processing and blending instead of doing it
> > > through GPU shaders.
> > >
> >
> > Thanks, Uma, for sending this. I've been working on something similar but you beat
> > me to it. :)
>
> Thanks Harry for the useful feedback and overall collaboration on this so far.
>
> > >
> > > Work done so far and relevant references
> > > ========================================
> > >
> > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > Broad consensus is there that we need a generic API at drm core to
> > > suffice the use case of various HW vendors. Below are the links
> > > capturing the discussion so far.
> > >
> > > Intel's Plane Color Implementation:
> > > https://patchwork.freedesktop.org/series/90825/
> > > AMD's Plane Color Implementation:
> > > https://patchwork.freedesktop.org/series/116862/
> > >
> > >
> > > Hackfest conclusions
> > > ====================
> > >
> > > HDR/Color Hackfest was organised by Redhat to bring all the industry
> > > stakeholders together and converge on a common uapi expectations.
> > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and
> > > other prominent user-space developers and maintainers.
> > >
> > > Discussions happened on the uapi expectations, opens, nature of
> > > hardware of multiple hardware vendors, challenges in generalizing the
> > > same and the path forward. Consensus was made that drm core should
> > > implement descriptive APIs and not go with prescriptive APIs. DRM core
> > > should just expose the hardware capabilities; enabling, customizing
> > > and programming the same should be done by the user-space. Driver should just
> > honor the user space request without doing any operations internally.
> > >
> > > Thanks to Simon Ser, for nicely documenting the design consensus and
> > > an UAPI RFC which can be referred to here:
> > >
> > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5
> > >
> > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1Q
> > Wn48
> > > 8=@emersion.fr/
> > >
> > >
> > > Design considerations
> > > =====================
> > >
> > > Following are the important aspects taken into account while designing
> > > the current RFC
> > > proposal:
> > >
> > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one
> > can be used)
> > > 2. Position of the HW block in the pipeline can be programmable
> > > 3. LUTs can be one dimentional or three dimentional
> > > 4. Number of LUT entries can vary across platforms
> > > 5. Precision of LUT entries can vary across platforms
> > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> > > Piece-Wise Linear(PWL) etc
> > > 7. There can be parameterized/non-parameterized fixed function HW
> > blocks.
> > > e.g. Just a hardware bit, to convert from one color space to another.
> > > 8. Custom non-standard HW implementation.
> > > 9. Leaving scope for some vendor defined pescriptive implementation if
> > required.
> > > 10.Scope to handle any modification in hardware as technology evolves
> > >
> > > The current proposal takes into account the above considerations while
> > > keeping the implementation as generic as possible leaving scope for future
> > additions or modifications.
> > >
> > > This proposal is also in line to the details mentioned by Simon's RFC
> > > covering all the aspects discussed in hackfest.
> > >
> > >
> > > Outline of the implementation
> > > ============================
> > >
> > > Each Color Hardware block will be represented by a data structure drm_color_op.
> > > These color operations will form the building blocks of a color
> > > pipeline which best represents the underlying Hardware. Color
> > > operations can be re-arranged, substracted or added to create distinct
> > > color pipelines to accurately describe the Hardware blocks present in the display
> > engine.
> >
> > Who is doing the arranging of color operations? IMO a driver should define one or
> > more respective pipelines that can be selected by userspace. This seems to be what
> > you're talking about after (I haven't reviewed the whole thing yet). Might be best to
> > drop this sentence or to add clarifications in order to avoid confusion.
>
> Yes it's the driver who will set the pipeline based on the underlying hardware arrangement
> and possible combinations. There can be multiple pipelines possible if hardware can be
> muxed or order can be re-arranged (all viable combinations should be defined as a pipeline in driver).
> Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
>
> > >
> > > In this proposal, a color pipeline is represented as an array of
> > > struct drm_color_op. For individual color operation, we add blobs to
> > > advertise the capability of the particular Hardware block.
> > >
> > > This color pipeline is then packaged within a blob for the user space
> > > to retrieve it.
> > >
> >
> > Would it be better to expose the drm_color_ops directly, instead of packing a array
> > of drm_color_ops into a blob and then giving that to userspace.
>
> Advantage we see in packing in blobs is that interface will be cleaner. There will be just
> one GET_COLOR_PIPELINE property to invoke by user and then its just parsing the data.
> This way the entire underlying hardware blocks with pipeline will be available to user.
I don't see how parsing a blob is easier than requesting the color ops
from the kernel. User space is already equiped with getting KMS objects
and the igt test code from Harry shows that this is all pretty trivial
plumbing.
> For a particular hardware block in pipeline, user can get the relevant details from blob
> representing that particular block. We have created IGT tests (links mentioned in cover-letter)
> to demonstrate how it can be done. This is just to clarify the idea.
The blob is also not introspectable with the usual tools whereas
exposing them as properties would be.
It also would, like Pekka correctly noted, bring a whole bunch of issues
about compatibility and versioning that are well understood with objects
+ properties.
> Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL
> hardware blocks. Also we have left scope for custom private hardware blocks as well which driver
> can work with respective HAL and get that implemented.
When color ops are real KMS objects they still can have properties which
are blobs that can store LUTs and other data.
And we should avoid private blocks at all cost. In fact, I don't think
the KMS rules have changed in that regard and it simply is not allowed.
> We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL
> agree.
>
> > > To advertise the available color pipelines, an immutable ENUM property
> > > "GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
> > > With each blob id representing a distinct color pipeline based on
> > > underlying HW capabilities and their respective combinations.
> > >
> > > Once the user space decides on a color pipeline, it can set the
> > > pipeline and the corresponding data for the hardware blocks within the
> > > pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > >
> >
> > When I discussed this at the hackfest with Simon he proposed a new DRM object,
> > (I've called it a drm_colorop) to represent a color operation.
> > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or NULL if its the
> > last in the pipeline. You can then have a mutable enum property on the plane to
> > discover and select a color pipeline.
>
> Yes, the proposal is inspired by this idea. Sure, we can work together to enhance the design.
> Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon
> and you Harry. Management of the pipeline, addition, deletion and flexibility to represent
> hardware is more with blobs with the relevant structures agreed in UAPI.
>
> > This seems a bit more transparent than a blob. You can see my changes
> > (unfortunately very WIP, don't look too closely at individual patches) at
> > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs
> >
> > libdrm changes:
> > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
>
> Sure, will check the same.
>
> > IGT changes:
> > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffs
> >
> > I'll take time to review your whole series and will see whether we can somehow
> > keep the best parts of each.
>
> Thanks and agree. Let's work together and get this implemented in DRM.
>
> > Curious to hear other opinions on the blob vs new DRM object question as well.
I can see a few advantages with the blob approach.
User space can store the entire state in one blob and just assign a new
blob to change to another pipeline configuration.
However, I would argue that changing a lot of properties is already
common practice and works well. User space can deal with it and has
abstractions to make this managable.
There also is no need for a new KMS object kindbu from Harrys work so
far the new KMS object kind is surprisingly simple and works just as
expected, which is a good sign.
With the blobs one can store general information about the pipeline and
not only about the color ops themselves. So far it's not clear if we
actually need this, and if we do, we could probably use color ops at the
start of the pipeline which apply to all ops.
There are also a few drawbacks to the blob design. It's not very KMS-y
which makes working with it different than with the rest of KMS and this
mismatch just adds more mental burden. It probably also introduces a new
set of problems that we might not even be aware of. It also makes it
less introspectable as Pekka noted.
When I came up with the KMS Color Pipeline API, I also started with the
blob approach because of the flexibility in it but it does add a fair
amount of hidden complexity. At the hackfest Simon managed to convince
me of the idea of using a new KMS object kind for it and it seemed to
simplify a bunch of things. With Harrys work I would argue that it is
ineed that case. It's pretty simple and still seems to do everything it
needs to.
So, yeah, I'm favoring the color op approach and not the blob approach.
>
> Yeah, request community and stakeholders to share feedback and suggestions.
> We will work on the feedback to improve the design.
>
> > > Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the
> > > patch
> > >
> > > IGT and test details
> > > ====================
> > >
> > > A set of IGT tests is written to demonstrate the usage of the proposed
> > > UAPIs along with some sanity validation.
> > >
> > > Details of the IGT test can be found here:
> > > https://patchwork.freedesktop.org/series/123018/
> > >
> > > Opens
> > > =====
> > >
> > > a. To come up with a list of common HW blocks which can be defined generically
> > by the DRM
> > > core in agreement with all the stakeholders b. To enhance/finalize
> > > the data structure to define segmented LUTs generically.
> > >
> >
> > It would be good to add some basic support in VKMS. My work has been based on
> > VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private
> > properties from Melissa through the API.
>
> Yeah sure Harry.
I think this is a great idea! Let's see if we can learn some lessons
from both patch series and then continue with the VKMS implementation
and igt tests for it before every vendor implements their view of how
this should all work.
>
> > >
> > > Out of Scope
> > > ============
> > >
> > > a. The coefficients for CTM and LUT value calculations are out of scope of the
> > proposal.
> > > b. The onus of programming the HW blocks and their values is on user-space.
> > Driver will
> > > just provide the interface for the same.
> > > c. In order to compute LUT values and coefficients, a helper library can be created
> > in
> > > user-space. However, it is out of scope for the current proposal.
> > >
> > > Acknowledgements and credits
> > > ============================
> > >
> > > There are multiple contributors who have helped us to reach to this
> > > proposal. Special mention to Ville
> > > Syrjala<ville.syrjala@linux.intel.com>, Pekka
> > > Paalanen<pekka.paalanen@collabora.com>,
> > > Simon Ser<contact@emersion.fr>, Harry
> > > Wentland<harry.wentland@amd.com>, Melissa Wen<mwen@igalia.com>,
> > > Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
> > Bhanu<bhanuprakash.modem@intel.com> and
> > Shashank<shashank.sharma@amd.com>.
> > >
> > > Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for
> > organizing the HDR hackfest.
> > >
> > >
> > > UAPI dependency and Usermode development
> > > ========================================
> > >
> > > The current KMS implementation requires a user space consumer for it to be
> > accepted upstream.
> > > Work is ongoing in weston and mutter community to get color management
> > > and HDR support implemented in the respective stacks.
> > >
> >
> > If we can get AMD properties encoded using a Color Pipeline API we can probably
> > use gamescope as the userspace vehicle.
>
> Yeah, nice.
>
> > I'm reviewing this in sequence, so there's a chance I'm missing context.
> > Please bear with me if some of my comments are answered later in the series.
>
> No worries, really appreciate the feedback and support.
>
> > Again, thanks for sending this.
>
> Always welcome.
>
> Regards,
> Uma Shankar
>
> > Harry
> >
> > >
> > ====================================================================
> > ==
> > > ===========
> > >
> > > We have tried to take care of all the scenarios and use-cases which
> > > possibly could exists in the current proposal. Thanks to everyone who
> > > has contributed in all color management discussions so far. Let's work
> > > together to improve the current proposal and get this thing implemented in
> > upstream linux. All the feedback and suggestions to enhance the design are
> > welcome.
> > >
> > > Regards,
> > > Uma Shankar
> > > Chaitanya Kumar Borah
> > >
> > > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > > Cc: Simon Ser <contact@emersion.fr>
> > > Cc: Harry Wentland <harry.wentland@amd.com>
> > > Cc: Melissa Wen <mwen@igalia.com>
> > > Cc: Jonas Ådahl <jadahl@redhat.com>
> > > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > > Cc: Alexander Goins <agoins@nvidia.com>
> > >
> > > Chaitanya Kumar Borah (14):
> > > drm: Add color operation structure
> > > drm: Add plane get color pipeline property
> > > drm: Add helper to add color pipeline
> > > drm: Manage color blob states
> > > drm: Replace individual color blobs
> > > drm: Reset pipeline when user sends NULL blob
> > > drm: Reset plane color state on pipeline switch request
> > > drm/i915/color: Add HDR plane LUT range data to color pipeline
> > > drm/i915/color: Add SDR plane LUT range data to color pipeline
> > > drm/i915/color: Add color pipelines to plane
> > > drm/i915/color: Create and attach set color pipeline property
> > > drm/i915/color: Enable plane color features
> > > drm/i915/color: Add a dummy pipeline with 3D LUT
> > > drm/i915/color: Add example implementation for vendor specific color
> > > operation
> > >
> > > Uma Shankar (19):
> > > drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> > > drm: Add structures for setting color pipeline
> > > drm: Add set colorpipeline property
> > > drm: Add Enhanced Gamma LUT precision structure
> > > drm: Add color lut range structure
> > > drm: Add color information to plane state
> > > drm/i915/color: Add lut range for SDR planes
> > > drm/i915/color: Add lut range for HDR planes
> > > drm/i915/color: Add color pipeline for HDR planes
> > > drm/i915/color: Add color pipeline for SDR planes
> > > drm/i915/color: Add plane color callbacks
> > > drm/i915/color: Load plane color luts from atomic flip
> > > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> > > drm/i915/xelpd: Add register definitions for Plane Degamma
> > > drm/i915/color: Add color functions for ADL
> > > drm/i915/color: Program Plane Pre-CSC Registers
> > > drm/i915/xelpd: Add register definitions for Plane Post CSC
> > > drm/i915/xelpd: Program Plane Post CSC Registers
> > > drm/i915/color: Enable Plane CSC
> > >
> > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> > > drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> > > drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> > > drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> > > drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> > > drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> > > .../drm/i915/display/skl_universal_plane.c | 21 +-
> > > drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> > > include/drm/drm_plane.h | 82 +++
> > > include/uapi/drm/drm_mode.h | 196 +++++
> > > include/uapi/drm/i915_drm.h | 25 +
> > > 11 files changed, 1899 insertions(+), 3 deletions(-)
> > > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-08-30 21:15 ` Sebastian Wick
0 siblings, 0 replies; 109+ messages in thread
From: Sebastian Wick @ 2023-08-30 21:15 UTC (permalink / raw)
To: Shankar, Uma
Cc: Alexander Goins, Pekka Paalanen, Simon Ser, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Harry Wentland, Christopher Braga
On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
>
>
> > -----Original Message-----
> > From: Harry Wentland <harry.wentland@amd.com>
> > Sent: Wednesday, August 30, 2023 12:56 AM
> > To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> > devel@lists.freedesktop.org
> > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > <ville.syrjala@linux.intel.com>; Pekka Paalanen <pekka.paalanen@collabora.com>;
> > Simon Ser <contact@emersion.fr>; Melissa Wen <mwen@igalia.com>; Jonas Ådahl
> > <jadahl@redhat.com>; Sebastian Wick <sebastian.wick@redhat.com>; Shashank
> > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > <quic_cbraga@quicinc.com>
> > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> >
> > +CC Naseer and Chris, FYI
> >
> > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> >
> > On 2023-08-29 12:03, Uma Shankar wrote:
> > > Introduction
> > > ============
> > >
> > > Modern hardwares have various color processing capabilities both at
> > > pre-blending and post-blending phases in the color pipeline.
> > > The current drm implementation exposes only the post-blending color
> > > hardware blocks. Support for pre-blending hardware is missing.
> > > There are multiple use cases where pre-blending color hardware will be
> > > useful:
> > > a) Linearization of input buffers encoded in various transfer
> > > functions.
> > > b) Color Space conversion
> > > c) Tone mapping
> > > d) Frame buffer format conversion
> > > e) Non-linearization of buffer(apply transfer function)
> > > f) 3D Luts
> > >
> > > and other miscellaneous color operations.
> > >
> > > Hence, there is a need to expose the color capabilities of the
> > > hardware to user-space. This will help userspace/middleware to use
> > > display hardware for color processing and blending instead of doing it
> > > through GPU shaders.
> > >
> >
> > Thanks, Uma, for sending this. I've been working on something similar but you beat
> > me to it. :)
>
> Thanks Harry for the useful feedback and overall collaboration on this so far.
>
> > >
> > > Work done so far and relevant references
> > > ========================================
> > >
> > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > Broad consensus is there that we need a generic API at drm core to
> > > suffice the use case of various HW vendors. Below are the links
> > > capturing the discussion so far.
> > >
> > > Intel's Plane Color Implementation:
> > > https://patchwork.freedesktop.org/series/90825/
> > > AMD's Plane Color Implementation:
> > > https://patchwork.freedesktop.org/series/116862/
> > >
> > >
> > > Hackfest conclusions
> > > ====================
> > >
> > > HDR/Color Hackfest was organised by Redhat to bring all the industry
> > > stakeholders together and converge on a common uapi expectations.
> > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and
> > > other prominent user-space developers and maintainers.
> > >
> > > Discussions happened on the uapi expectations, opens, nature of
> > > hardware of multiple hardware vendors, challenges in generalizing the
> > > same and the path forward. Consensus was made that drm core should
> > > implement descriptive APIs and not go with prescriptive APIs. DRM core
> > > should just expose the hardware capabilities; enabling, customizing
> > > and programming the same should be done by the user-space. Driver should just
> > honor the user space request without doing any operations internally.
> > >
> > > Thanks to Simon Ser, for nicely documenting the design consensus and
> > > an UAPI RFC which can be referred to here:
> > >
> > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5
> > >
> > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1Q
> > Wn48
> > > 8=@emersion.fr/
> > >
> > >
> > > Design considerations
> > > =====================
> > >
> > > Following are the important aspects taken into account while designing
> > > the current RFC
> > > proposal:
> > >
> > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one
> > can be used)
> > > 2. Position of the HW block in the pipeline can be programmable
> > > 3. LUTs can be one dimentional or three dimentional
> > > 4. Number of LUT entries can vary across platforms
> > > 5. Precision of LUT entries can vary across platforms
> > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> > > Piece-Wise Linear(PWL) etc
> > > 7. There can be parameterized/non-parameterized fixed function HW
> > blocks.
> > > e.g. Just a hardware bit, to convert from one color space to another.
> > > 8. Custom non-standard HW implementation.
> > > 9. Leaving scope for some vendor defined pescriptive implementation if
> > required.
> > > 10.Scope to handle any modification in hardware as technology evolves
> > >
> > > The current proposal takes into account the above considerations while
> > > keeping the implementation as generic as possible leaving scope for future
> > additions or modifications.
> > >
> > > This proposal is also in line to the details mentioned by Simon's RFC
> > > covering all the aspects discussed in hackfest.
> > >
> > >
> > > Outline of the implementation
> > > ============================
> > >
> > > Each Color Hardware block will be represented by a data structure drm_color_op.
> > > These color operations will form the building blocks of a color
> > > pipeline which best represents the underlying Hardware. Color
> > > operations can be re-arranged, substracted or added to create distinct
> > > color pipelines to accurately describe the Hardware blocks present in the display
> > engine.
> >
> > Who is doing the arranging of color operations? IMO a driver should define one or
> > more respective pipelines that can be selected by userspace. This seems to be what
> > you're talking about after (I haven't reviewed the whole thing yet). Might be best to
> > drop this sentence or to add clarifications in order to avoid confusion.
>
> Yes it's the driver who will set the pipeline based on the underlying hardware arrangement
> and possible combinations. There can be multiple pipelines possible if hardware can be
> muxed or order can be re-arranged (all viable combinations should be defined as a pipeline in driver).
> Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
>
> > >
> > > In this proposal, a color pipeline is represented as an array of
> > > struct drm_color_op. For individual color operation, we add blobs to
> > > advertise the capability of the particular Hardware block.
> > >
> > > This color pipeline is then packaged within a blob for the user space
> > > to retrieve it.
> > >
> >
> > Would it be better to expose the drm_color_ops directly, instead of packing a array
> > of drm_color_ops into a blob and then giving that to userspace.
>
> Advantage we see in packing in blobs is that interface will be cleaner. There will be just
> one GET_COLOR_PIPELINE property to invoke by user and then its just parsing the data.
> This way the entire underlying hardware blocks with pipeline will be available to user.
I don't see how parsing a blob is easier than requesting the color ops
from the kernel. User space is already equiped with getting KMS objects
and the igt test code from Harry shows that this is all pretty trivial
plumbing.
> For a particular hardware block in pipeline, user can get the relevant details from blob
> representing that particular block. We have created IGT tests (links mentioned in cover-letter)
> to demonstrate how it can be done. This is just to clarify the idea.
The blob is also not introspectable with the usual tools whereas
exposing them as properties would be.
It also would, like Pekka correctly noted, bring a whole bunch of issues
about compatibility and versioning that are well understood with objects
+ properties.
> Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL
> hardware blocks. Also we have left scope for custom private hardware blocks as well which driver
> can work with respective HAL and get that implemented.
When color ops are real KMS objects they still can have properties which
are blobs that can store LUTs and other data.
And we should avoid private blocks at all cost. In fact, I don't think
the KMS rules have changed in that regard and it simply is not allowed.
> We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL
> agree.
>
> > > To advertise the available color pipelines, an immutable ENUM property
> > > "GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values.
> > > With each blob id representing a distinct color pipeline based on
> > > underlying HW capabilities and their respective combinations.
> > >
> > > Once the user space decides on a color pipeline, it can set the
> > > pipeline and the corresponding data for the hardware blocks within the
> > > pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > >
> >
> > When I discussed this at the hackfest with Simon he proposed a new DRM object,
> > (I've called it a drm_colorop) to represent a color operation.
> > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or NULL if its the
> > last in the pipeline. You can then have a mutable enum property on the plane to
> > discover and select a color pipeline.
>
> Yes, the proposal is inspired by this idea. Sure, we can work together to enhance the design.
> Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon
> and you Harry. Management of the pipeline, addition, deletion and flexibility to represent
> hardware is more with blobs with the relevant structures agreed in UAPI.
>
> > This seems a bit more transparent than a blob. You can see my changes
> > (unfortunately very WIP, don't look too closely at individual patches) at
> > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs
> >
> > libdrm changes:
> > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
>
> Sure, will check the same.
>
> > IGT changes:
> > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffs
> >
> > I'll take time to review your whole series and will see whether we can somehow
> > keep the best parts of each.
>
> Thanks and agree. Let's work together and get this implemented in DRM.
>
> > Curious to hear other opinions on the blob vs new DRM object question as well.
I can see a few advantages with the blob approach.
User space can store the entire state in one blob and just assign a new
blob to change to another pipeline configuration.
However, I would argue that changing a lot of properties is already
common practice and works well. User space can deal with it and has
abstractions to make this managable.
There also is no need for a new KMS object kindbu from Harrys work so
far the new KMS object kind is surprisingly simple and works just as
expected, which is a good sign.
With the blobs one can store general information about the pipeline and
not only about the color ops themselves. So far it's not clear if we
actually need this, and if we do, we could probably use color ops at the
start of the pipeline which apply to all ops.
There are also a few drawbacks to the blob design. It's not very KMS-y
which makes working with it different than with the rest of KMS and this
mismatch just adds more mental burden. It probably also introduces a new
set of problems that we might not even be aware of. It also makes it
less introspectable as Pekka noted.
When I came up with the KMS Color Pipeline API, I also started with the
blob approach because of the flexibility in it but it does add a fair
amount of hidden complexity. At the hackfest Simon managed to convince
me of the idea of using a new KMS object kind for it and it seemed to
simplify a bunch of things. With Harrys work I would argue that it is
ineed that case. It's pretty simple and still seems to do everything it
needs to.
So, yeah, I'm favoring the color op approach and not the blob approach.
>
> Yeah, request community and stakeholders to share feedback and suggestions.
> We will work on the feedback to improve the design.
>
> > > Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the
> > > patch
> > >
> > > IGT and test details
> > > ====================
> > >
> > > A set of IGT tests is written to demonstrate the usage of the proposed
> > > UAPIs along with some sanity validation.
> > >
> > > Details of the IGT test can be found here:
> > > https://patchwork.freedesktop.org/series/123018/
> > >
> > > Opens
> > > =====
> > >
> > > a. To come up with a list of common HW blocks which can be defined generically
> > by the DRM
> > > core in agreement with all the stakeholders b. To enhance/finalize
> > > the data structure to define segmented LUTs generically.
> > >
> >
> > It would be good to add some basic support in VKMS. My work has been based on
> > VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private
> > properties from Melissa through the API.
>
> Yeah sure Harry.
I think this is a great idea! Let's see if we can learn some lessons
from both patch series and then continue with the VKMS implementation
and igt tests for it before every vendor implements their view of how
this should all work.
>
> > >
> > > Out of Scope
> > > ============
> > >
> > > a. The coefficients for CTM and LUT value calculations are out of scope of the
> > proposal.
> > > b. The onus of programming the HW blocks and their values is on user-space.
> > Driver will
> > > just provide the interface for the same.
> > > c. In order to compute LUT values and coefficients, a helper library can be created
> > in
> > > user-space. However, it is out of scope for the current proposal.
> > >
> > > Acknowledgements and credits
> > > ============================
> > >
> > > There are multiple contributors who have helped us to reach to this
> > > proposal. Special mention to Ville
> > > Syrjala<ville.syrjala@linux.intel.com>, Pekka
> > > Paalanen<pekka.paalanen@collabora.com>,
> > > Simon Ser<contact@emersion.fr>, Harry
> > > Wentland<harry.wentland@amd.com>, Melissa Wen<mwen@igalia.com>,
> > > Jonas<jadahl@redhat.com>, Sebastian Wick<sebastian.wick@redhat.com>,
> > Bhanu<bhanuprakash.modem@intel.com> and
> > Shashank<shashank.sharma@amd.com>.
> > >
> > > Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team for
> > organizing the HDR hackfest.
> > >
> > >
> > > UAPI dependency and Usermode development
> > > ========================================
> > >
> > > The current KMS implementation requires a user space consumer for it to be
> > accepted upstream.
> > > Work is ongoing in weston and mutter community to get color management
> > > and HDR support implemented in the respective stacks.
> > >
> >
> > If we can get AMD properties encoded using a Color Pipeline API we can probably
> > use gamescope as the userspace vehicle.
>
> Yeah, nice.
>
> > I'm reviewing this in sequence, so there's a chance I'm missing context.
> > Please bear with me if some of my comments are answered later in the series.
>
> No worries, really appreciate the feedback and support.
>
> > Again, thanks for sending this.
>
> Always welcome.
>
> Regards,
> Uma Shankar
>
> > Harry
> >
> > >
> > ====================================================================
> > ==
> > > ===========
> > >
> > > We have tried to take care of all the scenarios and use-cases which
> > > possibly could exists in the current proposal. Thanks to everyone who
> > > has contributed in all color management discussions so far. Let's work
> > > together to improve the current proposal and get this thing implemented in
> > upstream linux. All the feedback and suggestions to enhance the design are
> > welcome.
> > >
> > > Regards,
> > > Uma Shankar
> > > Chaitanya Kumar Borah
> > >
> > > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > > Cc: Simon Ser <contact@emersion.fr>
> > > Cc: Harry Wentland <harry.wentland@amd.com>
> > > Cc: Melissa Wen <mwen@igalia.com>
> > > Cc: Jonas Ådahl <jadahl@redhat.com>
> > > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > > Cc: Alexander Goins <agoins@nvidia.com>
> > >
> > > Chaitanya Kumar Borah (14):
> > > drm: Add color operation structure
> > > drm: Add plane get color pipeline property
> > > drm: Add helper to add color pipeline
> > > drm: Manage color blob states
> > > drm: Replace individual color blobs
> > > drm: Reset pipeline when user sends NULL blob
> > > drm: Reset plane color state on pipeline switch request
> > > drm/i915/color: Add HDR plane LUT range data to color pipeline
> > > drm/i915/color: Add SDR plane LUT range data to color pipeline
> > > drm/i915/color: Add color pipelines to plane
> > > drm/i915/color: Create and attach set color pipeline property
> > > drm/i915/color: Enable plane color features
> > > drm/i915/color: Add a dummy pipeline with 3D LUT
> > > drm/i915/color: Add example implementation for vendor specific color
> > > operation
> > >
> > > Uma Shankar (19):
> > > drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> > > drm: Add structures for setting color pipeline
> > > drm: Add set colorpipeline property
> > > drm: Add Enhanced Gamma LUT precision structure
> > > drm: Add color lut range structure
> > > drm: Add color information to plane state
> > > drm/i915/color: Add lut range for SDR planes
> > > drm/i915/color: Add lut range for HDR planes
> > > drm/i915/color: Add color pipeline for HDR planes
> > > drm/i915/color: Add color pipeline for SDR planes
> > > drm/i915/color: Add plane color callbacks
> > > drm/i915/color: Load plane color luts from atomic flip
> > > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> > > drm/i915/xelpd: Add register definitions for Plane Degamma
> > > drm/i915/color: Add color functions for ADL
> > > drm/i915/color: Program Plane Pre-CSC Registers
> > > drm/i915/xelpd: Add register definitions for Plane Post CSC
> > > drm/i915/xelpd: Program Plane Post CSC Registers
> > > drm/i915/color: Enable Plane CSC
> > >
> > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> > > drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> > > drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> > > drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> > > drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> > > drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> > > .../drm/i915/display/skl_universal_plane.c | 21 +-
> > > drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> > > include/drm/drm_plane.h | 82 +++
> > > include/uapi/drm/drm_mode.h | 196 +++++
> > > include/uapi/drm/i915_drm.h | 25 +
> > > 11 files changed, 1899 insertions(+), 3 deletions(-)
> > > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-08-30 12:28 ` [Intel-gfx] " Pekka Paalanen
@ 2023-09-04 13:44 ` Shankar, Uma
-1 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-04 13:44 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Harry Wentland
> -----Original Message-----
> From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> Paalanen
> Sent: Wednesday, August 30, 2023 5:59 PM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> Color Pipeline
>
> On Wed, 30 Aug 2023 08:59:36 +0000
> "Shankar, Uma" <uma.shankar@intel.com> wrote:
>
> > > -----Original Message-----
> > > From: Harry Wentland <harry.wentland@amd.com>
> > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > wayland- devel@lists.freedesktop.org
> > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > Plane Color Pipeline
> > >
> > >
> > >
> > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > >
> > > > Co-developed-by: Chaitanya Kumar Borah
> > > > <chaitanya.kumar.borah@intel.com>
> > > > Signed-off-by: Chaitanya Kumar Borah
> > > > <chaitanya.kumar.borah@intel.com>
> > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > ---
> > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > 1 file changed, 394 insertions(+)
> > > > create mode 100644
> > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > >
> > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > new file mode 100644
> > > > index 000000000000..60ce515b6ea7
> > > > --- /dev/null
> > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
>
> ...
>
> Hi Uma!
Thanks Pekka for the feedback and useful inputs.
> > > > +This color pipeline is then packaged within a blob for the user
> > > > +space to retrieve it. Details can be found in the next section
> > > > +
> > >
> > > Not sure I like blobs that contain other blob ids.
> >
> > It provides flexibility and helps with just one interface to
> > userspace. Its easy to handle and manage once we get the hang of it 😊.
> >
> > We can clearly define the steps of parsing and data structures to be
> > used while interpreting and parsing the blobs.
>
> Don't forget extendability. Possibly every single struct will need some kind of
> versioning, and then it's not simple to parse anymore. Add to that new/old kernel
> vs. old/new userspace, and it seems a bit nightmarish to design.
Structure to be used to interpret the blob should be defined as UAPI only and is not
expected to change once agreed upon. It should be interpreted like a standard property.
So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized
and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI
property should be applied.
New vs old userspace problem exists even today as you rightly highlighted in mail below,
however we are planning to propose that we clean the hardware state once the userspace
client switches or same client switches the pipeline.
> Also since it's records inside a single blob, it's like a new file
> format: every record needs a standard header that allows skipping it
> appropriately if userspace does not understand it, or you need a standard index
> telling where everything is. Making all records the same size would waste space,
> and extendability requires variable size.
The design currently implements 1 hardware block by a struct drm_color_op data structure.
Multiple such blocks make the pipeline. So userspace just needs to get the pipeline and then
parse blocks 1 by 1. For blocks which it doesn't understand it can just skip and move to the
next one. Each block is differentiated by a unique "name" standardized by an enum which will be
part of the UAPI. Thus we will have scope for variable size blob to represent the particular hardware
pipeline, userspace can parse and implement whichever blocks it understands. Only rule defined
by UAPI is the way the respective block is to be parsed and programmed.
> I also would not assume that we can declare a standard set of blocks and that
> nothing else will be needed. The existing hardware is too diverse for that from
> what I have understood. I assume that some hardware have blocks unique to
> them, and they want to at least expose that functionality through a UAPI that
> allows at least generic enumeration of functionality, even if it needs specialized
> userspace code to actually make use of.
Yeah, this is right and for that reason we came up with an idea of DRM_CB_PRIVATE
name for the hardware block. This will tell userspace that this is private hardware block
for a particular hardware vendor. Generic userspace will ignore this block. Vendor specific
HAL or compositor implementation can parse and use this block. To interpret the blob_id
and assign to a respective data structure, private_flags will be used. These private_flags will
be agreed upon by HAL and vendor implementation, generic userspace will ignore them.
For all other cases, private_flags is a don't care field.
> If we go with
>
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type;
> + u32 blob_id;
> + u32 private_flags;
> +};
>
> as in your proposal, I believe it can work (sorry, looking further down, I have
> assumed too much of 'type'), but the enumerations will become long, and the
> details blob_id is still specific to 'type'. This is unavoidable, but we can still choose
> the form between blobs and properties, integers and strings.
>
> I have a feeling that introspection will be valuable here, to help people
> understand what their hardware could do if they had the code to use it. 'name'
> and 'type' being integers require a translation table to strings before they are
> readable, and it would be best if the kernel itself provided that translation.
Name and type can be standardized in enum and well documented in the UAPI.
For all the standard hardware blocks common for all vendors and serving most of the
common usecases, we can define standard names in enum. These can be easily
interpreted by a table as done in many cases already in driver and userspace.
> I don't understand how 'private_flags' could be useful. There must not be any
> "hidden" features. Everything a block can be programmed to do via this UAPI must
> be clearly documented, there cannot be anything private. If two hardware
> versions of a block differ in a meaningful or significant way, they need to be
> exposed as different types of blocks.
The idea of private_flags partially explained above is to have an option for vendors to
expose hardware blocks unique or custom to their hardware. private_flags can help
differentiate incase multiple such hardware blocks exists on the platform.
For how we intend to use it, please refer below patch in the series:
https://patchwork.freedesktop.org/patch/554928/?series=123024&rev=1
Also, IGT showing the usage:
https://patchwork.freedesktop.org/patch/554831/?series=123018&rev=1
> OTOH, if one goes with a (new) DRM object with string named properties model,
> all that struct versioning and file format hassle has mostly a clear and well-
> understood solution. We only need to define the rules of how userspace needs to
> deal with properties or values it does not understand, so that the kernel can keep
> adding more.
With the current blob approach with fixed data structure for parsing, we can generalize
all the hardware blocks and custom implementations. It is future proof with scope to
add any new hardware block in future as well. Rules to interpret, parse, program the
hardware can be defined in the blob approach as well and ideally should be defined and
fixed in the UAPI documentation.
> Therefore, I'm not yet convinced with the "all blobs" design.
Looking forward to collaboratively solve the problem for the community.
Will improve the design based on all the feedback.
> > > > +Exposing a color pipeline to user space
> > > > +=======================================
> > > > +
> > > > +To advertise the available color pipelines, an immutable ENUM
> > > > +property "GET_COLOR_PIPELINE" is introduced.
> > > > +This enum property has blob id's as values. With each blob id
> > > > +representing a distinct color pipeline based on underlying HW
> > > > +capabilities and their respective combinations.
> > > > +
> > > > +The following output of drm_info [1], shows how color pipelines
> > > > +are visible to userspace.
> > > > +
> > > > +├───Plane 0
> > > > + │ ├───Object ID: 31
> > > > + │ ├───CRTCs: {0}
> > > > + │ ├───Legacy info
> > > > + ...
> > > > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color
> pipeline,
> > > > + color pipeline 1, color pipeline
> 2}=
> > > no color pipeline
> > > > +
> > > > +To understand the capabilities of individual pipelines, first the
> > > > +userspace need to retrieve the pipeline blob with the blob ids
> > > > +retrieved by reading the enum property.
> > > > +
> > > > +Once the color pipeline is retrieved, user can then parse through
> > > > +individual drm_color_op blocks to understand the capabilities of
> > > > +each hardware block.
> > > > +
> > > > +Check IGT series to see how user space can parse through color pipelines.
> > > > +Refer the IGT series here:
> > > > +https://patchwork.freedesktop.org/series/123018/
> > > > +
> > > > +Setting up a color pipeline
> > > > +===========================
> > > > +
> > > > +Once the user space decides on a color pipeline, it can set the
> > > > +pipeline and the corresponding data for the hardware blocks
> > > > +within the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > +
> > > > +To achieve this two structures are introduced
> > > > +
> > > > +1. struct drm_color_op_data: It represents data to be passed onto
> individual
> > > > + color hardware
> blocks. It
> > > contains three members
> > > > + a) name: to identify the color operation block
> > > > + b) blob_id: pointing to the blob with data for the
> > > > + corresponding hardware block
> > > > +
> > > > + struct drm_color_op_data {
> > > > + enum color_op_block name;
>
> Why is this a global fixed enum rather than a pipeline specific ordinal or a unique-
> per-device ID?
>
> There is no reason to believe that a 'name' always matches a hardware block 1:1.
> When drivers accumulate multiple different alternative pipelines due to
> backwards-compatibility reasons, the same 'name' could be implemented by
> different hardware blocks, or the same hardware block could implement different
> 'name's from different pipelines.
>
We have some fixed hardware operations in the color pipeline, its good to define and
identify them with name as identifier. For any other custom block we can use the PRIVATE type.
Driver should expose the name of the hardware block for a particular pipeline based on the
actual usage, and as long as that is standardized it should be ok if same hardware block is
named differently across pipeline and vice-versa.
> The names have also a problem. If you name something "pre-csc", then how do
> you name the thing that the next hardware version adds between "pre-csc" and
> "csc"?
Most of the common hardware blocks can be named based on the operation they are intended to
be used for. For some custom blocks it can be defined as PRIVATE. But this area can surely be improved
based on the suggestion to make it more generic.
> > > > + u32 blob_id;
> > > > + };
> > > > +
> > > > +2. struct drm_color_pipeline: This structure represents the
> aggregate
> > > > + pipeline to be set. it contains the following members
> > > > + a) num: pipeline number to be
> selected
> > > > + b) size: size of the data to be passed
> onto
> > > the driver
> > > > + c) data: array of struct
> > > drm_color_op_data with data for
> > > > + the hardware block/s that userspace wants to
> > > > + set values for.
> > > > +
> > > > + struct drm_color_pipeline {
> > > > + int num;
> > > > + int size;
> > > > + struct drm_color_op_data *data;
> > > > + };
> > > > +
> > > > + User can either:
> > > > + 1. send data for all the color operations in a pipeline as shown in [2].
> > > > + The color operation data need not be in order that the pipeline
> advertises
> > > > + however, it should not contain data for any
> > > > + color operation that is not present in the pipeline.
> > > > +
> > > > + Note: This check for pipeline is not yet implemented but if the
> > > > + wider proposal is accepted we have few solutions in mind.
> > > > +
> > > > + 2. send data for a subset of color operations as shown in [3].For the
> > > > + color operation that userspace does not send data will retain their
> > > > + older state.
>
> Retaining existing state, especially for operations that userspace does not
> understand, can lead to incorrect and unexpected results. That's why I say that
> userspace must understand all operations in a pipeline, and all parameters of all
> used operations before it can actually use that pipeline.
By retaining state here, we mean the state set by the same client while using the same
pipeline. If client wants to just alter one or subset of the hardware blocks in the pipeline,
it can just send that to driver. Rest of the pipeline which was previously programmed by the
same client will remain intact.
However once the client switches pipeline, driver will disable the pipeline and client will have
to program all the blocks again with the new pipeline.
> Otherwise we have the same problem as KMS properties have in general
> today: when new things are added that userspace does not understand, how will
> the userspace be able to maintain its old behaviour?
>
> That question has two answers today:
> - userspace learns to program everything, and accidentally
> (mal)functions until then
> - you do not switch between KMS clients that might leave incorrect
> state in not-understood properties
>
> Neither is a good answer, and the problem does not seem to have any practical
> traction either.
>
> For color pipelines I hope we can, and believe that we must, do better to maintain
> correct behaviour while extending functionality.
Yes agree, we are thinking to reset and disable the pipeline once client switches.
One of the ideas could be to use file_priv to achieve that.
> > > > +
> > > > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > > > + property to NULL as shown in both [2] and [3]
> > > > +
>
> Is it a reset and disable, or only disable?
It is reset and disable.
> How is the reset state defined, if that state becomes active when the pipeline is
> next enabled and data not set for the operations?
Idea is to set NULL to all blobs in the pipeline and disable the hardware blocks.
New pipeline should allocate new blobs and enable them back.
> > > > + 4. change the color pipeline as demonstrated in [3].
> > > > + On the new pipeline, the user is expected to setup all the
> > > > +color hardware
> > > block
> > > > + Once the user requests a pipeline change, the driver will
> > > > +provide it a clean
> > > slate
> > > > + which means that all the data previously set by the
> > > > + user will be discarded
> > > even if
> > > > + there are common hardware blocks between the
> > > > + two(previous and current)
> > > pipelines.
> > > > +
>
> Yes, alternative pipelines need to be completely independent.
Agree
> > > > +IGT implementation can be found here [4]
> > > > +
> > > > +Representing Fixed function hardware
> > > > +====================================
> > > > +
> > > > +To provide support for fixed function hardware, the driver could
> > > > +expose vendor specific struct drm_color_op with parameters that
> > > > +both the userspace and driver agrees on. To demonstrate, let's
> > > > +consider a hyphothetical fixed function hardware block that converts BT601
> to BT2020.
> > > > +The driver can choose to advertise the block as such.
> > > > +
> > > > +struct drm_color_op color_pipeline_X[] = {
> > > > + {
> > > > + .name = DRM_CB_PRIVATE,
>
> What if the hardware has 5 different custom blocks like this, multiple in the same
> pipeline. How do you 'name' them?
Partially explained above, but private_flags can help in differentiating this.
A link for implementation shared above for reference.
> > > > + .type = FIXED_FUNCTION,
>
> I have been assuming that 'type' uniquely defines both the operation and the
> contents of the parameter blob, but this does not look like it.
> What defines the operation and the parameters?
Statement is true for all other generic blocks, only for private blocks this is a
bit different. Here interpretation depends on the private_flags which can be
documented by the respective vendor for the custom HAL implementation.
> > > > + .blob_id = 45;
> > > > + },
> > > > +}
> > > > +
> > > > +Where the blob represents some vendor specific enums, strings or
> > > > +any other appropriate data types which both the user-space and
> > > > +drivers are aligned
> > > on.
>
> We have a word for that "data ... aligned on": UAPI.
Oh ok.
> > > > +
> > > > +blob:45 {
> > > > + VENDORXXX_BT602_TO_BT2020,
>
> Repeating the question from above, how will userspace know that the contents of
> this blob need to be VENDORXXX_BT602_TO_BT2020 and not something else?
Explained above.
> > > > +};
> > > > +
> > > > +For enabling or passing parameters to such blocks, the user can
> > > > +send data to the driver wrapped within a blob like any other color
> operation block.
> > > > +
> > > > + struct drm_color_op_data {
> > > > + .name = DRM_CB_PRIVATE;
> > > > + .blob_id = 46;
> > > > + } ;
> > > > +
> > > > +where blob with id 46 contains data to enable the fixed function
> hardware(s).
> > > > +
> > >
> > > One major thing missing from the RFC is an explanation on why we
> > > want to go with a prescriptive model, rather than a descriptive
> > > model. This question came up in Dave's response to Simon's RFC:
> > > https://lore.kernel.org/dri-
> > >
> devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmai
> l
> > > .co
> > > m/
> > >
> > > This is a rough attempt at such an explanation:
> > > https://gitlab.freedesktop.org/hwentland/linux/-
> > >
> /merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad
> > > 0b4d
> > > 8d
>
> Hey Harry, that's a good piece!
>
> >
> > Sure Harry, totally agree to this and will include in documentation to
> > highlight the rationale of going with prescriptive model.
>
> Uma, the cover letter had descriptive and prescriptive mixed up.
Agree, thanks for pointing out. Will fix the documentation.
Thanks Pekka for all the pointers.
Regards,
Uma Shankar
>
> Thanks,
> pq
>
> > > Harry
> > >
> > > > +References
> > > > +==========
> > > > +
> > > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > > +[2]
> > > > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev
> > > > +=1
> > > > +[3]
> > > > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev
> > > > +=1 [4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-09-04 13:44 ` Shankar, Uma
0 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-04 13:44 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Borah, Chaitanya Kumar
> -----Original Message-----
> From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> Paalanen
> Sent: Wednesday, August 30, 2023 5:59 PM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> Color Pipeline
>
> On Wed, 30 Aug 2023 08:59:36 +0000
> "Shankar, Uma" <uma.shankar@intel.com> wrote:
>
> > > -----Original Message-----
> > > From: Harry Wentland <harry.wentland@amd.com>
> > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > wayland- devel@lists.freedesktop.org
> > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > Plane Color Pipeline
> > >
> > >
> > >
> > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > >
> > > > Co-developed-by: Chaitanya Kumar Borah
> > > > <chaitanya.kumar.borah@intel.com>
> > > > Signed-off-by: Chaitanya Kumar Borah
> > > > <chaitanya.kumar.borah@intel.com>
> > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > ---
> > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > 1 file changed, 394 insertions(+)
> > > > create mode 100644
> > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > >
> > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > new file mode 100644
> > > > index 000000000000..60ce515b6ea7
> > > > --- /dev/null
> > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
>
> ...
>
> Hi Uma!
Thanks Pekka for the feedback and useful inputs.
> > > > +This color pipeline is then packaged within a blob for the user
> > > > +space to retrieve it. Details can be found in the next section
> > > > +
> > >
> > > Not sure I like blobs that contain other blob ids.
> >
> > It provides flexibility and helps with just one interface to
> > userspace. Its easy to handle and manage once we get the hang of it 😊.
> >
> > We can clearly define the steps of parsing and data structures to be
> > used while interpreting and parsing the blobs.
>
> Don't forget extendability. Possibly every single struct will need some kind of
> versioning, and then it's not simple to parse anymore. Add to that new/old kernel
> vs. old/new userspace, and it seems a bit nightmarish to design.
Structure to be used to interpret the blob should be defined as UAPI only and is not
expected to change once agreed upon. It should be interpreted like a standard property.
So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized
and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI
property should be applied.
New vs old userspace problem exists even today as you rightly highlighted in mail below,
however we are planning to propose that we clean the hardware state once the userspace
client switches or same client switches the pipeline.
> Also since it's records inside a single blob, it's like a new file
> format: every record needs a standard header that allows skipping it
> appropriately if userspace does not understand it, or you need a standard index
> telling where everything is. Making all records the same size would waste space,
> and extendability requires variable size.
The design currently implements 1 hardware block by a struct drm_color_op data structure.
Multiple such blocks make the pipeline. So userspace just needs to get the pipeline and then
parse blocks 1 by 1. For blocks which it doesn't understand it can just skip and move to the
next one. Each block is differentiated by a unique "name" standardized by an enum which will be
part of the UAPI. Thus we will have scope for variable size blob to represent the particular hardware
pipeline, userspace can parse and implement whichever blocks it understands. Only rule defined
by UAPI is the way the respective block is to be parsed and programmed.
> I also would not assume that we can declare a standard set of blocks and that
> nothing else will be needed. The existing hardware is too diverse for that from
> what I have understood. I assume that some hardware have blocks unique to
> them, and they want to at least expose that functionality through a UAPI that
> allows at least generic enumeration of functionality, even if it needs specialized
> userspace code to actually make use of.
Yeah, this is right and for that reason we came up with an idea of DRM_CB_PRIVATE
name for the hardware block. This will tell userspace that this is private hardware block
for a particular hardware vendor. Generic userspace will ignore this block. Vendor specific
HAL or compositor implementation can parse and use this block. To interpret the blob_id
and assign to a respective data structure, private_flags will be used. These private_flags will
be agreed upon by HAL and vendor implementation, generic userspace will ignore them.
For all other cases, private_flags is a don't care field.
> If we go with
>
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type;
> + u32 blob_id;
> + u32 private_flags;
> +};
>
> as in your proposal, I believe it can work (sorry, looking further down, I have
> assumed too much of 'type'), but the enumerations will become long, and the
> details blob_id is still specific to 'type'. This is unavoidable, but we can still choose
> the form between blobs and properties, integers and strings.
>
> I have a feeling that introspection will be valuable here, to help people
> understand what their hardware could do if they had the code to use it. 'name'
> and 'type' being integers require a translation table to strings before they are
> readable, and it would be best if the kernel itself provided that translation.
Name and type can be standardized in enum and well documented in the UAPI.
For all the standard hardware blocks common for all vendors and serving most of the
common usecases, we can define standard names in enum. These can be easily
interpreted by a table as done in many cases already in driver and userspace.
> I don't understand how 'private_flags' could be useful. There must not be any
> "hidden" features. Everything a block can be programmed to do via this UAPI must
> be clearly documented, there cannot be anything private. If two hardware
> versions of a block differ in a meaningful or significant way, they need to be
> exposed as different types of blocks.
The idea of private_flags partially explained above is to have an option for vendors to
expose hardware blocks unique or custom to their hardware. private_flags can help
differentiate incase multiple such hardware blocks exists on the platform.
For how we intend to use it, please refer below patch in the series:
https://patchwork.freedesktop.org/patch/554928/?series=123024&rev=1
Also, IGT showing the usage:
https://patchwork.freedesktop.org/patch/554831/?series=123018&rev=1
> OTOH, if one goes with a (new) DRM object with string named properties model,
> all that struct versioning and file format hassle has mostly a clear and well-
> understood solution. We only need to define the rules of how userspace needs to
> deal with properties or values it does not understand, so that the kernel can keep
> adding more.
With the current blob approach with fixed data structure for parsing, we can generalize
all the hardware blocks and custom implementations. It is future proof with scope to
add any new hardware block in future as well. Rules to interpret, parse, program the
hardware can be defined in the blob approach as well and ideally should be defined and
fixed in the UAPI documentation.
> Therefore, I'm not yet convinced with the "all blobs" design.
Looking forward to collaboratively solve the problem for the community.
Will improve the design based on all the feedback.
> > > > +Exposing a color pipeline to user space
> > > > +=======================================
> > > > +
> > > > +To advertise the available color pipelines, an immutable ENUM
> > > > +property "GET_COLOR_PIPELINE" is introduced.
> > > > +This enum property has blob id's as values. With each blob id
> > > > +representing a distinct color pipeline based on underlying HW
> > > > +capabilities and their respective combinations.
> > > > +
> > > > +The following output of drm_info [1], shows how color pipelines
> > > > +are visible to userspace.
> > > > +
> > > > +├───Plane 0
> > > > + │ ├───Object ID: 31
> > > > + │ ├───CRTCs: {0}
> > > > + │ ├───Legacy info
> > > > + ...
> > > > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color
> pipeline,
> > > > + color pipeline 1, color pipeline
> 2}=
> > > no color pipeline
> > > > +
> > > > +To understand the capabilities of individual pipelines, first the
> > > > +userspace need to retrieve the pipeline blob with the blob ids
> > > > +retrieved by reading the enum property.
> > > > +
> > > > +Once the color pipeline is retrieved, user can then parse through
> > > > +individual drm_color_op blocks to understand the capabilities of
> > > > +each hardware block.
> > > > +
> > > > +Check IGT series to see how user space can parse through color pipelines.
> > > > +Refer the IGT series here:
> > > > +https://patchwork.freedesktop.org/series/123018/
> > > > +
> > > > +Setting up a color pipeline
> > > > +===========================
> > > > +
> > > > +Once the user space decides on a color pipeline, it can set the
> > > > +pipeline and the corresponding data for the hardware blocks
> > > > +within the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > +
> > > > +To achieve this two structures are introduced
> > > > +
> > > > +1. struct drm_color_op_data: It represents data to be passed onto
> individual
> > > > + color hardware
> blocks. It
> > > contains three members
> > > > + a) name: to identify the color operation block
> > > > + b) blob_id: pointing to the blob with data for the
> > > > + corresponding hardware block
> > > > +
> > > > + struct drm_color_op_data {
> > > > + enum color_op_block name;
>
> Why is this a global fixed enum rather than a pipeline specific ordinal or a unique-
> per-device ID?
>
> There is no reason to believe that a 'name' always matches a hardware block 1:1.
> When drivers accumulate multiple different alternative pipelines due to
> backwards-compatibility reasons, the same 'name' could be implemented by
> different hardware blocks, or the same hardware block could implement different
> 'name's from different pipelines.
>
We have some fixed hardware operations in the color pipeline, its good to define and
identify them with name as identifier. For any other custom block we can use the PRIVATE type.
Driver should expose the name of the hardware block for a particular pipeline based on the
actual usage, and as long as that is standardized it should be ok if same hardware block is
named differently across pipeline and vice-versa.
> The names have also a problem. If you name something "pre-csc", then how do
> you name the thing that the next hardware version adds between "pre-csc" and
> "csc"?
Most of the common hardware blocks can be named based on the operation they are intended to
be used for. For some custom blocks it can be defined as PRIVATE. But this area can surely be improved
based on the suggestion to make it more generic.
> > > > + u32 blob_id;
> > > > + };
> > > > +
> > > > +2. struct drm_color_pipeline: This structure represents the
> aggregate
> > > > + pipeline to be set. it contains the following members
> > > > + a) num: pipeline number to be
> selected
> > > > + b) size: size of the data to be passed
> onto
> > > the driver
> > > > + c) data: array of struct
> > > drm_color_op_data with data for
> > > > + the hardware block/s that userspace wants to
> > > > + set values for.
> > > > +
> > > > + struct drm_color_pipeline {
> > > > + int num;
> > > > + int size;
> > > > + struct drm_color_op_data *data;
> > > > + };
> > > > +
> > > > + User can either:
> > > > + 1. send data for all the color operations in a pipeline as shown in [2].
> > > > + The color operation data need not be in order that the pipeline
> advertises
> > > > + however, it should not contain data for any
> > > > + color operation that is not present in the pipeline.
> > > > +
> > > > + Note: This check for pipeline is not yet implemented but if the
> > > > + wider proposal is accepted we have few solutions in mind.
> > > > +
> > > > + 2. send data for a subset of color operations as shown in [3].For the
> > > > + color operation that userspace does not send data will retain their
> > > > + older state.
>
> Retaining existing state, especially for operations that userspace does not
> understand, can lead to incorrect and unexpected results. That's why I say that
> userspace must understand all operations in a pipeline, and all parameters of all
> used operations before it can actually use that pipeline.
By retaining state here, we mean the state set by the same client while using the same
pipeline. If client wants to just alter one or subset of the hardware blocks in the pipeline,
it can just send that to driver. Rest of the pipeline which was previously programmed by the
same client will remain intact.
However once the client switches pipeline, driver will disable the pipeline and client will have
to program all the blocks again with the new pipeline.
> Otherwise we have the same problem as KMS properties have in general
> today: when new things are added that userspace does not understand, how will
> the userspace be able to maintain its old behaviour?
>
> That question has two answers today:
> - userspace learns to program everything, and accidentally
> (mal)functions until then
> - you do not switch between KMS clients that might leave incorrect
> state in not-understood properties
>
> Neither is a good answer, and the problem does not seem to have any practical
> traction either.
>
> For color pipelines I hope we can, and believe that we must, do better to maintain
> correct behaviour while extending functionality.
Yes agree, we are thinking to reset and disable the pipeline once client switches.
One of the ideas could be to use file_priv to achieve that.
> > > > +
> > > > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > > > + property to NULL as shown in both [2] and [3]
> > > > +
>
> Is it a reset and disable, or only disable?
It is reset and disable.
> How is the reset state defined, if that state becomes active when the pipeline is
> next enabled and data not set for the operations?
Idea is to set NULL to all blobs in the pipeline and disable the hardware blocks.
New pipeline should allocate new blobs and enable them back.
> > > > + 4. change the color pipeline as demonstrated in [3].
> > > > + On the new pipeline, the user is expected to setup all the
> > > > +color hardware
> > > block
> > > > + Once the user requests a pipeline change, the driver will
> > > > +provide it a clean
> > > slate
> > > > + which means that all the data previously set by the
> > > > + user will be discarded
> > > even if
> > > > + there are common hardware blocks between the
> > > > + two(previous and current)
> > > pipelines.
> > > > +
>
> Yes, alternative pipelines need to be completely independent.
Agree
> > > > +IGT implementation can be found here [4]
> > > > +
> > > > +Representing Fixed function hardware
> > > > +====================================
> > > > +
> > > > +To provide support for fixed function hardware, the driver could
> > > > +expose vendor specific struct drm_color_op with parameters that
> > > > +both the userspace and driver agrees on. To demonstrate, let's
> > > > +consider a hyphothetical fixed function hardware block that converts BT601
> to BT2020.
> > > > +The driver can choose to advertise the block as such.
> > > > +
> > > > +struct drm_color_op color_pipeline_X[] = {
> > > > + {
> > > > + .name = DRM_CB_PRIVATE,
>
> What if the hardware has 5 different custom blocks like this, multiple in the same
> pipeline. How do you 'name' them?
Partially explained above, but private_flags can help in differentiating this.
A link for implementation shared above for reference.
> > > > + .type = FIXED_FUNCTION,
>
> I have been assuming that 'type' uniquely defines both the operation and the
> contents of the parameter blob, but this does not look like it.
> What defines the operation and the parameters?
Statement is true for all other generic blocks, only for private blocks this is a
bit different. Here interpretation depends on the private_flags which can be
documented by the respective vendor for the custom HAL implementation.
> > > > + .blob_id = 45;
> > > > + },
> > > > +}
> > > > +
> > > > +Where the blob represents some vendor specific enums, strings or
> > > > +any other appropriate data types which both the user-space and
> > > > +drivers are aligned
> > > on.
>
> We have a word for that "data ... aligned on": UAPI.
Oh ok.
> > > > +
> > > > +blob:45 {
> > > > + VENDORXXX_BT602_TO_BT2020,
>
> Repeating the question from above, how will userspace know that the contents of
> this blob need to be VENDORXXX_BT602_TO_BT2020 and not something else?
Explained above.
> > > > +};
> > > > +
> > > > +For enabling or passing parameters to such blocks, the user can
> > > > +send data to the driver wrapped within a blob like any other color
> operation block.
> > > > +
> > > > + struct drm_color_op_data {
> > > > + .name = DRM_CB_PRIVATE;
> > > > + .blob_id = 46;
> > > > + } ;
> > > > +
> > > > +where blob with id 46 contains data to enable the fixed function
> hardware(s).
> > > > +
> > >
> > > One major thing missing from the RFC is an explanation on why we
> > > want to go with a prescriptive model, rather than a descriptive
> > > model. This question came up in Dave's response to Simon's RFC:
> > > https://lore.kernel.org/dri-
> > >
> devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmai
> l
> > > .co
> > > m/
> > >
> > > This is a rough attempt at such an explanation:
> > > https://gitlab.freedesktop.org/hwentland/linux/-
> > >
> /merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad
> > > 0b4d
> > > 8d
>
> Hey Harry, that's a good piece!
>
> >
> > Sure Harry, totally agree to this and will include in documentation to
> > highlight the rationale of going with prescriptive model.
>
> Uma, the cover letter had descriptive and prescriptive mixed up.
Agree, thanks for pointing out. Will fix the documentation.
Thanks Pekka for all the pointers.
Regards,
Uma Shankar
>
> Thanks,
> pq
>
> > > Harry
> > >
> > > > +References
> > > > +==========
> > > > +
> > > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > > +[2]
> > > > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev
> > > > +=1
> > > > +[3]
> > > > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev
> > > > +=1 [4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [RFC 02/33] drm: Add color operation structure
2023-08-30 13:00 ` [Intel-gfx] " Pekka Paalanen
@ 2023-09-04 14:10 ` Shankar, Uma
-1 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-04 14:10 UTC (permalink / raw)
To: Pekka Paalanen
Cc: intel-gfx, dri-devel, Borah, Chaitanya Kumar, wayland-devel
> -----Original Message-----
> From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> Paalanen
> Sent: Wednesday, August 30, 2023 6:30 PM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 02/33] drm: Add color operation structure
>
> On Tue, 29 Aug 2023 21:33:51 +0530
> Uma Shankar <uma.shankar@intel.com> wrote:
>
> > From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> >
> > Each Color Hardware block will be represented uniquely in the color
> > pipeline. Define the structure to represent the same.
> >
> > These color operations will form the building blocks of a color
> > pipeline which best represents the underlying Hardware. Color
> > operations can be re-arranged, substracted or added to create distinct
> > color pipelines to accurately describe the Hardware blocks present in
> > the display engine.
> >
> > Co-developed-by: Uma Shankar <uma.shankar@intel.com>
> > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > ---
> > include/uapi/drm/drm_mode.h | 72
> > +++++++++++++++++++++++++++++++++++++
> > 1 file changed, 72 insertions(+)
> >
> > diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> > index ea1b639bcb28..882479f41745 100644
> > --- a/include/uapi/drm/drm_mode.h
> > +++ b/include/uapi/drm/drm_mode.h
> > @@ -943,6 +943,78 @@ struct hdr_output_metadata {
> > };
> > };
> >
> > +/**
> > + * enum color_op_block
> > + *
> > + * Enums to identify hardware color blocks.
> > + *
> > + * @DRM_CB_PRE_CSC: LUT before the CTM unit
> > + * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
> > + * @DRM_CB_POST_CSC: LUT after the CTM unit
> > + * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
> > + * color components
> > + * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
> > + * can expose a custom hardware by defining a
> > + * color operation block with this name as
> > + * identifier
>
> This naming scheme does not seem to work. It assumes a far too rigid pipeline,
> just like the old KMS property design. What if you have two other operations
> between PRE_CSC and CSC?
>
> What sense do PRE_CSC and POST_CSC make if you don't happen to have a CSC
> operation?
Sure, we can re-look at the naming. However, it will be good to define some standard
operations common to all vendors and keep the rest as vendor private.
> What if a driver put POST_CSC before PRE_CSC in its pipeline?
>
> What if your CSC is actually a series of three independent operations, and in
> addition you have PRE_CSC and POST_CSC?
We should try to standardized the operations as much as possible and leave rest as
vendor private. Current proposal allows us to do that.
> 3D_LUT is an operation category, not a name. The same could be said about
> private.
Sure, will fix this.
> Given that all these are also UAPI, do we also need protect old userspace from
> seeing values it does not understand?
For the values userspace doesn't understand, it can ignore the blocks. We should ensure
that userspace always gets a clean state wrt color hardware state and no baggage from
another client should be there. With that there is no burden of disabling that particular
block will be there on an older userspace.
> > + */
> > +enum color_op_block {
> > + DRM_CB_INVAL = -1,
> > +
> > + DRM_CB_PRE_CSC = 0,
> > + DRM_CB_CSC,
> > + DRM_CB_POST_CSC,
> > + DRM_CB_3D_LUT,
> > +
> > + /* Any new generic hardware block can be updated here */
> > +
> > + /*
> > + * PRIVATE is kept at 255 to make it future proof and leave
> > + * scope for any new addition
> > + */
> > + DRM_CB_PRIVATE = 255,
> > + DRM_CB_MAX = DRM_CB_PRIVATE,
> > +};
> > +
> > +/**
> > + * enum color_op_type
> > + *
> > + * These enums are to identify the mathematical operation that
> > + * a hardware block is capable of.
> > + * @CURVE_1D: It represents a one dimensional lookup table
> > + * @CURVE_3D: Represents lut value for each color component for 3d
> > +lut capable hardware
> > + * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
> > + * @FIXED_FUNCTION: To enable and program any custom fixed function
> > +hardware unit */ enum color_op_type {
> > + CURVE_1D,
> > + CURVE_3D,
> > + MATRIX,
> > + FIXED_FUNCTION,
>
> My assumption was that a color_op_type would clearly and uniquely define the
> mathematical model of the operation and the UABI structure of the parameter
> blob. That means we need different values for uniform vs. exponentially vs.
> programmable distributed 1D LUT, etc.
In the hardware the LUTS are programmed as they are received from userspace.
So once the userspace gets to know the distribution of LUTS, segments, precision,
Number of lut samples, it can create the lut values to be programmed.
This information on the distribution of luts in the hardware can be extracted by the
drm_color_lut_range structure which is exposed as blob in the hardware block with
TYPE set as CURVE_1D.
> If there is a 1D curve with pre-programmed (fixed and named) curves, we need to
> enumerate all the curve types somehow. Probably each fixed curve type should
> not be a different operation type, because that would explode the number of
> alternative pipelines.
Exposing the lut distribution with drm_color_lut_range would do this job.
> A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D
> LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in three dimensional space.
Yes right, we can optimize and fine tune this.
> A matrix element could be with or without an offset vector I guess.
>
> FIXED_FUNCTION would need to be replaced with e.g. your example
> VENDORXXX_BT602_TO_BT2020 to work.
>
> Have I missed something, how did you intend this to work?
FIXED_FUNCTION is for some descriptive implementation. Some cases where
Hardware just have a bit to enable and rest of the implementation is within hardware.
Since type is used for other general hardware blocks, its not intended to cover vendor
specific implementation and a private_flags are used instead.
Thanks Pekka for the feedback.
Regards,
Uma Shankar
>
>
> Thanks,
> pq
>
> > +};
> > +
> > +/**
> > + * @struct drm_color_op
> > + *
> > + * This structure is used to represent the capability of
> > + * individual color hardware blocks.
> > + *
> > + * @name: a standardized enum to identify the color hardware block
> > + * @type: The type of mathematical operation it can perform
> > + * @blob_id: Id pointing to a blob containing information about
> > + * the hardware block which advertizes its capabilities
> > + * to the userspace. It can be an optional field depending
> > + * on the members "name" and "type".
> > + * @private_flags: This can be used to provide vendor specific hints
> > + * to user space
> > + */
> > +struct drm_color_op {
> > + enum color_op_block name;
> > + enum color_op_type type;
> > + __u32 blob_id;
> > + __u32 private_flags;
> > +};
> > +
> > /**
> > * DRM_MODE_PAGE_FLIP_EVENT
> > *
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 02/33] drm: Add color operation structure
@ 2023-09-04 14:10 ` Shankar, Uma
0 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-04 14:10 UTC (permalink / raw)
To: Pekka Paalanen; +Cc: intel-gfx, dri-devel, wayland-devel
> -----Original Message-----
> From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> Paalanen
> Sent: Wednesday, August 30, 2023 6:30 PM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 02/33] drm: Add color operation structure
>
> On Tue, 29 Aug 2023 21:33:51 +0530
> Uma Shankar <uma.shankar@intel.com> wrote:
>
> > From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> >
> > Each Color Hardware block will be represented uniquely in the color
> > pipeline. Define the structure to represent the same.
> >
> > These color operations will form the building blocks of a color
> > pipeline which best represents the underlying Hardware. Color
> > operations can be re-arranged, substracted or added to create distinct
> > color pipelines to accurately describe the Hardware blocks present in
> > the display engine.
> >
> > Co-developed-by: Uma Shankar <uma.shankar@intel.com>
> > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > ---
> > include/uapi/drm/drm_mode.h | 72
> > +++++++++++++++++++++++++++++++++++++
> > 1 file changed, 72 insertions(+)
> >
> > diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> > index ea1b639bcb28..882479f41745 100644
> > --- a/include/uapi/drm/drm_mode.h
> > +++ b/include/uapi/drm/drm_mode.h
> > @@ -943,6 +943,78 @@ struct hdr_output_metadata {
> > };
> > };
> >
> > +/**
> > + * enum color_op_block
> > + *
> > + * Enums to identify hardware color blocks.
> > + *
> > + * @DRM_CB_PRE_CSC: LUT before the CTM unit
> > + * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
> > + * @DRM_CB_POST_CSC: LUT after the CTM unit
> > + * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
> > + * color components
> > + * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
> > + * can expose a custom hardware by defining a
> > + * color operation block with this name as
> > + * identifier
>
> This naming scheme does not seem to work. It assumes a far too rigid pipeline,
> just like the old KMS property design. What if you have two other operations
> between PRE_CSC and CSC?
>
> What sense do PRE_CSC and POST_CSC make if you don't happen to have a CSC
> operation?
Sure, we can re-look at the naming. However, it will be good to define some standard
operations common to all vendors and keep the rest as vendor private.
> What if a driver put POST_CSC before PRE_CSC in its pipeline?
>
> What if your CSC is actually a series of three independent operations, and in
> addition you have PRE_CSC and POST_CSC?
We should try to standardized the operations as much as possible and leave rest as
vendor private. Current proposal allows us to do that.
> 3D_LUT is an operation category, not a name. The same could be said about
> private.
Sure, will fix this.
> Given that all these are also UAPI, do we also need protect old userspace from
> seeing values it does not understand?
For the values userspace doesn't understand, it can ignore the blocks. We should ensure
that userspace always gets a clean state wrt color hardware state and no baggage from
another client should be there. With that there is no burden of disabling that particular
block will be there on an older userspace.
> > + */
> > +enum color_op_block {
> > + DRM_CB_INVAL = -1,
> > +
> > + DRM_CB_PRE_CSC = 0,
> > + DRM_CB_CSC,
> > + DRM_CB_POST_CSC,
> > + DRM_CB_3D_LUT,
> > +
> > + /* Any new generic hardware block can be updated here */
> > +
> > + /*
> > + * PRIVATE is kept at 255 to make it future proof and leave
> > + * scope for any new addition
> > + */
> > + DRM_CB_PRIVATE = 255,
> > + DRM_CB_MAX = DRM_CB_PRIVATE,
> > +};
> > +
> > +/**
> > + * enum color_op_type
> > + *
> > + * These enums are to identify the mathematical operation that
> > + * a hardware block is capable of.
> > + * @CURVE_1D: It represents a one dimensional lookup table
> > + * @CURVE_3D: Represents lut value for each color component for 3d
> > +lut capable hardware
> > + * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
> > + * @FIXED_FUNCTION: To enable and program any custom fixed function
> > +hardware unit */ enum color_op_type {
> > + CURVE_1D,
> > + CURVE_3D,
> > + MATRIX,
> > + FIXED_FUNCTION,
>
> My assumption was that a color_op_type would clearly and uniquely define the
> mathematical model of the operation and the UABI structure of the parameter
> blob. That means we need different values for uniform vs. exponentially vs.
> programmable distributed 1D LUT, etc.
In the hardware the LUTS are programmed as they are received from userspace.
So once the userspace gets to know the distribution of LUTS, segments, precision,
Number of lut samples, it can create the lut values to be programmed.
This information on the distribution of luts in the hardware can be extracted by the
drm_color_lut_range structure which is exposed as blob in the hardware block with
TYPE set as CURVE_1D.
> If there is a 1D curve with pre-programmed (fixed and named) curves, we need to
> enumerate all the curve types somehow. Probably each fixed curve type should
> not be a different operation type, because that would explode the number of
> alternative pipelines.
Exposing the lut distribution with drm_color_lut_range would do this job.
> A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D
> LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in three dimensional space.
Yes right, we can optimize and fine tune this.
> A matrix element could be with or without an offset vector I guess.
>
> FIXED_FUNCTION would need to be replaced with e.g. your example
> VENDORXXX_BT602_TO_BT2020 to work.
>
> Have I missed something, how did you intend this to work?
FIXED_FUNCTION is for some descriptive implementation. Some cases where
Hardware just have a bit to enable and rest of the implementation is within hardware.
Since type is used for other general hardware blocks, its not intended to cover vendor
specific implementation and a private_flags are used instead.
Thanks Pekka for the feedback.
Regards,
Uma Shankar
>
>
> Thanks,
> pq
>
> > +};
> > +
> > +/**
> > + * @struct drm_color_op
> > + *
> > + * This structure is used to represent the capability of
> > + * individual color hardware blocks.
> > + *
> > + * @name: a standardized enum to identify the color hardware block
> > + * @type: The type of mathematical operation it can perform
> > + * @blob_id: Id pointing to a blob containing information about
> > + * the hardware block which advertizes its capabilities
> > + * to the userspace. It can be an optional field depending
> > + * on the members "name" and "type".
> > + * @private_flags: This can be used to provide vendor specific hints
> > + * to user space
> > + */
> > +struct drm_color_op {
> > + enum color_op_block name;
> > + enum color_op_type type;
> > + __u32 blob_id;
> > + __u32 private_flags;
> > +};
> > +
> > /**
> > * DRM_MODE_PAGE_FLIP_EVENT
> > *
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [RFC 00/33] Add Support for Plane Color Pipeline
2023-08-30 21:15 ` [Intel-gfx] " Sebastian Wick
@ 2023-09-04 14:29 ` Shankar, Uma
-1 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-04 14:29 UTC (permalink / raw)
To: Sebastian Wick
Cc: Pekka Paalanen, Shashank Sharma, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Christopher Braga
> -----Original Message-----
> From: Sebastian Wick <sebastian.wick@redhat.com>
> Sent: Thursday, August 31, 2023 2:46 AM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> <quic_cbraga@quicinc.com>
> Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
>
> On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> >
> >
> > > -----Original Message-----
> > > From: Harry Wentland <harry.wentland@amd.com>
> > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > <quic_cbraga@quicinc.com>
> > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > >
> > > +CC Naseer and Chris, FYI
> > >
> > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > >
> > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > Introduction
> > > > ============
> > > >
> > > > Modern hardwares have various color processing capabilities both
> > > > at pre-blending and post-blending phases in the color pipeline.
> > > > The current drm implementation exposes only the post-blending
> > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > There are multiple use cases where pre-blending color hardware
> > > > will be
> > > > useful:
> > > > a) Linearization of input buffers encoded in various transfer
> > > > functions.
> > > > b) Color Space conversion
> > > > c) Tone mapping
> > > > d) Frame buffer format conversion
> > > > e) Non-linearization of buffer(apply transfer function)
> > > > f) 3D Luts
> > > >
> > > > and other miscellaneous color operations.
> > > >
> > > > Hence, there is a need to expose the color capabilities of the
> > > > hardware to user-space. This will help userspace/middleware to use
> > > > display hardware for color processing and blending instead of
> > > > doing it through GPU shaders.
> > > >
> > >
> > > Thanks, Uma, for sending this. I've been working on something
> > > similar but you beat me to it. :)
> >
> > Thanks Harry for the useful feedback and overall collaboration on this so far.
> >
> > > >
> > > > Work done so far and relevant references
> > > > ========================================
> > > >
> > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > Broad consensus is there that we need a generic API at drm core to
> > > > suffice the use case of various HW vendors. Below are the links
> > > > capturing the discussion so far.
> > > >
> > > > Intel's Plane Color Implementation:
> > > > https://patchwork.freedesktop.org/series/90825/
> > > > AMD's Plane Color Implementation:
> > > > https://patchwork.freedesktop.org/series/116862/
> > > >
> > > >
> > > > Hackfest conclusions
> > > > ====================
> > > >
> > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > industry stakeholders together and converge on a common uapi
> expectations.
> > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > and other prominent user-space developers and maintainers.
> > > >
> > > > Discussions happened on the uapi expectations, opens, nature of
> > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > the same and the path forward. Consensus was made that drm core
> > > > should implement descriptive APIs and not go with prescriptive
> > > > APIs. DRM core should just expose the hardware capabilities;
> > > > enabling, customizing and programming the same should be done by
> > > > the user-space. Driver should just
> > > honor the user space request without doing any operations internally.
> > > >
> > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > and an UAPI RFC which can be referred to here:
> > > >
> > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > _hD5
> > > >
> > >
> nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> Q
> > > Wn48
> > > > 8=@emersion.fr/
> > > >
> > > >
> > > > Design considerations
> > > > =====================
> > > >
> > > > Following are the important aspects taken into account while
> > > > designing the current RFC
> > > > proposal:
> > > >
> > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > only one
> > > can be used)
> > > > 2. Position of the HW block in the pipeline can be programmable
> > > > 3. LUTs can be one dimentional or three dimentional
> > > > 4. Number of LUT entries can vary across platforms
> > > > 5. Precision of LUT entries can vary across platforms
> > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> Logarithmic,
> > > > Piece-Wise Linear(PWL) etc
> > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > blocks.
> > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > 8. Custom non-standard HW implementation.
> > > > 9. Leaving scope for some vendor defined pescriptive
> > > > implementation if
> > > required.
> > > > 10.Scope to handle any modification in hardware as technology
> > > > evolves
> > > >
> > > > The current proposal takes into account the above considerations
> > > > while keeping the implementation as generic as possible leaving
> > > > scope for future
> > > additions or modifications.
> > > >
> > > > This proposal is also in line to the details mentioned by Simon's
> > > > RFC covering all the aspects discussed in hackfest.
> > > >
> > > >
> > > > Outline of the implementation
> > > > ============================
> > > >
> > > > Each Color Hardware block will be represented by a data structure
> drm_color_op.
> > > > These color operations will form the building blocks of a color
> > > > pipeline which best represents the underlying Hardware. Color
> > > > operations can be re-arranged, substracted or added to create
> > > > distinct color pipelines to accurately describe the Hardware
> > > > blocks present in the display
> > > engine.
> > >
> > > Who is doing the arranging of color operations? IMO a driver should
> > > define one or more respective pipelines that can be selected by
> > > userspace. This seems to be what you're talking about after (I
> > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> to add clarifications in order to avoid confusion.
> >
> > Yes it's the driver who will set the pipeline based on the underlying
> > hardware arrangement and possible combinations. There can be multiple
> > pipelines possible if hardware can be muxed or order can be re-arranged (all
> viable combinations should be defined as a pipeline in driver).
> > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> >
> > > >
> > > > In this proposal, a color pipeline is represented as an array of
> > > > struct drm_color_op. For individual color operation, we add blobs
> > > > to advertise the capability of the particular Hardware block.
> > > >
> > > > This color pipeline is then packaged within a blob for the user
> > > > space to retrieve it.
> > > >
> > >
> > > Would it be better to expose the drm_color_ops directly, instead of
> > > packing a array of drm_color_ops into a blob and then giving that to
> userspace.
> >
> > Advantage we see in packing in blobs is that interface will be
> > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> and then its just parsing the data.
> > This way the entire underlying hardware blocks with pipeline will be available to
> user.
>
> I don't see how parsing a blob is easier than requesting the color ops from the
> kernel. User space is already equiped with getting KMS objects and the igt test
> code from Harry shows that this is all pretty trivial plumbing.
There are multiple color operations possible with unique lut distribution and
capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
Having all the information with one query and property and also be able to set the
same with just one property call using blobs is better than multiple calls to driver.
This can be useful in high refresh rate cases where not much time is there to program
the hardware state. Latency of multiple calls to driver can be avoided.
> > For a particular hardware block in pipeline, user can get the relevant
> > details from blob representing that particular block. We have created
> > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> This is just to clarify the idea.
>
> The blob is also not introspectable with the usual tools whereas exposing them as
> properties would be.
>
> It also would, like Pekka correctly noted, bring a whole bunch of issues about
> compatibility and versioning that are well understood with objects
> + properties.
The blob should be standardized in the UAPI and structures to parse them should be fixed.
With this compatibility issues can be prevented.
> > Also since info is passed via blobs we have the flexibility to even
> > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > for custom private hardware blocks as well which driver can work with
> respective HAL and get that implemented.
>
> When color ops are real KMS objects they still can have properties which are
> blobs that can store LUTs and other data.
>
> And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> have changed in that regard and it simply is not allowed.
Private blocks are not standardized but are vendor specific. So generic userspace will
ignore these. However vendor and its respective HAL can make use of this field and leaves
a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
standardized color operations which will be defined as enum in UAPI.
> > We can even define prescriptive operations as a private entry and
> > enable it if a certain driver and HAL agree.
> >
> > > > To advertise the available color pipelines, an immutable ENUM
> > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> blob id's as values.
> > > > With each blob id representing a distinct color pipeline based on
> > > > underlying HW capabilities and their respective combinations.
> > > >
> > > > Once the user space decides on a color pipeline, it can set the
> > > > pipeline and the corresponding data for the hardware blocks within
> > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > >
> > >
> > > When I discussed this at the hackfest with Simon he proposed a new
> > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > NULL if its the last in the pipeline. You can then have a mutable
> > > enum property on the plane to discover and select a color pipeline.
> >
> > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> the design.
> > Personally I feel the one proposed in the current RFC will do the same
> > thing envisioned by Simon and you Harry. Management of the pipeline,
> > addition, deletion and flexibility to represent hardware is more with blobs with
> the relevant structures agreed in UAPI.
> >
> > > This seems a bit more transparent than a blob. You can see my
> > > changes (unfortunately very WIP, don't look too closely at
> > > individual patches) at
> > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > ffs
> > >
> > > libdrm changes:
> > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > s
> >
> > Sure, will check the same.
> >
> > > IGT changes:
> > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > sts/1/diffs
> > >
> > > I'll take time to review your whole series and will see whether we
> > > can somehow keep the best parts of each.
> >
> > Thanks and agree. Let's work together and get this implemented in DRM.
> >
> > > Curious to hear other opinions on the blob vs new DRM object question as
> well.
>
> I can see a few advantages with the blob approach.
>
> User space can store the entire state in one blob and just assign a new blob to
> change to another pipeline configuration.
Agree
> However, I would argue that changing a lot of properties is already common
> practice and works well. User space can deal with it and has abstractions to
> make this managable.
Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
Lut distribution, number of segments, samples in each segment, precision of luts etc.
can be precisely defined and userspace will get a complete picture of the underlying
hardware and its capabilities. Also this is being done with just 2 properties. Leaving
scope for future addition of any standard color operation as well.
I feel blob approach once properly documented is a bit more flexible and scalable.
Maybe I am bit biased here 😊 but all ideas are welcome.
We have implemented some IGT's as well to explain the design better. Link below:
https://patchwork.freedesktop.org/series/123018/
Thanks Sebastian for the feedback.
Regards,
Uma Shankar
> There also is no need for a new KMS object kindbu from Harrys work so far the
> new KMS object kind is surprisingly simple and works just as expected, which is a
> good sign.
>
> With the blobs one can store general information about the pipeline and not only
> about the color ops themselves. So far it's not clear if we actually need this, and if
> we do, we could probably use color ops at the start of the pipeline which apply to
> all ops.
>
> There are also a few drawbacks to the blob design. It's not very KMS-y which
> makes working with it different than with the rest of KMS and this mismatch just
> adds more mental burden. It probably also introduces a new set of problems that
> we might not even be aware of. It also makes it less introspectable as Pekka
> noted.
>
> When I came up with the KMS Color Pipeline API, I also started with the blob
> approach because of the flexibility in it but it does add a fair amount of hidden
> complexity. At the hackfest Simon managed to convince me of the idea of using a
> new KMS object kind for it and it seemed to simplify a bunch of things. With
> Harrys work I would argue that it is ineed that case. It's pretty simple and still
> seems to do everything it needs to.
>
> So, yeah, I'm favoring the color op approach and not the blob approach.
>
> >
> > Yeah, request community and stakeholders to share feedback and suggestions.
> > We will work on the feedback to improve the design.
> >
> > > > Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in
> > > > the patch
> > > >
> > > > IGT and test details
> > > > ====================
> > > >
> > > > A set of IGT tests is written to demonstrate the usage of the
> > > > proposed UAPIs along with some sanity validation.
> > > >
> > > > Details of the IGT test can be found here:
> > > > https://patchwork.freedesktop.org/series/123018/
> > > >
> > > > Opens
> > > > =====
> > > >
> > > > a. To come up with a list of common HW blocks which can be defined
> > > > generically
> > > by the DRM
> > > > core in agreement with all the stakeholders b. To
> > > > enhance/finalize the data structure to define segmented LUTs generically.
> > > >
> > >
> > > It would be good to add some basic support in VKMS. My work has been
> > > based on VKMS. Once we kinda settle on an approach I'll look at
> > > exposing the AMD private properties from Melissa through the API.
> >
> > Yeah sure Harry.
>
> I think this is a great idea! Let's see if we can learn some lessons from both patch
> series and then continue with the VKMS implementation and igt tests for it before
> every vendor implements their view of how this should all work.
>
> >
> > > >
> > > > Out of Scope
> > > > ============
> > > >
> > > > a. The coefficients for CTM and LUT value calculations are out of
> > > > scope of the
> > > proposal.
> > > > b. The onus of programming the HW blocks and their values is on user-
> space.
> > > Driver will
> > > > just provide the interface for the same.
> > > > c. In order to compute LUT values and coefficients, a helper
> > > > library can be created
> > > in
> > > > user-space. However, it is out of scope for the current proposal.
> > > >
> > > > Acknowledgements and credits
> > > > ============================
> > > >
> > > > There are multiple contributors who have helped us to reach to
> > > > this proposal. Special mention to Ville
> > > > Syrjala<ville.syrjala@linux.intel.com>, Pekka
> > > > Paalanen<pekka.paalanen@collabora.com>,
> > > > Simon Ser<contact@emersion.fr>, Harry
> > > > Wentland<harry.wentland@amd.com>, Melissa Wen<mwen@igalia.com>,
> > > > Jonas<jadahl@redhat.com>, Sebastian
> > > > Wick<sebastian.wick@redhat.com>,
> > > Bhanu<bhanuprakash.modem@intel.com> and
> > > Shashank<shashank.sharma@amd.com>.
> > > >
> > > > Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team
> > > > for
> > > organizing the HDR hackfest.
> > > >
> > > >
> > > > UAPI dependency and Usermode development
> > > > ========================================
> > > >
> > > > The current KMS implementation requires a user space consumer for
> > > > it to be
> > > accepted upstream.
> > > > Work is ongoing in weston and mutter community to get color
> > > > management and HDR support implemented in the respective stacks.
> > > >
> > >
> > > If we can get AMD properties encoded using a Color Pipeline API we
> > > can probably use gamescope as the userspace vehicle.
> >
> > Yeah, nice.
> >
> > > I'm reviewing this in sequence, so there's a chance I'm missing context.
> > > Please bear with me if some of my comments are answered later in the series.
> >
> > No worries, really appreciate the feedback and support.
> >
> > > Again, thanks for sending this.
> >
> > Always welcome.
> >
> > Regards,
> > Uma Shankar
> >
> > > Harry
> > >
> > > >
> > >
> =================================================================
> ===
> > > ==
> > > > ===========
> > > >
> > > > We have tried to take care of all the scenarios and use-cases
> > > > which possibly could exists in the current proposal. Thanks to
> > > > everyone who has contributed in all color management discussions
> > > > so far. Let's work together to improve the current proposal and
> > > > get this thing implemented in
> > > upstream linux. All the feedback and suggestions to enhance the
> > > design are welcome.
> > > >
> > > > Regards,
> > > > Uma Shankar
> > > > Chaitanya Kumar Borah
> > > >
> > > > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > > > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > > > Cc: Simon Ser <contact@emersion.fr>
> > > > Cc: Harry Wentland <harry.wentland@amd.com>
> > > > Cc: Melissa Wen <mwen@igalia.com>
> > > > Cc: Jonas Ådahl <jadahl@redhat.com>
> > > > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > > > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > > > Cc: Alexander Goins <agoins@nvidia.com>
> > > >
> > > > Chaitanya Kumar Borah (14):
> > > > drm: Add color operation structure
> > > > drm: Add plane get color pipeline property
> > > > drm: Add helper to add color pipeline
> > > > drm: Manage color blob states
> > > > drm: Replace individual color blobs
> > > > drm: Reset pipeline when user sends NULL blob
> > > > drm: Reset plane color state on pipeline switch request
> > > > drm/i915/color: Add HDR plane LUT range data to color pipeline
> > > > drm/i915/color: Add SDR plane LUT range data to color pipeline
> > > > drm/i915/color: Add color pipelines to plane
> > > > drm/i915/color: Create and attach set color pipeline property
> > > > drm/i915/color: Enable plane color features
> > > > drm/i915/color: Add a dummy pipeline with 3D LUT
> > > > drm/i915/color: Add example implementation for vendor specific color
> > > > operation
> > > >
> > > > Uma Shankar (19):
> > > > drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> > > > drm: Add structures for setting color pipeline
> > > > drm: Add set colorpipeline property
> > > > drm: Add Enhanced Gamma LUT precision structure
> > > > drm: Add color lut range structure
> > > > drm: Add color information to plane state
> > > > drm/i915/color: Add lut range for SDR planes
> > > > drm/i915/color: Add lut range for HDR planes
> > > > drm/i915/color: Add color pipeline for HDR planes
> > > > drm/i915/color: Add color pipeline for SDR planes
> > > > drm/i915/color: Add plane color callbacks
> > > > drm/i915/color: Load plane color luts from atomic flip
> > > > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> > > > drm/i915/xelpd: Add register definitions for Plane Degamma
> > > > drm/i915/color: Add color functions for ADL
> > > > drm/i915/color: Program Plane Pre-CSC Registers
> > > > drm/i915/xelpd: Add register definitions for Plane Post CSC
> > > > drm/i915/xelpd: Program Plane Post CSC Registers
> > > > drm/i915/color: Enable Plane CSC
> > > >
> > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> > > > drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> > > > drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> > > > drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> > > > drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> > > > drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> > > > .../drm/i915/display/skl_universal_plane.c | 21 +-
> > > > drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> > > > include/drm/drm_plane.h | 82 +++
> > > > include/uapi/drm/drm_mode.h | 196 +++++
> > > > include/uapi/drm/i915_drm.h | 25 +
> > > > 11 files changed, 1899 insertions(+), 3 deletions(-)
> > > > create mode 100644
> > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > >
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-09-04 14:29 ` Shankar, Uma
0 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-04 14:29 UTC (permalink / raw)
To: Sebastian Wick
Cc: Alexander Goins, Pekka Paalanen, Simon Ser, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Harry Wentland, Christopher Braga
> -----Original Message-----
> From: Sebastian Wick <sebastian.wick@redhat.com>
> Sent: Thursday, August 31, 2023 2:46 AM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> <quic_cbraga@quicinc.com>
> Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
>
> On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> >
> >
> > > -----Original Message-----
> > > From: Harry Wentland <harry.wentland@amd.com>
> > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > <quic_cbraga@quicinc.com>
> > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > >
> > > +CC Naseer and Chris, FYI
> > >
> > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > >
> > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > Introduction
> > > > ============
> > > >
> > > > Modern hardwares have various color processing capabilities both
> > > > at pre-blending and post-blending phases in the color pipeline.
> > > > The current drm implementation exposes only the post-blending
> > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > There are multiple use cases where pre-blending color hardware
> > > > will be
> > > > useful:
> > > > a) Linearization of input buffers encoded in various transfer
> > > > functions.
> > > > b) Color Space conversion
> > > > c) Tone mapping
> > > > d) Frame buffer format conversion
> > > > e) Non-linearization of buffer(apply transfer function)
> > > > f) 3D Luts
> > > >
> > > > and other miscellaneous color operations.
> > > >
> > > > Hence, there is a need to expose the color capabilities of the
> > > > hardware to user-space. This will help userspace/middleware to use
> > > > display hardware for color processing and blending instead of
> > > > doing it through GPU shaders.
> > > >
> > >
> > > Thanks, Uma, for sending this. I've been working on something
> > > similar but you beat me to it. :)
> >
> > Thanks Harry for the useful feedback and overall collaboration on this so far.
> >
> > > >
> > > > Work done so far and relevant references
> > > > ========================================
> > > >
> > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > Broad consensus is there that we need a generic API at drm core to
> > > > suffice the use case of various HW vendors. Below are the links
> > > > capturing the discussion so far.
> > > >
> > > > Intel's Plane Color Implementation:
> > > > https://patchwork.freedesktop.org/series/90825/
> > > > AMD's Plane Color Implementation:
> > > > https://patchwork.freedesktop.org/series/116862/
> > > >
> > > >
> > > > Hackfest conclusions
> > > > ====================
> > > >
> > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > industry stakeholders together and converge on a common uapi
> expectations.
> > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > and other prominent user-space developers and maintainers.
> > > >
> > > > Discussions happened on the uapi expectations, opens, nature of
> > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > the same and the path forward. Consensus was made that drm core
> > > > should implement descriptive APIs and not go with prescriptive
> > > > APIs. DRM core should just expose the hardware capabilities;
> > > > enabling, customizing and programming the same should be done by
> > > > the user-space. Driver should just
> > > honor the user space request without doing any operations internally.
> > > >
> > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > and an UAPI RFC which can be referred to here:
> > > >
> > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > _hD5
> > > >
> > >
> nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> Q
> > > Wn48
> > > > 8=@emersion.fr/
> > > >
> > > >
> > > > Design considerations
> > > > =====================
> > > >
> > > > Following are the important aspects taken into account while
> > > > designing the current RFC
> > > > proposal:
> > > >
> > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > only one
> > > can be used)
> > > > 2. Position of the HW block in the pipeline can be programmable
> > > > 3. LUTs can be one dimentional or three dimentional
> > > > 4. Number of LUT entries can vary across platforms
> > > > 5. Precision of LUT entries can vary across platforms
> > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> Logarithmic,
> > > > Piece-Wise Linear(PWL) etc
> > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > blocks.
> > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > 8. Custom non-standard HW implementation.
> > > > 9. Leaving scope for some vendor defined pescriptive
> > > > implementation if
> > > required.
> > > > 10.Scope to handle any modification in hardware as technology
> > > > evolves
> > > >
> > > > The current proposal takes into account the above considerations
> > > > while keeping the implementation as generic as possible leaving
> > > > scope for future
> > > additions or modifications.
> > > >
> > > > This proposal is also in line to the details mentioned by Simon's
> > > > RFC covering all the aspects discussed in hackfest.
> > > >
> > > >
> > > > Outline of the implementation
> > > > ============================
> > > >
> > > > Each Color Hardware block will be represented by a data structure
> drm_color_op.
> > > > These color operations will form the building blocks of a color
> > > > pipeline which best represents the underlying Hardware. Color
> > > > operations can be re-arranged, substracted or added to create
> > > > distinct color pipelines to accurately describe the Hardware
> > > > blocks present in the display
> > > engine.
> > >
> > > Who is doing the arranging of color operations? IMO a driver should
> > > define one or more respective pipelines that can be selected by
> > > userspace. This seems to be what you're talking about after (I
> > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> to add clarifications in order to avoid confusion.
> >
> > Yes it's the driver who will set the pipeline based on the underlying
> > hardware arrangement and possible combinations. There can be multiple
> > pipelines possible if hardware can be muxed or order can be re-arranged (all
> viable combinations should be defined as a pipeline in driver).
> > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> >
> > > >
> > > > In this proposal, a color pipeline is represented as an array of
> > > > struct drm_color_op. For individual color operation, we add blobs
> > > > to advertise the capability of the particular Hardware block.
> > > >
> > > > This color pipeline is then packaged within a blob for the user
> > > > space to retrieve it.
> > > >
> > >
> > > Would it be better to expose the drm_color_ops directly, instead of
> > > packing a array of drm_color_ops into a blob and then giving that to
> userspace.
> >
> > Advantage we see in packing in blobs is that interface will be
> > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> and then its just parsing the data.
> > This way the entire underlying hardware blocks with pipeline will be available to
> user.
>
> I don't see how parsing a blob is easier than requesting the color ops from the
> kernel. User space is already equiped with getting KMS objects and the igt test
> code from Harry shows that this is all pretty trivial plumbing.
There are multiple color operations possible with unique lut distribution and
capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
Having all the information with one query and property and also be able to set the
same with just one property call using blobs is better than multiple calls to driver.
This can be useful in high refresh rate cases where not much time is there to program
the hardware state. Latency of multiple calls to driver can be avoided.
> > For a particular hardware block in pipeline, user can get the relevant
> > details from blob representing that particular block. We have created
> > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> This is just to clarify the idea.
>
> The blob is also not introspectable with the usual tools whereas exposing them as
> properties would be.
>
> It also would, like Pekka correctly noted, bring a whole bunch of issues about
> compatibility and versioning that are well understood with objects
> + properties.
The blob should be standardized in the UAPI and structures to parse them should be fixed.
With this compatibility issues can be prevented.
> > Also since info is passed via blobs we have the flexibility to even
> > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > for custom private hardware blocks as well which driver can work with
> respective HAL and get that implemented.
>
> When color ops are real KMS objects they still can have properties which are
> blobs that can store LUTs and other data.
>
> And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> have changed in that regard and it simply is not allowed.
Private blocks are not standardized but are vendor specific. So generic userspace will
ignore these. However vendor and its respective HAL can make use of this field and leaves
a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
standardized color operations which will be defined as enum in UAPI.
> > We can even define prescriptive operations as a private entry and
> > enable it if a certain driver and HAL agree.
> >
> > > > To advertise the available color pipelines, an immutable ENUM
> > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> blob id's as values.
> > > > With each blob id representing a distinct color pipeline based on
> > > > underlying HW capabilities and their respective combinations.
> > > >
> > > > Once the user space decides on a color pipeline, it can set the
> > > > pipeline and the corresponding data for the hardware blocks within
> > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > >
> > >
> > > When I discussed this at the hackfest with Simon he proposed a new
> > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > NULL if its the last in the pipeline. You can then have a mutable
> > > enum property on the plane to discover and select a color pipeline.
> >
> > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> the design.
> > Personally I feel the one proposed in the current RFC will do the same
> > thing envisioned by Simon and you Harry. Management of the pipeline,
> > addition, deletion and flexibility to represent hardware is more with blobs with
> the relevant structures agreed in UAPI.
> >
> > > This seems a bit more transparent than a blob. You can see my
> > > changes (unfortunately very WIP, don't look too closely at
> > > individual patches) at
> > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > ffs
> > >
> > > libdrm changes:
> > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > s
> >
> > Sure, will check the same.
> >
> > > IGT changes:
> > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > sts/1/diffs
> > >
> > > I'll take time to review your whole series and will see whether we
> > > can somehow keep the best parts of each.
> >
> > Thanks and agree. Let's work together and get this implemented in DRM.
> >
> > > Curious to hear other opinions on the blob vs new DRM object question as
> well.
>
> I can see a few advantages with the blob approach.
>
> User space can store the entire state in one blob and just assign a new blob to
> change to another pipeline configuration.
Agree
> However, I would argue that changing a lot of properties is already common
> practice and works well. User space can deal with it and has abstractions to
> make this managable.
Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
Lut distribution, number of segments, samples in each segment, precision of luts etc.
can be precisely defined and userspace will get a complete picture of the underlying
hardware and its capabilities. Also this is being done with just 2 properties. Leaving
scope for future addition of any standard color operation as well.
I feel blob approach once properly documented is a bit more flexible and scalable.
Maybe I am bit biased here 😊 but all ideas are welcome.
We have implemented some IGT's as well to explain the design better. Link below:
https://patchwork.freedesktop.org/series/123018/
Thanks Sebastian for the feedback.
Regards,
Uma Shankar
> There also is no need for a new KMS object kindbu from Harrys work so far the
> new KMS object kind is surprisingly simple and works just as expected, which is a
> good sign.
>
> With the blobs one can store general information about the pipeline and not only
> about the color ops themselves. So far it's not clear if we actually need this, and if
> we do, we could probably use color ops at the start of the pipeline which apply to
> all ops.
>
> There are also a few drawbacks to the blob design. It's not very KMS-y which
> makes working with it different than with the rest of KMS and this mismatch just
> adds more mental burden. It probably also introduces a new set of problems that
> we might not even be aware of. It also makes it less introspectable as Pekka
> noted.
>
> When I came up with the KMS Color Pipeline API, I also started with the blob
> approach because of the flexibility in it but it does add a fair amount of hidden
> complexity. At the hackfest Simon managed to convince me of the idea of using a
> new KMS object kind for it and it seemed to simplify a bunch of things. With
> Harrys work I would argue that it is ineed that case. It's pretty simple and still
> seems to do everything it needs to.
>
> So, yeah, I'm favoring the color op approach and not the blob approach.
>
> >
> > Yeah, request community and stakeholders to share feedback and suggestions.
> > We will work on the feedback to improve the design.
> >
> > > > Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in
> > > > the patch
> > > >
> > > > IGT and test details
> > > > ====================
> > > >
> > > > A set of IGT tests is written to demonstrate the usage of the
> > > > proposed UAPIs along with some sanity validation.
> > > >
> > > > Details of the IGT test can be found here:
> > > > https://patchwork.freedesktop.org/series/123018/
> > > >
> > > > Opens
> > > > =====
> > > >
> > > > a. To come up with a list of common HW blocks which can be defined
> > > > generically
> > > by the DRM
> > > > core in agreement with all the stakeholders b. To
> > > > enhance/finalize the data structure to define segmented LUTs generically.
> > > >
> > >
> > > It would be good to add some basic support in VKMS. My work has been
> > > based on VKMS. Once we kinda settle on an approach I'll look at
> > > exposing the AMD private properties from Melissa through the API.
> >
> > Yeah sure Harry.
>
> I think this is a great idea! Let's see if we can learn some lessons from both patch
> series and then continue with the VKMS implementation and igt tests for it before
> every vendor implements their view of how this should all work.
>
> >
> > > >
> > > > Out of Scope
> > > > ============
> > > >
> > > > a. The coefficients for CTM and LUT value calculations are out of
> > > > scope of the
> > > proposal.
> > > > b. The onus of programming the HW blocks and their values is on user-
> space.
> > > Driver will
> > > > just provide the interface for the same.
> > > > c. In order to compute LUT values and coefficients, a helper
> > > > library can be created
> > > in
> > > > user-space. However, it is out of scope for the current proposal.
> > > >
> > > > Acknowledgements and credits
> > > > ============================
> > > >
> > > > There are multiple contributors who have helped us to reach to
> > > > this proposal. Special mention to Ville
> > > > Syrjala<ville.syrjala@linux.intel.com>, Pekka
> > > > Paalanen<pekka.paalanen@collabora.com>,
> > > > Simon Ser<contact@emersion.fr>, Harry
> > > > Wentland<harry.wentland@amd.com>, Melissa Wen<mwen@igalia.com>,
> > > > Jonas<jadahl@redhat.com>, Sebastian
> > > > Wick<sebastian.wick@redhat.com>,
> > > Bhanu<bhanuprakash.modem@intel.com> and
> > > Shashank<shashank.sharma@amd.com>.
> > > >
> > > > Also, thanks to Carlos <csoriano@redhat.com> and the Redhat team
> > > > for
> > > organizing the HDR hackfest.
> > > >
> > > >
> > > > UAPI dependency and Usermode development
> > > > ========================================
> > > >
> > > > The current KMS implementation requires a user space consumer for
> > > > it to be
> > > accepted upstream.
> > > > Work is ongoing in weston and mutter community to get color
> > > > management and HDR support implemented in the respective stacks.
> > > >
> > >
> > > If we can get AMD properties encoded using a Color Pipeline API we
> > > can probably use gamescope as the userspace vehicle.
> >
> > Yeah, nice.
> >
> > > I'm reviewing this in sequence, so there's a chance I'm missing context.
> > > Please bear with me if some of my comments are answered later in the series.
> >
> > No worries, really appreciate the feedback and support.
> >
> > > Again, thanks for sending this.
> >
> > Always welcome.
> >
> > Regards,
> > Uma Shankar
> >
> > > Harry
> > >
> > > >
> > >
> =================================================================
> ===
> > > ==
> > > > ===========
> > > >
> > > > We have tried to take care of all the scenarios and use-cases
> > > > which possibly could exists in the current proposal. Thanks to
> > > > everyone who has contributed in all color management discussions
> > > > so far. Let's work together to improve the current proposal and
> > > > get this thing implemented in
> > > upstream linux. All the feedback and suggestions to enhance the
> > > design are welcome.
> > > >
> > > > Regards,
> > > > Uma Shankar
> > > > Chaitanya Kumar Borah
> > > >
> > > > Cc: Ville Syrjala <ville.syrjala@linux.intel.com>
> > > > Cc: Pekka Paalanen <pekka.paalanen@collabora.com>
> > > > Cc: Simon Ser <contact@emersion.fr>
> > > > Cc: Harry Wentland <harry.wentland@amd.com>
> > > > Cc: Melissa Wen <mwen@igalia.com>
> > > > Cc: Jonas Ådahl <jadahl@redhat.com>
> > > > Cc: Sebastian Wick <sebastian.wick@redhat.com>
> > > > Cc: Shashank Sharma <shashank.sharma@amd.com>
> > > > Cc: Alexander Goins <agoins@nvidia.com>
> > > >
> > > > Chaitanya Kumar Borah (14):
> > > > drm: Add color operation structure
> > > > drm: Add plane get color pipeline property
> > > > drm: Add helper to add color pipeline
> > > > drm: Manage color blob states
> > > > drm: Replace individual color blobs
> > > > drm: Reset pipeline when user sends NULL blob
> > > > drm: Reset plane color state on pipeline switch request
> > > > drm/i915/color: Add HDR plane LUT range data to color pipeline
> > > > drm/i915/color: Add SDR plane LUT range data to color pipeline
> > > > drm/i915/color: Add color pipelines to plane
> > > > drm/i915/color: Create and attach set color pipeline property
> > > > drm/i915/color: Enable plane color features
> > > > drm/i915/color: Add a dummy pipeline with 3D LUT
> > > > drm/i915/color: Add example implementation for vendor specific color
> > > > operation
> > > >
> > > > Uma Shankar (19):
> > > > drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> > > > drm: Add structures for setting color pipeline
> > > > drm: Add set colorpipeline property
> > > > drm: Add Enhanced Gamma LUT precision structure
> > > > drm: Add color lut range structure
> > > > drm: Add color information to plane state
> > > > drm/i915/color: Add lut range for SDR planes
> > > > drm/i915/color: Add lut range for HDR planes
> > > > drm/i915/color: Add color pipeline for HDR planes
> > > > drm/i915/color: Add color pipeline for SDR planes
> > > > drm/i915/color: Add plane color callbacks
> > > > drm/i915/color: Load plane color luts from atomic flip
> > > > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
> > > > drm/i915/xelpd: Add register definitions for Plane Degamma
> > > > drm/i915/color: Add color functions for ADL
> > > > drm/i915/color: Program Plane Pre-CSC Registers
> > > > drm/i915/xelpd: Add register definitions for Plane Post CSC
> > > > drm/i915/xelpd: Program Plane Post CSC Registers
> > > > drm/i915/color: Enable Plane CSC
> > > >
> > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++
> > > > drivers/gpu/drm/drm_atomic_state_helper.c | 21 +
> > > > drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++
> > > > drivers/gpu/drm/drm_color_mgmt.c | 130 ++++
> > > > drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++-
> > > > drivers/gpu/drm/i915/display/intel_color.h | 7 +-
> > > > .../drm/i915/display/skl_universal_plane.c | 21 +-
> > > > drivers/gpu/drm/i915/i915_reg.h | 124 ++++
> > > > include/drm/drm_plane.h | 82 +++
> > > > include/uapi/drm/drm_mode.h | 196 +++++
> > > > include/uapi/drm/i915_drm.h | 25 +
> > > > 11 files changed, 1899 insertions(+), 3 deletions(-)
> > > > create mode 100644
> > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > >
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-09-04 13:44 ` Shankar, Uma
@ 2023-09-05 11:32 ` Pekka Paalanen
-1 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-05 11:32 UTC (permalink / raw)
To: Shankar, Uma
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Borah, Chaitanya Kumar
[-- Attachment #1: Type: text/plain, Size: 13233 bytes --]
On Mon, 4 Sep 2023 13:44:49 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> > Paalanen
> > Sent: Wednesday, August 30, 2023 5:59 PM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> > Color Pipeline
> >
> > On Wed, 30 Aug 2023 08:59:36 +0000
> > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> >
> > > > -----Original Message-----
> > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > > wayland- devel@lists.freedesktop.org
> > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > > Plane Color Pipeline
> > > >
> > > >
> > > >
> > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > >
> > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > <chaitanya.kumar.borah@intel.com>
> > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > <chaitanya.kumar.borah@intel.com>
> > > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > > ---
> > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > > 1 file changed, 394 insertions(+)
> > > > > create mode 100644
> > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > >
> > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > new file mode 100644
> > > > > index 000000000000..60ce515b6ea7
> > > > > --- /dev/null
> > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> >
> > ...
> >
> > Hi Uma!
>
> Thanks Pekka for the feedback and useful inputs.
Hi Uma,
sorry to say, but the overall feeling I get from this proposal is that
it is just the current color related KMS properties wrapped in a
pipeline blob. That is not the re-design I believe we are looking for,
and the feeling is based on several details that are just copied from
the current property design. Also the "private" stuff has to go.
All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying
LUT tap distribution, and parametrized curves are good development, but
right now we are looking at things one step higher level: the overall
color pipeline design and how to represent any operation. Most of this
series is considering details below the current attention level, hence
I'm paying attention only to the first few patches.
More below.
> > > > > +This color pipeline is then packaged within a blob for the user
> > > > > +space to retrieve it. Details can be found in the next section
> > > > > +
> > > >
> > > > Not sure I like blobs that contain other blob ids.
> > >
> > > It provides flexibility and helps with just one interface to
> > > userspace. Its easy to handle and manage once we get the hang of it 😊.
> > >
> > > We can clearly define the steps of parsing and data structures to be
> > > used while interpreting and parsing the blobs.
> >
> > Don't forget extendability. Possibly every single struct will need some kind of
> > versioning, and then it's not simple to parse anymore. Add to that new/old kernel
> > vs. old/new userspace, and it seems a bit nightmarish to design.
>
> Structure to be used to interpret the blob should be defined as UAPI only and is not
> expected to change once agreed upon. It should be interpreted like a standard property.
> So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized
> and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI
> property should be applied.
That sounds like a UAPI that cannot be extended, either. So in a few
years we'd be looking at replacing it. Or maybe you are just
re-inventing the KMS object property system as blobs?
Replacing a single KMS property is much easier than replacing the
foundations of the whole color pipeline design.
> ...
> > I have a feeling that introspection will be valuable here, to help people
> > understand what their hardware could do if they had the code to use it. 'name'
> > and 'type' being integers require a translation table to strings before they are
> > readable, and it would be best if the kernel itself provided that translation.
>
> Name and type can be standardized in enum and well documented in the UAPI.
> For all the standard hardware blocks common for all vendors and serving most of the
> common usecases, we can define standard names in enum. These can be easily
> interpreted by a table as done in many cases already in driver and userspace.
Yeah, but it won't help with the type-specific parameter blobs that
are totally custom per each operation type. With the KMS property
system we could have more generic introspection into those as well, at
least naming all the parameters they have.
Plus, you don't have to patch drm_info every time something new gets
added in the kernel in order to see it named.
> ...
> > Therefore, I'm not yet convinced with the "all blobs" design.
>
> Looking forward to collaboratively solve the problem for the community.
> Will improve the design based on all the feedback.
I have the feeling that we should also see Harry's draft.
Intel's special LUT type is a whole another story, and we can always
have that fitted with any UAPI paradigm.
> ...
> > > > > + u32 blob_id;
> > > > > + };
> > > > > +
> > > > > +2. struct drm_color_pipeline: This structure represents the
> > aggregate
> > > > > + pipeline to be set. it contains the following members
> > > > > + a) num: pipeline number to be
> > selected
> > > > > + b) size: size of the data to be passed
> > onto
> > > > the driver
> > > > > + c) data: array of struct
> > > > drm_color_op_data with data for
> > > > > + the hardware block/s that userspace wants to
> > > > > + set values for.
> > > > > +
> > > > > + struct drm_color_pipeline {
> > > > > + int num;
> > > > > + int size;
> > > > > + struct drm_color_op_data *data;
> > > > > + };
> > > > > +
> > > > > + User can either:
> > > > > + 1. send data for all the color operations in a pipeline as shown in [2].
> > > > > + The color operation data need not be in order that the pipeline
> > advertises
> > > > > + however, it should not contain data for any
> > > > > + color operation that is not present in the pipeline.
> > > > > +
> > > > > + Note: This check for pipeline is not yet implemented but if the
> > > > > + wider proposal is accepted we have few solutions in mind.
> > > > > +
> > > > > + 2. send data for a subset of color operations as shown in [3].For the
> > > > > + color operation that userspace does not send data will retain their
> > > > > + older state.
> >
> > Retaining existing state, especially for operations that userspace does not
> > understand, can lead to incorrect and unexpected results. That's why I say that
> > userspace must understand all operations in a pipeline, and all parameters of all
> > used operations before it can actually use that pipeline.
>
> By retaining state here, we mean the state set by the same client while using the same
> pipeline. If client wants to just alter one or subset of the hardware blocks in the pipeline,
> it can just send that to driver. Rest of the pipeline which was previously programmed by the
> same client will remain intact.
>
> However once the client switches pipeline, driver will disable the pipeline and client will have
> to program all the blocks again with the new pipeline.
>
> > Otherwise we have the same problem as KMS properties have in general
> > today: when new things are added that userspace does not understand, how will
> > the userspace be able to maintain its old behaviour?
> >
> > That question has two answers today:
> > - userspace learns to program everything, and accidentally
> > (mal)functions until then
> > - you do not switch between KMS clients that might leave incorrect
> > state in not-understood properties
> >
> > Neither is a good answer, and the problem does not seem to have any practical
> > traction either.
> >
> > For color pipelines I hope we can, and believe that we must, do better to maintain
> > correct behaviour while extending functionality.
>
> Yes agree, we are thinking to reset and disable the pipeline once client switches.
> One of the ideas could be to use file_priv to achieve that.
I would assume that each color operation in each pipeline is already
independent. Userspace cannot know which color operation is mapped
to which hardware block, and should not need to care. That means the
kernel maintains software state with the UAPI objects, not with the
hardware blocks. Therefore, there is no inherent need to reset any
pipeline on switch. Hardware programming is a different matter.
In the hackfest or after it, people raised the possibility of having a
standard property in each KMS colorop object if possible: boolean
"pass-through", or any equivalent. If an object is set to pass-through,
it is as if it did not exist - it does not alter the values going
through it in any way. Not even clamping if that could make a
difference.
This allows adding new arbitrary blocks in old pipelines without
forcing userspace to abandon the pipeline, as long as userspace learns
to use the "pass-through" from the start. This reduces the number of
alternative pipelines that need to be advertised when adding new
features.
If an object is not pass-through, then userspace absolutely must
understand the values programmed into every property of that object, or
there is no knowing what the object actually does. If userspace does
not know what an object does, the object can be assumed to produce an
unexpected result, i.e. wrong result. No userspace is going to willingly
malfunction.
> ...
> > > > > +Representing Fixed function hardware
> > > > > +====================================
> > > > > +
> > > > > +To provide support for fixed function hardware, the driver could
> > > > > +expose vendor specific struct drm_color_op with parameters that
> > > > > +both the userspace and driver agrees on. To demonstrate, let's
> > > > > +consider a hyphothetical fixed function hardware block that converts BT601
> > to BT2020.
> > > > > +The driver can choose to advertise the block as such.
> > > > > +
> > > > > +struct drm_color_op color_pipeline_X[] = {
> > > > > + {
> > > > > + .name = DRM_CB_PRIVATE,
> >
> > What if the hardware has 5 different custom blocks like this, multiple in the same
> > pipeline. How do you 'name' them?
>
> Partially explained above, but private_flags can help in differentiating this.
> A link for implementation shared above for reference.
But they are all CB_PRIVATE, so what positions do they take in the pipeline?
The names CB_PRE_CSC and CB_POST_CSC imply that the name defines the
position in a pipeline. That is wrong with the names since the order is
defined by the record order in a pipeline blob, right?
> > > > > + .type = FIXED_FUNCTION,
> >
> > I have been assuming that 'type' uniquely defines both the operation and the
> > contents of the parameter blob, but this does not look like it.
> > What defines the operation and the parameters?
>
> Statement is true for all other generic blocks, only for private blocks this is a
> bit different. Here interpretation depends on the private_flags which can be
> documented by the respective vendor for the custom HAL implementation.
Exceptions to a rule are bad API design. In this case, it can easily be
avoided.
Now I'm actually confused about how 'name' and 'type' depend on
'private_flags'. You have to decipher 'private_flags' to understand
what 'name' means too?
Hmm, but isn't 'name' used for identifying the block/operation in the
blob that sets up the parameters for a whole pipeline? But then 'name'
does not uniquely identify a block?
Thanks,
pq
> ...
> > > > > +References
> > > > > +==========
> > > > > +
> > > > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > > > +[2]
> > > > > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev
> > > > > +=1
> > > > > +[3]
> > > > > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev
> > > > > +=1 [4] https://patchwork.freedesktop.org/series/123018/
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-09-05 11:32 ` Pekka Paalanen
0 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-05 11:32 UTC (permalink / raw)
To: Shankar, Uma
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Harry Wentland
[-- Attachment #1: Type: text/plain, Size: 13233 bytes --]
On Mon, 4 Sep 2023 13:44:49 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> > Paalanen
> > Sent: Wednesday, August 30, 2023 5:59 PM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> > Color Pipeline
> >
> > On Wed, 30 Aug 2023 08:59:36 +0000
> > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> >
> > > > -----Original Message-----
> > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > > wayland- devel@lists.freedesktop.org
> > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > > Plane Color Pipeline
> > > >
> > > >
> > > >
> > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > >
> > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > <chaitanya.kumar.borah@intel.com>
> > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > <chaitanya.kumar.borah@intel.com>
> > > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > > ---
> > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > > 1 file changed, 394 insertions(+)
> > > > > create mode 100644
> > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > >
> > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > new file mode 100644
> > > > > index 000000000000..60ce515b6ea7
> > > > > --- /dev/null
> > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> >
> > ...
> >
> > Hi Uma!
>
> Thanks Pekka for the feedback and useful inputs.
Hi Uma,
sorry to say, but the overall feeling I get from this proposal is that
it is just the current color related KMS properties wrapped in a
pipeline blob. That is not the re-design I believe we are looking for,
and the feeling is based on several details that are just copied from
the current property design. Also the "private" stuff has to go.
All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying
LUT tap distribution, and parametrized curves are good development, but
right now we are looking at things one step higher level: the overall
color pipeline design and how to represent any operation. Most of this
series is considering details below the current attention level, hence
I'm paying attention only to the first few patches.
More below.
> > > > > +This color pipeline is then packaged within a blob for the user
> > > > > +space to retrieve it. Details can be found in the next section
> > > > > +
> > > >
> > > > Not sure I like blobs that contain other blob ids.
> > >
> > > It provides flexibility and helps with just one interface to
> > > userspace. Its easy to handle and manage once we get the hang of it 😊.
> > >
> > > We can clearly define the steps of parsing and data structures to be
> > > used while interpreting and parsing the blobs.
> >
> > Don't forget extendability. Possibly every single struct will need some kind of
> > versioning, and then it's not simple to parse anymore. Add to that new/old kernel
> > vs. old/new userspace, and it seems a bit nightmarish to design.
>
> Structure to be used to interpret the blob should be defined as UAPI only and is not
> expected to change once agreed upon. It should be interpreted like a standard property.
> So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized
> and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI
> property should be applied.
That sounds like a UAPI that cannot be extended, either. So in a few
years we'd be looking at replacing it. Or maybe you are just
re-inventing the KMS object property system as blobs?
Replacing a single KMS property is much easier than replacing the
foundations of the whole color pipeline design.
> ...
> > I have a feeling that introspection will be valuable here, to help people
> > understand what their hardware could do if they had the code to use it. 'name'
> > and 'type' being integers require a translation table to strings before they are
> > readable, and it would be best if the kernel itself provided that translation.
>
> Name and type can be standardized in enum and well documented in the UAPI.
> For all the standard hardware blocks common for all vendors and serving most of the
> common usecases, we can define standard names in enum. These can be easily
> interpreted by a table as done in many cases already in driver and userspace.
Yeah, but it won't help with the type-specific parameter blobs that
are totally custom per each operation type. With the KMS property
system we could have more generic introspection into those as well, at
least naming all the parameters they have.
Plus, you don't have to patch drm_info every time something new gets
added in the kernel in order to see it named.
> ...
> > Therefore, I'm not yet convinced with the "all blobs" design.
>
> Looking forward to collaboratively solve the problem for the community.
> Will improve the design based on all the feedback.
I have the feeling that we should also see Harry's draft.
Intel's special LUT type is a whole another story, and we can always
have that fitted with any UAPI paradigm.
> ...
> > > > > + u32 blob_id;
> > > > > + };
> > > > > +
> > > > > +2. struct drm_color_pipeline: This structure represents the
> > aggregate
> > > > > + pipeline to be set. it contains the following members
> > > > > + a) num: pipeline number to be
> > selected
> > > > > + b) size: size of the data to be passed
> > onto
> > > > the driver
> > > > > + c) data: array of struct
> > > > drm_color_op_data with data for
> > > > > + the hardware block/s that userspace wants to
> > > > > + set values for.
> > > > > +
> > > > > + struct drm_color_pipeline {
> > > > > + int num;
> > > > > + int size;
> > > > > + struct drm_color_op_data *data;
> > > > > + };
> > > > > +
> > > > > + User can either:
> > > > > + 1. send data for all the color operations in a pipeline as shown in [2].
> > > > > + The color operation data need not be in order that the pipeline
> > advertises
> > > > > + however, it should not contain data for any
> > > > > + color operation that is not present in the pipeline.
> > > > > +
> > > > > + Note: This check for pipeline is not yet implemented but if the
> > > > > + wider proposal is accepted we have few solutions in mind.
> > > > > +
> > > > > + 2. send data for a subset of color operations as shown in [3].For the
> > > > > + color operation that userspace does not send data will retain their
> > > > > + older state.
> >
> > Retaining existing state, especially for operations that userspace does not
> > understand, can lead to incorrect and unexpected results. That's why I say that
> > userspace must understand all operations in a pipeline, and all parameters of all
> > used operations before it can actually use that pipeline.
>
> By retaining state here, we mean the state set by the same client while using the same
> pipeline. If client wants to just alter one or subset of the hardware blocks in the pipeline,
> it can just send that to driver. Rest of the pipeline which was previously programmed by the
> same client will remain intact.
>
> However once the client switches pipeline, driver will disable the pipeline and client will have
> to program all the blocks again with the new pipeline.
>
> > Otherwise we have the same problem as KMS properties have in general
> > today: when new things are added that userspace does not understand, how will
> > the userspace be able to maintain its old behaviour?
> >
> > That question has two answers today:
> > - userspace learns to program everything, and accidentally
> > (mal)functions until then
> > - you do not switch between KMS clients that might leave incorrect
> > state in not-understood properties
> >
> > Neither is a good answer, and the problem does not seem to have any practical
> > traction either.
> >
> > For color pipelines I hope we can, and believe that we must, do better to maintain
> > correct behaviour while extending functionality.
>
> Yes agree, we are thinking to reset and disable the pipeline once client switches.
> One of the ideas could be to use file_priv to achieve that.
I would assume that each color operation in each pipeline is already
independent. Userspace cannot know which color operation is mapped
to which hardware block, and should not need to care. That means the
kernel maintains software state with the UAPI objects, not with the
hardware blocks. Therefore, there is no inherent need to reset any
pipeline on switch. Hardware programming is a different matter.
In the hackfest or after it, people raised the possibility of having a
standard property in each KMS colorop object if possible: boolean
"pass-through", or any equivalent. If an object is set to pass-through,
it is as if it did not exist - it does not alter the values going
through it in any way. Not even clamping if that could make a
difference.
This allows adding new arbitrary blocks in old pipelines without
forcing userspace to abandon the pipeline, as long as userspace learns
to use the "pass-through" from the start. This reduces the number of
alternative pipelines that need to be advertised when adding new
features.
If an object is not pass-through, then userspace absolutely must
understand the values programmed into every property of that object, or
there is no knowing what the object actually does. If userspace does
not know what an object does, the object can be assumed to produce an
unexpected result, i.e. wrong result. No userspace is going to willingly
malfunction.
> ...
> > > > > +Representing Fixed function hardware
> > > > > +====================================
> > > > > +
> > > > > +To provide support for fixed function hardware, the driver could
> > > > > +expose vendor specific struct drm_color_op with parameters that
> > > > > +both the userspace and driver agrees on. To demonstrate, let's
> > > > > +consider a hyphothetical fixed function hardware block that converts BT601
> > to BT2020.
> > > > > +The driver can choose to advertise the block as such.
> > > > > +
> > > > > +struct drm_color_op color_pipeline_X[] = {
> > > > > + {
> > > > > + .name = DRM_CB_PRIVATE,
> >
> > What if the hardware has 5 different custom blocks like this, multiple in the same
> > pipeline. How do you 'name' them?
>
> Partially explained above, but private_flags can help in differentiating this.
> A link for implementation shared above for reference.
But they are all CB_PRIVATE, so what positions do they take in the pipeline?
The names CB_PRE_CSC and CB_POST_CSC imply that the name defines the
position in a pipeline. That is wrong with the names since the order is
defined by the record order in a pipeline blob, right?
> > > > > + .type = FIXED_FUNCTION,
> >
> > I have been assuming that 'type' uniquely defines both the operation and the
> > contents of the parameter blob, but this does not look like it.
> > What defines the operation and the parameters?
>
> Statement is true for all other generic blocks, only for private blocks this is a
> bit different. Here interpretation depends on the private_flags which can be
> documented by the respective vendor for the custom HAL implementation.
Exceptions to a rule are bad API design. In this case, it can easily be
avoided.
Now I'm actually confused about how 'name' and 'type' depend on
'private_flags'. You have to decipher 'private_flags' to understand
what 'name' means too?
Hmm, but isn't 'name' used for identifying the block/operation in the
blob that sets up the parameters for a whole pipeline? But then 'name'
does not uniquely identify a block?
Thanks,
pq
> ...
> > > > > +References
> > > > > +==========
> > > > > +
> > > > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > > > +[2]
> > > > > +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev
> > > > > +=1
> > > > > +[3]
> > > > > +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev
> > > > > +=1 [4] https://patchwork.freedesktop.org/series/123018/
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 02/33] drm: Add color operation structure
2023-09-04 14:10 ` [Intel-gfx] " Shankar, Uma
@ 2023-09-05 11:33 ` Pekka Paalanen
-1 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-05 11:33 UTC (permalink / raw)
To: Shankar, Uma; +Cc: intel-gfx, dri-devel, Borah, Chaitanya Kumar, wayland-devel
[-- Attachment #1: Type: text/plain, Size: 10114 bytes --]
On Mon, 4 Sep 2023 14:10:05 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> > Paalanen
> > Sent: Wednesday, August 30, 2023 6:30 PM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org
> > Subject: Re: [RFC 02/33] drm: Add color operation structure
> >
> > On Tue, 29 Aug 2023 21:33:51 +0530
> > Uma Shankar <uma.shankar@intel.com> wrote:
> >
> > > From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > >
> > > Each Color Hardware block will be represented uniquely in the color
> > > pipeline. Define the structure to represent the same.
> > >
> > > These color operations will form the building blocks of a color
> > > pipeline which best represents the underlying Hardware. Color
> > > operations can be re-arranged, substracted or added to create distinct
> > > color pipelines to accurately describe the Hardware blocks present in
> > > the display engine.
> > >
> > > Co-developed-by: Uma Shankar <uma.shankar@intel.com>
> > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > > ---
> > > include/uapi/drm/drm_mode.h | 72
> > > +++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 72 insertions(+)
> > >
> > > diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> > > index ea1b639bcb28..882479f41745 100644
> > > --- a/include/uapi/drm/drm_mode.h
> > > +++ b/include/uapi/drm/drm_mode.h
> > > @@ -943,6 +943,78 @@ struct hdr_output_metadata {
> > > };
> > > };
> > >
> > > +/**
> > > + * enum color_op_block
> > > + *
> > > + * Enums to identify hardware color blocks.
> > > + *
> > > + * @DRM_CB_PRE_CSC: LUT before the CTM unit
> > > + * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
> > > + * @DRM_CB_POST_CSC: LUT after the CTM unit
> > > + * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
> > > + * color components
> > > + * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
> > > + * can expose a custom hardware by defining a
> > > + * color operation block with this name as
> > > + * identifier
> >
> > This naming scheme does not seem to work. It assumes a far too rigid pipeline,
> > just like the old KMS property design. What if you have two other operations
> > between PRE_CSC and CSC?
> >
> > What sense do PRE_CSC and POST_CSC make if you don't happen to have a CSC
> > operation?
>
> Sure, we can re-look at the naming. However, it will be good to define some standard
> operations common to all vendors and keep the rest as vendor private.
My opinion is that these "names" should not even be an enum at all.
You're talking about operations, but operation is 'type', not 'name'.
There needs to be no pre-defined, hardcoded pipeline structure in the
UAPI design. Your naming scheme, and the old KMS color property design,
assume that there is such a rigid pre-defined structure that all
hardware and drivers must adhere to. That does not work.
Drivers need to be able to expose any arbitrary operations in any
arbitrary order in each pipeline, with no pre-determined or hinted
"intended use". We want to define the operations, and not say what they
should be used for. Usage examples are for accompanying documentation,
not API tokens.
This idea is at the core of the new color pipeline design we have been
discussing.
> > What if a driver put POST_CSC before PRE_CSC in its pipeline?
> >
> > What if your CSC is actually a series of three independent operations, and in
> > addition you have PRE_CSC and POST_CSC?
>
> We should try to standardized the operations as much as possible and leave rest as
> vendor private. Current proposal allows us to do that.
>
> > 3D_LUT is an operation category, not a name. The same could be said about
> > private.
>
> Sure, will fix this.
>
> > Given that all these are also UAPI, do we also need protect old userspace from
> > seeing values it does not understand?
>
> For the values userspace doesn't understand, it can ignore the blocks. We should ensure
> that userspace always gets a clean state wrt color hardware state and no baggage from
> another client should be there. With that there is no burden of disabling that particular
> block will be there on an older userspace.
>
> > > + */
> > > +enum color_op_block {
> > > + DRM_CB_INVAL = -1,
> > > +
> > > + DRM_CB_PRE_CSC = 0,
> > > + DRM_CB_CSC,
> > > + DRM_CB_POST_CSC,
> > > + DRM_CB_3D_LUT,
> > > +
> > > + /* Any new generic hardware block can be updated here */
> > > +
> > > + /*
> > > + * PRIVATE is kept at 255 to make it future proof and leave
> > > + * scope for any new addition
> > > + */
> > > + DRM_CB_PRIVATE = 255,
> > > + DRM_CB_MAX = DRM_CB_PRIVATE,
> > > +};
> > > +
> > > +/**
> > > + * enum color_op_type
> > > + *
> > > + * These enums are to identify the mathematical operation that
> > > + * a hardware block is capable of.
> > > + * @CURVE_1D: It represents a one dimensional lookup table
> > > + * @CURVE_3D: Represents lut value for each color component for 3d
> > > +lut capable hardware
> > > + * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
> > > + * @FIXED_FUNCTION: To enable and program any custom fixed function
> > > +hardware unit */ enum color_op_type {
> > > + CURVE_1D,
> > > + CURVE_3D,
> > > + MATRIX,
> > > + FIXED_FUNCTION,
> >
> > My assumption was that a color_op_type would clearly and uniquely define the
> > mathematical model of the operation and the UABI structure of the parameter
> > blob. That means we need different values for uniform vs. exponentially vs.
> > programmable distributed 1D LUT, etc.
>
> In the hardware the LUTS are programmed as they are received from userspace.
> So once the userspace gets to know the distribution of LUTS, segments, precision,
> Number of lut samples, it can create the lut values to be programmed.
>
> This information on the distribution of luts in the hardware can be extracted by the
> drm_color_lut_range structure which is exposed as blob in the hardware block with
> TYPE set as CURVE_1D.
I was referring to the special cases like FIXED_FUNCTION and PRIVATE.
If you have an operation of type PRIVATE, you have no idea what it does
until you decipher 'private_flags', but you cannot decipher
'private_flags' until you know the driver name because it is
vendor-specific.
Userspace hates keying things off driver name. It means no other driver
can expose the same and expect userspace to just start using it. It
prevents an originally single-vendor feature to become a common
multi-vendor feature in future hardware generations without patching
every KMS client. Think about OpenGL or Vulkan: they have
vendor-specific extensions, but if another vendor implements the same,
they don't make up another extension but claim to support the original
one.
> > If there is a 1D curve with pre-programmed (fixed and named) curves, we need to
> > enumerate all the curve types somehow. Probably each fixed curve type should
> > not be a different operation type, because that would explode the number of
> > alternative pipelines.
>
> Exposing the lut distribution with drm_color_lut_range would do this job.
I'm talking about named fixed curves, PQ EOTF for example. You'd
probably put it in FIXED_FUNCTION.
Or are you suggesting that userspace should take the LUT
parameterisation from the driver and check if it happens to match PQ
EOTF close enough?
We really need (string) names for fixed curves, so that they can be
readily identified. Maybe Intel does not have them then, but AMD has.
> > A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D
> > LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in three dimensional space.
>
> Yes right, we can optimize and fine tune this.
I mean you used a wrong term. It's not a 3D curve when you have a 3D
LUT.
> > A matrix element could be with or without an offset vector I guess.
> >
> > FIXED_FUNCTION would need to be replaced with e.g. your example
> > VENDORXXX_BT602_TO_BT2020 to work.
> >
> > Have I missed something, how did you intend this to work?
>
> FIXED_FUNCTION is for some descriptive implementation. Some cases where
> Hardware just have a bit to enable and rest of the implementation is within hardware.
> Since type is used for other general hardware blocks, its not intended to cover vendor
> specific implementation and a private_flags are used instead.
As I've said, 'private_flags' cannot exist.
Thanks,
pq
>
> Thanks Pekka for the feedback.
>
> Regards,
> Uma Shankar
>
> >
> >
> > Thanks,
> > pq
> >
> > > +};
> > > +
> > > +/**
> > > + * @struct drm_color_op
> > > + *
> > > + * This structure is used to represent the capability of
> > > + * individual color hardware blocks.
> > > + *
> > > + * @name: a standardized enum to identify the color hardware block
> > > + * @type: The type of mathematical operation it can perform
> > > + * @blob_id: Id pointing to a blob containing information about
> > > + * the hardware block which advertizes its capabilities
> > > + * to the userspace. It can be an optional field depending
> > > + * on the members "name" and "type".
> > > + * @private_flags: This can be used to provide vendor specific hints
> > > + * to user space
> > > + */
> > > +struct drm_color_op {
> > > + enum color_op_block name;
> > > + enum color_op_type type;
> > > + __u32 blob_id;
> > > + __u32 private_flags;
> > > +};
> > > +
> > > /**
> > > * DRM_MODE_PAGE_FLIP_EVENT
> > > *
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 02/33] drm: Add color operation structure
@ 2023-09-05 11:33 ` Pekka Paalanen
0 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-05 11:33 UTC (permalink / raw)
To: Shankar, Uma; +Cc: intel-gfx, dri-devel, wayland-devel
[-- Attachment #1: Type: text/plain, Size: 10114 bytes --]
On Mon, 4 Sep 2023 14:10:05 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Pekka
> > Paalanen
> > Sent: Wednesday, August 30, 2023 6:30 PM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org
> > Subject: Re: [RFC 02/33] drm: Add color operation structure
> >
> > On Tue, 29 Aug 2023 21:33:51 +0530
> > Uma Shankar <uma.shankar@intel.com> wrote:
> >
> > > From: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > >
> > > Each Color Hardware block will be represented uniquely in the color
> > > pipeline. Define the structure to represent the same.
> > >
> > > These color operations will form the building blocks of a color
> > > pipeline which best represents the underlying Hardware. Color
> > > operations can be re-arranged, substracted or added to create distinct
> > > color pipelines to accurately describe the Hardware blocks present in
> > > the display engine.
> > >
> > > Co-developed-by: Uma Shankar <uma.shankar@intel.com>
> > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > > ---
> > > include/uapi/drm/drm_mode.h | 72
> > > +++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 72 insertions(+)
> > >
> > > diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> > > index ea1b639bcb28..882479f41745 100644
> > > --- a/include/uapi/drm/drm_mode.h
> > > +++ b/include/uapi/drm/drm_mode.h
> > > @@ -943,6 +943,78 @@ struct hdr_output_metadata {
> > > };
> > > };
> > >
> > > +/**
> > > + * enum color_op_block
> > > + *
> > > + * Enums to identify hardware color blocks.
> > > + *
> > > + * @DRM_CB_PRE_CSC: LUT before the CTM unit
> > > + * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
> > > + * @DRM_CB_POST_CSC: LUT after the CTM unit
> > > + * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
> > > + * color components
> > > + * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
> > > + * can expose a custom hardware by defining a
> > > + * color operation block with this name as
> > > + * identifier
> >
> > This naming scheme does not seem to work. It assumes a far too rigid pipeline,
> > just like the old KMS property design. What if you have two other operations
> > between PRE_CSC and CSC?
> >
> > What sense do PRE_CSC and POST_CSC make if you don't happen to have a CSC
> > operation?
>
> Sure, we can re-look at the naming. However, it will be good to define some standard
> operations common to all vendors and keep the rest as vendor private.
My opinion is that these "names" should not even be an enum at all.
You're talking about operations, but operation is 'type', not 'name'.
There needs to be no pre-defined, hardcoded pipeline structure in the
UAPI design. Your naming scheme, and the old KMS color property design,
assume that there is such a rigid pre-defined structure that all
hardware and drivers must adhere to. That does not work.
Drivers need to be able to expose any arbitrary operations in any
arbitrary order in each pipeline, with no pre-determined or hinted
"intended use". We want to define the operations, and not say what they
should be used for. Usage examples are for accompanying documentation,
not API tokens.
This idea is at the core of the new color pipeline design we have been
discussing.
> > What if a driver put POST_CSC before PRE_CSC in its pipeline?
> >
> > What if your CSC is actually a series of three independent operations, and in
> > addition you have PRE_CSC and POST_CSC?
>
> We should try to standardized the operations as much as possible and leave rest as
> vendor private. Current proposal allows us to do that.
>
> > 3D_LUT is an operation category, not a name. The same could be said about
> > private.
>
> Sure, will fix this.
>
> > Given that all these are also UAPI, do we also need protect old userspace from
> > seeing values it does not understand?
>
> For the values userspace doesn't understand, it can ignore the blocks. We should ensure
> that userspace always gets a clean state wrt color hardware state and no baggage from
> another client should be there. With that there is no burden of disabling that particular
> block will be there on an older userspace.
>
> > > + */
> > > +enum color_op_block {
> > > + DRM_CB_INVAL = -1,
> > > +
> > > + DRM_CB_PRE_CSC = 0,
> > > + DRM_CB_CSC,
> > > + DRM_CB_POST_CSC,
> > > + DRM_CB_3D_LUT,
> > > +
> > > + /* Any new generic hardware block can be updated here */
> > > +
> > > + /*
> > > + * PRIVATE is kept at 255 to make it future proof and leave
> > > + * scope for any new addition
> > > + */
> > > + DRM_CB_PRIVATE = 255,
> > > + DRM_CB_MAX = DRM_CB_PRIVATE,
> > > +};
> > > +
> > > +/**
> > > + * enum color_op_type
> > > + *
> > > + * These enums are to identify the mathematical operation that
> > > + * a hardware block is capable of.
> > > + * @CURVE_1D: It represents a one dimensional lookup table
> > > + * @CURVE_3D: Represents lut value for each color component for 3d
> > > +lut capable hardware
> > > + * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
> > > + * @FIXED_FUNCTION: To enable and program any custom fixed function
> > > +hardware unit */ enum color_op_type {
> > > + CURVE_1D,
> > > + CURVE_3D,
> > > + MATRIX,
> > > + FIXED_FUNCTION,
> >
> > My assumption was that a color_op_type would clearly and uniquely define the
> > mathematical model of the operation and the UABI structure of the parameter
> > blob. That means we need different values for uniform vs. exponentially vs.
> > programmable distributed 1D LUT, etc.
>
> In the hardware the LUTS are programmed as they are received from userspace.
> So once the userspace gets to know the distribution of LUTS, segments, precision,
> Number of lut samples, it can create the lut values to be programmed.
>
> This information on the distribution of luts in the hardware can be extracted by the
> drm_color_lut_range structure which is exposed as blob in the hardware block with
> TYPE set as CURVE_1D.
I was referring to the special cases like FIXED_FUNCTION and PRIVATE.
If you have an operation of type PRIVATE, you have no idea what it does
until you decipher 'private_flags', but you cannot decipher
'private_flags' until you know the driver name because it is
vendor-specific.
Userspace hates keying things off driver name. It means no other driver
can expose the same and expect userspace to just start using it. It
prevents an originally single-vendor feature to become a common
multi-vendor feature in future hardware generations without patching
every KMS client. Think about OpenGL or Vulkan: they have
vendor-specific extensions, but if another vendor implements the same,
they don't make up another extension but claim to support the original
one.
> > If there is a 1D curve with pre-programmed (fixed and named) curves, we need to
> > enumerate all the curve types somehow. Probably each fixed curve type should
> > not be a different operation type, because that would explode the number of
> > alternative pipelines.
>
> Exposing the lut distribution with drm_color_lut_range would do this job.
I'm talking about named fixed curves, PQ EOTF for example. You'd
probably put it in FIXED_FUNCTION.
Or are you suggesting that userspace should take the LUT
parameterisation from the driver and check if it happens to match PQ
EOTF close enough?
We really need (string) names for fixed curves, so that they can be
readily identified. Maybe Intel does not have them then, but AMD has.
> > A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D
> > LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in three dimensional space.
>
> Yes right, we can optimize and fine tune this.
I mean you used a wrong term. It's not a 3D curve when you have a 3D
LUT.
> > A matrix element could be with or without an offset vector I guess.
> >
> > FIXED_FUNCTION would need to be replaced with e.g. your example
> > VENDORXXX_BT602_TO_BT2020 to work.
> >
> > Have I missed something, how did you intend this to work?
>
> FIXED_FUNCTION is for some descriptive implementation. Some cases where
> Hardware just have a bit to enable and rest of the implementation is within hardware.
> Since type is used for other general hardware blocks, its not intended to cover vendor
> specific implementation and a private_flags are used instead.
As I've said, 'private_flags' cannot exist.
Thanks,
pq
>
> Thanks Pekka for the feedback.
>
> Regards,
> Uma Shankar
>
> >
> >
> > Thanks,
> > pq
> >
> > > +};
> > > +
> > > +/**
> > > + * @struct drm_color_op
> > > + *
> > > + * This structure is used to represent the capability of
> > > + * individual color hardware blocks.
> > > + *
> > > + * @name: a standardized enum to identify the color hardware block
> > > + * @type: The type of mathematical operation it can perform
> > > + * @blob_id: Id pointing to a blob containing information about
> > > + * the hardware block which advertizes its capabilities
> > > + * to the userspace. It can be an optional field depending
> > > + * on the members "name" and "type".
> > > + * @private_flags: This can be used to provide vendor specific hints
> > > + * to user space
> > > + */
> > > +struct drm_color_op {
> > > + enum color_op_block name;
> > > + enum color_op_type type;
> > > + __u32 blob_id;
> > > + __u32 private_flags;
> > > +};
> > > +
> > > /**
> > > * DRM_MODE_PAGE_FLIP_EVENT
> > > *
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 00/33] Add Support for Plane Color Pipeline
2023-09-04 14:29 ` [Intel-gfx] " Shankar, Uma
@ 2023-09-05 11:33 ` Pekka Paalanen
-1 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-05 11:33 UTC (permalink / raw)
To: Shankar, Uma
Cc: Sebastian Wick, Pekka Paalanen, Shashank Sharma, intel-gfx,
dri-devel, wayland-devel, Melissa Wen, Jonas Ådahl,
Naseer Ahmed, Christopher Braga
[-- Attachment #1: Type: text/plain, Size: 16160 bytes --]
On Mon, 4 Sep 2023 14:29:56 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: Sebastian Wick <sebastian.wick@redhat.com>
> > Sent: Thursday, August 31, 2023 2:46 AM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> > gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> > Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > <quic_cbraga@quicinc.com>
> > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> >
> > On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > <quic_cbraga@quicinc.com>
> > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > >
> > > > +CC Naseer and Chris, FYI
> > > >
> > > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > > >
> > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > Introduction
> > > > > ============
> > > > >
> > > > > Modern hardwares have various color processing capabilities both
> > > > > at pre-blending and post-blending phases in the color pipeline.
> > > > > The current drm implementation exposes only the post-blending
> > > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > > There are multiple use cases where pre-blending color hardware
> > > > > will be
> > > > > useful:
> > > > > a) Linearization of input buffers encoded in various transfer
> > > > > functions.
> > > > > b) Color Space conversion
> > > > > c) Tone mapping
> > > > > d) Frame buffer format conversion
> > > > > e) Non-linearization of buffer(apply transfer function)
> > > > > f) 3D Luts
> > > > >
> > > > > and other miscellaneous color operations.
> > > > >
> > > > > Hence, there is a need to expose the color capabilities of the
> > > > > hardware to user-space. This will help userspace/middleware to use
> > > > > display hardware for color processing and blending instead of
> > > > > doing it through GPU shaders.
> > > > >
> > > >
> > > > Thanks, Uma, for sending this. I've been working on something
> > > > similar but you beat me to it. :)
> > >
> > > Thanks Harry for the useful feedback and overall collaboration on this so far.
> > >
> > > > >
> > > > > Work done so far and relevant references
> > > > > ========================================
> > > > >
> > > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > > Broad consensus is there that we need a generic API at drm core to
> > > > > suffice the use case of various HW vendors. Below are the links
> > > > > capturing the discussion so far.
> > > > >
> > > > > Intel's Plane Color Implementation:
> > > > > https://patchwork.freedesktop.org/series/90825/
> > > > > AMD's Plane Color Implementation:
> > > > > https://patchwork.freedesktop.org/series/116862/
> > > > >
> > > > >
> > > > > Hackfest conclusions
> > > > > ====================
> > > > >
> > > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > > industry stakeholders together and converge on a common uapi
> > expectations.
> > > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > > and other prominent user-space developers and maintainers.
> > > > >
> > > > > Discussions happened on the uapi expectations, opens, nature of
> > > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > > the same and the path forward. Consensus was made that drm core
> > > > > should implement descriptive APIs and not go with prescriptive
> > > > > APIs. DRM core should just expose the hardware capabilities;
> > > > > enabling, customizing and programming the same should be done by
> > > > > the user-space. Driver should just
> > > > honor the user space request without doing any operations internally.
> > > > >
> > > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > > and an UAPI RFC which can be referred to here:
> > > > >
> > > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > > _hD5
> > > > >
> > > >
> > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> > Q
> > > > Wn48
> > > > > 8=@emersion.fr/
> > > > >
> > > > >
> > > > > Design considerations
> > > > > =====================
> > > > >
> > > > > Following are the important aspects taken into account while
> > > > > designing the current RFC
> > > > > proposal:
> > > > >
> > > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > > only one
> > > > can be used)
> > > > > 2. Position of the HW block in the pipeline can be programmable
> > > > > 3. LUTs can be one dimentional or three dimentional
> > > > > 4. Number of LUT entries can vary across platforms
> > > > > 5. Precision of LUT entries can vary across platforms
> > > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> > Logarithmic,
> > > > > Piece-Wise Linear(PWL) etc
> > > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > > blocks.
> > > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > > 8. Custom non-standard HW implementation.
> > > > > 9. Leaving scope for some vendor defined pescriptive
> > > > > implementation if
> > > > required.
> > > > > 10.Scope to handle any modification in hardware as technology
> > > > > evolves
> > > > >
> > > > > The current proposal takes into account the above considerations
> > > > > while keeping the implementation as generic as possible leaving
> > > > > scope for future
> > > > additions or modifications.
> > > > >
> > > > > This proposal is also in line to the details mentioned by Simon's
> > > > > RFC covering all the aspects discussed in hackfest.
> > > > >
> > > > >
> > > > > Outline of the implementation
> > > > > ============================
> > > > >
> > > > > Each Color Hardware block will be represented by a data structure
> > drm_color_op.
> > > > > These color operations will form the building blocks of a color
> > > > > pipeline which best represents the underlying Hardware. Color
> > > > > operations can be re-arranged, substracted or added to create
> > > > > distinct color pipelines to accurately describe the Hardware
> > > > > blocks present in the display
> > > > engine.
> > > >
> > > > Who is doing the arranging of color operations? IMO a driver should
> > > > define one or more respective pipelines that can be selected by
> > > > userspace. This seems to be what you're talking about after (I
> > > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> > to add clarifications in order to avoid confusion.
> > >
> > > Yes it's the driver who will set the pipeline based on the underlying
> > > hardware arrangement and possible combinations. There can be multiple
> > > pipelines possible if hardware can be muxed or order can be re-arranged (all
> > viable combinations should be defined as a pipeline in driver).
> > > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> > >
> > > > >
> > > > > In this proposal, a color pipeline is represented as an array of
> > > > > struct drm_color_op. For individual color operation, we add blobs
> > > > > to advertise the capability of the particular Hardware block.
> > > > >
> > > > > This color pipeline is then packaged within a blob for the user
> > > > > space to retrieve it.
> > > > >
> > > >
> > > > Would it be better to expose the drm_color_ops directly, instead of
> > > > packing a array of drm_color_ops into a blob and then giving that to
> > userspace.
> > >
> > > Advantage we see in packing in blobs is that interface will be
> > > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> > and then its just parsing the data.
> > > This way the entire underlying hardware blocks with pipeline will be available to
> > user.
> >
> > I don't see how parsing a blob is easier than requesting the color ops from the
> > kernel. User space is already equiped with getting KMS objects and the igt test
> > code from Harry shows that this is all pretty trivial plumbing.
>
> There are multiple color operations possible with unique lut distribution and
> capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
> Having all the information with one query and property and also be able to set the
> same with just one property call using blobs is better than multiple calls to driver.
> This can be useful in high refresh rate cases where not much time is there to program
> the hardware state. Latency of multiple calls to driver can be avoided.
Hi,
querying all that information only happens once, at KMS client start-up.
Setting up a color pipeline is already a single call: the atomic commit ioctl.
>
> > > For a particular hardware block in pipeline, user can get the relevant
> > > details from blob representing that particular block. We have created
> > > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> > This is just to clarify the idea.
> >
> > The blob is also not introspectable with the usual tools whereas exposing them as
> > properties would be.
> >
> > It also would, like Pekka correctly noted, bring a whole bunch of issues about
> > compatibility and versioning that are well understood with objects
> > + properties.
>
> The blob should be standardized in the UAPI and structures to parse them should be fixed.
> With this compatibility issues can be prevented.
I think that is short-sighted.
> > > Also since info is passed via blobs we have the flexibility to even
> > > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > > for custom private hardware blocks as well which driver can work with
> > respective HAL and get that implemented.
> >
> > When color ops are real KMS objects they still can have properties which are
> > blobs that can store LUTs and other data.
> >
> > And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> > have changed in that regard and it simply is not allowed.
>
> Private blocks are not standardized but are vendor specific. So generic userspace will
> ignore these. However vendor and its respective HAL can make use of this field and leaves
> a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
> standardized color operations which will be defined as enum in UAPI.
Vendors can have and expose their own unique snowflake operations
without any "private" as well: pick an unused operation type code, and
document what it does. Advertise it in some pipelines.
Vendor HALs can make use of it, but it also allows generic userspace to
make use of it at will, and it allows other vendors to implement the
same and benefit from it without needing to patch every userspace.
Or does Intel not want other vendors to potentially make use of their
UAPIs?
> > > We can even define prescriptive operations as a private entry and
> > > enable it if a certain driver and HAL agree.
> > >
> > > > > To advertise the available color pipelines, an immutable ENUM
> > > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> > blob id's as values.
> > > > > With each blob id representing a distinct color pipeline based on
> > > > > underlying HW capabilities and their respective combinations.
> > > > >
> > > > > Once the user space decides on a color pipeline, it can set the
> > > > > pipeline and the corresponding data for the hardware blocks within
> > > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > >
> > > >
> > > > When I discussed this at the hackfest with Simon he proposed a new
> > > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > > NULL if its the last in the pipeline. You can then have a mutable
> > > > enum property on the plane to discover and select a color pipeline.
> > >
> > > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> > the design.
> > > Personally I feel the one proposed in the current RFC will do the same
> > > thing envisioned by Simon and you Harry. Management of the pipeline,
> > > addition, deletion and flexibility to represent hardware is more with blobs with
> > the relevant structures agreed in UAPI.
> > >
> > > > This seems a bit more transparent than a blob. You can see my
> > > > changes (unfortunately very WIP, don't look too closely at
> > > > individual patches) at
> > > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > > ffs
> > > >
> > > > libdrm changes:
> > > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > > s
> > >
> > > Sure, will check the same.
> > >
> > > > IGT changes:
> > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > > sts/1/diffs
> > > >
> > > > I'll take time to review your whole series and will see whether we
> > > > can somehow keep the best parts of each.
> > >
> > > Thanks and agree. Let's work together and get this implemented in DRM.
> > >
> > > > Curious to hear other opinions on the blob vs new DRM object question as
> > well.
> >
> > I can see a few advantages with the blob approach.
> >
> > User space can store the entire state in one blob and just assign a new blob to
> > change to another pipeline configuration.
>
> Agree
>
> > However, I would argue that changing a lot of properties is already common
> > practice and works well. User space can deal with it and has abstractions to
> > make this managable.
>
> Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
The structure of the atomic commit ioctl and the KMS property system is
even more flexible.
> Lut distribution, number of segments, samples in each segment, precision of luts etc.
> can be precisely defined and userspace will get a complete picture of the underlying
> hardware and its capabilities. Also this is being done with just 2 properties. Leaving
> scope for future addition of any standard color operation as well.
The number of properties does not seem too useful to strictly minimise
over other aspects.
Thanks,
pq
>
> I feel blob approach once properly documented is a bit more flexible and scalable.
> Maybe I am bit biased here 😊 but all ideas are welcome.
>
> We have implemented some IGT's as well to explain the design better. Link below:
> https://patchwork.freedesktop.org/series/123018/
>
> Thanks Sebastian for the feedback.
>
> Regards,
> Uma Shankar
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-09-05 11:33 ` Pekka Paalanen
0 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-05 11:33 UTC (permalink / raw)
To: Shankar, Uma
Cc: Sebastian Wick, Pekka Paalanen, intel-gfx, Alexander Goins,
dri-devel, wayland-devel, Melissa Wen, Jonas Ådahl,
Naseer Ahmed, Harry Wentland, Christopher Braga
[-- Attachment #1: Type: text/plain, Size: 16160 bytes --]
On Mon, 4 Sep 2023 14:29:56 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: Sebastian Wick <sebastian.wick@redhat.com>
> > Sent: Thursday, August 31, 2023 2:46 AM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> > gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> > Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > <quic_cbraga@quicinc.com>
> > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> >
> > On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > <quic_cbraga@quicinc.com>
> > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > >
> > > > +CC Naseer and Chris, FYI
> > > >
> > > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > > >
> > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > Introduction
> > > > > ============
> > > > >
> > > > > Modern hardwares have various color processing capabilities both
> > > > > at pre-blending and post-blending phases in the color pipeline.
> > > > > The current drm implementation exposes only the post-blending
> > > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > > There are multiple use cases where pre-blending color hardware
> > > > > will be
> > > > > useful:
> > > > > a) Linearization of input buffers encoded in various transfer
> > > > > functions.
> > > > > b) Color Space conversion
> > > > > c) Tone mapping
> > > > > d) Frame buffer format conversion
> > > > > e) Non-linearization of buffer(apply transfer function)
> > > > > f) 3D Luts
> > > > >
> > > > > and other miscellaneous color operations.
> > > > >
> > > > > Hence, there is a need to expose the color capabilities of the
> > > > > hardware to user-space. This will help userspace/middleware to use
> > > > > display hardware for color processing and blending instead of
> > > > > doing it through GPU shaders.
> > > > >
> > > >
> > > > Thanks, Uma, for sending this. I've been working on something
> > > > similar but you beat me to it. :)
> > >
> > > Thanks Harry for the useful feedback and overall collaboration on this so far.
> > >
> > > > >
> > > > > Work done so far and relevant references
> > > > > ========================================
> > > > >
> > > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > > Broad consensus is there that we need a generic API at drm core to
> > > > > suffice the use case of various HW vendors. Below are the links
> > > > > capturing the discussion so far.
> > > > >
> > > > > Intel's Plane Color Implementation:
> > > > > https://patchwork.freedesktop.org/series/90825/
> > > > > AMD's Plane Color Implementation:
> > > > > https://patchwork.freedesktop.org/series/116862/
> > > > >
> > > > >
> > > > > Hackfest conclusions
> > > > > ====================
> > > > >
> > > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > > industry stakeholders together and converge on a common uapi
> > expectations.
> > > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > > and other prominent user-space developers and maintainers.
> > > > >
> > > > > Discussions happened on the uapi expectations, opens, nature of
> > > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > > the same and the path forward. Consensus was made that drm core
> > > > > should implement descriptive APIs and not go with prescriptive
> > > > > APIs. DRM core should just expose the hardware capabilities;
> > > > > enabling, customizing and programming the same should be done by
> > > > > the user-space. Driver should just
> > > > honor the user space request without doing any operations internally.
> > > > >
> > > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > > and an UAPI RFC which can be referred to here:
> > > > >
> > > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > > _hD5
> > > > >
> > > >
> > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> > Q
> > > > Wn48
> > > > > 8=@emersion.fr/
> > > > >
> > > > >
> > > > > Design considerations
> > > > > =====================
> > > > >
> > > > > Following are the important aspects taken into account while
> > > > > designing the current RFC
> > > > > proposal:
> > > > >
> > > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > > only one
> > > > can be used)
> > > > > 2. Position of the HW block in the pipeline can be programmable
> > > > > 3. LUTs can be one dimentional or three dimentional
> > > > > 4. Number of LUT entries can vary across platforms
> > > > > 5. Precision of LUT entries can vary across platforms
> > > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> > Logarithmic,
> > > > > Piece-Wise Linear(PWL) etc
> > > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > > blocks.
> > > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > > 8. Custom non-standard HW implementation.
> > > > > 9. Leaving scope for some vendor defined pescriptive
> > > > > implementation if
> > > > required.
> > > > > 10.Scope to handle any modification in hardware as technology
> > > > > evolves
> > > > >
> > > > > The current proposal takes into account the above considerations
> > > > > while keeping the implementation as generic as possible leaving
> > > > > scope for future
> > > > additions or modifications.
> > > > >
> > > > > This proposal is also in line to the details mentioned by Simon's
> > > > > RFC covering all the aspects discussed in hackfest.
> > > > >
> > > > >
> > > > > Outline of the implementation
> > > > > ============================
> > > > >
> > > > > Each Color Hardware block will be represented by a data structure
> > drm_color_op.
> > > > > These color operations will form the building blocks of a color
> > > > > pipeline which best represents the underlying Hardware. Color
> > > > > operations can be re-arranged, substracted or added to create
> > > > > distinct color pipelines to accurately describe the Hardware
> > > > > blocks present in the display
> > > > engine.
> > > >
> > > > Who is doing the arranging of color operations? IMO a driver should
> > > > define one or more respective pipelines that can be selected by
> > > > userspace. This seems to be what you're talking about after (I
> > > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> > to add clarifications in order to avoid confusion.
> > >
> > > Yes it's the driver who will set the pipeline based on the underlying
> > > hardware arrangement and possible combinations. There can be multiple
> > > pipelines possible if hardware can be muxed or order can be re-arranged (all
> > viable combinations should be defined as a pipeline in driver).
> > > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> > >
> > > > >
> > > > > In this proposal, a color pipeline is represented as an array of
> > > > > struct drm_color_op. For individual color operation, we add blobs
> > > > > to advertise the capability of the particular Hardware block.
> > > > >
> > > > > This color pipeline is then packaged within a blob for the user
> > > > > space to retrieve it.
> > > > >
> > > >
> > > > Would it be better to expose the drm_color_ops directly, instead of
> > > > packing a array of drm_color_ops into a blob and then giving that to
> > userspace.
> > >
> > > Advantage we see in packing in blobs is that interface will be
> > > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> > and then its just parsing the data.
> > > This way the entire underlying hardware blocks with pipeline will be available to
> > user.
> >
> > I don't see how parsing a blob is easier than requesting the color ops from the
> > kernel. User space is already equiped with getting KMS objects and the igt test
> > code from Harry shows that this is all pretty trivial plumbing.
>
> There are multiple color operations possible with unique lut distribution and
> capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
> Having all the information with one query and property and also be able to set the
> same with just one property call using blobs is better than multiple calls to driver.
> This can be useful in high refresh rate cases where not much time is there to program
> the hardware state. Latency of multiple calls to driver can be avoided.
Hi,
querying all that information only happens once, at KMS client start-up.
Setting up a color pipeline is already a single call: the atomic commit ioctl.
>
> > > For a particular hardware block in pipeline, user can get the relevant
> > > details from blob representing that particular block. We have created
> > > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> > This is just to clarify the idea.
> >
> > The blob is also not introspectable with the usual tools whereas exposing them as
> > properties would be.
> >
> > It also would, like Pekka correctly noted, bring a whole bunch of issues about
> > compatibility and versioning that are well understood with objects
> > + properties.
>
> The blob should be standardized in the UAPI and structures to parse them should be fixed.
> With this compatibility issues can be prevented.
I think that is short-sighted.
> > > Also since info is passed via blobs we have the flexibility to even
> > > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > > for custom private hardware blocks as well which driver can work with
> > respective HAL and get that implemented.
> >
> > When color ops are real KMS objects they still can have properties which are
> > blobs that can store LUTs and other data.
> >
> > And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> > have changed in that regard and it simply is not allowed.
>
> Private blocks are not standardized but are vendor specific. So generic userspace will
> ignore these. However vendor and its respective HAL can make use of this field and leaves
> a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
> standardized color operations which will be defined as enum in UAPI.
Vendors can have and expose their own unique snowflake operations
without any "private" as well: pick an unused operation type code, and
document what it does. Advertise it in some pipelines.
Vendor HALs can make use of it, but it also allows generic userspace to
make use of it at will, and it allows other vendors to implement the
same and benefit from it without needing to patch every userspace.
Or does Intel not want other vendors to potentially make use of their
UAPIs?
> > > We can even define prescriptive operations as a private entry and
> > > enable it if a certain driver and HAL agree.
> > >
> > > > > To advertise the available color pipelines, an immutable ENUM
> > > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> > blob id's as values.
> > > > > With each blob id representing a distinct color pipeline based on
> > > > > underlying HW capabilities and their respective combinations.
> > > > >
> > > > > Once the user space decides on a color pipeline, it can set the
> > > > > pipeline and the corresponding data for the hardware blocks within
> > > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > >
> > > >
> > > > When I discussed this at the hackfest with Simon he proposed a new
> > > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > > NULL if its the last in the pipeline. You can then have a mutable
> > > > enum property on the plane to discover and select a color pipeline.
> > >
> > > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> > the design.
> > > Personally I feel the one proposed in the current RFC will do the same
> > > thing envisioned by Simon and you Harry. Management of the pipeline,
> > > addition, deletion and flexibility to represent hardware is more with blobs with
> > the relevant structures agreed in UAPI.
> > >
> > > > This seems a bit more transparent than a blob. You can see my
> > > > changes (unfortunately very WIP, don't look too closely at
> > > > individual patches) at
> > > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > > ffs
> > > >
> > > > libdrm changes:
> > > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > > s
> > >
> > > Sure, will check the same.
> > >
> > > > IGT changes:
> > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > > sts/1/diffs
> > > >
> > > > I'll take time to review your whole series and will see whether we
> > > > can somehow keep the best parts of each.
> > >
> > > Thanks and agree. Let's work together and get this implemented in DRM.
> > >
> > > > Curious to hear other opinions on the blob vs new DRM object question as
> > well.
> >
> > I can see a few advantages with the blob approach.
> >
> > User space can store the entire state in one blob and just assign a new blob to
> > change to another pipeline configuration.
>
> Agree
>
> > However, I would argue that changing a lot of properties is already common
> > practice and works well. User space can deal with it and has abstractions to
> > make this managable.
>
> Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
The structure of the atomic commit ioctl and the KMS property system is
even more flexible.
> Lut distribution, number of segments, samples in each segment, precision of luts etc.
> can be precisely defined and userspace will get a complete picture of the underlying
> hardware and its capabilities. Also this is being done with just 2 properties. Leaving
> scope for future addition of any standard color operation as well.
The number of properties does not seem too useful to strictly minimise
over other aspects.
Thanks,
pq
>
> I feel blob approach once properly documented is a bit more flexible and scalable.
> Maybe I am bit biased here 😊 but all ideas are welcome.
>
> We have implemented some IGT's as well to explain the design better. Link below:
> https://patchwork.freedesktop.org/series/123018/
>
> Thanks Sebastian for the feedback.
>
> Regards,
> Uma Shankar
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 00/33] Add Support for Plane Color Pipeline
2023-09-05 11:33 ` [Intel-gfx] " Pekka Paalanen
@ 2023-09-05 12:33 ` Sebastian Wick
-1 siblings, 0 replies; 109+ messages in thread
From: Sebastian Wick @ 2023-09-05 12:33 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Pekka Paalanen, Shashank Sharma, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Shankar, Uma,
Naseer Ahmed, Christopher Braga
On Tue, Sep 05, 2023 at 02:33:26PM +0300, Pekka Paalanen wrote:
> On Mon, 4 Sep 2023 14:29:56 +0000
> "Shankar, Uma" <uma.shankar@intel.com> wrote:
>
> > > -----Original Message-----
> > > From: Sebastian Wick <sebastian.wick@redhat.com>
> > > Sent: Thursday, August 31, 2023 2:46 AM
> > > To: Shankar, Uma <uma.shankar@intel.com>
> > > Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> > > gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> > > devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> > > Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> > > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > <quic_cbraga@quicinc.com>
> > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > >
> > > On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > > <quic_cbraga@quicinc.com>
> > > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > > >
> > > > > +CC Naseer and Chris, FYI
> > > > >
> > > > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > > > >
> > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > Introduction
> > > > > > ============
> > > > > >
> > > > > > Modern hardwares have various color processing capabilities both
> > > > > > at pre-blending and post-blending phases in the color pipeline.
> > > > > > The current drm implementation exposes only the post-blending
> > > > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > > > There are multiple use cases where pre-blending color hardware
> > > > > > will be
> > > > > > useful:
> > > > > > a) Linearization of input buffers encoded in various transfer
> > > > > > functions.
> > > > > > b) Color Space conversion
> > > > > > c) Tone mapping
> > > > > > d) Frame buffer format conversion
> > > > > > e) Non-linearization of buffer(apply transfer function)
> > > > > > f) 3D Luts
> > > > > >
> > > > > > and other miscellaneous color operations.
> > > > > >
> > > > > > Hence, there is a need to expose the color capabilities of the
> > > > > > hardware to user-space. This will help userspace/middleware to use
> > > > > > display hardware for color processing and blending instead of
> > > > > > doing it through GPU shaders.
> > > > > >
> > > > >
> > > > > Thanks, Uma, for sending this. I've been working on something
> > > > > similar but you beat me to it. :)
> > > >
> > > > Thanks Harry for the useful feedback and overall collaboration on this so far.
> > > >
> > > > > >
> > > > > > Work done so far and relevant references
> > > > > > ========================================
> > > > > >
> > > > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > > > Broad consensus is there that we need a generic API at drm core to
> > > > > > suffice the use case of various HW vendors. Below are the links
> > > > > > capturing the discussion so far.
> > > > > >
> > > > > > Intel's Plane Color Implementation:
> > > > > > https://patchwork.freedesktop.org/series/90825/
> > > > > > AMD's Plane Color Implementation:
> > > > > > https://patchwork.freedesktop.org/series/116862/
> > > > > >
> > > > > >
> > > > > > Hackfest conclusions
> > > > > > ====================
> > > > > >
> > > > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > > > industry stakeholders together and converge on a common uapi
> > > expectations.
> > > > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > > > and other prominent user-space developers and maintainers.
> > > > > >
> > > > > > Discussions happened on the uapi expectations, opens, nature of
> > > > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > > > the same and the path forward. Consensus was made that drm core
> > > > > > should implement descriptive APIs and not go with prescriptive
> > > > > > APIs. DRM core should just expose the hardware capabilities;
> > > > > > enabling, customizing and programming the same should be done by
> > > > > > the user-space. Driver should just
> > > > > honor the user space request without doing any operations internally.
> > > > > >
> > > > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > > > and an UAPI RFC which can be referred to here:
> > > > > >
> > > > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > > > _hD5
> > > > > >
> > > > >
> > > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> > > Q
> > > > > Wn48
> > > > > > 8=@emersion.fr/
> > > > > >
> > > > > >
> > > > > > Design considerations
> > > > > > =====================
> > > > > >
> > > > > > Following are the important aspects taken into account while
> > > > > > designing the current RFC
> > > > > > proposal:
> > > > > >
> > > > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > > > only one
> > > > > can be used)
> > > > > > 2. Position of the HW block in the pipeline can be programmable
> > > > > > 3. LUTs can be one dimentional or three dimentional
> > > > > > 4. Number of LUT entries can vary across platforms
> > > > > > 5. Precision of LUT entries can vary across platforms
> > > > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> > > Logarithmic,
> > > > > > Piece-Wise Linear(PWL) etc
> > > > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > > > blocks.
> > > > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > > > 8. Custom non-standard HW implementation.
> > > > > > 9. Leaving scope for some vendor defined pescriptive
> > > > > > implementation if
> > > > > required.
> > > > > > 10.Scope to handle any modification in hardware as technology
> > > > > > evolves
> > > > > >
> > > > > > The current proposal takes into account the above considerations
> > > > > > while keeping the implementation as generic as possible leaving
> > > > > > scope for future
> > > > > additions or modifications.
> > > > > >
> > > > > > This proposal is also in line to the details mentioned by Simon's
> > > > > > RFC covering all the aspects discussed in hackfest.
> > > > > >
> > > > > >
> > > > > > Outline of the implementation
> > > > > > ============================
> > > > > >
> > > > > > Each Color Hardware block will be represented by a data structure
> > > drm_color_op.
> > > > > > These color operations will form the building blocks of a color
> > > > > > pipeline which best represents the underlying Hardware. Color
> > > > > > operations can be re-arranged, substracted or added to create
> > > > > > distinct color pipelines to accurately describe the Hardware
> > > > > > blocks present in the display
> > > > > engine.
> > > > >
> > > > > Who is doing the arranging of color operations? IMO a driver should
> > > > > define one or more respective pipelines that can be selected by
> > > > > userspace. This seems to be what you're talking about after (I
> > > > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> > > to add clarifications in order to avoid confusion.
> > > >
> > > > Yes it's the driver who will set the pipeline based on the underlying
> > > > hardware arrangement and possible combinations. There can be multiple
> > > > pipelines possible if hardware can be muxed or order can be re-arranged (all
> > > viable combinations should be defined as a pipeline in driver).
> > > > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> > > >
> > > > > >
> > > > > > In this proposal, a color pipeline is represented as an array of
> > > > > > struct drm_color_op. For individual color operation, we add blobs
> > > > > > to advertise the capability of the particular Hardware block.
> > > > > >
> > > > > > This color pipeline is then packaged within a blob for the user
> > > > > > space to retrieve it.
> > > > > >
> > > > >
> > > > > Would it be better to expose the drm_color_ops directly, instead of
> > > > > packing a array of drm_color_ops into a blob and then giving that to
> > > userspace.
> > > >
> > > > Advantage we see in packing in blobs is that interface will be
> > > > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> > > and then its just parsing the data.
> > > > This way the entire underlying hardware blocks with pipeline will be available to
> > > user.
> > >
> > > I don't see how parsing a blob is easier than requesting the color ops from the
> > > kernel. User space is already equiped with getting KMS objects and the igt test
> > > code from Harry shows that this is all pretty trivial plumbing.
> >
> > There are multiple color operations possible with unique lut distribution and
> > capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
> > Having all the information with one query and property and also be able to set the
> > same with just one property call using blobs is better than multiple calls to driver.
> > This can be useful in high refresh rate cases where not much time is there to program
> > the hardware state. Latency of multiple calls to driver can be avoided.
>
> Hi,
>
> querying all that information only happens once, at KMS client start-up.
>
> Setting up a color pipeline is already a single call: the atomic commit ioctl.
Well, clients also issue a bunch of ioctl to set some properties to the
desired state and then you might run through a whole bunch of
configurations to find one that actually works, so there is a case to be
made that there are a lot of ioctls involved.
I just don't think this is an issue right now. Nobody has been
complaining about the ioctls being a limiting factor so why should we
optimize for this? Especially because it brings with it a bunch of
disadvantages.
Anyway, I agree with the sentiment here: this is not something we should
optimize for.
>
> >
> > > > For a particular hardware block in pipeline, user can get the relevant
> > > > details from blob representing that particular block. We have created
> > > > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> > > This is just to clarify the idea.
> > >
> > > The blob is also not introspectable with the usual tools whereas exposing them as
> > > properties would be.
> > >
> > > It also would, like Pekka correctly noted, bring a whole bunch of issues about
> > > compatibility and versioning that are well understood with objects
> > > + properties.
> >
> > The blob should be standardized in the UAPI and structures to parse them should be fixed.
> > With this compatibility issues can be prevented.
>
> I think that is short-sighted.
>
> > > > Also since info is passed via blobs we have the flexibility to even
> > > > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > > > for custom private hardware blocks as well which driver can work with
> > > respective HAL and get that implemented.
> > >
> > > When color ops are real KMS objects they still can have properties which are
> > > blobs that can store LUTs and other data.
> > >
> > > And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> > > have changed in that regard and it simply is not allowed.
> >
> > Private blocks are not standardized but are vendor specific. So generic userspace will
> > ignore these. However vendor and its respective HAL can make use of this field and leaves
> > a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
> > standardized color operations which will be defined as enum in UAPI.
>
> Vendors can have and expose their own unique snowflake operations
> without any "private" as well: pick an unused operation type code, and
> document what it does. Advertise it in some pipelines.
>
> Vendor HALs can make use of it, but it also allows generic userspace to
> make use of it at will, and it allows other vendors to implement the
> same and benefit from it without needing to patch every userspace.
>
> Or does Intel not want other vendors to potentially make use of their
> UAPIs?
>
> > > > We can even define prescriptive operations as a private entry and
> > > > enable it if a certain driver and HAL agree.
> > > >
> > > > > > To advertise the available color pipelines, an immutable ENUM
> > > > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> > > blob id's as values.
> > > > > > With each blob id representing a distinct color pipeline based on
> > > > > > underlying HW capabilities and their respective combinations.
> > > > > >
> > > > > > Once the user space decides on a color pipeline, it can set the
> > > > > > pipeline and the corresponding data for the hardware blocks within
> > > > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > > >
> > > > >
> > > > > When I discussed this at the hackfest with Simon he proposed a new
> > > > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > > > NULL if its the last in the pipeline. You can then have a mutable
> > > > > enum property on the plane to discover and select a color pipeline.
> > > >
> > > > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> > > the design.
> > > > Personally I feel the one proposed in the current RFC will do the same
> > > > thing envisioned by Simon and you Harry. Management of the pipeline,
> > > > addition, deletion and flexibility to represent hardware is more with blobs with
> > > the relevant structures agreed in UAPI.
> > > >
> > > > > This seems a bit more transparent than a blob. You can see my
> > > > > changes (unfortunately very WIP, don't look too closely at
> > > > > individual patches) at
> > > > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > > > ffs
> > > > >
> > > > > libdrm changes:
> > > > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > > > s
> > > >
> > > > Sure, will check the same.
> > > >
> > > > > IGT changes:
> > > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > > > sts/1/diffs
> > > > >
> > > > > I'll take time to review your whole series and will see whether we
> > > > > can somehow keep the best parts of each.
> > > >
> > > > Thanks and agree. Let's work together and get this implemented in DRM.
> > > >
> > > > > Curious to hear other opinions on the blob vs new DRM object question as
> > > well.
> > >
> > > I can see a few advantages with the blob approach.
> > >
> > > User space can store the entire state in one blob and just assign a new blob to
> > > change to another pipeline configuration.
> >
> > Agree
> >
> > > However, I would argue that changing a lot of properties is already common
> > > practice and works well. User space can deal with it and has abstractions to
> > > make this managable.
> >
> > Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
>
> The structure of the atomic commit ioctl and the KMS property system is
> even more flexible.
>
> > Lut distribution, number of segments, samples in each segment, precision of luts etc.
> > can be precisely defined and userspace will get a complete picture of the underlying
> > hardware and its capabilities. Also this is being done with just 2 properties. Leaving
> > scope for future addition of any standard color operation as well.
>
> The number of properties does not seem too useful to strictly minimise
> over other aspects.
>
>
> Thanks,
> pq
>
> >
> > I feel blob approach once properly documented is a bit more flexible and scalable.
> > Maybe I am bit biased here 😊 but all ideas are welcome.
> >
> > We have implemented some IGT's as well to explain the design better. Link below:
> > https://patchwork.freedesktop.org/series/123018/
> >
> > Thanks Sebastian for the feedback.
> >
> > Regards,
> > Uma Shankar
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-09-05 12:33 ` Sebastian Wick
0 siblings, 0 replies; 109+ messages in thread
From: Sebastian Wick @ 2023-09-05 12:33 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Alexander Goins, Pekka Paalanen, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Harry Wentland, Christopher Braga
On Tue, Sep 05, 2023 at 02:33:26PM +0300, Pekka Paalanen wrote:
> On Mon, 4 Sep 2023 14:29:56 +0000
> "Shankar, Uma" <uma.shankar@intel.com> wrote:
>
> > > -----Original Message-----
> > > From: Sebastian Wick <sebastian.wick@redhat.com>
> > > Sent: Thursday, August 31, 2023 2:46 AM
> > > To: Shankar, Uma <uma.shankar@intel.com>
> > > Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> > > gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> > > devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> > > Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> > > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > <quic_cbraga@quicinc.com>
> > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > >
> > > On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > > <quic_cbraga@quicinc.com>
> > > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > > >
> > > > > +CC Naseer and Chris, FYI
> > > > >
> > > > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > > > >
> > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > Introduction
> > > > > > ============
> > > > > >
> > > > > > Modern hardwares have various color processing capabilities both
> > > > > > at pre-blending and post-blending phases in the color pipeline.
> > > > > > The current drm implementation exposes only the post-blending
> > > > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > > > There are multiple use cases where pre-blending color hardware
> > > > > > will be
> > > > > > useful:
> > > > > > a) Linearization of input buffers encoded in various transfer
> > > > > > functions.
> > > > > > b) Color Space conversion
> > > > > > c) Tone mapping
> > > > > > d) Frame buffer format conversion
> > > > > > e) Non-linearization of buffer(apply transfer function)
> > > > > > f) 3D Luts
> > > > > >
> > > > > > and other miscellaneous color operations.
> > > > > >
> > > > > > Hence, there is a need to expose the color capabilities of the
> > > > > > hardware to user-space. This will help userspace/middleware to use
> > > > > > display hardware for color processing and blending instead of
> > > > > > doing it through GPU shaders.
> > > > > >
> > > > >
> > > > > Thanks, Uma, for sending this. I've been working on something
> > > > > similar but you beat me to it. :)
> > > >
> > > > Thanks Harry for the useful feedback and overall collaboration on this so far.
> > > >
> > > > > >
> > > > > > Work done so far and relevant references
> > > > > > ========================================
> > > > > >
> > > > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > > > Broad consensus is there that we need a generic API at drm core to
> > > > > > suffice the use case of various HW vendors. Below are the links
> > > > > > capturing the discussion so far.
> > > > > >
> > > > > > Intel's Plane Color Implementation:
> > > > > > https://patchwork.freedesktop.org/series/90825/
> > > > > > AMD's Plane Color Implementation:
> > > > > > https://patchwork.freedesktop.org/series/116862/
> > > > > >
> > > > > >
> > > > > > Hackfest conclusions
> > > > > > ====================
> > > > > >
> > > > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > > > industry stakeholders together and converge on a common uapi
> > > expectations.
> > > > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > > > and other prominent user-space developers and maintainers.
> > > > > >
> > > > > > Discussions happened on the uapi expectations, opens, nature of
> > > > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > > > the same and the path forward. Consensus was made that drm core
> > > > > > should implement descriptive APIs and not go with prescriptive
> > > > > > APIs. DRM core should just expose the hardware capabilities;
> > > > > > enabling, customizing and programming the same should be done by
> > > > > > the user-space. Driver should just
> > > > > honor the user space request without doing any operations internally.
> > > > > >
> > > > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > > > and an UAPI RFC which can be referred to here:
> > > > > >
> > > > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > > > _hD5
> > > > > >
> > > > >
> > > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> > > Q
> > > > > Wn48
> > > > > > 8=@emersion.fr/
> > > > > >
> > > > > >
> > > > > > Design considerations
> > > > > > =====================
> > > > > >
> > > > > > Following are the important aspects taken into account while
> > > > > > designing the current RFC
> > > > > > proposal:
> > > > > >
> > > > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > > > only one
> > > > > can be used)
> > > > > > 2. Position of the HW block in the pipeline can be programmable
> > > > > > 3. LUTs can be one dimentional or three dimentional
> > > > > > 4. Number of LUT entries can vary across platforms
> > > > > > 5. Precision of LUT entries can vary across platforms
> > > > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> > > Logarithmic,
> > > > > > Piece-Wise Linear(PWL) etc
> > > > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > > > blocks.
> > > > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > > > 8. Custom non-standard HW implementation.
> > > > > > 9. Leaving scope for some vendor defined pescriptive
> > > > > > implementation if
> > > > > required.
> > > > > > 10.Scope to handle any modification in hardware as technology
> > > > > > evolves
> > > > > >
> > > > > > The current proposal takes into account the above considerations
> > > > > > while keeping the implementation as generic as possible leaving
> > > > > > scope for future
> > > > > additions or modifications.
> > > > > >
> > > > > > This proposal is also in line to the details mentioned by Simon's
> > > > > > RFC covering all the aspects discussed in hackfest.
> > > > > >
> > > > > >
> > > > > > Outline of the implementation
> > > > > > ============================
> > > > > >
> > > > > > Each Color Hardware block will be represented by a data structure
> > > drm_color_op.
> > > > > > These color operations will form the building blocks of a color
> > > > > > pipeline which best represents the underlying Hardware. Color
> > > > > > operations can be re-arranged, substracted or added to create
> > > > > > distinct color pipelines to accurately describe the Hardware
> > > > > > blocks present in the display
> > > > > engine.
> > > > >
> > > > > Who is doing the arranging of color operations? IMO a driver should
> > > > > define one or more respective pipelines that can be selected by
> > > > > userspace. This seems to be what you're talking about after (I
> > > > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> > > to add clarifications in order to avoid confusion.
> > > >
> > > > Yes it's the driver who will set the pipeline based on the underlying
> > > > hardware arrangement and possible combinations. There can be multiple
> > > > pipelines possible if hardware can be muxed or order can be re-arranged (all
> > > viable combinations should be defined as a pipeline in driver).
> > > > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> > > >
> > > > > >
> > > > > > In this proposal, a color pipeline is represented as an array of
> > > > > > struct drm_color_op. For individual color operation, we add blobs
> > > > > > to advertise the capability of the particular Hardware block.
> > > > > >
> > > > > > This color pipeline is then packaged within a blob for the user
> > > > > > space to retrieve it.
> > > > > >
> > > > >
> > > > > Would it be better to expose the drm_color_ops directly, instead of
> > > > > packing a array of drm_color_ops into a blob and then giving that to
> > > userspace.
> > > >
> > > > Advantage we see in packing in blobs is that interface will be
> > > > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> > > and then its just parsing the data.
> > > > This way the entire underlying hardware blocks with pipeline will be available to
> > > user.
> > >
> > > I don't see how parsing a blob is easier than requesting the color ops from the
> > > kernel. User space is already equiped with getting KMS objects and the igt test
> > > code from Harry shows that this is all pretty trivial plumbing.
> >
> > There are multiple color operations possible with unique lut distribution and
> > capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
> > Having all the information with one query and property and also be able to set the
> > same with just one property call using blobs is better than multiple calls to driver.
> > This can be useful in high refresh rate cases where not much time is there to program
> > the hardware state. Latency of multiple calls to driver can be avoided.
>
> Hi,
>
> querying all that information only happens once, at KMS client start-up.
>
> Setting up a color pipeline is already a single call: the atomic commit ioctl.
Well, clients also issue a bunch of ioctl to set some properties to the
desired state and then you might run through a whole bunch of
configurations to find one that actually works, so there is a case to be
made that there are a lot of ioctls involved.
I just don't think this is an issue right now. Nobody has been
complaining about the ioctls being a limiting factor so why should we
optimize for this? Especially because it brings with it a bunch of
disadvantages.
Anyway, I agree with the sentiment here: this is not something we should
optimize for.
>
> >
> > > > For a particular hardware block in pipeline, user can get the relevant
> > > > details from blob representing that particular block. We have created
> > > > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> > > This is just to clarify the idea.
> > >
> > > The blob is also not introspectable with the usual tools whereas exposing them as
> > > properties would be.
> > >
> > > It also would, like Pekka correctly noted, bring a whole bunch of issues about
> > > compatibility and versioning that are well understood with objects
> > > + properties.
> >
> > The blob should be standardized in the UAPI and structures to parse them should be fixed.
> > With this compatibility issues can be prevented.
>
> I think that is short-sighted.
>
> > > > Also since info is passed via blobs we have the flexibility to even
> > > > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > > > for custom private hardware blocks as well which driver can work with
> > > respective HAL and get that implemented.
> > >
> > > When color ops are real KMS objects they still can have properties which are
> > > blobs that can store LUTs and other data.
> > >
> > > And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> > > have changed in that regard and it simply is not allowed.
> >
> > Private blocks are not standardized but are vendor specific. So generic userspace will
> > ignore these. However vendor and its respective HAL can make use of this field and leaves
> > a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
> > standardized color operations which will be defined as enum in UAPI.
>
> Vendors can have and expose their own unique snowflake operations
> without any "private" as well: pick an unused operation type code, and
> document what it does. Advertise it in some pipelines.
>
> Vendor HALs can make use of it, but it also allows generic userspace to
> make use of it at will, and it allows other vendors to implement the
> same and benefit from it without needing to patch every userspace.
>
> Or does Intel not want other vendors to potentially make use of their
> UAPIs?
>
> > > > We can even define prescriptive operations as a private entry and
> > > > enable it if a certain driver and HAL agree.
> > > >
> > > > > > To advertise the available color pipelines, an immutable ENUM
> > > > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> > > blob id's as values.
> > > > > > With each blob id representing a distinct color pipeline based on
> > > > > > underlying HW capabilities and their respective combinations.
> > > > > >
> > > > > > Once the user space decides on a color pipeline, it can set the
> > > > > > pipeline and the corresponding data for the hardware blocks within
> > > > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > > >
> > > > >
> > > > > When I discussed this at the hackfest with Simon he proposed a new
> > > > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > > > NULL if its the last in the pipeline. You can then have a mutable
> > > > > enum property on the plane to discover and select a color pipeline.
> > > >
> > > > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> > > the design.
> > > > Personally I feel the one proposed in the current RFC will do the same
> > > > thing envisioned by Simon and you Harry. Management of the pipeline,
> > > > addition, deletion and flexibility to represent hardware is more with blobs with
> > > the relevant structures agreed in UAPI.
> > > >
> > > > > This seems a bit more transparent than a blob. You can see my
> > > > > changes (unfortunately very WIP, don't look too closely at
> > > > > individual patches) at
> > > > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > > > ffs
> > > > >
> > > > > libdrm changes:
> > > > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > > > s
> > > >
> > > > Sure, will check the same.
> > > >
> > > > > IGT changes:
> > > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > > > sts/1/diffs
> > > > >
> > > > > I'll take time to review your whole series and will see whether we
> > > > > can somehow keep the best parts of each.
> > > >
> > > > Thanks and agree. Let's work together and get this implemented in DRM.
> > > >
> > > > > Curious to hear other opinions on the blob vs new DRM object question as
> > > well.
> > >
> > > I can see a few advantages with the blob approach.
> > >
> > > User space can store the entire state in one blob and just assign a new blob to
> > > change to another pipeline configuration.
> >
> > Agree
> >
> > > However, I would argue that changing a lot of properties is already common
> > > practice and works well. User space can deal with it and has abstractions to
> > > make this managable.
> >
> > Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
>
> The structure of the atomic commit ioctl and the KMS property system is
> even more flexible.
>
> > Lut distribution, number of segments, samples in each segment, precision of luts etc.
> > can be precisely defined and userspace will get a complete picture of the underlying
> > hardware and its capabilities. Also this is being done with just 2 properties. Leaving
> > scope for future addition of any standard color operation as well.
>
> The number of properties does not seem too useful to strictly minimise
> over other aspects.
>
>
> Thanks,
> pq
>
> >
> > I feel blob approach once properly documented is a bit more flexible and scalable.
> > Maybe I am bit biased here 😊 but all ideas are welcome.
> >
> > We have implemented some IGT's as well to explain the design better. Link below:
> > https://patchwork.freedesktop.org/series/123018/
> >
> > Thanks Sebastian for the feedback.
> >
> > Regards,
> > Uma Shankar
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 00/33] Add Support for Plane Color Pipeline
2023-09-05 12:33 ` [Intel-gfx] " Sebastian Wick
@ 2023-09-05 12:57 ` Sebastian Wick
-1 siblings, 0 replies; 109+ messages in thread
From: Sebastian Wick @ 2023-09-05 12:57 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Pekka Paalanen, Shashank Sharma, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Shankar, Uma,
Naseer Ahmed, Christopher Braga
On Tue, Sep 05, 2023 at 02:33:04PM +0200, Sebastian Wick wrote:
> On Tue, Sep 05, 2023 at 02:33:26PM +0300, Pekka Paalanen wrote:
> > On Mon, 4 Sep 2023 14:29:56 +0000
> > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> >
> > > > -----Original Message-----
> > > > From: Sebastian Wick <sebastian.wick@redhat.com>
> > > > Sent: Thursday, August 31, 2023 2:46 AM
> > > > To: Shankar, Uma <uma.shankar@intel.com>
> > > > Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> > > > gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> > > > devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> > > > Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> > > > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > <quic_cbraga@quicinc.com>
> > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > >
> > > > On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > > > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > > > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > > > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > > > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > > > <quic_cbraga@quicinc.com>
> > > > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > > > >
> > > > > > +CC Naseer and Chris, FYI
> > > > > >
> > > > > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > > > > >
> > > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > > Introduction
> > > > > > > ============
> > > > > > >
> > > > > > > Modern hardwares have various color processing capabilities both
> > > > > > > at pre-blending and post-blending phases in the color pipeline.
> > > > > > > The current drm implementation exposes only the post-blending
> > > > > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > > > > There are multiple use cases where pre-blending color hardware
> > > > > > > will be
> > > > > > > useful:
> > > > > > > a) Linearization of input buffers encoded in various transfer
> > > > > > > functions.
> > > > > > > b) Color Space conversion
> > > > > > > c) Tone mapping
> > > > > > > d) Frame buffer format conversion
> > > > > > > e) Non-linearization of buffer(apply transfer function)
> > > > > > > f) 3D Luts
> > > > > > >
> > > > > > > and other miscellaneous color operations.
> > > > > > >
> > > > > > > Hence, there is a need to expose the color capabilities of the
> > > > > > > hardware to user-space. This will help userspace/middleware to use
> > > > > > > display hardware for color processing and blending instead of
> > > > > > > doing it through GPU shaders.
> > > > > > >
> > > > > >
> > > > > > Thanks, Uma, for sending this. I've been working on something
> > > > > > similar but you beat me to it. :)
> > > > >
> > > > > Thanks Harry for the useful feedback and overall collaboration on this so far.
> > > > >
> > > > > > >
> > > > > > > Work done so far and relevant references
> > > > > > > ========================================
> > > > > > >
> > > > > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > > > > Broad consensus is there that we need a generic API at drm core to
> > > > > > > suffice the use case of various HW vendors. Below are the links
> > > > > > > capturing the discussion so far.
> > > > > > >
> > > > > > > Intel's Plane Color Implementation:
> > > > > > > https://patchwork.freedesktop.org/series/90825/
> > > > > > > AMD's Plane Color Implementation:
> > > > > > > https://patchwork.freedesktop.org/series/116862/
> > > > > > >
> > > > > > >
> > > > > > > Hackfest conclusions
> > > > > > > ====================
> > > > > > >
> > > > > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > > > > industry stakeholders together and converge on a common uapi
> > > > expectations.
> > > > > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > > > > and other prominent user-space developers and maintainers.
> > > > > > >
> > > > > > > Discussions happened on the uapi expectations, opens, nature of
> > > > > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > > > > the same and the path forward. Consensus was made that drm core
> > > > > > > should implement descriptive APIs and not go with prescriptive
> > > > > > > APIs. DRM core should just expose the hardware capabilities;
> > > > > > > enabling, customizing and programming the same should be done by
> > > > > > > the user-space. Driver should just
> > > > > > honor the user space request without doing any operations internally.
> > > > > > >
> > > > > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > > > > and an UAPI RFC which can be referred to here:
> > > > > > >
> > > > > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > > > > _hD5
> > > > > > >
> > > > > >
> > > > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> > > > Q
> > > > > > Wn48
> > > > > > > 8=@emersion.fr/
> > > > > > >
> > > > > > >
> > > > > > > Design considerations
> > > > > > > =====================
> > > > > > >
> > > > > > > Following are the important aspects taken into account while
> > > > > > > designing the current RFC
> > > > > > > proposal:
> > > > > > >
> > > > > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > > > > only one
> > > > > > can be used)
> > > > > > > 2. Position of the HW block in the pipeline can be programmable
> > > > > > > 3. LUTs can be one dimentional or three dimentional
> > > > > > > 4. Number of LUT entries can vary across platforms
> > > > > > > 5. Precision of LUT entries can vary across platforms
> > > > > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> > > > Logarithmic,
> > > > > > > Piece-Wise Linear(PWL) etc
> > > > > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > > > > blocks.
> > > > > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > > > > 8. Custom non-standard HW implementation.
> > > > > > > 9. Leaving scope for some vendor defined pescriptive
> > > > > > > implementation if
> > > > > > required.
> > > > > > > 10.Scope to handle any modification in hardware as technology
> > > > > > > evolves
> > > > > > >
> > > > > > > The current proposal takes into account the above considerations
> > > > > > > while keeping the implementation as generic as possible leaving
> > > > > > > scope for future
> > > > > > additions or modifications.
> > > > > > >
> > > > > > > This proposal is also in line to the details mentioned by Simon's
> > > > > > > RFC covering all the aspects discussed in hackfest.
> > > > > > >
> > > > > > >
> > > > > > > Outline of the implementation
> > > > > > > ============================
> > > > > > >
> > > > > > > Each Color Hardware block will be represented by a data structure
> > > > drm_color_op.
> > > > > > > These color operations will form the building blocks of a color
> > > > > > > pipeline which best represents the underlying Hardware. Color
> > > > > > > operations can be re-arranged, substracted or added to create
> > > > > > > distinct color pipelines to accurately describe the Hardware
> > > > > > > blocks present in the display
> > > > > > engine.
> > > > > >
> > > > > > Who is doing the arranging of color operations? IMO a driver should
> > > > > > define one or more respective pipelines that can be selected by
> > > > > > userspace. This seems to be what you're talking about after (I
> > > > > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> > > > to add clarifications in order to avoid confusion.
> > > > >
> > > > > Yes it's the driver who will set the pipeline based on the underlying
> > > > > hardware arrangement and possible combinations. There can be multiple
> > > > > pipelines possible if hardware can be muxed or order can be re-arranged (all
> > > > viable combinations should be defined as a pipeline in driver).
> > > > > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> > > > >
> > > > > > >
> > > > > > > In this proposal, a color pipeline is represented as an array of
> > > > > > > struct drm_color_op. For individual color operation, we add blobs
> > > > > > > to advertise the capability of the particular Hardware block.
> > > > > > >
> > > > > > > This color pipeline is then packaged within a blob for the user
> > > > > > > space to retrieve it.
> > > > > > >
> > > > > >
> > > > > > Would it be better to expose the drm_color_ops directly, instead of
> > > > > > packing a array of drm_color_ops into a blob and then giving that to
> > > > userspace.
> > > > >
> > > > > Advantage we see in packing in blobs is that interface will be
> > > > > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> > > > and then its just parsing the data.
> > > > > This way the entire underlying hardware blocks with pipeline will be available to
> > > > user.
> > > >
> > > > I don't see how parsing a blob is easier than requesting the color ops from the
> > > > kernel. User space is already equiped with getting KMS objects and the igt test
> > > > code from Harry shows that this is all pretty trivial plumbing.
> > >
> > > There are multiple color operations possible with unique lut distribution and
> > > capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
> > > Having all the information with one query and property and also be able to set the
> > > same with just one property call using blobs is better than multiple calls to driver.
> > > This can be useful in high refresh rate cases where not much time is there to program
> > > the hardware state. Latency of multiple calls to driver can be avoided.
> >
> > Hi,
> >
> > querying all that information only happens once, at KMS client start-up.
> >
> > Setting up a color pipeline is already a single call: the atomic commit ioctl.
>
> Well, clients also issue a bunch of ioctl to set some properties to the
> desired state and then you might run through a whole bunch of
> configurations to find one that actually works, so there is a case to be
> made that there are a lot of ioctls involved.
pq pointed at that for atomic this is actually really just a single
ioctl so really there is no advantage here at all.
> I just don't think this is an issue right now. Nobody has been
> complaining about the ioctls being a limiting factor so why should we
> optimize for this? Especially because it brings with it a bunch of
> disadvantages.
>
> Anyway, I agree with the sentiment here: this is not something we should
> optimize for.
>
> >
> > >
> > > > > For a particular hardware block in pipeline, user can get the relevant
> > > > > details from blob representing that particular block. We have created
> > > > > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> > > > This is just to clarify the idea.
> > > >
> > > > The blob is also not introspectable with the usual tools whereas exposing them as
> > > > properties would be.
> > > >
> > > > It also would, like Pekka correctly noted, bring a whole bunch of issues about
> > > > compatibility and versioning that are well understood with objects
> > > > + properties.
> > >
> > > The blob should be standardized in the UAPI and structures to parse them should be fixed.
> > > With this compatibility issues can be prevented.
> >
> > I think that is short-sighted.
> >
> > > > > Also since info is passed via blobs we have the flexibility to even
> > > > > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > > > > for custom private hardware blocks as well which driver can work with
> > > > respective HAL and get that implemented.
> > > >
> > > > When color ops are real KMS objects they still can have properties which are
> > > > blobs that can store LUTs and other data.
> > > >
> > > > And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> > > > have changed in that regard and it simply is not allowed.
> > >
> > > Private blocks are not standardized but are vendor specific. So generic userspace will
> > > ignore these. However vendor and its respective HAL can make use of this field and leaves
> > > a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
> > > standardized color operations which will be defined as enum in UAPI.
> >
> > Vendors can have and expose their own unique snowflake operations
> > without any "private" as well: pick an unused operation type code, and
> > document what it does. Advertise it in some pipelines.
> >
> > Vendor HALs can make use of it, but it also allows generic userspace to
> > make use of it at will, and it allows other vendors to implement the
> > same and benefit from it without needing to patch every userspace.
> >
> > Or does Intel not want other vendors to potentially make use of their
> > UAPIs?
> >
> > > > > We can even define prescriptive operations as a private entry and
> > > > > enable it if a certain driver and HAL agree.
> > > > >
> > > > > > > To advertise the available color pipelines, an immutable ENUM
> > > > > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> > > > blob id's as values.
> > > > > > > With each blob id representing a distinct color pipeline based on
> > > > > > > underlying HW capabilities and their respective combinations.
> > > > > > >
> > > > > > > Once the user space decides on a color pipeline, it can set the
> > > > > > > pipeline and the corresponding data for the hardware blocks within
> > > > > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > > > >
> > > > > >
> > > > > > When I discussed this at the hackfest with Simon he proposed a new
> > > > > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > > > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > > > > NULL if its the last in the pipeline. You can then have a mutable
> > > > > > enum property on the plane to discover and select a color pipeline.
> > > > >
> > > > > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> > > > the design.
> > > > > Personally I feel the one proposed in the current RFC will do the same
> > > > > thing envisioned by Simon and you Harry. Management of the pipeline,
> > > > > addition, deletion and flexibility to represent hardware is more with blobs with
> > > > the relevant structures agreed in UAPI.
> > > > >
> > > > > > This seems a bit more transparent than a blob. You can see my
> > > > > > changes (unfortunately very WIP, don't look too closely at
> > > > > > individual patches) at
> > > > > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > > > > ffs
> > > > > >
> > > > > > libdrm changes:
> > > > > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > > > > s
> > > > >
> > > > > Sure, will check the same.
> > > > >
> > > > > > IGT changes:
> > > > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > > > > sts/1/diffs
> > > > > >
> > > > > > I'll take time to review your whole series and will see whether we
> > > > > > can somehow keep the best parts of each.
> > > > >
> > > > > Thanks and agree. Let's work together and get this implemented in DRM.
> > > > >
> > > > > > Curious to hear other opinions on the blob vs new DRM object question as
> > > > well.
> > > >
> > > > I can see a few advantages with the blob approach.
> > > >
> > > > User space can store the entire state in one blob and just assign a new blob to
> > > > change to another pipeline configuration.
> > >
> > > Agree
> > >
> > > > However, I would argue that changing a lot of properties is already common
> > > > practice and works well. User space can deal with it and has abstractions to
> > > > make this managable.
> > >
> > > Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
> >
> > The structure of the atomic commit ioctl and the KMS property system is
> > even more flexible.
> >
> > > Lut distribution, number of segments, samples in each segment, precision of luts etc.
> > > can be precisely defined and userspace will get a complete picture of the underlying
> > > hardware and its capabilities. Also this is being done with just 2 properties. Leaving
> > > scope for future addition of any standard color operation as well.
> >
> > The number of properties does not seem too useful to strictly minimise
> > over other aspects.
> >
> >
> > Thanks,
> > pq
> >
> > >
> > > I feel blob approach once properly documented is a bit more flexible and scalable.
> > > Maybe I am bit biased here 😊 but all ideas are welcome.
> > >
> > > We have implemented some IGT's as well to explain the design better. Link below:
> > > https://patchwork.freedesktop.org/series/123018/
> > >
> > > Thanks Sebastian for the feedback.
> > >
> > > Regards,
> > > Uma Shankar
>
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 00/33] Add Support for Plane Color Pipeline
@ 2023-09-05 12:57 ` Sebastian Wick
0 siblings, 0 replies; 109+ messages in thread
From: Sebastian Wick @ 2023-09-05 12:57 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Alexander Goins, Pekka Paalanen, intel-gfx, dri-devel,
wayland-devel, Melissa Wen, Jonas Ådahl, Naseer Ahmed,
Harry Wentland, Christopher Braga
On Tue, Sep 05, 2023 at 02:33:04PM +0200, Sebastian Wick wrote:
> On Tue, Sep 05, 2023 at 02:33:26PM +0300, Pekka Paalanen wrote:
> > On Mon, 4 Sep 2023 14:29:56 +0000
> > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> >
> > > > -----Original Message-----
> > > > From: Sebastian Wick <sebastian.wick@redhat.com>
> > > > Sent: Thursday, August 31, 2023 2:46 AM
> > > > To: Shankar, Uma <uma.shankar@intel.com>
> > > > Cc: Harry Wentland <harry.wentland@amd.com>; intel-
> > > > gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; wayland-
> > > > devel@lists.freedesktop.org; Ville Syrjala <ville.syrjala@linux.intel.com>; Pekka
> > > > Paalanen <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>; Shashank
> > > > Sharma <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > <quic_cbraga@quicinc.com>
> > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > >
> > > > On Wed, Aug 30, 2023 at 08:47:37AM +0000, Shankar, Uma wrote:
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > > Sent: Wednesday, August 30, 2023 12:56 AM
> > > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > > intel-gfx@lists.freedesktop.org; dri- devel@lists.freedesktop.org
> > > > > > Cc: wayland-devel@lists.freedesktop.org; Ville Syrjala
> > > > > > <ville.syrjala@linux.intel.com>; Pekka Paalanen
> > > > > > <pekka.paalanen@collabora.com>; Simon Ser <contact@emersion.fr>;
> > > > > > Melissa Wen <mwen@igalia.com>; Jonas Ådahl <jadahl@redhat.com>;
> > > > > > Sebastian Wick <sebastian.wick@redhat.com>; Shashank Sharma
> > > > > > <shashank.sharma@amd.com>; Alexander Goins <agoins@nvidia.com>;
> > > > > > Naseer Ahmed <quic_naseer@quicinc.com>; Christopher Braga
> > > > > > <quic_cbraga@quicinc.com>
> > > > > > Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline
> > > > > >
> > > > > > +CC Naseer and Chris, FYI
> > > > > >
> > > > > > See https://patchwork.freedesktop.org/series/123024/ for whole series.
> > > > > >
> > > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > > Introduction
> > > > > > > ============
> > > > > > >
> > > > > > > Modern hardwares have various color processing capabilities both
> > > > > > > at pre-blending and post-blending phases in the color pipeline.
> > > > > > > The current drm implementation exposes only the post-blending
> > > > > > > color hardware blocks. Support for pre-blending hardware is missing.
> > > > > > > There are multiple use cases where pre-blending color hardware
> > > > > > > will be
> > > > > > > useful:
> > > > > > > a) Linearization of input buffers encoded in various transfer
> > > > > > > functions.
> > > > > > > b) Color Space conversion
> > > > > > > c) Tone mapping
> > > > > > > d) Frame buffer format conversion
> > > > > > > e) Non-linearization of buffer(apply transfer function)
> > > > > > > f) 3D Luts
> > > > > > >
> > > > > > > and other miscellaneous color operations.
> > > > > > >
> > > > > > > Hence, there is a need to expose the color capabilities of the
> > > > > > > hardware to user-space. This will help userspace/middleware to use
> > > > > > > display hardware for color processing and blending instead of
> > > > > > > doing it through GPU shaders.
> > > > > > >
> > > > > >
> > > > > > Thanks, Uma, for sending this. I've been working on something
> > > > > > similar but you beat me to it. :)
> > > > >
> > > > > Thanks Harry for the useful feedback and overall collaboration on this so far.
> > > > >
> > > > > > >
> > > > > > > Work done so far and relevant references
> > > > > > > ========================================
> > > > > > >
> > > > > > > Some implementation is done by Intel and AMD/Igalia to address the same.
> > > > > > > Broad consensus is there that we need a generic API at drm core to
> > > > > > > suffice the use case of various HW vendors. Below are the links
> > > > > > > capturing the discussion so far.
> > > > > > >
> > > > > > > Intel's Plane Color Implementation:
> > > > > > > https://patchwork.freedesktop.org/series/90825/
> > > > > > > AMD's Plane Color Implementation:
> > > > > > > https://patchwork.freedesktop.org/series/116862/
> > > > > > >
> > > > > > >
> > > > > > > Hackfest conclusions
> > > > > > > ====================
> > > > > > >
> > > > > > > HDR/Color Hackfest was organised by Redhat to bring all the
> > > > > > > industry stakeholders together and converge on a common uapi
> > > > expectations.
> > > > > > > Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia
> > > > > > > and other prominent user-space developers and maintainers.
> > > > > > >
> > > > > > > Discussions happened on the uapi expectations, opens, nature of
> > > > > > > hardware of multiple hardware vendors, challenges in generalizing
> > > > > > > the same and the path forward. Consensus was made that drm core
> > > > > > > should implement descriptive APIs and not go with prescriptive
> > > > > > > APIs. DRM core should just expose the hardware capabilities;
> > > > > > > enabling, customizing and programming the same should be done by
> > > > > > > the user-space. Driver should just
> > > > > > honor the user space request without doing any operations internally.
> > > > > > >
> > > > > > > Thanks to Simon Ser, for nicely documenting the design consensus
> > > > > > > and an UAPI RFC which can be referred to here:
> > > > > > >
> > > > > > > https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
> > > > > > > _hD5
> > > > > > >
> > > > > >
> > > > nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1
> > > > Q
> > > > > > Wn48
> > > > > > > 8=@emersion.fr/
> > > > > > >
> > > > > > >
> > > > > > > Design considerations
> > > > > > > =====================
> > > > > > >
> > > > > > > Following are the important aspects taken into account while
> > > > > > > designing the current RFC
> > > > > > > proposal:
> > > > > > >
> > > > > > > 1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
> > > > > > > only one
> > > > > > can be used)
> > > > > > > 2. Position of the HW block in the pipeline can be programmable
> > > > > > > 3. LUTs can be one dimentional or three dimentional
> > > > > > > 4. Number of LUT entries can vary across platforms
> > > > > > > 5. Precision of LUT entries can vary across platforms
> > > > > > > 6. Distribution of LUT entries may vary. e.g Mutli-segmented,
> > > > Logarithmic,
> > > > > > > Piece-Wise Linear(PWL) etc
> > > > > > > 7. There can be parameterized/non-parameterized fixed function HW
> > > > > > blocks.
> > > > > > > e.g. Just a hardware bit, to convert from one color space to another.
> > > > > > > 8. Custom non-standard HW implementation.
> > > > > > > 9. Leaving scope for some vendor defined pescriptive
> > > > > > > implementation if
> > > > > > required.
> > > > > > > 10.Scope to handle any modification in hardware as technology
> > > > > > > evolves
> > > > > > >
> > > > > > > The current proposal takes into account the above considerations
> > > > > > > while keeping the implementation as generic as possible leaving
> > > > > > > scope for future
> > > > > > additions or modifications.
> > > > > > >
> > > > > > > This proposal is also in line to the details mentioned by Simon's
> > > > > > > RFC covering all the aspects discussed in hackfest.
> > > > > > >
> > > > > > >
> > > > > > > Outline of the implementation
> > > > > > > ============================
> > > > > > >
> > > > > > > Each Color Hardware block will be represented by a data structure
> > > > drm_color_op.
> > > > > > > These color operations will form the building blocks of a color
> > > > > > > pipeline which best represents the underlying Hardware. Color
> > > > > > > operations can be re-arranged, substracted or added to create
> > > > > > > distinct color pipelines to accurately describe the Hardware
> > > > > > > blocks present in the display
> > > > > > engine.
> > > > > >
> > > > > > Who is doing the arranging of color operations? IMO a driver should
> > > > > > define one or more respective pipelines that can be selected by
> > > > > > userspace. This seems to be what you're talking about after (I
> > > > > > haven't reviewed the whole thing yet). Might be best to drop this sentence or
> > > > to add clarifications in order to avoid confusion.
> > > > >
> > > > > Yes it's the driver who will set the pipeline based on the underlying
> > > > > hardware arrangement and possible combinations. There can be multiple
> > > > > pipelines possible if hardware can be muxed or order can be re-arranged (all
> > > > viable combinations should be defined as a pipeline in driver).
> > > > > Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
> > > > >
> > > > > > >
> > > > > > > In this proposal, a color pipeline is represented as an array of
> > > > > > > struct drm_color_op. For individual color operation, we add blobs
> > > > > > > to advertise the capability of the particular Hardware block.
> > > > > > >
> > > > > > > This color pipeline is then packaged within a blob for the user
> > > > > > > space to retrieve it.
> > > > > > >
> > > > > >
> > > > > > Would it be better to expose the drm_color_ops directly, instead of
> > > > > > packing a array of drm_color_ops into a blob and then giving that to
> > > > userspace.
> > > > >
> > > > > Advantage we see in packing in blobs is that interface will be
> > > > > cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user
> > > > and then its just parsing the data.
> > > > > This way the entire underlying hardware blocks with pipeline will be available to
> > > > user.
> > > >
> > > > I don't see how parsing a blob is easier than requesting the color ops from the
> > > > kernel. User space is already equiped with getting KMS objects and the igt test
> > > > code from Harry shows that this is all pretty trivial plumbing.
> > >
> > > There are multiple color operations possible with unique lut distribution and
> > > capabilities. Also the order of hardware blocks and possibility of multiple pipelines.
> > > Having all the information with one query and property and also be able to set the
> > > same with just one property call using blobs is better than multiple calls to driver.
> > > This can be useful in high refresh rate cases where not much time is there to program
> > > the hardware state. Latency of multiple calls to driver can be avoided.
> >
> > Hi,
> >
> > querying all that information only happens once, at KMS client start-up.
> >
> > Setting up a color pipeline is already a single call: the atomic commit ioctl.
>
> Well, clients also issue a bunch of ioctl to set some properties to the
> desired state and then you might run through a whole bunch of
> configurations to find one that actually works, so there is a case to be
> made that there are a lot of ioctls involved.
pq pointed at that for atomic this is actually really just a single
ioctl so really there is no advantage here at all.
> I just don't think this is an issue right now. Nobody has been
> complaining about the ioctls being a limiting factor so why should we
> optimize for this? Especially because it brings with it a bunch of
> disadvantages.
>
> Anyway, I agree with the sentiment here: this is not something we should
> optimize for.
>
> >
> > >
> > > > > For a particular hardware block in pipeline, user can get the relevant
> > > > > details from blob representing that particular block. We have created
> > > > > IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
> > > > This is just to clarify the idea.
> > > >
> > > > The blob is also not introspectable with the usual tools whereas exposing them as
> > > > properties would be.
> > > >
> > > > It also would, like Pekka correctly noted, bring a whole bunch of issues about
> > > > compatibility and versioning that are well understood with objects
> > > > + properties.
> > >
> > > The blob should be standardized in the UAPI and structures to parse them should be fixed.
> > > With this compatibility issues can be prevented.
> >
> > I think that is short-sighted.
> >
> > > > > Also since info is passed via blobs we have the flexibility to even
> > > > > define segmented LUTs and PWL hardware blocks. Also we have left scope
> > > > > for custom private hardware blocks as well which driver can work with
> > > > respective HAL and get that implemented.
> > > >
> > > > When color ops are real KMS objects they still can have properties which are
> > > > blobs that can store LUTs and other data.
> > > >
> > > > And we should avoid private blocks at all cost. In fact, I don't think the KMS rules
> > > > have changed in that regard and it simply is not allowed.
> > >
> > > Private blocks are not standardized but are vendor specific. So generic userspace will
> > > ignore these. However vendor and its respective HAL can make use of this field and leaves
> > > a scope to cater to such hardware vendors need. This doesn't affect the expectation of the
> > > standardized color operations which will be defined as enum in UAPI.
> >
> > Vendors can have and expose their own unique snowflake operations
> > without any "private" as well: pick an unused operation type code, and
> > document what it does. Advertise it in some pipelines.
> >
> > Vendor HALs can make use of it, but it also allows generic userspace to
> > make use of it at will, and it allows other vendors to implement the
> > same and benefit from it without needing to patch every userspace.
> >
> > Or does Intel not want other vendors to potentially make use of their
> > UAPIs?
> >
> > > > > We can even define prescriptive operations as a private entry and
> > > > > enable it if a certain driver and HAL agree.
> > > > >
> > > > > > > To advertise the available color pipelines, an immutable ENUM
> > > > > > > property "GET_COLOR_PIPELINE" is introduced. This enum property has
> > > > blob id's as values.
> > > > > > > With each blob id representing a distinct color pipeline based on
> > > > > > > underlying HW capabilities and their respective combinations.
> > > > > > >
> > > > > > > Once the user space decides on a color pipeline, it can set the
> > > > > > > pipeline and the corresponding data for the hardware blocks within
> > > > > > > the pipeline with the BLOB property "SET_COLOR_PIPELINE".
> > > > > > >
> > > > > >
> > > > > > When I discussed this at the hackfest with Simon he proposed a new
> > > > > > DRM object, (I've called it a drm_colorop) to represent a color operation.
> > > > > > Each drm_colorop has a "NEXT" pointer to another drm_colorop, or
> > > > > > NULL if its the last in the pipeline. You can then have a mutable
> > > > > > enum property on the plane to discover and select a color pipeline.
> > > > >
> > > > > Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
> > > > the design.
> > > > > Personally I feel the one proposed in the current RFC will do the same
> > > > > thing envisioned by Simon and you Harry. Management of the pipeline,
> > > > > addition, deletion and flexibility to represent hardware is more with blobs with
> > > > the relevant structures agreed in UAPI.
> > > > >
> > > > > > This seems a bit more transparent than a blob. You can see my
> > > > > > changes (unfortunately very WIP, don't look too closely at
> > > > > > individual patches) at
> > > > > > https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
> > > > > > ffs
> > > > > >
> > > > > > libdrm changes:
> > > > > > https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
> > > > > > s
> > > > >
> > > > > Sure, will check the same.
> > > > >
> > > > > > IGT changes:
> > > > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
> > > > > > sts/1/diffs
> > > > > >
> > > > > > I'll take time to review your whole series and will see whether we
> > > > > > can somehow keep the best parts of each.
> > > > >
> > > > > Thanks and agree. Let's work together and get this implemented in DRM.
> > > > >
> > > > > > Curious to hear other opinions on the blob vs new DRM object question as
> > > > well.
> > > >
> > > > I can see a few advantages with the blob approach.
> > > >
> > > > User space can store the entire state in one blob and just assign a new blob to
> > > > change to another pipeline configuration.
> > >
> > > Agree
> > >
> > > > However, I would argue that changing a lot of properties is already common
> > > > practice and works well. User space can deal with it and has abstractions to
> > > > make this managable.
> > >
> > > Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
> >
> > The structure of the atomic commit ioctl and the KMS property system is
> > even more flexible.
> >
> > > Lut distribution, number of segments, samples in each segment, precision of luts etc.
> > > can be precisely defined and userspace will get a complete picture of the underlying
> > > hardware and its capabilities. Also this is being done with just 2 properties. Leaving
> > > scope for future addition of any standard color operation as well.
> >
> > The number of properties does not seem too useful to strictly minimise
> > over other aspects.
> >
> >
> > Thanks,
> > pq
> >
> > >
> > > I feel blob approach once properly documented is a bit more flexible and scalable.
> > > Maybe I am bit biased here 😊 but all ideas are welcome.
> > >
> > > We have implemented some IGT's as well to explain the design better. Link below:
> > > https://patchwork.freedesktop.org/series/123018/
> > >
> > > Thanks Sebastian for the feedback.
> > >
> > > Regards,
> > > Uma Shankar
>
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-09-05 11:32 ` [Intel-gfx] " Pekka Paalanen
@ 2023-09-07 12:31 ` Shankar, Uma
-1 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-07 12:31 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Borah, Chaitanya Kumar
> -----Original Message-----
> From: Pekka Paalanen <ppaalanen@gmail.com>
> Sent: Tuesday, September 5, 2023 5:03 PM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org; Harry Wentland <harry.wentland@amd.com>;
> Sebastian Wick <sebastian.wick@redhat.com>; ville.syrjala@linux.intel.com;
> Jonas Adahl <jadahl@redhat.com>
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> Color Pipeline
>
> On Mon, 4 Sep 2023 13:44:49 +0000
> "Shankar, Uma" <uma.shankar@intel.com> wrote:
>
> > > -----Original Message-----
> > > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf
> > > Of Pekka Paalanen
> > > Sent: Wednesday, August 30, 2023 5:59 PM
> > > To: Shankar, Uma <uma.shankar@intel.com>
> > > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org;
> > > wayland- devel@lists.freedesktop.org
> > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > Plane Color Pipeline
> > >
> > > On Wed, 30 Aug 2023 08:59:36 +0000
> > > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> > >
> > > > > -----Original Message-----
> > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > intel-gfx@lists.freedesktop.org; dri-
> > > > > devel@lists.freedesktop.org
> > > > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > > > wayland- devel@lists.freedesktop.org
> > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for
> > > > > proposed Plane Color Pipeline
> > > > >
> > > > >
> > > > >
> > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > > >
> > > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > > > ---
> > > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > > > 1 file changed, 394 insertions(+)
> > > > > > create mode 100644
> > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > >
> > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > new file mode 100644
> > > > > > index 000000000000..60ce515b6ea7
> > > > > > --- /dev/null
> > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
> > > ...
> > >
> > > Hi Uma!
> >
> > Thanks Pekka for the feedback and useful inputs.
>
> Hi Uma,
>
> sorry to say, but the overall feeling I get from this proposal is that it is just the
> current color related KMS properties wrapped in a pipeline blob. That is not the
> re-design I believe we are looking for, and the feeling is based on several details
> that are just copied from the current property design. Also the "private" stuff has
> to go.
Hi Pekka,
Ok, got the concerns in general. We will try to evaluate in deeper detail the
property based design and come back if there are some issues or inputs.
At Intel we don't need private as of now, but we thought of having an option to
enable any custom hardware or vendor. But we can drop the same for now if
community doesn't feel the need for it.
> All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap
> distribution, and parametrized curves are good development, but right now we
> are looking at things one step higher level: the overall color pipeline design and
> how to represent any operation. Most of this series is considering details below
> the current attention level, hence I'm paying attention only to the first few
> patches.
We will need to precisely describe the hardware in userspace. Number of luts, precision,
segments etc.., we can't just pass EOTF's as enum from userspace and let driver put
hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour.
This will be needed as there are multiple colorspaces possible and even LUTS can be
used to perform tone mapping. So, we need userspace to be able to program luts directly.
This is something we must expose to userspace. We will check if this can be fitted in
property based approach.
Thanks for all the feedback and comments on the design. We will work on it and get back.
Regards,
Uma Shankar
> More below.
>
> > > > > > +This color pipeline is then packaged within a blob for the
> > > > > > +user space to retrieve it. Details can be found in the next
> > > > > > +section
> > > > > > +
> > > > >
> > > > > Not sure I like blobs that contain other blob ids.
> > > >
> > > > It provides flexibility and helps with just one interface to
> > > > userspace. Its easy to handle and manage once we get the hang of it 😊.
> > > >
> > > > We can clearly define the steps of parsing and data structures to
> > > > be used while interpreting and parsing the blobs.
> > >
> > > Don't forget extendability. Possibly every single struct will need
> > > some kind of versioning, and then it's not simple to parse anymore.
> > > Add to that new/old kernel vs. old/new userspace, and it seems a bit
> nightmarish to design.
> >
> > Structure to be used to interpret the blob should be defined as UAPI
> > only and is not expected to change once agreed upon. It should be interpreted
> like a standard property.
> > So structure to be used, say for 3dLut or 1dlut or CTM operations
> > should be standardized and fixed. No versioning of structure should be
> > done and same rules/restrictions as of UAPI property should be applied.
>
> That sounds like a UAPI that cannot be extended, either. So in a few years we'd be
> looking at replacing it. Or maybe you are just re-inventing the KMS object
> property system as blobs?
>
> Replacing a single KMS property is much easier than replacing the foundations of
> the whole color pipeline design.
>
>
> > ...
>
> > > I have a feeling that introspection will be valuable here, to help
> > > people understand what their hardware could do if they had the code to use
> it. 'name'
> > > and 'type' being integers require a translation table to strings
> > > before they are readable, and it would be best if the kernel itself provided that
> translation.
> >
> > Name and type can be standardized in enum and well documented in the UAPI.
> > For all the standard hardware blocks common for all vendors and
> > serving most of the common usecases, we can define standard names in
> > enum. These can be easily interpreted by a table as done in many cases already
> in driver and userspace.
>
> Yeah, but it won't help with the type-specific parameter blobs that are totally
> custom per each operation type. With the KMS property system we could have
> more generic introspection into those as well, at least naming all the parameters
> they have.
>
> Plus, you don't have to patch drm_info every time something new gets added in
> the kernel in order to see it named.
>
> > ...
>
> > > Therefore, I'm not yet convinced with the "all blobs" design.
> >
> > Looking forward to collaboratively solve the problem for the community.
> > Will improve the design based on all the feedback.
>
> I have the feeling that we should also see Harry's draft.
>
> Intel's special LUT type is a whole another story, and we can always have that
> fitted with any UAPI paradigm.
>
> > ...
>
> > > > > > + u32 blob_id;
> > > > > > + };
> > > > > > +
> > > > > > +2. struct drm_color_pipeline: This structure represents the
> > > aggregate
> > > > > > + pipeline to be set. it contains the following
> members
> > > > > > + a) num: pipeline number to be
> > > selected
> > > > > > + b) size: size of the data to be
> passed
> > > onto
> > > > > the driver
> > > > > > + c) data: array of struct
> > > > > drm_color_op_data with data for
> > > > > > + the hardware block/s that userspace
> wants to
> > > > > > + set values for.
> > > > > > +
> > > > > > + struct drm_color_pipeline {
> > > > > > + int num;
> > > > > > + int size;
> > > > > > + struct drm_color_op_data *data;
> > > > > > + };
> > > > > > +
> > > > > > + User can either:
> > > > > > + 1. send data for all the color operations in a pipeline as shown in
> [2].
> > > > > > + The color operation data need not be in order that the
> > > > > > +pipeline
> > > advertises
> > > > > > + however, it should not contain data for any
> > > > > > + color operation that is not present in the pipeline.
> > > > > > +
> > > > > > + Note: This check for pipeline is not yet implemented but if the
> > > > > > + wider proposal is accepted we have few solutions in mind.
> > > > > > +
> > > > > > + 2. send data for a subset of color operations as shown in [3].For
> the
> > > > > > + color operation that userspace does not send data will retain
> their
> > > > > > + older state.
> > >
> > > Retaining existing state, especially for operations that userspace
> > > does not understand, can lead to incorrect and unexpected results.
> > > That's why I say that userspace must understand all operations in a
> > > pipeline, and all parameters of all used operations before it can actually use
> that pipeline.
> >
> > By retaining state here, we mean the state set by the same client
> > while using the same pipeline. If client wants to just alter one or
> > subset of the hardware blocks in the pipeline, it can just send that
> > to driver. Rest of the pipeline which was previously programmed by the same
> client will remain intact.
> >
> > However once the client switches pipeline, driver will disable the
> > pipeline and client will have to program all the blocks again with the new
> pipeline.
> >
> > > Otherwise we have the same problem as KMS properties have in general
> > > today: when new things are added that userspace does not understand,
> > > how will the userspace be able to maintain its old behaviour?
> > >
> > > That question has two answers today:
> > > - userspace learns to program everything, and accidentally
> > > (mal)functions until then
> > > - you do not switch between KMS clients that might leave incorrect
> > > state in not-understood properties
> > >
> > > Neither is a good answer, and the problem does not seem to have any
> > > practical traction either.
> > >
> > > For color pipelines I hope we can, and believe that we must, do
> > > better to maintain correct behaviour while extending functionality.
> >
> > Yes agree, we are thinking to reset and disable the pipeline once client
> switches.
> > One of the ideas could be to use file_priv to achieve that.
>
> I would assume that each color operation in each pipeline is already independent.
> Userspace cannot know which color operation is mapped to which hardware
> block, and should not need to care. That means the kernel maintains software
> state with the UAPI objects, not with the hardware blocks. Therefore, there is no
> inherent need to reset any pipeline on switch. Hardware programming is a
> different matter.
>
> In the hackfest or after it, people raised the possibility of having a standard
> property in each KMS colorop object if possible: boolean "pass-through", or any
> equivalent. If an object is set to pass-through, it is as if it did not exist - it does not
> alter the values going through it in any way. Not even clamping if that could make
> a difference.
>
> This allows adding new arbitrary blocks in old pipelines without forcing userspace
> to abandon the pipeline, as long as userspace learns to use the "pass-through"
> from the start. This reduces the number of alternative pipelines that need to be
> advertised when adding new features.
>
> If an object is not pass-through, then userspace absolutely must understand the
> values programmed into every property of that object, or there is no knowing
> what the object actually does. If userspace does not know what an object does,
> the object can be assumed to produce an unexpected result, i.e. wrong result. No
> userspace is going to willingly malfunction.
>
> > ...
>
> > > > > > +Representing Fixed function hardware
> > > > > > +====================================
> > > > > > +
> > > > > > +To provide support for fixed function hardware, the driver
> > > > > > +could expose vendor specific struct drm_color_op with
> > > > > > +parameters that both the userspace and driver agrees on. To
> > > > > > +demonstrate, let's consider a hyphothetical fixed function
> > > > > > +hardware block that converts BT601
> > > to BT2020.
> > > > > > +The driver can choose to advertise the block as such.
> > > > > > +
> > > > > > +struct drm_color_op color_pipeline_X[] = {
> > > > > > + {
> > > > > > + .name = DRM_CB_PRIVATE,
> > >
> > > What if the hardware has 5 different custom blocks like this,
> > > multiple in the same pipeline. How do you 'name' them?
> >
> > Partially explained above, but private_flags can help in differentiating this.
> > A link for implementation shared above for reference.
>
> But they are all CB_PRIVATE, so what positions do they take in the pipeline?
>
> The names CB_PRE_CSC and CB_POST_CSC imply that the name defines the
> position in a pipeline. That is wrong with the names since the order is defined by
> the record order in a pipeline blob, right?
>
> > > > > > + .type = FIXED_FUNCTION,
> > >
> > > I have been assuming that 'type' uniquely defines both the operation
> > > and the contents of the parameter blob, but this does not look like it.
> > > What defines the operation and the parameters?
> >
> > Statement is true for all other generic blocks, only for private
> > blocks this is a bit different. Here interpretation depends on the
> > private_flags which can be documented by the respective vendor for the
> custom HAL implementation.
>
> Exceptions to a rule are bad API design. In this case, it can easily be avoided.
>
> Now I'm actually confused about how 'name' and 'type' depend on
> 'private_flags'. You have to decipher 'private_flags' to understand what 'name'
> means too?
>
> Hmm, but isn't 'name' used for identifying the block/operation in the blob that
> sets up the parameters for a whole pipeline? But then 'name'
> does not uniquely identify a block?
>
>
> Thanks,
> pq
>
> > ...
>
> > > > > > +References
> > > > > > +==========
> > > > > > +
> > > > > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > > > > +[2]
> > > > > > +https://patchwork.freedesktop.org/patch/554827/?series=123018
> > > > > > +&rev
> > > > > > +=1
> > > > > > +[3]
> > > > > > +https://patchwork.freedesktop.org/patch/554826/?series=123018
> > > > > > +&rev
> > > > > > +=1 [4] https://patchwork.freedesktop.org/series/123018/
> >
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-09-07 12:31 ` Shankar, Uma
0 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-09-07 12:31 UTC (permalink / raw)
To: Pekka Paalanen
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Harry Wentland
> -----Original Message-----
> From: Pekka Paalanen <ppaalanen@gmail.com>
> Sent: Tuesday, September 5, 2023 5:03 PM
> To: Shankar, Uma <uma.shankar@intel.com>
> Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> devel@lists.freedesktop.org; Harry Wentland <harry.wentland@amd.com>;
> Sebastian Wick <sebastian.wick@redhat.com>; ville.syrjala@linux.intel.com;
> Jonas Adahl <jadahl@redhat.com>
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> Color Pipeline
>
> On Mon, 4 Sep 2023 13:44:49 +0000
> "Shankar, Uma" <uma.shankar@intel.com> wrote:
>
> > > -----Original Message-----
> > > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf
> > > Of Pekka Paalanen
> > > Sent: Wednesday, August 30, 2023 5:59 PM
> > > To: Shankar, Uma <uma.shankar@intel.com>
> > > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org;
> > > wayland- devel@lists.freedesktop.org
> > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > Plane Color Pipeline
> > >
> > > On Wed, 30 Aug 2023 08:59:36 +0000
> > > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> > >
> > > > > -----Original Message-----
> > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > intel-gfx@lists.freedesktop.org; dri-
> > > > > devel@lists.freedesktop.org
> > > > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > > > wayland- devel@lists.freedesktop.org
> > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for
> > > > > proposed Plane Color Pipeline
> > > > >
> > > > >
> > > > >
> > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > > >
> > > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > > > ---
> > > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > > > 1 file changed, 394 insertions(+)
> > > > > > create mode 100644
> > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > >
> > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > new file mode 100644
> > > > > > index 000000000000..60ce515b6ea7
> > > > > > --- /dev/null
> > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
> > > ...
> > >
> > > Hi Uma!
> >
> > Thanks Pekka for the feedback and useful inputs.
>
> Hi Uma,
>
> sorry to say, but the overall feeling I get from this proposal is that it is just the
> current color related KMS properties wrapped in a pipeline blob. That is not the
> re-design I believe we are looking for, and the feeling is based on several details
> that are just copied from the current property design. Also the "private" stuff has
> to go.
Hi Pekka,
Ok, got the concerns in general. We will try to evaluate in deeper detail the
property based design and come back if there are some issues or inputs.
At Intel we don't need private as of now, but we thought of having an option to
enable any custom hardware or vendor. But we can drop the same for now if
community doesn't feel the need for it.
> All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap
> distribution, and parametrized curves are good development, but right now we
> are looking at things one step higher level: the overall color pipeline design and
> how to represent any operation. Most of this series is considering details below
> the current attention level, hence I'm paying attention only to the first few
> patches.
We will need to precisely describe the hardware in userspace. Number of luts, precision,
segments etc.., we can't just pass EOTF's as enum from userspace and let driver put
hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour.
This will be needed as there are multiple colorspaces possible and even LUTS can be
used to perform tone mapping. So, we need userspace to be able to program luts directly.
This is something we must expose to userspace. We will check if this can be fitted in
property based approach.
Thanks for all the feedback and comments on the design. We will work on it and get back.
Regards,
Uma Shankar
> More below.
>
> > > > > > +This color pipeline is then packaged within a blob for the
> > > > > > +user space to retrieve it. Details can be found in the next
> > > > > > +section
> > > > > > +
> > > > >
> > > > > Not sure I like blobs that contain other blob ids.
> > > >
> > > > It provides flexibility and helps with just one interface to
> > > > userspace. Its easy to handle and manage once we get the hang of it 😊.
> > > >
> > > > We can clearly define the steps of parsing and data structures to
> > > > be used while interpreting and parsing the blobs.
> > >
> > > Don't forget extendability. Possibly every single struct will need
> > > some kind of versioning, and then it's not simple to parse anymore.
> > > Add to that new/old kernel vs. old/new userspace, and it seems a bit
> nightmarish to design.
> >
> > Structure to be used to interpret the blob should be defined as UAPI
> > only and is not expected to change once agreed upon. It should be interpreted
> like a standard property.
> > So structure to be used, say for 3dLut or 1dlut or CTM operations
> > should be standardized and fixed. No versioning of structure should be
> > done and same rules/restrictions as of UAPI property should be applied.
>
> That sounds like a UAPI that cannot be extended, either. So in a few years we'd be
> looking at replacing it. Or maybe you are just re-inventing the KMS object
> property system as blobs?
>
> Replacing a single KMS property is much easier than replacing the foundations of
> the whole color pipeline design.
>
>
> > ...
>
> > > I have a feeling that introspection will be valuable here, to help
> > > people understand what their hardware could do if they had the code to use
> it. 'name'
> > > and 'type' being integers require a translation table to strings
> > > before they are readable, and it would be best if the kernel itself provided that
> translation.
> >
> > Name and type can be standardized in enum and well documented in the UAPI.
> > For all the standard hardware blocks common for all vendors and
> > serving most of the common usecases, we can define standard names in
> > enum. These can be easily interpreted by a table as done in many cases already
> in driver and userspace.
>
> Yeah, but it won't help with the type-specific parameter blobs that are totally
> custom per each operation type. With the KMS property system we could have
> more generic introspection into those as well, at least naming all the parameters
> they have.
>
> Plus, you don't have to patch drm_info every time something new gets added in
> the kernel in order to see it named.
>
> > ...
>
> > > Therefore, I'm not yet convinced with the "all blobs" design.
> >
> > Looking forward to collaboratively solve the problem for the community.
> > Will improve the design based on all the feedback.
>
> I have the feeling that we should also see Harry's draft.
>
> Intel's special LUT type is a whole another story, and we can always have that
> fitted with any UAPI paradigm.
>
> > ...
>
> > > > > > + u32 blob_id;
> > > > > > + };
> > > > > > +
> > > > > > +2. struct drm_color_pipeline: This structure represents the
> > > aggregate
> > > > > > + pipeline to be set. it contains the following
> members
> > > > > > + a) num: pipeline number to be
> > > selected
> > > > > > + b) size: size of the data to be
> passed
> > > onto
> > > > > the driver
> > > > > > + c) data: array of struct
> > > > > drm_color_op_data with data for
> > > > > > + the hardware block/s that userspace
> wants to
> > > > > > + set values for.
> > > > > > +
> > > > > > + struct drm_color_pipeline {
> > > > > > + int num;
> > > > > > + int size;
> > > > > > + struct drm_color_op_data *data;
> > > > > > + };
> > > > > > +
> > > > > > + User can either:
> > > > > > + 1. send data for all the color operations in a pipeline as shown in
> [2].
> > > > > > + The color operation data need not be in order that the
> > > > > > +pipeline
> > > advertises
> > > > > > + however, it should not contain data for any
> > > > > > + color operation that is not present in the pipeline.
> > > > > > +
> > > > > > + Note: This check for pipeline is not yet implemented but if the
> > > > > > + wider proposal is accepted we have few solutions in mind.
> > > > > > +
> > > > > > + 2. send data for a subset of color operations as shown in [3].For
> the
> > > > > > + color operation that userspace does not send data will retain
> their
> > > > > > + older state.
> > >
> > > Retaining existing state, especially for operations that userspace
> > > does not understand, can lead to incorrect and unexpected results.
> > > That's why I say that userspace must understand all operations in a
> > > pipeline, and all parameters of all used operations before it can actually use
> that pipeline.
> >
> > By retaining state here, we mean the state set by the same client
> > while using the same pipeline. If client wants to just alter one or
> > subset of the hardware blocks in the pipeline, it can just send that
> > to driver. Rest of the pipeline which was previously programmed by the same
> client will remain intact.
> >
> > However once the client switches pipeline, driver will disable the
> > pipeline and client will have to program all the blocks again with the new
> pipeline.
> >
> > > Otherwise we have the same problem as KMS properties have in general
> > > today: when new things are added that userspace does not understand,
> > > how will the userspace be able to maintain its old behaviour?
> > >
> > > That question has two answers today:
> > > - userspace learns to program everything, and accidentally
> > > (mal)functions until then
> > > - you do not switch between KMS clients that might leave incorrect
> > > state in not-understood properties
> > >
> > > Neither is a good answer, and the problem does not seem to have any
> > > practical traction either.
> > >
> > > For color pipelines I hope we can, and believe that we must, do
> > > better to maintain correct behaviour while extending functionality.
> >
> > Yes agree, we are thinking to reset and disable the pipeline once client
> switches.
> > One of the ideas could be to use file_priv to achieve that.
>
> I would assume that each color operation in each pipeline is already independent.
> Userspace cannot know which color operation is mapped to which hardware
> block, and should not need to care. That means the kernel maintains software
> state with the UAPI objects, not with the hardware blocks. Therefore, there is no
> inherent need to reset any pipeline on switch. Hardware programming is a
> different matter.
>
> In the hackfest or after it, people raised the possibility of having a standard
> property in each KMS colorop object if possible: boolean "pass-through", or any
> equivalent. If an object is set to pass-through, it is as if it did not exist - it does not
> alter the values going through it in any way. Not even clamping if that could make
> a difference.
>
> This allows adding new arbitrary blocks in old pipelines without forcing userspace
> to abandon the pipeline, as long as userspace learns to use the "pass-through"
> from the start. This reduces the number of alternative pipelines that need to be
> advertised when adding new features.
>
> If an object is not pass-through, then userspace absolutely must understand the
> values programmed into every property of that object, or there is no knowing
> what the object actually does. If userspace does not know what an object does,
> the object can be assumed to produce an unexpected result, i.e. wrong result. No
> userspace is going to willingly malfunction.
>
> > ...
>
> > > > > > +Representing Fixed function hardware
> > > > > > +====================================
> > > > > > +
> > > > > > +To provide support for fixed function hardware, the driver
> > > > > > +could expose vendor specific struct drm_color_op with
> > > > > > +parameters that both the userspace and driver agrees on. To
> > > > > > +demonstrate, let's consider a hyphothetical fixed function
> > > > > > +hardware block that converts BT601
> > > to BT2020.
> > > > > > +The driver can choose to advertise the block as such.
> > > > > > +
> > > > > > +struct drm_color_op color_pipeline_X[] = {
> > > > > > + {
> > > > > > + .name = DRM_CB_PRIVATE,
> > >
> > > What if the hardware has 5 different custom blocks like this,
> > > multiple in the same pipeline. How do you 'name' them?
> >
> > Partially explained above, but private_flags can help in differentiating this.
> > A link for implementation shared above for reference.
>
> But they are all CB_PRIVATE, so what positions do they take in the pipeline?
>
> The names CB_PRE_CSC and CB_POST_CSC imply that the name defines the
> position in a pipeline. That is wrong with the names since the order is defined by
> the record order in a pipeline blob, right?
>
> > > > > > + .type = FIXED_FUNCTION,
> > >
> > > I have been assuming that 'type' uniquely defines both the operation
> > > and the contents of the parameter blob, but this does not look like it.
> > > What defines the operation and the parameters?
> >
> > Statement is true for all other generic blocks, only for private
> > blocks this is a bit different. Here interpretation depends on the
> > private_flags which can be documented by the respective vendor for the
> custom HAL implementation.
>
> Exceptions to a rule are bad API design. In this case, it can easily be avoided.
>
> Now I'm actually confused about how 'name' and 'type' depend on
> 'private_flags'. You have to decipher 'private_flags' to understand what 'name'
> means too?
>
> Hmm, but isn't 'name' used for identifying the block/operation in the blob that
> sets up the parameters for a whole pipeline? But then 'name'
> does not uniquely identify a block?
>
>
> Thanks,
> pq
>
> > ...
>
> > > > > > +References
> > > > > > +==========
> > > > > > +
> > > > > > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > > > > > +[2]
> > > > > > +https://patchwork.freedesktop.org/patch/554827/?series=123018
> > > > > > +&rev
> > > > > > +=1
> > > > > > +[3]
> > > > > > +https://patchwork.freedesktop.org/patch/554826/?series=123018
> > > > > > +&rev
> > > > > > +=1 [4] https://patchwork.freedesktop.org/series/123018/
> >
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
@ 2023-09-07 20:08 ` Christopher Braga
-1 siblings, 0 replies; 109+ messages in thread
From: Christopher Braga @ 2023-09-07 20:08 UTC (permalink / raw)
To: Uma Shankar, intel-gfx, dri-devel; +Cc: Chaitanya Kumar Borah, wayland-devel
On 8/29/2023 12:03 PM, Uma Shankar wrote:
> Add the documentation for the new proposed Plane Color Pipeline.
>
> Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> ---
> .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> 1 file changed, 394 insertions(+)
> create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
>
> diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst
> new file mode 100644
> index 000000000000..60ce515b6ea7
> --- /dev/null
> +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> @@ -0,0 +1,394 @@
> +=======================================
> + Plane Color Pipeline: A UAPI proposal
> +=======================================
> +
> +To build the proposal on, lets take the premise of a color pipeline as shown
> +below.
> +
Hi Uma,
Thanks for posting this. A few comments, with some echoing the
sentiments of other commenters on the patch set.
> + +-------------------------------------------+
> + | RAM |
> + | +------+ +---------+ +---------+ |
> + | | FB 1 | | FB 2 | | FB N | |
> + | +------+ +---------+ +---------+ |
> + +-------------------------------------------+
> + | Plane Color Hardware Block |
> + +--------------------------------------------+
> + | +---v-----+ +---v-------+ +---v------+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> + | +---+-----+ +---+-------+ +---+------+ |
> + | | | | |
> + | +---v-----+ +---v-------+ +---v------+ |
> + | |Plane A | | Plane B | | Plane N | |
> + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + | +---v-----+ +----v------+ +----v-----+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | |Post-CSC | | Post-CSC | | Post-CSC | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + +--------------------------------------------+
> ++------v--------------v---------------v-------|
> +|| ||
> +|| Pipe Blender ||
> ++--------------------+------------------------+
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Pre-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | Pipe Color |
> +| +-----------v----------+ Hardware |
> +| | Pipe CSC/CTM | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Post-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> ++---------------------------------------------+
> + |
> + v
> + Pipe Output
> +
> +Each plane consists of the following color blocks
> + * Pre-CSC : This block can used to linearize the input frame buffer data.
> + The linear data then can be further acted on by the following
> + color hardware blocks in the display hardware pipeline
> +
> + * CSC/CTM: Used to program color transformation matrix, this block is used
> + to perform color space conversions like BT2020 to BT709 or BT601
> + etc. This block acts on the linearized data coming from the
> + Pre-CSC HW block.
> +
> + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> + match the sink. Another use case of it could be to perform Tone
> + mapping for HDR use-cases.
> +
> +Data from multiple planes will then be fed to pipe/crtc where it will get blended.
> +There is a similar set of HW blocks available at pipe/crtc level which acts on
> +this blended data.
> +
> +Below is a sample usecase fo video playback with sub-titles and playback
> +controls
> +
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
> +(subtitles) │
> + │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
> +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
> +(Playback controls UI) │ │
> + │ │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
> +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │
BT601 to pass-through seems odd here. Shouldn't this be BT2020 pass-through?
> +(video frame) │ │ │
> + │ │ │
> +┌────────────────────────────────────────────────────────────────────────┴─┴─┘
> +│
> +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> +│ │ CRTC │ │ CRTC │ │ CRTC │
> +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port
> + │ │ │ │ │ │
> + └─────────────┘ └─────────────┘ └───────────────┘
> +
> +This RFC is intended to propose an uAPI for the pre-blending color pipeline
> +(however, can be also extended to post blending pipeline).
> +
> +Below are the design considerations while designing the uAPI.
> +
> +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
> +2. Position of the HW block can be programmable
> +3. LUTs can be one dimentional or three dimentional
> +4. Number of LUT entries can vary across platforms
> +5. Precision of LUT entries can vary across platforms
> +6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> + Piece-Wise Linear(PWL) etc
> +7. There can be parameterized/non-parameterized fixed function HW blocks.
> + e.g. Just a hardware bit, to convert from one color space to another.
> +
> +Plane Color Pipeline: Design Proposal
> +=====================================
> +Each Color Hardware block will be represented by the structure drm_color_op.
> +
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type; > + u32 blob_id;
> + u32 private_flags;
> +};
> +
> +The members of which will constitute:
> +1. name: A standardised enum for the color hardware block
> +2. type: The type of mathematical operation done by the hardware block.
> + e.g. 1D Curve, 3D Curve, Matrix etc.
Using both 'type' and 'name' to identify the block sounds like an over
complication to me. From a UAPI perspective we mainly care about the
functionality the block provides, and type should be able to cover this
on its own. If name is needed to declare a divergence from the standard
definition for the type, why wouldn't a new type just be declared?
I'm currently not seeing any benefit that name provides.
> +3. blob id: Id pointing to a blob containing information about the hardware
> + block advertising the respective capabilities to the userspace.
> + It can be an optional field depending on the members "name" and "type".
> +4. private_flags: This can be used to provide vendor specific hints
> + to user space
> +
If OEMs are declaring essentially hints in this property, how is user
space going to handle this when they don't know how to interpret or
handle them? To give any compositor a chance to use the color block
these hints would have to result in no modification of the behavior
agreed upon by the 'type', at which point I don't see what purpose they
serve. The blob UAPI being fixed for a given color block type would also
pose an issue here.
On the other hand, if these flags are only used for vendor specific
hardware blocks it sounds like we are clubbing a proprietary block +
definition into the color pipeline. I don't think this co-exists well
with the intent of the original proposal.
> +
> + For example to represent LUTs, we introduce the drm_color_lut_range
> + structure. It can represent LUTs with varied number of entries and
> + distributions (Multi segmented, Logarithmic etc).
> +
> + struct drm_color_lut_range {
> + /* DRM_MODE_LUT_* */
> + __u32 flags;
> + /* number of points on the curve */
> + __u16 count;
> + /* input/output bits per component */
> + __u8 input_bpc, output_bpc;
> + /* input start/end values */
> + __s32 start, end;
> + /* output min/max values */
> + __s32 min, max;
> + };
> +
> +Note: More details on exact usage and implementation of this structure can be
> + found in the RFC. This structure is taken as is from the series.
> + https://patchwork.freedesktop.org/series/90825/
> + However, we can add more members to it to encompass all use-cases.
> + For example. we can add a precision field to advertise the
> + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> + field for vendor specific use cases.
> +
> + At the same time, we don't need to pass any additional information for the
> + CSC block as userspace and driver already agrees struct drm_color_ctm as
> + a uAPI to pass on data.
> +
> +These color operations will form the building blocks of a color pipeline which
> +best represents the underlying Hardware. Color operations can be re-arranged,
> +substracted or added to create distinct color pipelines to accurately describe
> +the Hardware blocks present in the display engine.
> +
> +In this proposal, a color pipeline is represented as an array of
> +struct drm_color_op.
> +
> +struct drm_color_op color_pipeline_1[]
> +
> +For example to represent the pre-blending color pipeline as described above
> +
> +We can create a color pipeline like this.
> +
> +struct drm_color_op color_pipeline_1[] = {
> + {
> + .name = DRM_CB_PRE_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0; /* actual value to be populated during plane
> + initialization*/
> + },
> + {
> + .name = DRM_CB_CSC,
> + .type = MATRIX,
> + .blob_id = 0;
> + },
> + {
> + .name = DRM_CB_POST_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0;
> + },
> +};
I'm noticing that in the shared IGT patch set these names are being used
to determine position instead of, well, the actual position in the color
pipeline. We definitely need to be careful to avoid coupling
functionality with usage, and I think having a name field is going to
cause developers to make that mistake / shortcut.
> +
> +Then, for individual color operation, we add blobs to advertise the capability
> +of the particular Hardware block. In case of the example pipeline, we add
> +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
> +For the "csc" block we pass no blob id to user space as it is known to both
> +user space and driver that drm_color_ctm structure is to be used for such
> +operation.
> +
> +To represent, this in a diagram.
> +
> + struct drm_color_op color_pipeline_1[]
> + +---------------------------+
> + | | drm_color_op
> + | +---------------------+--+-----------+---------------------+
> + | | | | | |
> + | | | | | +-----------------+ |
> + | | | | | | name | |
> + | | | | | +-----------------+ |
> + | | | | | | type | |
> + | | color_op_1 | | | +-----------------+ |
> + | | | | | | blob id | +--------+
> + | | | | | +-----------------+ | |
> + | | | | | | private | | |
> + | | | | | | flags | | |
> + | | | | | +-----------------+ | |
> + | | | | | | |
> + | +---------------------+--+-----------+---------------------+ |
> + | | |
> + | | |
> + | +---------------------+ | |
> + | | | | drm_color_lut_range |
> + | | color_op_2 | | +-------------------------+ |
> + | | | | | | |
> + | | | | | +---------------------+ | |
> + | +---------------------+ | | | segment 1 { | |<---+
> + | | | | ... | |
> + | +---------------------+ | | | .input_bpc = 16, | |
> + | | | | | | .output_bpc = 16, | |
> + | | color_op_3 | | | | ... | |
> + | | | | | | } | |
> + | | | | | +---------------------+ |
> + | +---------------------+ | | |
> + | . | | +---------------------+ |
> + | . | | | segment 2 { | |
> + | . | | | ... | |
> + +---------------------------+ | | } | |
> + | | | |
> + | | | |
> + | | | |
> + | +---------------------+ |
> + | . |
> + | . |
> + | . |
> + +-------------------------+
> +
> +
> +
> +This color pipeline is then packaged within a blob for the user space to
> +retrieve it. Details can be found in the next section
While I get the intent of the blob nesting, as others have mentioned
this does feel like a recreation of what we already have with DRM
objects. DRM objects would also have the advantage of allowing for
easier OEM extension. Instead of private flags, an OEM could introduce
behavior modifiers or additional features on a color block object via
additional properties (these properties would obviously have to default
to a disabled state to ensure default behavior is what is defined by the
base color block type).
> +
> +Exposing a color pipeline to user space
> +=======================================
> +
> +To advertise the available color pipelines, an immutable ENUM property
> +"GET_COLOR_PIPELINE" is introduced.
> +This enum property has blob id's as values. With each blob id representing
> +a distinct color pipeline based on underlying HW capabilities and their
> +respective combinations.
> +
> +The following output of drm_info [1], shows how color pipelines are visible
> +to userspace.
> +
> +├───Plane 0
> + │ ├───Object ID: 31
> + │ ├───CRTCs: {0}
> + │ ├───Legacy info
> + ...
> + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> + color pipeline 1, color pipeline 2}= no color pipeline
> +
> +To understand the capabilities of individual pipelines, first the userspace
> +need to retrieve the pipeline blob with the blob ids retrieved by reading the
> +enum property.
> +
> +Once the color pipeline is retrieved, user can then parse through
> +individual drm_color_op blocks to understand the capabilities of each
> +hardware block.
> +
> +Check IGT series to see how user space can parse through color pipelines.
> +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
> +
> +Setting up a color pipeline
> +===========================
> +
> +Once the user space decides on a color pipeline, it can set the pipeline and
> +the corresponding data for the hardware blocks within the pipeline with
> +the BLOB property "SET_COLOR_PIPELINE".
> +
> +To achieve this two structures are introduced
> +
> +1. struct drm_color_op_data: It represents data to be passed onto individual
> + color hardware blocks. It contains three members
> + a) name: to identify the color operation block
> + b) blob_id: pointing to the blob with data for the
> + corresponding hardware block
> +
> + struct drm_color_op_data {
> + enum color_op_block name;
> + u32 blob_id;
> + };
> +
> +2. struct drm_color_pipeline: This structure represents the aggregate
> + pipeline to be set. it contains the following members
> + a) num: pipeline number to be selected
> + b) size: size of the data to be passed onto the driver
> + c) data: array of struct drm_color_op_data with data for
> + the hardware block/s that userspace wants to
> + set values for.
> +
> + struct drm_color_pipeline {
> + int num;
> + int size;
> + struct drm_color_op_data *data;
> + };
> +
> + User can either:
> + 1. send data for all the color operations in a pipeline as shown in [2].
> + The color operation data need not be in order that the pipeline advertises
> + however, it should not contain data for any
> + color operation that is not present in the pipeline.
> +
> + Note: This check for pipeline is not yet implemented but if the
> + wider proposal is accepted we have few solutions in mind.
> +
> + 2. send data for a subset of color operations as shown in [3].For the
> + color operation that userspace does not send data will retain their
> + older state.
> +
> + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> + property to NULL as shown in both [2] and [3]
> +
> + 4. change the color pipeline as demonstrated in [3].
> + On the new pipeline, the user is expected to setup all the color hardware block
> + Once the user requests a pipeline change, the driver will provide it a clean slate
> + which means that all the data previously set by the user will be discarded even if
> + there are common hardware blocks between the two(previous and current) pipelines.
> +
> +IGT implementation can be found here [4]
> +
> +Representing Fixed function hardware
> +====================================
> +
> +To provide support for fixed function hardware, the driver could expose vendor
> +specific struct drm_color_op with parameters that both the userspace and
> +driver agrees on. To demonstrate, let's consider a hyphothetical fixed
> +function hardware block that converts BT601 to BT2020.
> +The driver can choose to advertise the block as such.
> +
> +struct drm_color_op color_pipeline_X[] = {
> + {
> + .name = DRM_CB_PRIVATE,
> + .type = FIXED_FUNCTION,
> + .blob_id = 45;
> + },
> +}
> +
> +Where the blob represents some vendor specific enums, strings or any other
> +appropriate data types which both the user-space and drivers are aligned on.
> +
> +blob:45 {
> + VENDORXXX_BT602_TO_BT2020,
> +};
As far as I see it, one of the primary advantages of moving to this new
color pipeline is the standardization of the color block definition.
User space looks at the 'type', and if it supports it then it can look
at other expected information such as 'lut_size', 'bit_depth', etc to
determine the programming considerations.
The second we start having color blocks fully defined by private_flags
and vendor specific blobs we end up with the same scenario we have today
with vendor specific properties on our DRM CRTC and DRM PLANE objects.
While we aren't going to make all the different OEM hardware match, we
still want to encourage common approaches to block declaration so user
space can easily identify functionality it supports across a wide range
of hardware.
> +
> +For enabling or passing parameters to such blocks, the user can send data
> +to the driver wrapped within a blob like any other color operation block.
> +
> + struct drm_color_op_data {
> + .name = DRM_CB_PRIVATE;
> + .blob_id = 46;
> + } ;
> +
> +where blob with id 46 contains data to enable the fixed function hardware(s).
> +
All that said, thanks again for working on this. I'm definitely curious
to see how this will all come together.
- Christopher
> +References
> +==========
> +
> +[1] https://gitlab.freedesktop.org/emersion/drm_info
> +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-09-07 20:08 ` Christopher Braga
0 siblings, 0 replies; 109+ messages in thread
From: Christopher Braga @ 2023-09-07 20:08 UTC (permalink / raw)
To: Uma Shankar, intel-gfx, dri-devel; +Cc: wayland-devel
On 8/29/2023 12:03 PM, Uma Shankar wrote:
> Add the documentation for the new proposed Plane Color Pipeline.
>
> Co-developed-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> ---
> .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> 1 file changed, 394 insertions(+)
> create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
>
> diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst
> new file mode 100644
> index 000000000000..60ce515b6ea7
> --- /dev/null
> +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> @@ -0,0 +1,394 @@
> +=======================================
> + Plane Color Pipeline: A UAPI proposal
> +=======================================
> +
> +To build the proposal on, lets take the premise of a color pipeline as shown
> +below.
> +
Hi Uma,
Thanks for posting this. A few comments, with some echoing the
sentiments of other commenters on the patch set.
> + +-------------------------------------------+
> + | RAM |
> + | +------+ +---------+ +---------+ |
> + | | FB 1 | | FB 2 | | FB N | |
> + | +------+ +---------+ +---------+ |
> + +-------------------------------------------+
> + | Plane Color Hardware Block |
> + +--------------------------------------------+
> + | +---v-----+ +---v-------+ +---v------+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> + | +---+-----+ +---+-------+ +---+------+ |
> + | | | | |
> + | +---v-----+ +---v-------+ +---v------+ |
> + | |Plane A | | Plane B | | Plane N | |
> + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + | +---v-----+ +----v------+ +----v-----+ |
> + | | Plane A | | Plane B | | Plane N | |
> + | |Post-CSC | | Post-CSC | | Post-CSC | |
> + | +---+-----+ +----+------+ +----+-----+ |
> + | | | | |
> + +--------------------------------------------+
> ++------v--------------v---------------v-------|
> +|| ||
> +|| Pipe Blender ||
> ++--------------------+------------------------+
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Pre-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | Pipe Color |
> +| +-----------v----------+ Hardware |
> +| | Pipe CSC/CTM | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> +| +-----------v----------+ |
> +| | Pipe Post-CSC | |
> +| | | |
> +| +-----------+----------+ |
> +| | |
> ++---------------------------------------------+
> + |
> + v
> + Pipe Output
> +
> +Each plane consists of the following color blocks
> + * Pre-CSC : This block can used to linearize the input frame buffer data.
> + The linear data then can be further acted on by the following
> + color hardware blocks in the display hardware pipeline
> +
> + * CSC/CTM: Used to program color transformation matrix, this block is used
> + to perform color space conversions like BT2020 to BT709 or BT601
> + etc. This block acts on the linearized data coming from the
> + Pre-CSC HW block.
> +
> + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> + match the sink. Another use case of it could be to perform Tone
> + mapping for HDR use-cases.
> +
> +Data from multiple planes will then be fed to pipe/crtc where it will get blended.
> +There is a similar set of HW blocks available at pipe/crtc level which acts on
> +this blended data.
> +
> +Below is a sample usecase fo video playback with sub-titles and playback
> +controls
> +
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
> +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
> +(subtitles) │
> + │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │
> +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
> +(Playback controls UI) │ │
> + │ │
> +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
> +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │
> +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │
BT601 to pass-through seems odd here. Shouldn't this be BT2020 pass-through?
> +(video frame) │ │ │
> + │ │ │
> +┌────────────────────────────────────────────────────────────────────────┴─┴─┘
> +│
> +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> +│ │ CRTC │ │ CRTC │ │ CRTC │
> +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port
> + │ │ │ │ │ │
> + └─────────────┘ └─────────────┘ └───────────────┘
> +
> +This RFC is intended to propose an uAPI for the pre-blending color pipeline
> +(however, can be also extended to post blending pipeline).
> +
> +Below are the design considerations while designing the uAPI.
> +
> +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
> +2. Position of the HW block can be programmable
> +3. LUTs can be one dimentional or three dimentional
> +4. Number of LUT entries can vary across platforms
> +5. Precision of LUT entries can vary across platforms
> +6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic,
> + Piece-Wise Linear(PWL) etc
> +7. There can be parameterized/non-parameterized fixed function HW blocks.
> + e.g. Just a hardware bit, to convert from one color space to another.
> +
> +Plane Color Pipeline: Design Proposal
> +=====================================
> +Each Color Hardware block will be represented by the structure drm_color_op.
> +
> +struct drm_color_op {
> + enum color_op_block name;
> + enum color_op_type type; > + u32 blob_id;
> + u32 private_flags;
> +};
> +
> +The members of which will constitute:
> +1. name: A standardised enum for the color hardware block
> +2. type: The type of mathematical operation done by the hardware block.
> + e.g. 1D Curve, 3D Curve, Matrix etc.
Using both 'type' and 'name' to identify the block sounds like an over
complication to me. From a UAPI perspective we mainly care about the
functionality the block provides, and type should be able to cover this
on its own. If name is needed to declare a divergence from the standard
definition for the type, why wouldn't a new type just be declared?
I'm currently not seeing any benefit that name provides.
> +3. blob id: Id pointing to a blob containing information about the hardware
> + block advertising the respective capabilities to the userspace.
> + It can be an optional field depending on the members "name" and "type".
> +4. private_flags: This can be used to provide vendor specific hints
> + to user space
> +
If OEMs are declaring essentially hints in this property, how is user
space going to handle this when they don't know how to interpret or
handle them? To give any compositor a chance to use the color block
these hints would have to result in no modification of the behavior
agreed upon by the 'type', at which point I don't see what purpose they
serve. The blob UAPI being fixed for a given color block type would also
pose an issue here.
On the other hand, if these flags are only used for vendor specific
hardware blocks it sounds like we are clubbing a proprietary block +
definition into the color pipeline. I don't think this co-exists well
with the intent of the original proposal.
> +
> + For example to represent LUTs, we introduce the drm_color_lut_range
> + structure. It can represent LUTs with varied number of entries and
> + distributions (Multi segmented, Logarithmic etc).
> +
> + struct drm_color_lut_range {
> + /* DRM_MODE_LUT_* */
> + __u32 flags;
> + /* number of points on the curve */
> + __u16 count;
> + /* input/output bits per component */
> + __u8 input_bpc, output_bpc;
> + /* input start/end values */
> + __s32 start, end;
> + /* output min/max values */
> + __s32 min, max;
> + };
> +
> +Note: More details on exact usage and implementation of this structure can be
> + found in the RFC. This structure is taken as is from the series.
> + https://patchwork.freedesktop.org/series/90825/
> + However, we can add more members to it to encompass all use-cases.
> + For example. we can add a precision field to advertise the
> + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> + field for vendor specific use cases.
> +
> + At the same time, we don't need to pass any additional information for the
> + CSC block as userspace and driver already agrees struct drm_color_ctm as
> + a uAPI to pass on data.
> +
> +These color operations will form the building blocks of a color pipeline which
> +best represents the underlying Hardware. Color operations can be re-arranged,
> +substracted or added to create distinct color pipelines to accurately describe
> +the Hardware blocks present in the display engine.
> +
> +In this proposal, a color pipeline is represented as an array of
> +struct drm_color_op.
> +
> +struct drm_color_op color_pipeline_1[]
> +
> +For example to represent the pre-blending color pipeline as described above
> +
> +We can create a color pipeline like this.
> +
> +struct drm_color_op color_pipeline_1[] = {
> + {
> + .name = DRM_CB_PRE_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0; /* actual value to be populated during plane
> + initialization*/
> + },
> + {
> + .name = DRM_CB_CSC,
> + .type = MATRIX,
> + .blob_id = 0;
> + },
> + {
> + .name = DRM_CB_POST_CSC,
> + .type = CURVE_1D,
> + .blob_id = 0;
> + },
> +};
I'm noticing that in the shared IGT patch set these names are being used
to determine position instead of, well, the actual position in the color
pipeline. We definitely need to be careful to avoid coupling
functionality with usage, and I think having a name field is going to
cause developers to make that mistake / shortcut.
> +
> +Then, for individual color operation, we add blobs to advertise the capability
> +of the particular Hardware block. In case of the example pipeline, we add
> +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
> +For the "csc" block we pass no blob id to user space as it is known to both
> +user space and driver that drm_color_ctm structure is to be used for such
> +operation.
> +
> +To represent, this in a diagram.
> +
> + struct drm_color_op color_pipeline_1[]
> + +---------------------------+
> + | | drm_color_op
> + | +---------------------+--+-----------+---------------------+
> + | | | | | |
> + | | | | | +-----------------+ |
> + | | | | | | name | |
> + | | | | | +-----------------+ |
> + | | | | | | type | |
> + | | color_op_1 | | | +-----------------+ |
> + | | | | | | blob id | +--------+
> + | | | | | +-----------------+ | |
> + | | | | | | private | | |
> + | | | | | | flags | | |
> + | | | | | +-----------------+ | |
> + | | | | | | |
> + | +---------------------+--+-----------+---------------------+ |
> + | | |
> + | | |
> + | +---------------------+ | |
> + | | | | drm_color_lut_range |
> + | | color_op_2 | | +-------------------------+ |
> + | | | | | | |
> + | | | | | +---------------------+ | |
> + | +---------------------+ | | | segment 1 { | |<---+
> + | | | | ... | |
> + | +---------------------+ | | | .input_bpc = 16, | |
> + | | | | | | .output_bpc = 16, | |
> + | | color_op_3 | | | | ... | |
> + | | | | | | } | |
> + | | | | | +---------------------+ |
> + | +---------------------+ | | |
> + | . | | +---------------------+ |
> + | . | | | segment 2 { | |
> + | . | | | ... | |
> + +---------------------------+ | | } | |
> + | | | |
> + | | | |
> + | | | |
> + | +---------------------+ |
> + | . |
> + | . |
> + | . |
> + +-------------------------+
> +
> +
> +
> +This color pipeline is then packaged within a blob for the user space to
> +retrieve it. Details can be found in the next section
While I get the intent of the blob nesting, as others have mentioned
this does feel like a recreation of what we already have with DRM
objects. DRM objects would also have the advantage of allowing for
easier OEM extension. Instead of private flags, an OEM could introduce
behavior modifiers or additional features on a color block object via
additional properties (these properties would obviously have to default
to a disabled state to ensure default behavior is what is defined by the
base color block type).
> +
> +Exposing a color pipeline to user space
> +=======================================
> +
> +To advertise the available color pipelines, an immutable ENUM property
> +"GET_COLOR_PIPELINE" is introduced.
> +This enum property has blob id's as values. With each blob id representing
> +a distinct color pipeline based on underlying HW capabilities and their
> +respective combinations.
> +
> +The following output of drm_info [1], shows how color pipelines are visible
> +to userspace.
> +
> +├───Plane 0
> + │ ├───Object ID: 31
> + │ ├───CRTCs: {0}
> + │ ├───Legacy info
> + ...
> + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> + color pipeline 1, color pipeline 2}= no color pipeline
> +
> +To understand the capabilities of individual pipelines, first the userspace
> +need to retrieve the pipeline blob with the blob ids retrieved by reading the
> +enum property.
> +
> +Once the color pipeline is retrieved, user can then parse through
> +individual drm_color_op blocks to understand the capabilities of each
> +hardware block.
> +
> +Check IGT series to see how user space can parse through color pipelines.
> +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
> +
> +Setting up a color pipeline
> +===========================
> +
> +Once the user space decides on a color pipeline, it can set the pipeline and
> +the corresponding data for the hardware blocks within the pipeline with
> +the BLOB property "SET_COLOR_PIPELINE".
> +
> +To achieve this two structures are introduced
> +
> +1. struct drm_color_op_data: It represents data to be passed onto individual
> + color hardware blocks. It contains three members
> + a) name: to identify the color operation block
> + b) blob_id: pointing to the blob with data for the
> + corresponding hardware block
> +
> + struct drm_color_op_data {
> + enum color_op_block name;
> + u32 blob_id;
> + };
> +
> +2. struct drm_color_pipeline: This structure represents the aggregate
> + pipeline to be set. it contains the following members
> + a) num: pipeline number to be selected
> + b) size: size of the data to be passed onto the driver
> + c) data: array of struct drm_color_op_data with data for
> + the hardware block/s that userspace wants to
> + set values for.
> +
> + struct drm_color_pipeline {
> + int num;
> + int size;
> + struct drm_color_op_data *data;
> + };
> +
> + User can either:
> + 1. send data for all the color operations in a pipeline as shown in [2].
> + The color operation data need not be in order that the pipeline advertises
> + however, it should not contain data for any
> + color operation that is not present in the pipeline.
> +
> + Note: This check for pipeline is not yet implemented but if the
> + wider proposal is accepted we have few solutions in mind.
> +
> + 2. send data for a subset of color operations as shown in [3].For the
> + color operation that userspace does not send data will retain their
> + older state.
> +
> + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> + property to NULL as shown in both [2] and [3]
> +
> + 4. change the color pipeline as demonstrated in [3].
> + On the new pipeline, the user is expected to setup all the color hardware block
> + Once the user requests a pipeline change, the driver will provide it a clean slate
> + which means that all the data previously set by the user will be discarded even if
> + there are common hardware blocks between the two(previous and current) pipelines.
> +
> +IGT implementation can be found here [4]
> +
> +Representing Fixed function hardware
> +====================================
> +
> +To provide support for fixed function hardware, the driver could expose vendor
> +specific struct drm_color_op with parameters that both the userspace and
> +driver agrees on. To demonstrate, let's consider a hyphothetical fixed
> +function hardware block that converts BT601 to BT2020.
> +The driver can choose to advertise the block as such.
> +
> +struct drm_color_op color_pipeline_X[] = {
> + {
> + .name = DRM_CB_PRIVATE,
> + .type = FIXED_FUNCTION,
> + .blob_id = 45;
> + },
> +}
> +
> +Where the blob represents some vendor specific enums, strings or any other
> +appropriate data types which both the user-space and drivers are aligned on.
> +
> +blob:45 {
> + VENDORXXX_BT602_TO_BT2020,
> +};
As far as I see it, one of the primary advantages of moving to this new
color pipeline is the standardization of the color block definition.
User space looks at the 'type', and if it supports it then it can look
at other expected information such as 'lut_size', 'bit_depth', etc to
determine the programming considerations.
The second we start having color blocks fully defined by private_flags
and vendor specific blobs we end up with the same scenario we have today
with vendor specific properties on our DRM CRTC and DRM PLANE objects.
While we aren't going to make all the different OEM hardware match, we
still want to encourage common approaches to block declaration so user
space can easily identify functionality it supports across a wide range
of hardware.
> +
> +For enabling or passing parameters to such blocks, the user can send data
> +to the driver wrapped within a blob like any other color operation block.
> +
> + struct drm_color_op_data {
> + .name = DRM_CB_PRIVATE;
> + .blob_id = 46;
> + } ;
> +
> +where blob with id 46 contains data to enable the fixed function hardware(s).
> +
All that said, thanks again for working on this. I'm definitely curious
to see how this will all come together.
- Christopher
> +References
> +==========
> +
> +[1] https://gitlab.freedesktop.org/emersion/drm_info
> +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-09-07 12:31 ` [Intel-gfx] " Shankar, Uma
@ 2023-09-08 8:31 ` Pekka Paalanen
-1 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-08 8:31 UTC (permalink / raw)
To: Shankar, Uma
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Borah, Chaitanya Kumar
[-- Attachment #1: Type: text/plain, Size: 7966 bytes --]
On Thu, 7 Sep 2023 12:31:47 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: Pekka Paalanen <ppaalanen@gmail.com>
> > Sent: Tuesday, September 5, 2023 5:03 PM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org; Harry Wentland <harry.wentland@amd.com>;
> > Sebastian Wick <sebastian.wick@redhat.com>; ville.syrjala@linux.intel.com;
> > Jonas Adahl <jadahl@redhat.com>
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> > Color Pipeline
> >
> > On Mon, 4 Sep 2023 13:44:49 +0000
> > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> >
> > > > -----Original Message-----
> > > > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf
> > > > Of Pekka Paalanen
> > > > Sent: Wednesday, August 30, 2023 5:59 PM
> > > > To: Shankar, Uma <uma.shankar@intel.com>
> > > > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > > > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org;
> > > > wayland- devel@lists.freedesktop.org
> > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > > Plane Color Pipeline
> > > >
> > > > On Wed, 30 Aug 2023 08:59:36 +0000
> > > > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> > > >
> > > > > > -----Original Message-----
> > > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > > intel-gfx@lists.freedesktop.org; dri-
> > > > > > devel@lists.freedesktop.org
> > > > > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > > > > wayland- devel@lists.freedesktop.org
> > > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for
> > > > > > proposed Plane Color Pipeline
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > > > >
> > > > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > > > > ---
> > > > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > > > > 1 file changed, 394 insertions(+)
> > > > > > > create mode 100644
> > > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > >
> > > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > > new file mode 100644
> > > > > > > index 000000000000..60ce515b6ea7
> > > > > > > --- /dev/null
> > > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > >
> > > > ...
> > > >
> > > > Hi Uma!
> > >
> > > Thanks Pekka for the feedback and useful inputs.
> >
> > Hi Uma,
> >
> > sorry to say, but the overall feeling I get from this proposal is that it is just the
> > current color related KMS properties wrapped in a pipeline blob. That is not the
> > re-design I believe we are looking for, and the feeling is based on several details
> > that are just copied from the current property design. Also the "private" stuff has
> > to go.
>
> Hi Pekka,
> Ok, got the concerns in general. We will try to evaluate in deeper detail the
> property based design and come back if there are some issues or inputs.
>
> At Intel we don't need private as of now, but we thought of having an option to
> enable any custom hardware or vendor. But we can drop the same for now if
> community doesn't feel the need for it.
>
> > All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap
> > distribution, and parametrized curves are good development, but right now we
> > are looking at things one step higher level: the overall color pipeline design and
> > how to represent any operation. Most of this series is considering details below
> > the current attention level, hence I'm paying attention only to the first few
> > patches.
>
> We will need to precisely describe the hardware in userspace. Number of luts, precision,
> segments etc.., we can't just pass EOTF's as enum from userspace and let driver put
> hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour.
> This will be needed as there are multiple colorspaces possible and even LUTS can be
> used to perform tone mapping. So, we need userspace to be able to program luts directly.
Hi Uma,
yes, we do need to expose freely programmable LUTs when hardware has
them. That's why I say it is good development.
However, this is not an either-or situation.
We must also be able to expose fixed-function curve blocks when
hardware has them. Please, do not confuse this with a descriptive
design. This is not about saying "this FB is using PQ encoding, convert
it to NNN for me".
This is about defining an operation, that is mathematically defined as
"the PQ EOTF with normalized domain and range", for example. This is
prescriptive, because the exact mathematical formula of the operation
is defined, and it does not depend on any properties of the block's
input or output. It contains no opinions on how to convert something to
another, like a conversion from a color system and viewing environment
to another does (e.g. sRGB<->BT.2100/PQ). Every driver exposing this
operation must implement it exactly the same, with a small allowed error
tolerance.
There are no limitations on how it can be used. Userspace can choose to
apply that formula on anything it likes, and use the result in any way
it likes, even if it is an utterly non-standard pipeline not making any
sense *to us*.
You could argue that an operation to "convert PQ to HLG" is also
prescriptive for example, and yes, it is if implementations have to
adhere to a single specific formula for it. But if implementations are
allowed choose any formula they believe is the best to implement that
operation, then it is descriptive: "I have PQ content, I want HLG out,
do whatever, I don't care".
If you really have a fixed-function hardware block that literally uses
one specific formula to convert PQ to HLG, you can and probably should
expose that as a colorop. It is prescriptive, because the formula is
fully known to userspace, and userspace will choose to use it for its
formula, and not because it converts PQ to HLG.
However, if you had to combine multiple hardware blocks to achieve the
PQ-to-HLG formula, then that does not make sense to expose as a colorop
(other than for backward compatibility if your previous hardware
generation had it as a single fixed-function block). In practise, it
would be too rigid to be useful in more than few specific situations.
It would be much more flexible to expose the actual hardware blocks you
have, and let userspace use them any way it likes. This is about
finding the right balance in abstraction for UAPI.
> This is something we must expose to userspace. We will check if this can be fitted in
> property based approach.
I'm sure it can be exposed. The trivial option is to define a colorop
with the specific operation type that defines a property that will hold
the LUT configuration blob you have designed.
You do not need to force literally everything into a million
properties, you can still use blobs where they make sense, like needing
a variable or great number of elements of some type.
As such, the overall color pipeline UAPI design has little effect
on how you would design your LUT UAPI structures.
Thanks,
pq
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-09-08 8:31 ` Pekka Paalanen
0 siblings, 0 replies; 109+ messages in thread
From: Pekka Paalanen @ 2023-09-08 8:31 UTC (permalink / raw)
To: Shankar, Uma
Cc: Sebastian Wick, intel-gfx, dri-devel, wayland-devel, Jonas Adahl,
Harry Wentland
[-- Attachment #1: Type: text/plain, Size: 7966 bytes --]
On Thu, 7 Sep 2023 12:31:47 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:
> > -----Original Message-----
> > From: Pekka Paalanen <ppaalanen@gmail.com>
> > Sent: Tuesday, September 5, 2023 5:03 PM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org; Harry Wentland <harry.wentland@amd.com>;
> > Sebastian Wick <sebastian.wick@redhat.com>; ville.syrjala@linux.intel.com;
> > Jonas Adahl <jadahl@redhat.com>
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> > Color Pipeline
> >
> > On Mon, 4 Sep 2023 13:44:49 +0000
> > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> >
> > > > -----Original Message-----
> > > > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf
> > > > Of Pekka Paalanen
> > > > Sent: Wednesday, August 30, 2023 5:59 PM
> > > > To: Shankar, Uma <uma.shankar@intel.com>
> > > > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > > > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org;
> > > > wayland- devel@lists.freedesktop.org
> > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > > Plane Color Pipeline
> > > >
> > > > On Wed, 30 Aug 2023 08:59:36 +0000
> > > > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> > > >
> > > > > > -----Original Message-----
> > > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > > intel-gfx@lists.freedesktop.org; dri-
> > > > > > devel@lists.freedesktop.org
> > > > > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > > > > wayland- devel@lists.freedesktop.org
> > > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for
> > > > > > proposed Plane Color Pipeline
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > > > >
> > > > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > > > > ---
> > > > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > > > > > > 1 file changed, 394 insertions(+)
> > > > > > > create mode 100644
> > > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > >
> > > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > > new file mode 100644
> > > > > > > index 000000000000..60ce515b6ea7
> > > > > > > --- /dev/null
> > > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > >
> > > > ...
> > > >
> > > > Hi Uma!
> > >
> > > Thanks Pekka for the feedback and useful inputs.
> >
> > Hi Uma,
> >
> > sorry to say, but the overall feeling I get from this proposal is that it is just the
> > current color related KMS properties wrapped in a pipeline blob. That is not the
> > re-design I believe we are looking for, and the feeling is based on several details
> > that are just copied from the current property design. Also the "private" stuff has
> > to go.
>
> Hi Pekka,
> Ok, got the concerns in general. We will try to evaluate in deeper detail the
> property based design and come back if there are some issues or inputs.
>
> At Intel we don't need private as of now, but we thought of having an option to
> enable any custom hardware or vendor. But we can drop the same for now if
> community doesn't feel the need for it.
>
> > All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap
> > distribution, and parametrized curves are good development, but right now we
> > are looking at things one step higher level: the overall color pipeline design and
> > how to represent any operation. Most of this series is considering details below
> > the current attention level, hence I'm paying attention only to the first few
> > patches.
>
> We will need to precisely describe the hardware in userspace. Number of luts, precision,
> segments etc.., we can't just pass EOTF's as enum from userspace and let driver put
> hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour.
> This will be needed as there are multiple colorspaces possible and even LUTS can be
> used to perform tone mapping. So, we need userspace to be able to program luts directly.
Hi Uma,
yes, we do need to expose freely programmable LUTs when hardware has
them. That's why I say it is good development.
However, this is not an either-or situation.
We must also be able to expose fixed-function curve blocks when
hardware has them. Please, do not confuse this with a descriptive
design. This is not about saying "this FB is using PQ encoding, convert
it to NNN for me".
This is about defining an operation, that is mathematically defined as
"the PQ EOTF with normalized domain and range", for example. This is
prescriptive, because the exact mathematical formula of the operation
is defined, and it does not depend on any properties of the block's
input or output. It contains no opinions on how to convert something to
another, like a conversion from a color system and viewing environment
to another does (e.g. sRGB<->BT.2100/PQ). Every driver exposing this
operation must implement it exactly the same, with a small allowed error
tolerance.
There are no limitations on how it can be used. Userspace can choose to
apply that formula on anything it likes, and use the result in any way
it likes, even if it is an utterly non-standard pipeline not making any
sense *to us*.
You could argue that an operation to "convert PQ to HLG" is also
prescriptive for example, and yes, it is if implementations have to
adhere to a single specific formula for it. But if implementations are
allowed choose any formula they believe is the best to implement that
operation, then it is descriptive: "I have PQ content, I want HLG out,
do whatever, I don't care".
If you really have a fixed-function hardware block that literally uses
one specific formula to convert PQ to HLG, you can and probably should
expose that as a colorop. It is prescriptive, because the formula is
fully known to userspace, and userspace will choose to use it for its
formula, and not because it converts PQ to HLG.
However, if you had to combine multiple hardware blocks to achieve the
PQ-to-HLG formula, then that does not make sense to expose as a colorop
(other than for backward compatibility if your previous hardware
generation had it as a single fixed-function block). In practise, it
would be too rigid to be useful in more than few specific situations.
It would be much more flexible to expose the actual hardware blocks you
have, and let userspace use them any way it likes. This is about
finding the right balance in abstraction for UAPI.
> This is something we must expose to userspace. We will check if this can be fitted in
> property based approach.
I'm sure it can be exposed. The trivial option is to define a colorop
with the specific operation type that defines a property that will hold
the LUT configuration blob you have designed.
You do not need to force literally everything into a million
properties, you can still use blobs where they make sense, like needing
a variable or great number of elements of some type.
As such, the overall color pipeline UAPI design has little effect
on how you would design your LUT UAPI structures.
Thanks,
pq
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
2023-09-07 20:08 ` [Intel-gfx] " Christopher Braga
@ 2023-10-13 5:46 ` Shankar, Uma
-1 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-10-13 5:46 UTC (permalink / raw)
To: Christopher Braga, intel-gfx, dri-devel
Cc: Borah, Chaitanya Kumar, wayland-devel
> -----Original Message-----
> From: Christopher Braga <quic_cbraga@quicinc.com>
> Sent: Friday, September 8, 2023 1:39 AM
> To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> devel@lists.freedesktop.org
> Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> Color Pipeline
>
>
>
> On 8/29/2023 12:03 PM, Uma Shankar wrote:
> > Add the documentation for the new proposed Plane Color Pipeline.
> >
> > Co-developed-by: Chaitanya Kumar Borah
> > <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > ---
> > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > 1 file changed, 394 insertions(+)
> > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> >
> > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > new file mode 100644
> > index 000000000000..60ce515b6ea7
> > --- /dev/null
> > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > @@ -0,0 +1,394 @@
> > +=======================================
> > + Plane Color Pipeline: A UAPI proposal
> > +=======================================
> > +
> > +To build the proposal on, lets take the premise of a color pipeline
> > +as shown below.
> > +
> Hi Uma,
> Thanks for posting this. A few comments, with some echoing the sentiments of
> other commenters on the patch set.
Hi Christopher,
Thanks for your inputs on the series and my apologies for a late reply.
> > + +-------------------------------------------+
> > + | RAM |
> > + | +------+ +---------+ +---------+ |
> > + | | FB 1 | | FB 2 | | FB N | |
> > + | +------+ +---------+ +---------+ |
> > + +-------------------------------------------+
> > + | Plane Color Hardware Block |
> > + +--------------------------------------------+
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> > + | +---+-----+ +---+-------+ +---+------+ |
> > + | | | | |
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | |Plane A | | Plane B | | Plane N | |
> > + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + | +---v-----+ +----v------+ +----v-----+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | |Post-CSC | | Post-CSC | | Post-CSC | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + +--------------------------------------------+
> > ++------v--------------v---------------v-------|
> > +|| ||
> > +|| Pipe Blender ||
> > ++--------------------+------------------------+
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Pre-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | Pipe Color |
> > +| +-----------v----------+ Hardware |
> > +| | Pipe CSC/CTM | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Post-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > ++---------------------------------------------+
> > + |
> > + v
> > + Pipe Output
> > +
> > +Each plane consists of the following color blocks
> > + * Pre-CSC : This block can used to linearize the input frame buffer data.
> > + The linear data then can be further acted on by the following
> > + color hardware blocks in the display hardware pipeline
> > +
> > + * CSC/CTM: Used to program color transformation matrix, this block is used
> > + to perform color space conversions like BT2020 to BT709 or BT601
> > + etc. This block acts on the linearized data coming from the
> > + Pre-CSC HW block.
> > +
> > + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> > + match the sink. Another use case of it could be to perform Tone
> > + mapping for HDR use-cases.
> > +
> > +Data from multiple planes will then be fed to pipe/crtc where it will get
> blended.
> > +There is a similar set of HW blocks available at pipe/crtc level
> > +which acts on this blended data.
> > +
> > +Below is a sample usecase fo video playback with sub-titles and
> > +playback controls
> > +
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐
> ┌─────────────┐
> > +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> > +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> > +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> > +└────────────┘ └─────────────┘ └─────────────┘
> └─────────────┘ │
> > +(subtitles) │
> > + │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐
> ┌─────────────┐ │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR
> ├───┐ │
> > +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘
> └─────────────┘ │ │
> > +(Playback controls UI) │ │
> > + │ │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐
> ┌─────────────┐ │ │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐
> │ │
> > +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘
> └─────────────┘ │ │ │
> BT601 to pass-through seems odd here. Shouldn't this be BT2020 pass-through?
> > +(video frame) │ │ │
> > +
> > +│ │ │
> >
> +┌───────────────────────────────────────────────────────────────
> ─────
> > +────┴─┴─┘
> > +│
> > +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> > +│ │ CRTC │ │ CRTC │ │ CRTC │
> > +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────►
> TO Port
> > + │ │ │ │ │ │
> > + └─────────────┘ └─────────────┘ └───────────────┘
> > +
> > +This RFC is intended to propose an uAPI for the pre-blending color
> > +pipeline (however, can be also extended to post blending pipeline).
> > +
> > +Below are the design considerations while designing the uAPI.
> > +
> > +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only
> > +one can be used) 2. Position of the HW block can be programmable 3.
> > +LUTs can be one dimentional or three dimentional 4. Number of LUT
> > +entries can vary across platforms 5. Precision of LUT entries can
> > +vary across platforms 6. Distribution of LUT entries may vary. e.g
> > +Mutli-segmented, Logarithmic,
> > + Piece-Wise Linear(PWL) etc
> > +7. There can be parameterized/non-parameterized fixed function HW blocks.
> > + e.g. Just a hardware bit, to convert from one color space to another.
> > +
> > +Plane Color Pipeline: Design Proposal
> > +=====================================
> > +Each Color Hardware block will be represented by the structure drm_color_op.
> > +
> > +struct drm_color_op {
> > + enum color_op_block name;
> > + enum color_op_type type; > + u32 blob_id;
> > + u32 private_flags;
> > +};
> > +
> > +The members of which will constitute:
> > +1. name: A standardised enum for the color hardware block 2. type:
> > +The type of mathematical operation done by the hardware block.
> > + e.g. 1D Curve, 3D Curve, Matrix etc.
> Using both 'type' and 'name' to identify the block sounds like an over complication
> to me. From a UAPI perspective we mainly care about the functionality the block
> provides, and type should be able to cover this on its own. If name is needed to
> declare a divergence from the standard definition for the type, why wouldn't a
> new type just be declared?
>
> I'm currently not seeing any benefit that name provides.
We kept name just to give some logical identification to the hardware block with a
human readable name. But I agree given the combinations which could arise, its
hard to come up with a fixed name for a block. We can drop it.
> > +3. blob id: Id pointing to a blob containing information about the hardware
> > + block advertising the respective capabilities to the userspace.
> > + It can be an optional field depending on the members "name" and "type".
> > +4. private_flags: This can be used to provide vendor specific hints
> > + to user space
> > +
> If OEMs are declaring essentially hints in this property, how is user space going to
> handle this when they don't know how to interpret or handle them? To give any
> compositor a chance to use the color block these hints would have to result in no
> modification of the behavior agreed upon by the 'type', at which point I don't see
> what purpose they serve. The blob UAPI being fixed for a given color block type
> would also pose an issue here.
>
> On the other hand, if these flags are only used for vendor specific hardware
> blocks it sounds like we are clubbing a proprietary block + definition into the color
> pipeline. I don't think this co-exists well with the intent of the original proposal.
Our idea was to have an option to allow some custom hardware implementation which
can't be represented with generic design. However with the feedback received, we can surely
drop this from the generic structure. For future a custom PRIVATE_TYPE can be made
for which respective driver can implement the relevant data structures and blob contained
within the driver and supported by its custom userspace. This way generic userspace can
totally ignore it and implementation will be standardized.
> > +
> > + For example to represent LUTs, we introduce the drm_color_lut_range
> > + structure. It can represent LUTs with varied number of entries and
> > + distributions (Multi segmented, Logarithmic etc).
> > +
> > + struct drm_color_lut_range {
> > + /* DRM_MODE_LUT_* */
> > + __u32 flags;
> > + /* number of points on the curve */
> > + __u16 count;
> > + /* input/output bits per component */
> > + __u8 input_bpc, output_bpc;
> > + /* input start/end values */
> > + __s32 start, end;
> > + /* output min/max values */
> > + __s32 min, max;
> > + };
> > +
> > +Note: More details on exact usage and implementation of this structure can be
> > + found in the RFC. This structure is taken as is from the series.
> > + https://patchwork.freedesktop.org/series/90825/
> > + However, we can add more members to it to encompass all use-cases.
> > + For example. we can add a precision field to advertise the
> > + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> > + field for vendor specific use cases.
> > +
> > + At the same time, we don't need to pass any additional information for the
> > + CSC block as userspace and driver already agrees struct drm_color_ctm as
> > + a uAPI to pass on data.
> > +
> > +These color operations will form the building blocks of a color
> > +pipeline which best represents the underlying Hardware. Color
> > +operations can be re-arranged, substracted or added to create
> > +distinct color pipelines to accurately describe the Hardware blocks present in
> the display engine.
> > +
> > +In this proposal, a color pipeline is represented as an array of
> > +struct drm_color_op.
> > +
> > +struct drm_color_op color_pipeline_1[]
> > +
> > +For example to represent the pre-blending color pipeline as described
> > +above
> > +
> > +We can create a color pipeline like this.
> > +
> > +struct drm_color_op color_pipeline_1[] = {
> > + {
> > + .name = DRM_CB_PRE_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0; /* actual value to be populated during plane
> > + initialization*/
> > + },
> > + {
> > + .name = DRM_CB_CSC,
> > + .type = MATRIX,
> > + .blob_id = 0;
> > + },
> > + {
> > + .name = DRM_CB_POST_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0;
> > + },
> > +};
>
> I'm noticing that in the shared IGT patch set these names are being used to
> determine position instead of, well, the actual position in the color pipeline. We
> definitely need to be careful to avoid coupling functionality with usage, and I think
> having a name field is going to cause developers to make that mistake / shortcut.
Yeah we can drop the idea of name and go with type only instead to avoid any confusion
> > +
> > +Then, for individual color operation, we add blobs to advertise the capability
> > +of the particular Hardware block. In case of the example pipeline, we add
> > +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
> > +For the "csc" block we pass no blob id to user space as it is known to both
> > +user space and driver that drm_color_ctm structure is to be used for such
> > +operation.
> > +
> > +To represent, this in a diagram.
> > +
> > + struct drm_color_op color_pipeline_1[]
> > + +---------------------------+
> > + | | drm_color_op
> > + | +---------------------+--+-----------+---------------------+
> > + | | | | | |
> > + | | | | | +-----------------+ |
> > + | | | | | | name | |
> > + | | | | | +-----------------+ |
> > + | | | | | | type | |
> > + | | color_op_1 | | | +-----------------+ |
> > + | | | | | | blob id | +--------+
> > + | | | | | +-----------------+ | |
> > + | | | | | | private | | |
> > + | | | | | | flags | | |
> > + | | | | | +-----------------+ | |
> > + | | | | | | |
> > + | +---------------------+--+-----------+---------------------+ |
> > + | | |
> > + | | |
> > + | +---------------------+ | |
> > + | | | | drm_color_lut_range |
> > + | | color_op_2 | | +-------------------------+ |
> > + | | | | | | |
> > + | | | | | +---------------------+ | |
> > + | +---------------------+ | | | segment 1 { | |<---+
> > + | | | | ... | |
> > + | +---------------------+ | | | .input_bpc = 16, | |
> > + | | | | | | .output_bpc = 16, | |
> > + | | color_op_3 | | | | ... | |
> > + | | | | | | } | |
> > + | | | | | +---------------------+ |
> > + | +---------------------+ | | |
> > + | . | | +---------------------+ |
> > + | . | | | segment 2 { | |
> > + | . | | | ... | |
> > + +---------------------------+ | | } | |
> > + | | | |
> > + | | | |
> > + | | | |
> > + | +---------------------+ |
> > + | . |
> > + | . |
> > + | . |
> > + +-------------------------+
> > +
> > +
> > +
> > +This color pipeline is then packaged within a blob for the user space to
> > +retrieve it. Details can be found in the next section
> While I get the intent of the blob nesting, as others have mentioned
> this does feel like a recreation of what we already have with DRM
> objects. DRM objects would also have the advantage of allowing for
> easier OEM extension. Instead of private flags, an OEM could introduce
> behavior modifiers or additional features on a color block object via
> additional properties (these properties would obviously have to default
> to a disabled state to ensure default behavior is what is defined by the
> base color block type).
There are pros and cons to both the approaches, however we respect the
feedback of community and will try to adhere to a universally acceptable and
scalable design. We will work with Harry Wentland and other community developers
to come up with good color implementation in upstream, which meets the needs
of all stakeholders and is generic and scalable.
> > +
> > +Exposing a color pipeline to user space
> > +=======================================
> > +
> > +To advertise the available color pipelines, an immutable ENUM property
> > +"GET_COLOR_PIPELINE" is introduced.
> > +This enum property has blob id's as values. With each blob id representing
> > +a distinct color pipeline based on underlying HW capabilities and their
> > +respective combinations.
> > +
> > +The following output of drm_info [1], shows how color pipelines are visible
> > +to userspace.
> > +
> > +├───Plane 0
> > + │ ├───Object ID: 31
> > + │ ├───CRTCs: {0}
> > + │ ├───Legacy info
> > + ...
> > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> > + color pipeline 1, color pipeline
> 2}= no color pipeline
> > +
> > +To understand the capabilities of individual pipelines, first the userspace
> > +need to retrieve the pipeline blob with the blob ids retrieved by reading the
> > +enum property.
> > +
> > +Once the color pipeline is retrieved, user can then parse through
> > +individual drm_color_op blocks to understand the capabilities of each
> > +hardware block.
> > +
> > +Check IGT series to see how user space can parse through color pipelines.
> > +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
> > +
> > +Setting up a color pipeline
> > +===========================
> > +
> > +Once the user space decides on a color pipeline, it can set the pipeline and
> > +the corresponding data for the hardware blocks within the pipeline with
> > +the BLOB property "SET_COLOR_PIPELINE".
> > +
> > +To achieve this two structures are introduced
> > +
> > +1. struct drm_color_op_data: It represents data to be passed onto
> individual
> > + color hardware
> blocks. It contains three members
> > + a) name: to identify the color operation block
> > + b) blob_id: pointing to the blob with data for the
> > + corresponding hardware block
> > +
> > + struct drm_color_op_data {
> > + enum color_op_block name;
> > + u32 blob_id;
> > + };
> > +
> > +2. struct drm_color_pipeline: This structure represents the aggregate
> > + pipeline to be set. it contains the following members
> > + a) num: pipeline number to be
> selected
> > + b) size: size of the data to be passed
> onto the driver
> > + c) data: array of struct
> drm_color_op_data with data for
> > + the hardware block/s that userspace wants to
> > + set values for.
> > +
> > + struct drm_color_pipeline {
> > + int num;
> > + int size;
> > + struct drm_color_op_data *data;
> > + };
> > +
> > + User can either:
> > + 1. send data for all the color operations in a pipeline as shown in [2].
> > + The color operation data need not be in order that the pipeline
> advertises
> > + however, it should not contain data for any
> > + color operation that is not present in the pipeline.
> > +
> > + Note: This check for pipeline is not yet implemented but if the
> > + wider proposal is accepted we have few solutions in mind.
> > +
> > + 2. send data for a subset of color operations as shown in [3].For the
> > + color operation that userspace does not send data will retain their
> > + older state.
> > +
> > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > + property to NULL as shown in both [2] and [3]
> > +
> > + 4. change the color pipeline as demonstrated in [3].
> > + On the new pipeline, the user is expected to setup all the color
> hardware block
> > + Once the user requests a pipeline change, the driver will provide it a
> clean slate
> > + which means that all the data previously set by the user will be
> discarded even if
> > + there are common hardware blocks between the two(previous and
> current) pipelines.
> > +
> > +IGT implementation can be found here [4]
> > +
> > +Representing Fixed function hardware
> > +====================================
> > +
> > +To provide support for fixed function hardware, the driver could expose vendor
> > +specific struct drm_color_op with parameters that both the userspace and
> > +driver agrees on. To demonstrate, let's consider a hyphothetical fixed
> > +function hardware block that converts BT601 to BT2020.
> > +The driver can choose to advertise the block as such.
> > +
> > +struct drm_color_op color_pipeline_X[] = {
> > + {
> > + .name = DRM_CB_PRIVATE,
> > + .type = FIXED_FUNCTION,
> > + .blob_id = 45;
> > + },
> > +}
> > +
> > +Where the blob represents some vendor specific enums, strings or any other
> > +appropriate data types which both the user-space and drivers are aligned on.
> > +
> > +blob:45 {
> > + VENDORXXX_BT602_TO_BT2020,
> > +};
>
> As far as I see it, one of the primary advantages of moving to this new
> color pipeline is the standardization of the color block definition.
> User space looks at the 'type', and if it supports it then it can look
> at other expected information such as 'lut_size', 'bit_depth', etc to
> determine the programming considerations.
>
> The second we start having color blocks fully defined by private_flags
> and vendor specific blobs we end up with the same scenario we have today
> with vendor specific properties on our DRM CRTC and DRM PLANE objects.
> While we aren't going to make all the different OEM hardware match, we
> still want to encourage common approaches to block declaration so user
> space can easily identify functionality it supports across a wide range
> of hardware.
Yes we can skip this PRIVATE flag to avoid such situations. For Intel, we don’t need
it and generic types should suffice.
>
> > +
> > +For enabling or passing parameters to such blocks, the user can send data
> > +to the driver wrapped within a blob like any other color operation block.
> > +
> > + struct drm_color_op_data {
> > + .name = DRM_CB_PRIVATE;
> > + .blob_id = 46;
> > + } ;
> > +
> > +where blob with id 46 contains data to enable the fixed function hardware(s).
> > +
>
> All that said, thanks again for working on this. I'm definitely curious
> to see how this will all come together.
Thanks Christopher for your review and constructive feedback. We will work
with rest of the community to make this happen.
Apologies again for being late on my reply.
Regards,
Uma Shankar
> - Christopher
>
> > +References
> > +==========
> > +
> > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> > +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> > +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
@ 2023-10-13 5:46 ` Shankar, Uma
0 siblings, 0 replies; 109+ messages in thread
From: Shankar, Uma @ 2023-10-13 5:46 UTC (permalink / raw)
To: Christopher Braga, intel-gfx, dri-devel; +Cc: Harry Wentland, wayland-devel
> -----Original Message-----
> From: Christopher Braga <quic_cbraga@quicinc.com>
> Sent: Friday, September 8, 2023 1:39 AM
> To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri-
> devel@lists.freedesktop.org
> Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>; wayland-
> devel@lists.freedesktop.org
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> Color Pipeline
>
>
>
> On 8/29/2023 12:03 PM, Uma Shankar wrote:
> > Add the documentation for the new proposed Plane Color Pipeline.
> >
> > Co-developed-by: Chaitanya Kumar Borah
> > <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > ---
> > .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++
> > 1 file changed, 394 insertions(+)
> > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> >
> > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > new file mode 100644
> > index 000000000000..60ce515b6ea7
> > --- /dev/null
> > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > @@ -0,0 +1,394 @@
> > +=======================================
> > + Plane Color Pipeline: A UAPI proposal
> > +=======================================
> > +
> > +To build the proposal on, lets take the premise of a color pipeline
> > +as shown below.
> > +
> Hi Uma,
> Thanks for posting this. A few comments, with some echoing the sentiments of
> other commenters on the patch set.
Hi Christopher,
Thanks for your inputs on the series and my apologies for a late reply.
> > + +-------------------------------------------+
> > + | RAM |
> > + | +------+ +---------+ +---------+ |
> > + | | FB 1 | | FB 2 | | FB N | |
> > + | +------+ +---------+ +---------+ |
> > + +-------------------------------------------+
> > + | Plane Color Hardware Block |
> > + +--------------------------------------------+
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | | Pre-CSC | | Pre-CSC | | Pre-CSC | |
> > + | +---+-----+ +---+-------+ +---+------+ |
> > + | | | | |
> > + | +---v-----+ +---v-------+ +---v------+ |
> > + | |Plane A | | Plane B | | Plane N | |
> > + | |CSC/CTM | | CSC/CTM | | CSC/CTM | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + | +---v-----+ +----v------+ +----v-----+ |
> > + | | Plane A | | Plane B | | Plane N | |
> > + | |Post-CSC | | Post-CSC | | Post-CSC | |
> > + | +---+-----+ +----+------+ +----+-----+ |
> > + | | | | |
> > + +--------------------------------------------+
> > ++------v--------------v---------------v-------|
> > +|| ||
> > +|| Pipe Blender ||
> > ++--------------------+------------------------+
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Pre-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | Pipe Color |
> > +| +-----------v----------+ Hardware |
> > +| | Pipe CSC/CTM | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > +| +-----------v----------+ |
> > +| | Pipe Post-CSC | |
> > +| | | |
> > +| +-----------+----------+ |
> > +| | |
> > ++---------------------------------------------+
> > + |
> > + v
> > + Pipe Output
> > +
> > +Each plane consists of the following color blocks
> > + * Pre-CSC : This block can used to linearize the input frame buffer data.
> > + The linear data then can be further acted on by the following
> > + color hardware blocks in the display hardware pipeline
> > +
> > + * CSC/CTM: Used to program color transformation matrix, this block is used
> > + to perform color space conversions like BT2020 to BT709 or BT601
> > + etc. This block acts on the linearized data coming from the
> > + Pre-CSC HW block.
> > +
> > + * Post-CSC: This HW block can be used to non-linearize frame buffer data to
> > + match the sink. Another use case of it could be to perform Tone
> > + mapping for HDR use-cases.
> > +
> > +Data from multiple planes will then be fed to pipe/crtc where it will get
> blended.
> > +There is a similar set of HW blocks available at pipe/crtc level
> > +which acts on this blended data.
> > +
> > +Below is a sample usecase fo video playback with sub-titles and
> > +playback controls
> > +
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐
> ┌─────────────┐
> > +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │
> > +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │
> > +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐
> > +└────────────┘ └─────────────┘ └─────────────┘
> └─────────────┘ │
> > +(subtitles) │
> > + │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐
> ┌─────────────┐ │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR
> ├───┐ │
> > +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘
> └─────────────┘ │ │
> > +(Playback controls UI) │ │
> > + │ │
> > +┌────────────┐ ┌─────────────┐ ┌─────────────┐
> ┌─────────────┐ │ │
> > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │
> > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐
> │ │
> > +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │
> > +└────────────┘ └─────────────┘ └─────────────┘
> └─────────────┘ │ │ │
> BT601 to pass-through seems odd here. Shouldn't this be BT2020 pass-through?
> > +(video frame) │ │ │
> > +
> > +│ │ │
> >
> +┌───────────────────────────────────────────────────────────────
> ─────
> > +────┴─┴─┘
> > +│
> > +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐
> > +│ │ CRTC │ │ CRTC │ │ CRTC │
> > +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────►
> TO Port
> > + │ │ │ │ │ │
> > + └─────────────┘ └─────────────┘ └───────────────┘
> > +
> > +This RFC is intended to propose an uAPI for the pre-blending color
> > +pipeline (however, can be also extended to post blending pipeline).
> > +
> > +Below are the design considerations while designing the uAPI.
> > +
> > +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only
> > +one can be used) 2. Position of the HW block can be programmable 3.
> > +LUTs can be one dimentional or three dimentional 4. Number of LUT
> > +entries can vary across platforms 5. Precision of LUT entries can
> > +vary across platforms 6. Distribution of LUT entries may vary. e.g
> > +Mutli-segmented, Logarithmic,
> > + Piece-Wise Linear(PWL) etc
> > +7. There can be parameterized/non-parameterized fixed function HW blocks.
> > + e.g. Just a hardware bit, to convert from one color space to another.
> > +
> > +Plane Color Pipeline: Design Proposal
> > +=====================================
> > +Each Color Hardware block will be represented by the structure drm_color_op.
> > +
> > +struct drm_color_op {
> > + enum color_op_block name;
> > + enum color_op_type type; > + u32 blob_id;
> > + u32 private_flags;
> > +};
> > +
> > +The members of which will constitute:
> > +1. name: A standardised enum for the color hardware block 2. type:
> > +The type of mathematical operation done by the hardware block.
> > + e.g. 1D Curve, 3D Curve, Matrix etc.
> Using both 'type' and 'name' to identify the block sounds like an over complication
> to me. From a UAPI perspective we mainly care about the functionality the block
> provides, and type should be able to cover this on its own. If name is needed to
> declare a divergence from the standard definition for the type, why wouldn't a
> new type just be declared?
>
> I'm currently not seeing any benefit that name provides.
We kept name just to give some logical identification to the hardware block with a
human readable name. But I agree given the combinations which could arise, its
hard to come up with a fixed name for a block. We can drop it.
> > +3. blob id: Id pointing to a blob containing information about the hardware
> > + block advertising the respective capabilities to the userspace.
> > + It can be an optional field depending on the members "name" and "type".
> > +4. private_flags: This can be used to provide vendor specific hints
> > + to user space
> > +
> If OEMs are declaring essentially hints in this property, how is user space going to
> handle this when they don't know how to interpret or handle them? To give any
> compositor a chance to use the color block these hints would have to result in no
> modification of the behavior agreed upon by the 'type', at which point I don't see
> what purpose they serve. The blob UAPI being fixed for a given color block type
> would also pose an issue here.
>
> On the other hand, if these flags are only used for vendor specific hardware
> blocks it sounds like we are clubbing a proprietary block + definition into the color
> pipeline. I don't think this co-exists well with the intent of the original proposal.
Our idea was to have an option to allow some custom hardware implementation which
can't be represented with generic design. However with the feedback received, we can surely
drop this from the generic structure. For future a custom PRIVATE_TYPE can be made
for which respective driver can implement the relevant data structures and blob contained
within the driver and supported by its custom userspace. This way generic userspace can
totally ignore it and implementation will be standardized.
> > +
> > + For example to represent LUTs, we introduce the drm_color_lut_range
> > + structure. It can represent LUTs with varied number of entries and
> > + distributions (Multi segmented, Logarithmic etc).
> > +
> > + struct drm_color_lut_range {
> > + /* DRM_MODE_LUT_* */
> > + __u32 flags;
> > + /* number of points on the curve */
> > + __u16 count;
> > + /* input/output bits per component */
> > + __u8 input_bpc, output_bpc;
> > + /* input start/end values */
> > + __s32 start, end;
> > + /* output min/max values */
> > + __s32 min, max;
> > + };
> > +
> > +Note: More details on exact usage and implementation of this structure can be
> > + found in the RFC. This structure is taken as is from the series.
> > + https://patchwork.freedesktop.org/series/90825/
> > + However, we can add more members to it to encompass all use-cases.
> > + For example. we can add a precision field to advertise the
> > + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag
> > + field for vendor specific use cases.
> > +
> > + At the same time, we don't need to pass any additional information for the
> > + CSC block as userspace and driver already agrees struct drm_color_ctm as
> > + a uAPI to pass on data.
> > +
> > +These color operations will form the building blocks of a color
> > +pipeline which best represents the underlying Hardware. Color
> > +operations can be re-arranged, substracted or added to create
> > +distinct color pipelines to accurately describe the Hardware blocks present in
> the display engine.
> > +
> > +In this proposal, a color pipeline is represented as an array of
> > +struct drm_color_op.
> > +
> > +struct drm_color_op color_pipeline_1[]
> > +
> > +For example to represent the pre-blending color pipeline as described
> > +above
> > +
> > +We can create a color pipeline like this.
> > +
> > +struct drm_color_op color_pipeline_1[] = {
> > + {
> > + .name = DRM_CB_PRE_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0; /* actual value to be populated during plane
> > + initialization*/
> > + },
> > + {
> > + .name = DRM_CB_CSC,
> > + .type = MATRIX,
> > + .blob_id = 0;
> > + },
> > + {
> > + .name = DRM_CB_POST_CSC,
> > + .type = CURVE_1D,
> > + .blob_id = 0;
> > + },
> > +};
>
> I'm noticing that in the shared IGT patch set these names are being used to
> determine position instead of, well, the actual position in the color pipeline. We
> definitely need to be careful to avoid coupling functionality with usage, and I think
> having a name field is going to cause developers to make that mistake / shortcut.
Yeah we can drop the idea of name and go with type only instead to avoid any confusion
> > +
> > +Then, for individual color operation, we add blobs to advertise the capability
> > +of the particular Hardware block. In case of the example pipeline, we add
> > +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
> > +For the "csc" block we pass no blob id to user space as it is known to both
> > +user space and driver that drm_color_ctm structure is to be used for such
> > +operation.
> > +
> > +To represent, this in a diagram.
> > +
> > + struct drm_color_op color_pipeline_1[]
> > + +---------------------------+
> > + | | drm_color_op
> > + | +---------------------+--+-----------+---------------------+
> > + | | | | | |
> > + | | | | | +-----------------+ |
> > + | | | | | | name | |
> > + | | | | | +-----------------+ |
> > + | | | | | | type | |
> > + | | color_op_1 | | | +-----------------+ |
> > + | | | | | | blob id | +--------+
> > + | | | | | +-----------------+ | |
> > + | | | | | | private | | |
> > + | | | | | | flags | | |
> > + | | | | | +-----------------+ | |
> > + | | | | | | |
> > + | +---------------------+--+-----------+---------------------+ |
> > + | | |
> > + | | |
> > + | +---------------------+ | |
> > + | | | | drm_color_lut_range |
> > + | | color_op_2 | | +-------------------------+ |
> > + | | | | | | |
> > + | | | | | +---------------------+ | |
> > + | +---------------------+ | | | segment 1 { | |<---+
> > + | | | | ... | |
> > + | +---------------------+ | | | .input_bpc = 16, | |
> > + | | | | | | .output_bpc = 16, | |
> > + | | color_op_3 | | | | ... | |
> > + | | | | | | } | |
> > + | | | | | +---------------------+ |
> > + | +---------------------+ | | |
> > + | . | | +---------------------+ |
> > + | . | | | segment 2 { | |
> > + | . | | | ... | |
> > + +---------------------------+ | | } | |
> > + | | | |
> > + | | | |
> > + | | | |
> > + | +---------------------+ |
> > + | . |
> > + | . |
> > + | . |
> > + +-------------------------+
> > +
> > +
> > +
> > +This color pipeline is then packaged within a blob for the user space to
> > +retrieve it. Details can be found in the next section
> While I get the intent of the blob nesting, as others have mentioned
> this does feel like a recreation of what we already have with DRM
> objects. DRM objects would also have the advantage of allowing for
> easier OEM extension. Instead of private flags, an OEM could introduce
> behavior modifiers or additional features on a color block object via
> additional properties (these properties would obviously have to default
> to a disabled state to ensure default behavior is what is defined by the
> base color block type).
There are pros and cons to both the approaches, however we respect the
feedback of community and will try to adhere to a universally acceptable and
scalable design. We will work with Harry Wentland and other community developers
to come up with good color implementation in upstream, which meets the needs
of all stakeholders and is generic and scalable.
> > +
> > +Exposing a color pipeline to user space
> > +=======================================
> > +
> > +To advertise the available color pipelines, an immutable ENUM property
> > +"GET_COLOR_PIPELINE" is introduced.
> > +This enum property has blob id's as values. With each blob id representing
> > +a distinct color pipeline based on underlying HW capabilities and their
> > +respective combinations.
> > +
> > +The following output of drm_info [1], shows how color pipelines are visible
> > +to userspace.
> > +
> > +├───Plane 0
> > + │ ├───Object ID: 31
> > + │ ├───CRTCs: {0}
> > + │ ├───Legacy info
> > + ...
> > + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline,
> > + color pipeline 1, color pipeline
> 2}= no color pipeline
> > +
> > +To understand the capabilities of individual pipelines, first the userspace
> > +need to retrieve the pipeline blob with the blob ids retrieved by reading the
> > +enum property.
> > +
> > +Once the color pipeline is retrieved, user can then parse through
> > +individual drm_color_op blocks to understand the capabilities of each
> > +hardware block.
> > +
> > +Check IGT series to see how user space can parse through color pipelines.
> > +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/
> > +
> > +Setting up a color pipeline
> > +===========================
> > +
> > +Once the user space decides on a color pipeline, it can set the pipeline and
> > +the corresponding data for the hardware blocks within the pipeline with
> > +the BLOB property "SET_COLOR_PIPELINE".
> > +
> > +To achieve this two structures are introduced
> > +
> > +1. struct drm_color_op_data: It represents data to be passed onto
> individual
> > + color hardware
> blocks. It contains three members
> > + a) name: to identify the color operation block
> > + b) blob_id: pointing to the blob with data for the
> > + corresponding hardware block
> > +
> > + struct drm_color_op_data {
> > + enum color_op_block name;
> > + u32 blob_id;
> > + };
> > +
> > +2. struct drm_color_pipeline: This structure represents the aggregate
> > + pipeline to be set. it contains the following members
> > + a) num: pipeline number to be
> selected
> > + b) size: size of the data to be passed
> onto the driver
> > + c) data: array of struct
> drm_color_op_data with data for
> > + the hardware block/s that userspace wants to
> > + set values for.
> > +
> > + struct drm_color_pipeline {
> > + int num;
> > + int size;
> > + struct drm_color_op_data *data;
> > + };
> > +
> > + User can either:
> > + 1. send data for all the color operations in a pipeline as shown in [2].
> > + The color operation data need not be in order that the pipeline
> advertises
> > + however, it should not contain data for any
> > + color operation that is not present in the pipeline.
> > +
> > + Note: This check for pipeline is not yet implemented but if the
> > + wider proposal is accepted we have few solutions in mind.
> > +
> > + 2. send data for a subset of color operations as shown in [3].For the
> > + color operation that userspace does not send data will retain their
> > + older state.
> > +
> > + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob
> > + property to NULL as shown in both [2] and [3]
> > +
> > + 4. change the color pipeline as demonstrated in [3].
> > + On the new pipeline, the user is expected to setup all the color
> hardware block
> > + Once the user requests a pipeline change, the driver will provide it a
> clean slate
> > + which means that all the data previously set by the user will be
> discarded even if
> > + there are common hardware blocks between the two(previous and
> current) pipelines.
> > +
> > +IGT implementation can be found here [4]
> > +
> > +Representing Fixed function hardware
> > +====================================
> > +
> > +To provide support for fixed function hardware, the driver could expose vendor
> > +specific struct drm_color_op with parameters that both the userspace and
> > +driver agrees on. To demonstrate, let's consider a hyphothetical fixed
> > +function hardware block that converts BT601 to BT2020.
> > +The driver can choose to advertise the block as such.
> > +
> > +struct drm_color_op color_pipeline_X[] = {
> > + {
> > + .name = DRM_CB_PRIVATE,
> > + .type = FIXED_FUNCTION,
> > + .blob_id = 45;
> > + },
> > +}
> > +
> > +Where the blob represents some vendor specific enums, strings or any other
> > +appropriate data types which both the user-space and drivers are aligned on.
> > +
> > +blob:45 {
> > + VENDORXXX_BT602_TO_BT2020,
> > +};
>
> As far as I see it, one of the primary advantages of moving to this new
> color pipeline is the standardization of the color block definition.
> User space looks at the 'type', and if it supports it then it can look
> at other expected information such as 'lut_size', 'bit_depth', etc to
> determine the programming considerations.
>
> The second we start having color blocks fully defined by private_flags
> and vendor specific blobs we end up with the same scenario we have today
> with vendor specific properties on our DRM CRTC and DRM PLANE objects.
> While we aren't going to make all the different OEM hardware match, we
> still want to encourage common approaches to block declaration so user
> space can easily identify functionality it supports across a wide range
> of hardware.
Yes we can skip this PRIVATE flag to avoid such situations. For Intel, we don’t need
it and generic types should suffice.
>
> > +
> > +For enabling or passing parameters to such blocks, the user can send data
> > +to the driver wrapped within a blob like any other color operation block.
> > +
> > + struct drm_color_op_data {
> > + .name = DRM_CB_PRIVATE;
> > + .blob_id = 46;
> > + } ;
> > +
> > +where blob with id 46 contains data to enable the fixed function hardware(s).
> > +
>
> All that said, thanks again for working on this. I'm definitely curious
> to see how this will all come together.
Thanks Christopher for your review and constructive feedback. We will work
with rest of the community to make this happen.
Apologies again for being late on my reply.
Regards,
Uma Shankar
> - Christopher
>
> > +References
> > +==========
> > +
> > +[1] https://gitlab.freedesktop.org/emersion/drm_info
> > +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1
> > +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1
> > +[4] https://patchwork.freedesktop.org/series/123018/
^ permalink raw reply [flat|nested] 109+ messages in thread
end of thread, other threads:[~2023-10-13 5:47 UTC | newest]
Thread overview: 109+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-29 16:03 [RFC 00/33] Add Support for Plane Color Pipeline Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:03 ` [RFC 01/33] drm/doc/rfc: Add RFC document for proposed " Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 19:40 ` Harry Wentland
2023-08-29 19:40 ` [Intel-gfx] " Harry Wentland
2023-08-30 8:59 ` Shankar, Uma
2023-08-30 8:59 ` [Intel-gfx] " Shankar, Uma
2023-08-30 12:28 ` Pekka Paalanen
2023-08-30 12:28 ` [Intel-gfx] " Pekka Paalanen
2023-09-04 13:44 ` Shankar, Uma
2023-09-04 13:44 ` Shankar, Uma
2023-09-05 11:32 ` Pekka Paalanen
2023-09-05 11:32 ` [Intel-gfx] " Pekka Paalanen
2023-09-07 12:31 ` Shankar, Uma
2023-09-07 12:31 ` [Intel-gfx] " Shankar, Uma
2023-09-08 8:31 ` Pekka Paalanen
2023-09-08 8:31 ` [Intel-gfx] " Pekka Paalanen
2023-09-07 20:08 ` Christopher Braga
2023-09-07 20:08 ` [Intel-gfx] " Christopher Braga
2023-10-13 5:46 ` Shankar, Uma
2023-10-13 5:46 ` [Intel-gfx] " Shankar, Uma
2023-08-29 16:03 ` [RFC 02/33] drm: Add color operation structure Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-30 13:00 ` Pekka Paalanen
2023-08-30 13:00 ` [Intel-gfx] " Pekka Paalanen
2023-09-04 14:10 ` Shankar, Uma
2023-09-04 14:10 ` [Intel-gfx] " Shankar, Uma
2023-09-05 11:33 ` Pekka Paalanen
2023-09-05 11:33 ` [Intel-gfx] " Pekka Paalanen
2023-08-29 16:03 ` [RFC 03/33] drm: Add plane get color pipeline property Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:03 ` [RFC 04/33] drm: Add helper to add color pipeline Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:03 ` [RFC 05/33] drm: Add structures for setting " Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] [RFC 06/33] drm: Add set colorpipeline property Uma Shankar
2023-08-29 16:03 ` Uma Shankar
2023-08-29 16:03 ` [RFC 07/33] drm: Add Enhanced Gamma LUT precision structure Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:03 ` [RFC 08/33] drm: Add color lut range structure Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:03 ` [RFC 09/33] drm: Add color information to plane state Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:03 ` [RFC 10/33] drm: Manage color blob states Uma Shankar
2023-08-29 16:03 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 11/33] drm: Replace individual color blobs Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 12/33] drm: Reset pipeline when user sends NULL blob Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 13/33] drm: Reset plane color state on pipeline switch request Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 14/33] drm/i915/color: Add lut range for SDR planes Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 15/33] drm/i915/color: Add lut range for HDR planes Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 16/33] drm/i915/color: Add color pipeline " Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 17/33] drm/i915/color: Add color pipeline for SDR planes Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 18/33] drm/i915/color: Add HDR plane LUT range data to color pipeline Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 19/33] drm/i915/color: Add SDR " Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 20/33] drm/i915/color: Add color pipelines to plane Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 21/33] drm/i915/color: Create and attach set color pipeline property Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 22/33] drm/i915/color: Add plane color callbacks Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 23/33] drm/i915/color: Load plane color luts from atomic flip Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 24/33] drm/i915/xelpd: Add plane color check to glk_plane_color_ctl Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 25/33] drm/i915/xelpd: Add register definitions for Plane Degamma Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 26/33] drm/i915/color: Add color functions for ADL Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 27/33] drm/i915/color: Program Plane Pre-CSC Registers Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 28/33] drm/i915/xelpd: Add register definitions for Plane Post CSC Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 29/33] drm/i915/xelpd: Program Plane Post CSC Registers Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 30/33] drm/i915/color: Enable Plane CSC Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 31/33] drm/i915/color: Enable plane color features Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 32/33] drm/i915/color: Add a dummy pipeline with 3D LUT Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 16:04 ` [RFC 33/33] drm/i915/color: Add example implementation for vendor specific color operation Uma Shankar
2023-08-29 16:04 ` [Intel-gfx] " Uma Shankar
2023-08-29 19:26 ` [RFC 00/33] Add Support for Plane Color Pipeline Harry Wentland
2023-08-29 19:26 ` [Intel-gfx] " Harry Wentland
2023-08-30 8:47 ` Shankar, Uma
2023-08-30 8:47 ` [Intel-gfx] " Shankar, Uma
2023-08-30 21:15 ` Sebastian Wick
2023-08-30 21:15 ` [Intel-gfx] " Sebastian Wick
2023-09-04 14:29 ` Shankar, Uma
2023-09-04 14:29 ` [Intel-gfx] " Shankar, Uma
2023-09-05 11:33 ` Pekka Paalanen
2023-09-05 11:33 ` [Intel-gfx] " Pekka Paalanen
2023-09-05 12:33 ` Sebastian Wick
2023-09-05 12:33 ` [Intel-gfx] " Sebastian Wick
2023-09-05 12:57 ` Sebastian Wick
2023-09-05 12:57 ` [Intel-gfx] " Sebastian Wick
2023-08-29 20:02 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for " Patchwork
2023-08-29 20:02 ` [Intel-gfx] ✗ Fi.CI.SPARSE: " Patchwork
2023-08-29 20:17 ` [Intel-gfx] ✗ Fi.CI.BAT: failure " Patchwork
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.