[PATCH v3] Documentation: gpu: Mention the requirements for new properties

[PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Maxime Ripard]

New KMS properties come with a bunch of requirements to avoid each
driver from running their own, inconsistent, set of properties,
eventually leading to issues like property conflicts, inconsistencies
between drivers and semantics, etc.

Let's document what we expect.

Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Alison Wang <alison.wang@nxp.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: Andrew Jeffery <andrew@aj.id.au>
Cc: Andrzej Hajda <a.hajda@samsung.com>
Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Boris Brezillon <bbrezillon@kernel.org>
Cc: Brian Starkey <brian.starkey@arm.com>
Cc: Chen Feng <puck.chen@hisilicon.com>
Cc: Chen-Yu Tsai <wens@csie.org>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
Cc: Edmund Dea <edmund.j.dea@intel.com>
Cc: Eric Anholt <eric@anholt.net>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: "Heiko Stübner" <heiko@sntech.de>
Cc: Huang Rui <ray.huang@amd.com>
Cc: Hyun Kwon <hyun.kwon@xilinx.com>
Cc: Inki Dae <inki.dae@samsung.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Jernej Skrabec <jernej.skrabec@siol.net>
Cc: Jerome Brunet <jbrunet@baylibre.com>
Cc: Joel Stanley <joel@jms.id.au>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Jonas Karlman <jonas@kwiboo.se>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Joonyoung Shim <jy0922.shim@samsung.com>
Cc: Jyri Sarha <jyri.sarha@iki.fi>
Cc: Kevin Hilman <khilman@baylibre.com>
Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
Cc: Kyungmin Park <kyungmin.park@samsung.com>
Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Liviu Dudau <liviu.dudau@arm.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
Cc: Marek Vasut <marex@denx.de>
Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Melissa Wen <melissa.srw@gmail.com>
Cc: Neil Armstrong <narmstrong@baylibre.com>
Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
Cc: "Noralf Trønnes" <noralf@tronnes.org>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
Cc: Paul Cercueil <paul@crapouillou.net>
Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
Cc: Philippe Cornu <philippe.cornu@foss.st.com>
Cc: Philipp Zabel <p.zabel@pengutronix.de>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Robert Foss <robert.foss@linaro.org>
Cc: Rob Herring <robh@kernel.org>
Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Roland Scheidegger <sroland@vmware.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Sandy Huang <hjc@rock-chips.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Sean Paul <sean@poorly.run>
Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Stefan Agner <stefan@agner.ch>
Cc: Steven Price <steven.price@arm.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Tian Tao <tiantao6@hisilicon.com>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Tomi Valkeinen <tomba@kernel.org>
Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
Cc: Xinliang Liu <xinliang.liu@linaro.org>
Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
Cc: Yannick Fertre <yannick.fertre@foss.st.com>
Cc: Zack Rusin <zackr@vmware.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Maxime Ripard <maxime@cerno.tech>

---

Changes from v2:
  - Take into account the feedback from Laurent and Lidiu to no longer
    force generic properties, but prefix vendor-specific properties with
    the vendor name

Changes from v1:
  - Typos and wording reported by Daniel and Alex
---
 Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 87e5023e3f55..bbe254dca635 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -463,6 +463,33 @@ KMS Properties
 This section of the documentation is primarily aimed at user-space developers.
 For the driver APIs, see the other sections.
 
+Requirements
+------------
+
+KMS drivers might need to add extra properties to support new features.
+Each new property introduced in a driver need to meet a few
+requirements, in addition to the one mentioned above.:
+
+- Before the introduction of any vendor-specific properties, they must
+  be first checked against the generic ones to avoid any conflict or
+  redundancy.
+
+- Vendor-specific properties must be prefixed by the vendor's name,
+  following the syntax "$vendor:$property".
+
+- Generic properties must be standardized, with some documentation to
+  describe how the property can be used.
+
+- Generic properties must provide a generic helper in the core code to
+  register that property on the object it attaches to.
+
+- Generic properties content must be decoded by the core and provided in
+  the object's associated state structure. That includes anything
+  drivers might want to precompute, like :c:type:`struct drm_clip_rect
+  <drm_clip_rect>` for planes.
+
+- An IGT test should be submitted.
+
 Property Types and Blob Property Support
 ----------------------------------------
 
-- 
2.31.1

[PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Maxime Ripard]

New KMS properties come with a bunch of requirements to avoid each
driver from running their own, inconsistent, set of properties,
eventually leading to issues like property conflicts, inconsistencies
between drivers and semantics, etc.

Let's document what we expect.

Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Alison Wang <alison.wang@nxp.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: Andrew Jeffery <andrew@aj.id.au>
Cc: Andrzej Hajda <a.hajda@samsung.com>
Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Boris Brezillon <bbrezillon@kernel.org>
Cc: Brian Starkey <brian.starkey@arm.com>
Cc: Chen Feng <puck.chen@hisilicon.com>
Cc: Chen-Yu Tsai <wens@csie.org>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
Cc: Edmund Dea <edmund.j.dea@intel.com>
Cc: Eric Anholt <eric@anholt.net>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: "Heiko Stübner" <heiko@sntech.de>
Cc: Huang Rui <ray.huang@amd.com>
Cc: Hyun Kwon <hyun.kwon@xilinx.com>
Cc: Inki Dae <inki.dae@samsung.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Jernej Skrabec <jernej.skrabec@siol.net>
Cc: Jerome Brunet <jbrunet@baylibre.com>
Cc: Joel Stanley <joel@jms.id.au>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Jonas Karlman <jonas@kwiboo.se>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Joonyoung Shim <jy0922.shim@samsung.com>
Cc: Jyri Sarha <jyri.sarha@iki.fi>
Cc: Kevin Hilman <khilman@baylibre.com>
Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
Cc: Kyungmin Park <kyungmin.park@samsung.com>
Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Liviu Dudau <liviu.dudau@arm.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
Cc: Marek Vasut <marex@denx.de>
Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Melissa Wen <melissa.srw@gmail.com>
Cc: Neil Armstrong <narmstrong@baylibre.com>
Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
Cc: "Noralf Trønnes" <noralf@tronnes.org>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
Cc: Paul Cercueil <paul@crapouillou.net>
Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
Cc: Philippe Cornu <philippe.cornu@foss.st.com>
Cc: Philipp Zabel <p.zabel@pengutronix.de>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Robert Foss <robert.foss@linaro.org>
Cc: Rob Herring <robh@kernel.org>
Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Roland Scheidegger <sroland@vmware.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Sandy Huang <hjc@rock-chips.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Sean Paul <sean@poorly.run>
Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Stefan Agner <stefan@agner.ch>
Cc: Steven Price <steven.price@arm.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Tian Tao <tiantao6@hisilicon.com>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Tomi Valkeinen <tomba@kernel.org>
Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
Cc: Xinliang Liu <xinliang.liu@linaro.org>
Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
Cc: Yannick Fertre <yannick.fertre@foss.st.com>
Cc: Zack Rusin <zackr@vmware.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Maxime Ripard <maxime@cerno.tech>

---

Changes from v2:
  - Take into account the feedback from Laurent and Lidiu to no longer
    force generic properties, but prefix vendor-specific properties with
    the vendor name

Changes from v1:
  - Typos and wording reported by Daniel and Alex
---
 Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 87e5023e3f55..bbe254dca635 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -463,6 +463,33 @@ KMS Properties
 This section of the documentation is primarily aimed at user-space developers.
 For the driver APIs, see the other sections.
 
+Requirements
+------------
+
+KMS drivers might need to add extra properties to support new features.
+Each new property introduced in a driver need to meet a few
+requirements, in addition to the one mentioned above.:
+
+- Before the introduction of any vendor-specific properties, they must
+  be first checked against the generic ones to avoid any conflict or
+  redundancy.
+
+- Vendor-specific properties must be prefixed by the vendor's name,
+  following the syntax "$vendor:$property".
+
+- Generic properties must be standardized, with some documentation to
+  describe how the property can be used.
+
+- Generic properties must provide a generic helper in the core code to
+  register that property on the object it attaches to.
+
+- Generic properties content must be decoded by the core and provided in
+  the object's associated state structure. That includes anything
+  drivers might want to precompute, like :c:type:`struct drm_clip_rect
+  <drm_clip_rect>` for planes.
+
+- An IGT test should be submitted.
+
 Property Types and Blob Property Support
 ----------------------------------------
 
-- 
2.31.1

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

Hi Maxime,

On Thu, Jun 10, 2021 at 07:47:31PM +0200, Maxime Ripard wrote:
> New KMS properties come with a bunch of requirements to avoid each
> driver from running their own, inconsistent, set of properties,
> eventually leading to issues like property conflicts, inconsistencies
> between drivers and semantics, etc.
> 
> Let's document what we expect.
> 
> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Alison Wang <alison.wang@nxp.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Andrew Jeffery <andrew@aj.id.au>
> Cc: Andrzej Hajda <a.hajda@samsung.com>
> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> Cc: Ben Skeggs <bskeggs@redhat.com>
> Cc: Boris Brezillon <bbrezillon@kernel.org>
> Cc: Brian Starkey <brian.starkey@arm.com>
> Cc: Chen Feng <puck.chen@hisilicon.com>
> Cc: Chen-Yu Tsai <wens@csie.org>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> Cc: Edmund Dea <edmund.j.dea@intel.com>
> Cc: Eric Anholt <eric@anholt.net>
> Cc: Fabio Estevam <festevam@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: "Heiko Stübner" <heiko@sntech.de>
> Cc: Huang Rui <ray.huang@amd.com>
> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> Cc: Inki Dae <inki.dae@samsung.com>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> Cc: Jerome Brunet <jbrunet@baylibre.com>
> Cc: Joel Stanley <joel@jms.id.au>
> Cc: John Stultz <john.stultz@linaro.org>
> Cc: Jonas Karlman <jonas@kwiboo.se>
> Cc: Jonathan Hunter <jonathanh@nvidia.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> Cc: Jyri Sarha <jyri.sarha@iki.fi>
> Cc: Kevin Hilman <khilman@baylibre.com>
> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> Cc: Kyungmin Park <kyungmin.park@samsung.com>
> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Liviu Dudau <liviu.dudau@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> Cc: Matthias Brugger <matthias.bgg@gmail.com>
> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Cc: Neil Armstrong <narmstrong@baylibre.com>
> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> Cc: "Noralf Trønnes" <noralf@tronnes.org>
> Cc: NXP Linux Team <linux-imx@nxp.com>
> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> Cc: Paul Cercueil <paul@crapouillou.net>
> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> Cc: Philipp Zabel <p.zabel@pengutronix.de>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Robert Foss <robert.foss@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Roland Scheidegger <sroland@vmware.com>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Sam Ravnborg <sam@ravnborg.org>
> Cc: Sandy Huang <hjc@rock-chips.com>
> Cc: Sascha Hauer <s.hauer@pengutronix.de>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> Cc: Shawn Guo <shawnguo@kernel.org>
> Cc: Stefan Agner <stefan@agner.ch>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: Thierry Reding <thierry.reding@gmail.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Tomi Valkeinen <tomba@kernel.org>
> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> Cc: Xinliang Liu <xinliang.liu@linaro.org>
> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> Cc: Zack Rusin <zackr@vmware.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> 
> ---
> 
> Changes from v2:
>   - Take into account the feedback from Laurent and Lidiu to no longer
>     force generic properties, but prefix vendor-specific properties with
>     the vendor name
> 
> Changes from v1:
>   - Typos and wording reported by Daniel and Alex
> ---
>  Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index 87e5023e3f55..bbe254dca635 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -463,6 +463,33 @@ KMS Properties
>  This section of the documentation is primarily aimed at user-space developers.
>  For the driver APIs, see the other sections.
>  
> +Requirements
> +------------
> +
> +KMS drivers might need to add extra properties to support new features.
> +Each new property introduced in a driver need to meet a few
> +requirements, in addition to the one mentioned above.:
> +
> +- Before the introduction of any vendor-specific properties, they must
> +  be first checked against the generic ones to avoid any conflict or
> +  redundancy.
> +
> +- Vendor-specific properties must be prefixed by the vendor's name,
> +  following the syntax "$vendor:$property".
> +
> +- Generic properties must be standardized, with some documentation to
> +  describe how the property can be used.
> +
> +- Generic properties must provide a generic helper in the core code to
> +  register that property on the object it attaches to.
> +
> +- Generic properties content must be decoded by the core and provided in
> +  the object's associated state structure. That includes anything
> +  drivers might want to precompute, like :c:type:`struct drm_clip_rect
> +  <drm_clip_rect>` for planes.
> +
> +- An IGT test should be submitted.
> +
>  Property Types and Blob Property Support
>  ----------------------------------------

Looks nice, thanks for the work!

Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>

Best regards,
Liviu

>  
> -- 
> 2.31.1
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

Hi Maxime,

On Thu, Jun 10, 2021 at 07:47:31PM +0200, Maxime Ripard wrote:
> New KMS properties come with a bunch of requirements to avoid each
> driver from running their own, inconsistent, set of properties,
> eventually leading to issues like property conflicts, inconsistencies
> between drivers and semantics, etc.
> 
> Let's document what we expect.
> 
> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Alison Wang <alison.wang@nxp.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Andrew Jeffery <andrew@aj.id.au>
> Cc: Andrzej Hajda <a.hajda@samsung.com>
> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> Cc: Ben Skeggs <bskeggs@redhat.com>
> Cc: Boris Brezillon <bbrezillon@kernel.org>
> Cc: Brian Starkey <brian.starkey@arm.com>
> Cc: Chen Feng <puck.chen@hisilicon.com>
> Cc: Chen-Yu Tsai <wens@csie.org>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> Cc: Edmund Dea <edmund.j.dea@intel.com>
> Cc: Eric Anholt <eric@anholt.net>
> Cc: Fabio Estevam <festevam@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: "Heiko Stübner" <heiko@sntech.de>
> Cc: Huang Rui <ray.huang@amd.com>
> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> Cc: Inki Dae <inki.dae@samsung.com>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> Cc: Jerome Brunet <jbrunet@baylibre.com>
> Cc: Joel Stanley <joel@jms.id.au>
> Cc: John Stultz <john.stultz@linaro.org>
> Cc: Jonas Karlman <jonas@kwiboo.se>
> Cc: Jonathan Hunter <jonathanh@nvidia.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> Cc: Jyri Sarha <jyri.sarha@iki.fi>
> Cc: Kevin Hilman <khilman@baylibre.com>
> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> Cc: Kyungmin Park <kyungmin.park@samsung.com>
> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Liviu Dudau <liviu.dudau@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> Cc: Matthias Brugger <matthias.bgg@gmail.com>
> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Cc: Neil Armstrong <narmstrong@baylibre.com>
> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> Cc: "Noralf Trønnes" <noralf@tronnes.org>
> Cc: NXP Linux Team <linux-imx@nxp.com>
> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> Cc: Paul Cercueil <paul@crapouillou.net>
> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> Cc: Philipp Zabel <p.zabel@pengutronix.de>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Robert Foss <robert.foss@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Roland Scheidegger <sroland@vmware.com>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Sam Ravnborg <sam@ravnborg.org>
> Cc: Sandy Huang <hjc@rock-chips.com>
> Cc: Sascha Hauer <s.hauer@pengutronix.de>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> Cc: Shawn Guo <shawnguo@kernel.org>
> Cc: Stefan Agner <stefan@agner.ch>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: Thierry Reding <thierry.reding@gmail.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Tomi Valkeinen <tomba@kernel.org>
> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> Cc: Xinliang Liu <xinliang.liu@linaro.org>
> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> Cc: Zack Rusin <zackr@vmware.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> 
> ---
> 
> Changes from v2:
>   - Take into account the feedback from Laurent and Lidiu to no longer
>     force generic properties, but prefix vendor-specific properties with
>     the vendor name
> 
> Changes from v1:
>   - Typos and wording reported by Daniel and Alex
> ---
>  Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index 87e5023e3f55..bbe254dca635 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -463,6 +463,33 @@ KMS Properties
>  This section of the documentation is primarily aimed at user-space developers.
>  For the driver APIs, see the other sections.
>  
> +Requirements
> +------------
> +
> +KMS drivers might need to add extra properties to support new features.
> +Each new property introduced in a driver need to meet a few
> +requirements, in addition to the one mentioned above.:
> +
> +- Before the introduction of any vendor-specific properties, they must
> +  be first checked against the generic ones to avoid any conflict or
> +  redundancy.
> +
> +- Vendor-specific properties must be prefixed by the vendor's name,
> +  following the syntax "$vendor:$property".
> +
> +- Generic properties must be standardized, with some documentation to
> +  describe how the property can be used.
> +
> +- Generic properties must provide a generic helper in the core code to
> +  register that property on the object it attaches to.
> +
> +- Generic properties content must be decoded by the core and provided in
> +  the object's associated state structure. That includes anything
> +  drivers might want to precompute, like :c:type:`struct drm_clip_rect
> +  <drm_clip_rect>` for planes.
> +
> +- An IGT test should be submitted.
> +
>  Property Types and Blob Property Support
>  ----------------------------------------

Looks nice, thanks for the work!

Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>

Best regards,
Liviu

>  
> -- 
> 2.31.1
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Rodrigo Vivi]

On Thu, Jun 10, 2021 at 07:47:31PM +0200, Maxime Ripard wrote:
> New KMS properties come with a bunch of requirements to avoid each
> driver from running their own, inconsistent, set of properties,
> eventually leading to issues like property conflicts, inconsistencies
> between drivers and semantics, etc.
> 
> Let's document what we expect.
> 
> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Alison Wang <alison.wang@nxp.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Andrew Jeffery <andrew@aj.id.au>
> Cc: Andrzej Hajda <a.hajda@samsung.com>
> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> Cc: Ben Skeggs <bskeggs@redhat.com>
> Cc: Boris Brezillon <bbrezillon@kernel.org>
> Cc: Brian Starkey <brian.starkey@arm.com>
> Cc: Chen Feng <puck.chen@hisilicon.com>
> Cc: Chen-Yu Tsai <wens@csie.org>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> Cc: Edmund Dea <edmund.j.dea@intel.com>
> Cc: Eric Anholt <eric@anholt.net>
> Cc: Fabio Estevam <festevam@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: "Heiko Stübner" <heiko@sntech.de>
> Cc: Huang Rui <ray.huang@amd.com>
> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> Cc: Inki Dae <inki.dae@samsung.com>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> Cc: Jerome Brunet <jbrunet@baylibre.com>
> Cc: Joel Stanley <joel@jms.id.au>
> Cc: John Stultz <john.stultz@linaro.org>
> Cc: Jonas Karlman <jonas@kwiboo.se>
> Cc: Jonathan Hunter <jonathanh@nvidia.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> Cc: Jyri Sarha <jyri.sarha@iki.fi>
> Cc: Kevin Hilman <khilman@baylibre.com>
> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> Cc: Kyungmin Park <kyungmin.park@samsung.com>
> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Liviu Dudau <liviu.dudau@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> Cc: Matthias Brugger <matthias.bgg@gmail.com>
> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Cc: Neil Armstrong <narmstrong@baylibre.com>
> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> Cc: "Noralf Trønnes" <noralf@tronnes.org>
> Cc: NXP Linux Team <linux-imx@nxp.com>
> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> Cc: Paul Cercueil <paul@crapouillou.net>
> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> Cc: Philipp Zabel <p.zabel@pengutronix.de>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Robert Foss <robert.foss@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Roland Scheidegger <sroland@vmware.com>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Sam Ravnborg <sam@ravnborg.org>
> Cc: Sandy Huang <hjc@rock-chips.com>
> Cc: Sascha Hauer <s.hauer@pengutronix.de>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> Cc: Shawn Guo <shawnguo@kernel.org>
> Cc: Stefan Agner <stefan@agner.ch>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: Thierry Reding <thierry.reding@gmail.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Tomi Valkeinen <tomba@kernel.org>
> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> Cc: Xinliang Liu <xinliang.liu@linaro.org>
> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> Cc: Zack Rusin <zackr@vmware.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> 
> ---
> 
> Changes from v2:
>   - Take into account the feedback from Laurent and Lidiu to no longer
>     force generic properties, but prefix vendor-specific properties with
>     the vendor name
> 
> Changes from v1:
>   - Typos and wording reported by Daniel and Alex
> ---
>  Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index 87e5023e3f55..bbe254dca635 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -463,6 +463,33 @@ KMS Properties
>  This section of the documentation is primarily aimed at user-space developers.
>  For the driver APIs, see the other sections.
>  
> +Requirements
> +------------
> +
> +KMS drivers might need to add extra properties to support new features.
> +Each new property introduced in a driver need to meet a few
> +requirements, in addition to the one mentioned above.:
> +
> +- Before the introduction of any vendor-specific properties, they must
> +  be first checked against the generic ones to avoid any conflict or
> +  redundancy.
> +
> +- Vendor-specific properties must be prefixed by the vendor's name,
> +  following the syntax "$vendor:$property".
> +
> +- Generic properties must be standardized, with some documentation to
> +  describe how the property can be used.
> +
> +- Generic properties must provide a generic helper in the core code to
> +  register that property on the object it attaches to.
> +
> +- Generic properties content must be decoded by the core and provided in
> +  the object's associated state structure. That includes anything
> +  drivers might want to precompute, like :c:type:`struct drm_clip_rect
> +  <drm_clip_rect>` for planes.
> +
> +- An IGT test should be submitted.
> +
>  Property Types and Blob Property Support
>  ----------------------------------------

Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

>  
> -- 
> 2.31.1
> 

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Rodrigo Vivi]

On Thu, Jun 10, 2021 at 07:47:31PM +0200, Maxime Ripard wrote:
> New KMS properties come with a bunch of requirements to avoid each
> driver from running their own, inconsistent, set of properties,
> eventually leading to issues like property conflicts, inconsistencies
> between drivers and semantics, etc.
> 
> Let's document what we expect.
> 
> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Alison Wang <alison.wang@nxp.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Andrew Jeffery <andrew@aj.id.au>
> Cc: Andrzej Hajda <a.hajda@samsung.com>
> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> Cc: Ben Skeggs <bskeggs@redhat.com>
> Cc: Boris Brezillon <bbrezillon@kernel.org>
> Cc: Brian Starkey <brian.starkey@arm.com>
> Cc: Chen Feng <puck.chen@hisilicon.com>
> Cc: Chen-Yu Tsai <wens@csie.org>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> Cc: Edmund Dea <edmund.j.dea@intel.com>
> Cc: Eric Anholt <eric@anholt.net>
> Cc: Fabio Estevam <festevam@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: "Heiko Stübner" <heiko@sntech.de>
> Cc: Huang Rui <ray.huang@amd.com>
> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> Cc: Inki Dae <inki.dae@samsung.com>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> Cc: Jerome Brunet <jbrunet@baylibre.com>
> Cc: Joel Stanley <joel@jms.id.au>
> Cc: John Stultz <john.stultz@linaro.org>
> Cc: Jonas Karlman <jonas@kwiboo.se>
> Cc: Jonathan Hunter <jonathanh@nvidia.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> Cc: Jyri Sarha <jyri.sarha@iki.fi>
> Cc: Kevin Hilman <khilman@baylibre.com>
> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> Cc: Kyungmin Park <kyungmin.park@samsung.com>
> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Liviu Dudau <liviu.dudau@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> Cc: Matthias Brugger <matthias.bgg@gmail.com>
> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Cc: Neil Armstrong <narmstrong@baylibre.com>
> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> Cc: "Noralf Trønnes" <noralf@tronnes.org>
> Cc: NXP Linux Team <linux-imx@nxp.com>
> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> Cc: Paul Cercueil <paul@crapouillou.net>
> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> Cc: Philipp Zabel <p.zabel@pengutronix.de>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Robert Foss <robert.foss@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Roland Scheidegger <sroland@vmware.com>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Sam Ravnborg <sam@ravnborg.org>
> Cc: Sandy Huang <hjc@rock-chips.com>
> Cc: Sascha Hauer <s.hauer@pengutronix.de>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> Cc: Shawn Guo <shawnguo@kernel.org>
> Cc: Stefan Agner <stefan@agner.ch>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: Thierry Reding <thierry.reding@gmail.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Tomi Valkeinen <tomba@kernel.org>
> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> Cc: Xinliang Liu <xinliang.liu@linaro.org>
> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> Cc: Zack Rusin <zackr@vmware.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> 
> ---
> 
> Changes from v2:
>   - Take into account the feedback from Laurent and Lidiu to no longer
>     force generic properties, but prefix vendor-specific properties with
>     the vendor name
> 
> Changes from v1:
>   - Typos and wording reported by Daniel and Alex
> ---
>  Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index 87e5023e3f55..bbe254dca635 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -463,6 +463,33 @@ KMS Properties
>  This section of the documentation is primarily aimed at user-space developers.
>  For the driver APIs, see the other sections.
>  
> +Requirements
> +------------
> +
> +KMS drivers might need to add extra properties to support new features.
> +Each new property introduced in a driver need to meet a few
> +requirements, in addition to the one mentioned above.:
> +
> +- Before the introduction of any vendor-specific properties, they must
> +  be first checked against the generic ones to avoid any conflict or
> +  redundancy.
> +
> +- Vendor-specific properties must be prefixed by the vendor's name,
> +  following the syntax "$vendor:$property".
> +
> +- Generic properties must be standardized, with some documentation to
> +  describe how the property can be used.
> +
> +- Generic properties must provide a generic helper in the core code to
> +  register that property on the object it attaches to.
> +
> +- Generic properties content must be decoded by the core and provided in
> +  the object's associated state structure. That includes anything
> +  drivers might want to precompute, like :c:type:`struct drm_clip_rect
> +  <drm_clip_rect>` for planes.
> +
> +- An IGT test should be submitted.
> +
>  Property Types and Blob Property Support
>  ----------------------------------------

Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

>  
> -- 
> 2.31.1
> 

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Daniel Vetter]

On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
>
> New KMS properties come with a bunch of requirements to avoid each
> driver from running their own, inconsistent, set of properties,
> eventually leading to issues like property conflicts, inconsistencies
> between drivers and semantics, etc.
>
> Let's document what we expect.
>
> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Alison Wang <alison.wang@nxp.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Andrew Jeffery <andrew@aj.id.au>
> Cc: Andrzej Hajda <a.hajda@samsung.com>
> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> Cc: Ben Skeggs <bskeggs@redhat.com>
> Cc: Boris Brezillon <bbrezillon@kernel.org>
> Cc: Brian Starkey <brian.starkey@arm.com>
> Cc: Chen Feng <puck.chen@hisilicon.com>
> Cc: Chen-Yu Tsai <wens@csie.org>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> Cc: Edmund Dea <edmund.j.dea@intel.com>
> Cc: Eric Anholt <eric@anholt.net>
> Cc: Fabio Estevam <festevam@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: "Heiko Stübner" <heiko@sntech.de>
> Cc: Huang Rui <ray.huang@amd.com>
> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> Cc: Inki Dae <inki.dae@samsung.com>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> Cc: Jerome Brunet <jbrunet@baylibre.com>
> Cc: Joel Stanley <joel@jms.id.au>
> Cc: John Stultz <john.stultz@linaro.org>
> Cc: Jonas Karlman <jonas@kwiboo.se>
> Cc: Jonathan Hunter <jonathanh@nvidia.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> Cc: Jyri Sarha <jyri.sarha@iki.fi>
> Cc: Kevin Hilman <khilman@baylibre.com>
> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> Cc: Kyungmin Park <kyungmin.park@samsung.com>
> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Liviu Dudau <liviu.dudau@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> Cc: Matthias Brugger <matthias.bgg@gmail.com>
> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Cc: Neil Armstrong <narmstrong@baylibre.com>
> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> Cc: "Noralf Trønnes" <noralf@tronnes.org>
> Cc: NXP Linux Team <linux-imx@nxp.com>
> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> Cc: Paul Cercueil <paul@crapouillou.net>
> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> Cc: Philipp Zabel <p.zabel@pengutronix.de>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Robert Foss <robert.foss@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Roland Scheidegger <sroland@vmware.com>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Sam Ravnborg <sam@ravnborg.org>
> Cc: Sandy Huang <hjc@rock-chips.com>
> Cc: Sascha Hauer <s.hauer@pengutronix.de>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> Cc: Shawn Guo <shawnguo@kernel.org>
> Cc: Stefan Agner <stefan@agner.ch>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: Thierry Reding <thierry.reding@gmail.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Tomi Valkeinen <tomba@kernel.org>
> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> Cc: Xinliang Liu <xinliang.liu@linaro.org>
> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> Cc: Zack Rusin <zackr@vmware.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
>
> ---
>
> Changes from v2:
>   - Take into account the feedback from Laurent and Lidiu to no longer
>     force generic properties, but prefix vendor-specific properties with
>     the vendor name

I'm pretty sure my r-b was without this ... Why exactly do we need
this? KMS is meant to be fairly generic (bugs throw a wrench around
here sometimes, and semantics can be tricky). If we open up the door
to yolo vendor properties in upstream, then that goal is pretty much
written off. And we've been there with vendor properties, it's a
giantic mess.

Minimally drop my r-b, I'm definitely not in support of this idea.

If there's a strong consensus that we really need this then I'm not
going to nack this, but this really needs a pile of acks from
compositor folks that they're willing to live with the resulting
fallout this will likely bring. Your cc list seems to have an absence
of compositor folks, but instead every driver maintainer. That's
backwards. We make uapi for userspace, not for kernel driver
maintainers!

ltdr; I'd go back to v2. And then cc compositor folks on this to get their ack.
-Daniel

> Changes from v1:
>   - Typos and wording reported by Daniel and Alex
> ---
>  Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index 87e5023e3f55..bbe254dca635 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -463,6 +463,33 @@ KMS Properties
>  This section of the documentation is primarily aimed at user-space developers.
>  For the driver APIs, see the other sections.
>
> +Requirements
> +------------
> +
> +KMS drivers might need to add extra properties to support new features.
> +Each new property introduced in a driver need to meet a few
> +requirements, in addition to the one mentioned above.:
> +
> +- Before the introduction of any vendor-specific properties, they must
> +  be first checked against the generic ones to avoid any conflict or
> +  redundancy.
> +
> +- Vendor-specific properties must be prefixed by the vendor's name,
> +  following the syntax "$vendor:$property".
> +
> +- Generic properties must be standardized, with some documentation to
> +  describe how the property can be used.
> +
> +- Generic properties must provide a generic helper in the core code to
> +  register that property on the object it attaches to.
> +
> +- Generic properties content must be decoded by the core and provided in
> +  the object's associated state structure. That includes anything
> +  drivers might want to precompute, like :c:type:`struct drm_clip_rect
> +  <drm_clip_rect>` for planes.
> +
> +- An IGT test should be submitted.
> +
>  Property Types and Blob Property Support
>  ----------------------------------------
>
> --
> 2.31.1
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Daniel Vetter]

On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
>
> New KMS properties come with a bunch of requirements to avoid each
> driver from running their own, inconsistent, set of properties,
> eventually leading to issues like property conflicts, inconsistencies
> between drivers and semantics, etc.
>
> Let's document what we expect.
>
> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Alison Wang <alison.wang@nxp.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Andrew Jeffery <andrew@aj.id.au>
> Cc: Andrzej Hajda <a.hajda@samsung.com>
> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> Cc: Ben Skeggs <bskeggs@redhat.com>
> Cc: Boris Brezillon <bbrezillon@kernel.org>
> Cc: Brian Starkey <brian.starkey@arm.com>
> Cc: Chen Feng <puck.chen@hisilicon.com>
> Cc: Chen-Yu Tsai <wens@csie.org>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> Cc: Edmund Dea <edmund.j.dea@intel.com>
> Cc: Eric Anholt <eric@anholt.net>
> Cc: Fabio Estevam <festevam@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: "Heiko Stübner" <heiko@sntech.de>
> Cc: Huang Rui <ray.huang@amd.com>
> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> Cc: Inki Dae <inki.dae@samsung.com>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> Cc: Jerome Brunet <jbrunet@baylibre.com>
> Cc: Joel Stanley <joel@jms.id.au>
> Cc: John Stultz <john.stultz@linaro.org>
> Cc: Jonas Karlman <jonas@kwiboo.se>
> Cc: Jonathan Hunter <jonathanh@nvidia.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> Cc: Jyri Sarha <jyri.sarha@iki.fi>
> Cc: Kevin Hilman <khilman@baylibre.com>
> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> Cc: Kyungmin Park <kyungmin.park@samsung.com>
> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Liviu Dudau <liviu.dudau@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> Cc: Matthias Brugger <matthias.bgg@gmail.com>
> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Cc: Neil Armstrong <narmstrong@baylibre.com>
> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> Cc: "Noralf Trønnes" <noralf@tronnes.org>
> Cc: NXP Linux Team <linux-imx@nxp.com>
> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> Cc: Paul Cercueil <paul@crapouillou.net>
> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> Cc: Philipp Zabel <p.zabel@pengutronix.de>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Robert Foss <robert.foss@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Roland Scheidegger <sroland@vmware.com>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Sam Ravnborg <sam@ravnborg.org>
> Cc: Sandy Huang <hjc@rock-chips.com>
> Cc: Sascha Hauer <s.hauer@pengutronix.de>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> Cc: Shawn Guo <shawnguo@kernel.org>
> Cc: Stefan Agner <stefan@agner.ch>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: Thierry Reding <thierry.reding@gmail.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Tomi Valkeinen <tomba@kernel.org>
> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> Cc: Xinliang Liu <xinliang.liu@linaro.org>
> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> Cc: Zack Rusin <zackr@vmware.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
>
> ---
>
> Changes from v2:
>   - Take into account the feedback from Laurent and Lidiu to no longer
>     force generic properties, but prefix vendor-specific properties with
>     the vendor name

I'm pretty sure my r-b was without this ... Why exactly do we need
this? KMS is meant to be fairly generic (bugs throw a wrench around
here sometimes, and semantics can be tricky). If we open up the door
to yolo vendor properties in upstream, then that goal is pretty much
written off. And we've been there with vendor properties, it's a
giantic mess.

Minimally drop my r-b, I'm definitely not in support of this idea.

If there's a strong consensus that we really need this then I'm not
going to nack this, but this really needs a pile of acks from
compositor folks that they're willing to live with the resulting
fallout this will likely bring. Your cc list seems to have an absence
of compositor folks, but instead every driver maintainer. That's
backwards. We make uapi for userspace, not for kernel driver
maintainers!

ltdr; I'd go back to v2. And then cc compositor folks on this to get their ack.
-Daniel

> Changes from v1:
>   - Typos and wording reported by Daniel and Alex
> ---
>  Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index 87e5023e3f55..bbe254dca635 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -463,6 +463,33 @@ KMS Properties
>  This section of the documentation is primarily aimed at user-space developers.
>  For the driver APIs, see the other sections.
>
> +Requirements
> +------------
> +
> +KMS drivers might need to add extra properties to support new features.
> +Each new property introduced in a driver need to meet a few
> +requirements, in addition to the one mentioned above.:
> +
> +- Before the introduction of any vendor-specific properties, they must
> +  be first checked against the generic ones to avoid any conflict or
> +  redundancy.
> +
> +- Vendor-specific properties must be prefixed by the vendor's name,
> +  following the syntax "$vendor:$property".
> +
> +- Generic properties must be standardized, with some documentation to
> +  describe how the property can be used.
> +
> +- Generic properties must provide a generic helper in the core code to
> +  register that property on the object it attaches to.
> +
> +- Generic properties content must be decoded by the core and provided in
> +  the object's associated state structure. That includes anything
> +  drivers might want to precompute, like :c:type:`struct drm_clip_rect
> +  <drm_clip_rect>` for planes.
> +
> +- An IGT test should be submitted.
> +
>  Property Types and Blob Property Support
>  ----------------------------------------
>
> --
> 2.31.1
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 6686 bytes --]

Hi,

On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
> On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
> >
> > New KMS properties come with a bunch of requirements to avoid each
> > driver from running their own, inconsistent, set of properties,
> > eventually leading to issues like property conflicts, inconsistencies
> > between drivers and semantics, etc.
> >
> > Let's document what we expect.
> >
> > Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
> > Cc: Alison Wang <alison.wang@nxp.com>
> > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > Cc: Andrew Jeffery <andrew@aj.id.au>
> > Cc: Andrzej Hajda <a.hajda@samsung.com>
> > Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> > Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> > Cc: Ben Skeggs <bskeggs@redhat.com>
> > Cc: Boris Brezillon <bbrezillon@kernel.org>
> > Cc: Brian Starkey <brian.starkey@arm.com>
> > Cc: Chen Feng <puck.chen@hisilicon.com>
> > Cc: Chen-Yu Tsai <wens@csie.org>
> > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> > Cc: Edmund Dea <edmund.j.dea@intel.com>
> > Cc: Eric Anholt <eric@anholt.net>
> > Cc: Fabio Estevam <festevam@gmail.com>
> > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> > Cc: Hans de Goede <hdegoede@redhat.com>
> > Cc: "Heiko Stübner" <heiko@sntech.de>
> > Cc: Huang Rui <ray.huang@amd.com>
> > Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> > Cc: Inki Dae <inki.dae@samsung.com>
> > Cc: Jani Nikula <jani.nikula@linux.intel.com>
> > Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> > Cc: Jerome Brunet <jbrunet@baylibre.com>
> > Cc: Joel Stanley <joel@jms.id.au>
> > Cc: John Stultz <john.stultz@linaro.org>
> > Cc: Jonas Karlman <jonas@kwiboo.se>
> > Cc: Jonathan Hunter <jonathanh@nvidia.com>
> > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> > Cc: Jyri Sarha <jyri.sarha@iki.fi>
> > Cc: Kevin Hilman <khilman@baylibre.com>
> > Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> > Cc: Kyungmin Park <kyungmin.park@samsung.com>
> > Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> > Cc: Linus Walleij <linus.walleij@linaro.org>
> > Cc: Liviu Dudau <liviu.dudau@arm.com>
> > Cc: Lucas Stach <l.stach@pengutronix.de>
> > Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> > Cc: Marek Vasut <marex@denx.de>
> > Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> > Cc: Matthias Brugger <matthias.bgg@gmail.com>
> > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > Cc: Maxime Ripard <mripard@kernel.org>
> > Cc: Melissa Wen <melissa.srw@gmail.com>
> > Cc: Neil Armstrong <narmstrong@baylibre.com>
> > Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> > Cc: "Noralf Trønnes" <noralf@tronnes.org>
> > Cc: NXP Linux Team <linux-imx@nxp.com>
> > Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> > Cc: Paul Cercueil <paul@crapouillou.net>
> > Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> > Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> > Cc: Philipp Zabel <p.zabel@pengutronix.de>
> > Cc: Qiang Yu <yuq825@gmail.com>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Robert Foss <robert.foss@linaro.org>
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Cc: Roland Scheidegger <sroland@vmware.com>
> > Cc: Russell King <linux@armlinux.org.uk>
> > Cc: Sam Ravnborg <sam@ravnborg.org>
> > Cc: Sandy Huang <hjc@rock-chips.com>
> > Cc: Sascha Hauer <s.hauer@pengutronix.de>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> > Cc: Shawn Guo <shawnguo@kernel.org>
> > Cc: Stefan Agner <stefan@agner.ch>
> > Cc: Steven Price <steven.price@arm.com>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: Thierry Reding <thierry.reding@gmail.com>
> > Cc: Tian Tao <tiantao6@hisilicon.com>
> > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > Cc: Tomi Valkeinen <tomba@kernel.org>
> > Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> > Cc: Xinliang Liu <xinliang.liu@linaro.org>
> > Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> > Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> > Cc: Zack Rusin <zackr@vmware.com>
> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> >
> > ---
> >
> > Changes from v2:
> >   - Take into account the feedback from Laurent and Lidiu to no longer
> >     force generic properties, but prefix vendor-specific properties with
> >     the vendor name
> 
> I'm pretty sure my r-b was without this ...

Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
second look, but I shouldn't have kept it and caught you by surprise
indeed.

> Why exactly do we need this? KMS is meant to be fairly generic (bugs
> throw a wrench around here sometimes, and semantics can be tricky). If
> we open up the door to yolo vendor properties in upstream, then that
> goal is pretty much written off. And we've been there with vendor
> properties, it's a giantic mess.
> 
> Minimally drop my r-b, I'm definitely not in support of this idea.

So the argument Lidiu and Laurent made was that in some cases, getting a
generic property right with only a couple of users is hard. So they
advocated for the right to keep non-generic properties. I can get the
argument, and no-one else said that was wrong, so it felt like the
consensus was there.

> If there's a strong consensus that we really need this then I'm not
> going to nack this, but this really needs a pile of acks from
> compositor folks that they're willing to live with the resulting
> fallout this will likely bring. Your cc list seems to have an absence
> of compositor folks, but instead every driver maintainer. That's
> backwards. We make uapi for userspace, not for kernel driver
> maintainers!

Right, but it's mostly about in-kernel rules though? And you're the one
who mentionned CC'ing the driver maintainers in the first iteration?

> ltdr; I'd go back to v2. And then cc compositor folks on this to get
> their ack.

So, Pekka, Simon, is there anyone else I should Cc?

Thanks!
Maxime

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 6686 bytes --]

Hi,

On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
> On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
> >
> > New KMS properties come with a bunch of requirements to avoid each
> > driver from running their own, inconsistent, set of properties,
> > eventually leading to issues like property conflicts, inconsistencies
> > between drivers and semantics, etc.
> >
> > Let's document what we expect.
> >
> > Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
> > Cc: Alison Wang <alison.wang@nxp.com>
> > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > Cc: Andrew Jeffery <andrew@aj.id.au>
> > Cc: Andrzej Hajda <a.hajda@samsung.com>
> > Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> > Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> > Cc: Ben Skeggs <bskeggs@redhat.com>
> > Cc: Boris Brezillon <bbrezillon@kernel.org>
> > Cc: Brian Starkey <brian.starkey@arm.com>
> > Cc: Chen Feng <puck.chen@hisilicon.com>
> > Cc: Chen-Yu Tsai <wens@csie.org>
> > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> > Cc: Edmund Dea <edmund.j.dea@intel.com>
> > Cc: Eric Anholt <eric@anholt.net>
> > Cc: Fabio Estevam <festevam@gmail.com>
> > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> > Cc: Hans de Goede <hdegoede@redhat.com>
> > Cc: "Heiko Stübner" <heiko@sntech.de>
> > Cc: Huang Rui <ray.huang@amd.com>
> > Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> > Cc: Inki Dae <inki.dae@samsung.com>
> > Cc: Jani Nikula <jani.nikula@linux.intel.com>
> > Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> > Cc: Jerome Brunet <jbrunet@baylibre.com>
> > Cc: Joel Stanley <joel@jms.id.au>
> > Cc: John Stultz <john.stultz@linaro.org>
> > Cc: Jonas Karlman <jonas@kwiboo.se>
> > Cc: Jonathan Hunter <jonathanh@nvidia.com>
> > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> > Cc: Jyri Sarha <jyri.sarha@iki.fi>
> > Cc: Kevin Hilman <khilman@baylibre.com>
> > Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> > Cc: Kyungmin Park <kyungmin.park@samsung.com>
> > Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> > Cc: Linus Walleij <linus.walleij@linaro.org>
> > Cc: Liviu Dudau <liviu.dudau@arm.com>
> > Cc: Lucas Stach <l.stach@pengutronix.de>
> > Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> > Cc: Marek Vasut <marex@denx.de>
> > Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> > Cc: Matthias Brugger <matthias.bgg@gmail.com>
> > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > Cc: Maxime Ripard <mripard@kernel.org>
> > Cc: Melissa Wen <melissa.srw@gmail.com>
> > Cc: Neil Armstrong <narmstrong@baylibre.com>
> > Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> > Cc: "Noralf Trønnes" <noralf@tronnes.org>
> > Cc: NXP Linux Team <linux-imx@nxp.com>
> > Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> > Cc: Paul Cercueil <paul@crapouillou.net>
> > Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> > Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> > Cc: Philipp Zabel <p.zabel@pengutronix.de>
> > Cc: Qiang Yu <yuq825@gmail.com>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Robert Foss <robert.foss@linaro.org>
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Cc: Roland Scheidegger <sroland@vmware.com>
> > Cc: Russell King <linux@armlinux.org.uk>
> > Cc: Sam Ravnborg <sam@ravnborg.org>
> > Cc: Sandy Huang <hjc@rock-chips.com>
> > Cc: Sascha Hauer <s.hauer@pengutronix.de>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> > Cc: Shawn Guo <shawnguo@kernel.org>
> > Cc: Stefan Agner <stefan@agner.ch>
> > Cc: Steven Price <steven.price@arm.com>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: Thierry Reding <thierry.reding@gmail.com>
> > Cc: Tian Tao <tiantao6@hisilicon.com>
> > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > Cc: Tomi Valkeinen <tomba@kernel.org>
> > Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> > Cc: Xinliang Liu <xinliang.liu@linaro.org>
> > Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> > Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> > Cc: Zack Rusin <zackr@vmware.com>
> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> >
> > ---
> >
> > Changes from v2:
> >   - Take into account the feedback from Laurent and Lidiu to no longer
> >     force generic properties, but prefix vendor-specific properties with
> >     the vendor name
> 
> I'm pretty sure my r-b was without this ...

Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
second look, but I shouldn't have kept it and caught you by surprise
indeed.

> Why exactly do we need this? KMS is meant to be fairly generic (bugs
> throw a wrench around here sometimes, and semantics can be tricky). If
> we open up the door to yolo vendor properties in upstream, then that
> goal is pretty much written off. And we've been there with vendor
> properties, it's a giantic mess.
> 
> Minimally drop my r-b, I'm definitely not in support of this idea.

So the argument Lidiu and Laurent made was that in some cases, getting a
generic property right with only a couple of users is hard. So they
advocated for the right to keep non-generic properties. I can get the
argument, and no-one else said that was wrong, so it felt like the
consensus was there.

> If there's a strong consensus that we really need this then I'm not
> going to nack this, but this really needs a pile of acks from
> compositor folks that they're willing to live with the resulting
> fallout this will likely bring. Your cc list seems to have an absence
> of compositor folks, but instead every driver maintainer. That's
> backwards. We make uapi for userspace, not for kernel driver
> maintainers!

Right, but it's mostly about in-kernel rules though? And you're the one
who mentionned CC'ing the driver maintainers in the first iteration?

> ltdr; I'd go back to v2. And then cc compositor folks on this to get
> their ack.

So, Pekka, Simon, is there anyone else I should Cc?

Thanks!
Maxime

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Tomi Valkeinen]

On 11/06/2021 08:54, Maxime Ripard wrote:
> Hi,
> 
> On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
>> On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
>>>
>>> New KMS properties come with a bunch of requirements to avoid each
>>> driver from running their own, inconsistent, set of properties,
>>> eventually leading to issues like property conflicts, inconsistencies
>>> between drivers and semantics, etc.
>>>
>>> Let's document what we expect.
>>>
>>> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
>>> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
>>> Cc: Alex Deucher <alexander.deucher@amd.com>
>>> Cc: Alison Wang <alison.wang@nxp.com>
>>> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
>>> Cc: Andrew Jeffery <andrew@aj.id.au>
>>> Cc: Andrzej Hajda <a.hajda@samsung.com>
>>> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
>>> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
>>> Cc: Ben Skeggs <bskeggs@redhat.com>
>>> Cc: Boris Brezillon <bbrezillon@kernel.org>
>>> Cc: Brian Starkey <brian.starkey@arm.com>
>>> Cc: Chen Feng <puck.chen@hisilicon.com>
>>> Cc: Chen-Yu Tsai <wens@csie.org>
>>> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
>>> Cc: "Christian König" <christian.koenig@amd.com>
>>> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
>>> Cc: Edmund Dea <edmund.j.dea@intel.com>
>>> Cc: Eric Anholt <eric@anholt.net>
>>> Cc: Fabio Estevam <festevam@gmail.com>
>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
>>> Cc: Hans de Goede <hdegoede@redhat.com>
>>> Cc: "Heiko Stübner" <heiko@sntech.de>
>>> Cc: Huang Rui <ray.huang@amd.com>
>>> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
>>> Cc: Inki Dae <inki.dae@samsung.com>
>>> Cc: Jani Nikula <jani.nikula@linux.intel.com>
>>> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
>>> Cc: Jerome Brunet <jbrunet@baylibre.com>
>>> Cc: Joel Stanley <joel@jms.id.au>
>>> Cc: John Stultz <john.stultz@linaro.org>
>>> Cc: Jonas Karlman <jonas@kwiboo.se>
>>> Cc: Jonathan Hunter <jonathanh@nvidia.com>
>>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>>> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
>>> Cc: Jyri Sarha <jyri.sarha@iki.fi>
>>> Cc: Kevin Hilman <khilman@baylibre.com>
>>> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
>>> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
>>> Cc: Kyungmin Park <kyungmin.park@samsung.com>
>>> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
>>> Cc: Linus Walleij <linus.walleij@linaro.org>
>>> Cc: Liviu Dudau <liviu.dudau@arm.com>
>>> Cc: Lucas Stach <l.stach@pengutronix.de>
>>> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
>>> Cc: Marek Vasut <marex@denx.de>
>>> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
>>> Cc: Matthias Brugger <matthias.bgg@gmail.com>
>>> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
>>> Cc: Maxime Ripard <mripard@kernel.org>
>>> Cc: Melissa Wen <melissa.srw@gmail.com>
>>> Cc: Neil Armstrong <narmstrong@baylibre.com>
>>> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
>>> Cc: "Noralf Trønnes" <noralf@tronnes.org>
>>> Cc: NXP Linux Team <linux-imx@nxp.com>
>>> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
>>> Cc: Paul Cercueil <paul@crapouillou.net>
>>> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
>>> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
>>> Cc: Philipp Zabel <p.zabel@pengutronix.de>
>>> Cc: Qiang Yu <yuq825@gmail.com>
>>> Cc: Rob Clark <robdclark@gmail.com>
>>> Cc: Robert Foss <robert.foss@linaro.org>
>>> Cc: Rob Herring <robh@kernel.org>
>>> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
>>> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
>>> Cc: Roland Scheidegger <sroland@vmware.com>
>>> Cc: Russell King <linux@armlinux.org.uk>
>>> Cc: Sam Ravnborg <sam@ravnborg.org>
>>> Cc: Sandy Huang <hjc@rock-chips.com>
>>> Cc: Sascha Hauer <s.hauer@pengutronix.de>
>>> Cc: Sean Paul <sean@poorly.run>
>>> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
>>> Cc: Shawn Guo <shawnguo@kernel.org>
>>> Cc: Stefan Agner <stefan@agner.ch>
>>> Cc: Steven Price <steven.price@arm.com>
>>> Cc: Sumit Semwal <sumit.semwal@linaro.org>
>>> Cc: Thierry Reding <thierry.reding@gmail.com>
>>> Cc: Tian Tao <tiantao6@hisilicon.com>
>>> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>>> Cc: Tomi Valkeinen <tomba@kernel.org>
>>> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
>>> Cc: Xinliang Liu <xinliang.liu@linaro.org>
>>> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
>>> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
>>> Cc: Zack Rusin <zackr@vmware.com>
>>> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>>> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
>>>
>>> ---
>>>
>>> Changes from v2:
>>>    - Take into account the feedback from Laurent and Lidiu to no longer
>>>      force generic properties, but prefix vendor-specific properties with
>>>      the vendor name
>>
>> I'm pretty sure my r-b was without this ...
> 
> Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
> second look, but I shouldn't have kept it and caught you by surprise
> indeed.
> 
>> Why exactly do we need this? KMS is meant to be fairly generic (bugs
>> throw a wrench around here sometimes, and semantics can be tricky). If
>> we open up the door to yolo vendor properties in upstream, then that
>> goal is pretty much written off. And we've been there with vendor
>> properties, it's a giantic mess.
>>
>> Minimally drop my r-b, I'm definitely not in support of this idea.
> 
> So the argument Lidiu and Laurent made was that in some cases, getting a
> generic property right with only a couple of users is hard. So they
> advocated for the right to keep non-generic properties. I can get the
> argument, and no-one else said that was wrong, so it felt like the
> consensus was there.

I also think that (maybe mainly on embedded side) we may have 1) 
esoteric HW features which perhaps can't even be made generic, and 2) 
features which may or may not be generic, but for which support cannot 
be added to any common opensource userspace projects like X or Weston, 
as the only use cases for the features are specialized low level apps 
(often customer's closed-source apps).

While I agree with Daniel's "gigantic mess" problem, it would also be 
quite nice to have a way to support all the HW features upstream instead 
of carrying them in vendor trees.

  Tomi

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Tomi Valkeinen]

On 11/06/2021 08:54, Maxime Ripard wrote:
> Hi,
> 
> On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
>> On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
>>>
>>> New KMS properties come with a bunch of requirements to avoid each
>>> driver from running their own, inconsistent, set of properties,
>>> eventually leading to issues like property conflicts, inconsistencies
>>> between drivers and semantics, etc.
>>>
>>> Let's document what we expect.
>>>
>>> Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
>>> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
>>> Cc: Alex Deucher <alexander.deucher@amd.com>
>>> Cc: Alison Wang <alison.wang@nxp.com>
>>> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
>>> Cc: Andrew Jeffery <andrew@aj.id.au>
>>> Cc: Andrzej Hajda <a.hajda@samsung.com>
>>> Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
>>> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
>>> Cc: Ben Skeggs <bskeggs@redhat.com>
>>> Cc: Boris Brezillon <bbrezillon@kernel.org>
>>> Cc: Brian Starkey <brian.starkey@arm.com>
>>> Cc: Chen Feng <puck.chen@hisilicon.com>
>>> Cc: Chen-Yu Tsai <wens@csie.org>
>>> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
>>> Cc: "Christian König" <christian.koenig@amd.com>
>>> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
>>> Cc: Edmund Dea <edmund.j.dea@intel.com>
>>> Cc: Eric Anholt <eric@anholt.net>
>>> Cc: Fabio Estevam <festevam@gmail.com>
>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>> Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
>>> Cc: Hans de Goede <hdegoede@redhat.com>
>>> Cc: "Heiko Stübner" <heiko@sntech.de>
>>> Cc: Huang Rui <ray.huang@amd.com>
>>> Cc: Hyun Kwon <hyun.kwon@xilinx.com>
>>> Cc: Inki Dae <inki.dae@samsung.com>
>>> Cc: Jani Nikula <jani.nikula@linux.intel.com>
>>> Cc: Jernej Skrabec <jernej.skrabec@siol.net>
>>> Cc: Jerome Brunet <jbrunet@baylibre.com>
>>> Cc: Joel Stanley <joel@jms.id.au>
>>> Cc: John Stultz <john.stultz@linaro.org>
>>> Cc: Jonas Karlman <jonas@kwiboo.se>
>>> Cc: Jonathan Hunter <jonathanh@nvidia.com>
>>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>>> Cc: Joonyoung Shim <jy0922.shim@samsung.com>
>>> Cc: Jyri Sarha <jyri.sarha@iki.fi>
>>> Cc: Kevin Hilman <khilman@baylibre.com>
>>> Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
>>> Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
>>> Cc: Kyungmin Park <kyungmin.park@samsung.com>
>>> Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
>>> Cc: Linus Walleij <linus.walleij@linaro.org>
>>> Cc: Liviu Dudau <liviu.dudau@arm.com>
>>> Cc: Lucas Stach <l.stach@pengutronix.de>
>>> Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
>>> Cc: Marek Vasut <marex@denx.de>
>>> Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
>>> Cc: Matthias Brugger <matthias.bgg@gmail.com>
>>> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
>>> Cc: Maxime Ripard <mripard@kernel.org>
>>> Cc: Melissa Wen <melissa.srw@gmail.com>
>>> Cc: Neil Armstrong <narmstrong@baylibre.com>
>>> Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
>>> Cc: "Noralf Trønnes" <noralf@tronnes.org>
>>> Cc: NXP Linux Team <linux-imx@nxp.com>
>>> Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>> Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
>>> Cc: Paul Cercueil <paul@crapouillou.net>
>>> Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
>>> Cc: Philippe Cornu <philippe.cornu@foss.st.com>
>>> Cc: Philipp Zabel <p.zabel@pengutronix.de>
>>> Cc: Qiang Yu <yuq825@gmail.com>
>>> Cc: Rob Clark <robdclark@gmail.com>
>>> Cc: Robert Foss <robert.foss@linaro.org>
>>> Cc: Rob Herring <robh@kernel.org>
>>> Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
>>> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
>>> Cc: Roland Scheidegger <sroland@vmware.com>
>>> Cc: Russell King <linux@armlinux.org.uk>
>>> Cc: Sam Ravnborg <sam@ravnborg.org>
>>> Cc: Sandy Huang <hjc@rock-chips.com>
>>> Cc: Sascha Hauer <s.hauer@pengutronix.de>
>>> Cc: Sean Paul <sean@poorly.run>
>>> Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
>>> Cc: Shawn Guo <shawnguo@kernel.org>
>>> Cc: Stefan Agner <stefan@agner.ch>
>>> Cc: Steven Price <steven.price@arm.com>
>>> Cc: Sumit Semwal <sumit.semwal@linaro.org>
>>> Cc: Thierry Reding <thierry.reding@gmail.com>
>>> Cc: Tian Tao <tiantao6@hisilicon.com>
>>> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>>> Cc: Tomi Valkeinen <tomba@kernel.org>
>>> Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
>>> Cc: Xinliang Liu <xinliang.liu@linaro.org>
>>> Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
>>> Cc: Yannick Fertre <yannick.fertre@foss.st.com>
>>> Cc: Zack Rusin <zackr@vmware.com>
>>> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>>> Signed-off-by: Maxime Ripard <maxime@cerno.tech>
>>>
>>> ---
>>>
>>> Changes from v2:
>>>    - Take into account the feedback from Laurent and Lidiu to no longer
>>>      force generic properties, but prefix vendor-specific properties with
>>>      the vendor name
>>
>> I'm pretty sure my r-b was without this ...
> 
> Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
> second look, but I shouldn't have kept it and caught you by surprise
> indeed.
> 
>> Why exactly do we need this? KMS is meant to be fairly generic (bugs
>> throw a wrench around here sometimes, and semantics can be tricky). If
>> we open up the door to yolo vendor properties in upstream, then that
>> goal is pretty much written off. And we've been there with vendor
>> properties, it's a giantic mess.
>>
>> Minimally drop my r-b, I'm definitely not in support of this idea.
> 
> So the argument Lidiu and Laurent made was that in some cases, getting a
> generic property right with only a couple of users is hard. So they
> advocated for the right to keep non-generic properties. I can get the
> argument, and no-one else said that was wrong, so it felt like the
> consensus was there.

I also think that (maybe mainly on embedded side) we may have 1) 
esoteric HW features which perhaps can't even be made generic, and 2) 
features which may or may not be generic, but for which support cannot 
be added to any common opensource userspace projects like X or Weston, 
as the only use cases for the features are specialized low level apps 
(often customer's closed-source apps).

While I agree with Daniel's "gigantic mess" problem, it would also be 
quite nice to have a way to support all the HW features upstream instead 
of carrying them in vendor trees.

  Tomi

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Simon Ser]

On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:

> If there's a strong consensus that we really need this then I'm not
> going to nack this, but this really needs a pile of acks from
> compositor folks that they're willing to live with the resulting
> fallout this will likely bring. Your cc list seems to have an absence
> of compositor folks, but instead every driver maintainer. That's
> backwards. We make uapi for userspace, not for kernel driver
> maintainers!

In wlroots we have a policy of only allowing standard KMS properties to
be used. Any vendor-specific property is going to be less well-defined,
less widely useful, potentially have more design issues, potentially
overlap in functionality with other vendor-specific properties, likely
have some hardware-specific assumptions, etc.

What matters here is discussing with other driver & user-space folks to
make sure the new property's design is sound. Designing uAPI is hard.

If kernel folks are struggling with a user-space implementation, they
should discuss with user-space folks to see which project would be
interested. There's a chance a compositor will be interested in the new
property and will just do the user-space part for you, if not we can
suggest candidate projects.

tl;dr strong agree with Daniel here.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Simon Ser]

On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:

> If there's a strong consensus that we really need this then I'm not
> going to nack this, but this really needs a pile of acks from
> compositor folks that they're willing to live with the resulting
> fallout this will likely bring. Your cc list seems to have an absence
> of compositor folks, but instead every driver maintainer. That's
> backwards. We make uapi for userspace, not for kernel driver
> maintainers!

In wlroots we have a policy of only allowing standard KMS properties to
be used. Any vendor-specific property is going to be less well-defined,
less widely useful, potentially have more design issues, potentially
overlap in functionality with other vendor-specific properties, likely
have some hardware-specific assumptions, etc.

What matters here is discussing with other driver & user-space folks to
make sure the new property's design is sound. Designing uAPI is hard.

If kernel folks are struggling with a user-space implementation, they
should discuss with user-space folks to see which project would be
interested. There's a chance a compositor will be interested in the new
property and will just do the user-space part for you, if not we can
suggest candidate projects.

tl;dr strong agree with Daniel here.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> 
> > If there's a strong consensus that we really need this then I'm not
> > going to nack this, but this really needs a pile of acks from
> > compositor folks that they're willing to live with the resulting
> > fallout this will likely bring. Your cc list seems to have an absence
> > of compositor folks, but instead every driver maintainer. That's
> > backwards. We make uapi for userspace, not for kernel driver
> > maintainers!
> 
> In wlroots we have a policy of only allowing standard KMS properties to
> be used. Any vendor-specific property is going to be less well-defined,
> less widely useful, potentially have more design issues, potentially
> overlap in functionality with other vendor-specific properties, likely
> have some hardware-specific assumptions, etc.
> 
> What matters here is discussing with other driver & user-space folks to
> make sure the new property's design is sound. Designing uAPI is hard.
> 
> If kernel folks are struggling with a user-space implementation, they
> should discuss with user-space folks to see which project would be
> interested. There's a chance a compositor will be interested in the new
> property and will just do the user-space part for you, if not we can
> suggest candidate projects.
> 
> tl;dr strong agree with Daniel here.

I think the assumption you and Daniel are making is that the first implementation of
a new KMS property can be made standard from day one and that it will work for any
late comer driver as is, without having to make changes to its behaviour in a
significant way. In my experience that is not the case.

I think we have moved from the times when we were trying to implement in the Linux
world features that were available in the hardware but needed a kernel and userspace
API. The set of properties that exist in KMS cover a lot of needed functionality and
I don't expect to see new properties for stuff that is already supported by hardware.

What I'm expected to see in the future is new functionality that gets implemented by
one hardware vendor and the kernel developers trying to enable that for userspace. It
could be that the new property is generic, but there is no way of testing that on
more than one implementation yet, so I'd say we are generous calling it "standard
property". When the second or third hardware vendor comes along and starts supporting
that property with their own set of extra requirements, then we can call it
"standard". Then comes the effort cost: would it be easier to start with a vendor
property that only the vendor needs to support (and can submit patches into the
compositors to do so) and when the standard property gets added moves to that, or
should we start with a generic property that gets implemented by the compositors
(maybe, but then only one vendor supports it) and then later when we actually
standardise the property we will have to carry backwards compatibility code in the
kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
the former option to be reflected in the documentation, but I would like to hear your
thoughts.

Best regards,
Liviu


-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> 
> > If there's a strong consensus that we really need this then I'm not
> > going to nack this, but this really needs a pile of acks from
> > compositor folks that they're willing to live with the resulting
> > fallout this will likely bring. Your cc list seems to have an absence
> > of compositor folks, but instead every driver maintainer. That's
> > backwards. We make uapi for userspace, not for kernel driver
> > maintainers!
> 
> In wlroots we have a policy of only allowing standard KMS properties to
> be used. Any vendor-specific property is going to be less well-defined,
> less widely useful, potentially have more design issues, potentially
> overlap in functionality with other vendor-specific properties, likely
> have some hardware-specific assumptions, etc.
> 
> What matters here is discussing with other driver & user-space folks to
> make sure the new property's design is sound. Designing uAPI is hard.
> 
> If kernel folks are struggling with a user-space implementation, they
> should discuss with user-space folks to see which project would be
> interested. There's a chance a compositor will be interested in the new
> property and will just do the user-space part for you, if not we can
> suggest candidate projects.
> 
> tl;dr strong agree with Daniel here.

I think the assumption you and Daniel are making is that the first implementation of
a new KMS property can be made standard from day one and that it will work for any
late comer driver as is, without having to make changes to its behaviour in a
significant way. In my experience that is not the case.

I think we have moved from the times when we were trying to implement in the Linux
world features that were available in the hardware but needed a kernel and userspace
API. The set of properties that exist in KMS cover a lot of needed functionality and
I don't expect to see new properties for stuff that is already supported by hardware.

What I'm expected to see in the future is new functionality that gets implemented by
one hardware vendor and the kernel developers trying to enable that for userspace. It
could be that the new property is generic, but there is no way of testing that on
more than one implementation yet, so I'd say we are generous calling it "standard
property". When the second or third hardware vendor comes along and starts supporting
that property with their own set of extra requirements, then we can call it
"standard". Then comes the effort cost: would it be easier to start with a vendor
property that only the vendor needs to support (and can submit patches into the
compositors to do so) and when the standard property gets added moves to that, or
should we start with a generic property that gets implemented by the compositors
(maybe, but then only one vendor supports it) and then later when we actually
standardise the property we will have to carry backwards compatibility code in the
kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
the former option to be reflected in the documentation, but I would like to hear your
thoughts.

Best regards,
Liviu


-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Alyssa Rosenzweig]

> What I'm expected to see in the future is new functionality that gets implemented by
> one hardware vendor and the kernel developers trying to enable that for userspace. It
> could be that the new property is generic, but there is no way of testing that on
> more than one implementation yet, so I'd say we are generous calling it "standard
> property". When the second or third hardware vendor comes along and starts supporting
> that property with their own set of extra requirements, then we can call it
> "standard". Then comes the effort cost: would it be easier to start with a vendor
> property that only the vendor needs to support (and can submit patches into the
> compositors to do so) and when the standard property gets added moves to that, or
> should we start with a generic property that gets implemented by the compositors
> (maybe, but then only one vendor supports it) and then later when we actually
> standardise the property we will have to carry backwards compatibility code in the
> kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> the former option to be reflected in the documentation, but I would like to hear your
> thoughts.

Just my 2c - if the mainline kernel isn't willing to commit to a feature
for upstream userspace to use, why does that feature belong in the
kernel at all? I don't see much value in exposing hardware for the sake
of exposing it when, practically, Linux userspace /can't/ use it as-is.

Might these vendor properties be used on downstream Android userspaces?
That's not generally an upstream goal to support.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Alyssa Rosenzweig]

> What I'm expected to see in the future is new functionality that gets implemented by
> one hardware vendor and the kernel developers trying to enable that for userspace. It
> could be that the new property is generic, but there is no way of testing that on
> more than one implementation yet, so I'd say we are generous calling it "standard
> property". When the second or third hardware vendor comes along and starts supporting
> that property with their own set of extra requirements, then we can call it
> "standard". Then comes the effort cost: would it be easier to start with a vendor
> property that only the vendor needs to support (and can submit patches into the
> compositors to do so) and when the standard property gets added moves to that, or
> should we start with a generic property that gets implemented by the compositors
> (maybe, but then only one vendor supports it) and then later when we actually
> standardise the property we will have to carry backwards compatibility code in the
> kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> the former option to be reflected in the documentation, but I would like to hear your
> thoughts.

Just my 2c - if the mainline kernel isn't willing to commit to a feature
for upstream userspace to use, why does that feature belong in the
kernel at all? I don't see much value in exposing hardware for the sake
of exposing it when, practically, Linux userspace /can't/ use it as-is.

Might these vendor properties be used on downstream Android userspaces?
That's not generally an upstream goal to support.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

On Fri, Jun 11, 2021 at 08:56:04AM -0400, Alyssa Rosenzweig wrote:
> > What I'm expected to see in the future is new functionality that gets implemented by
> > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > could be that the new property is generic, but there is no way of testing that on
> > more than one implementation yet, so I'd say we are generous calling it "standard
> > property". When the second or third hardware vendor comes along and starts supporting
> > that property with their own set of extra requirements, then we can call it
> > "standard". Then comes the effort cost: would it be easier to start with a vendor
> > property that only the vendor needs to support (and can submit patches into the
> > compositors to do so) and when the standard property gets added moves to that, or
> > should we start with a generic property that gets implemented by the compositors
> > (maybe, but then only one vendor supports it) and then later when we actually
> > standardise the property we will have to carry backwards compatibility code in the
> > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > the former option to be reflected in the documentation, but I would like to hear your
> > thoughts.
> 
> Just my 2c - if the mainline kernel isn't willing to commit to a feature
> for upstream userspace to use, why does that feature belong in the
> kernel at all? I don't see much value in exposing hardware for the sake
> of exposing it when, practically, Linux userspace /can't/ use it as-is.
> 
> Might these vendor properties be used on downstream Android userspaces?
> That's not generally an upstream goal to support.

I think the assumption is that we are willing to commit to supporting a feature for
userspace, just that (I personally) lack the confidence that I will be getting the
feature right on the first attempt and using only one vendor hardware. And that
supporting potential mistakes I might've made in the first version is harder if the
feature was deemed "standard".

I'm talking from my experience with the writeback connector. We almost committed the
feature twice before more people chipped in and asked us for changes, but that was lucky.

Best regards,
Liviu

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

On Fri, Jun 11, 2021 at 08:56:04AM -0400, Alyssa Rosenzweig wrote:
> > What I'm expected to see in the future is new functionality that gets implemented by
> > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > could be that the new property is generic, but there is no way of testing that on
> > more than one implementation yet, so I'd say we are generous calling it "standard
> > property". When the second or third hardware vendor comes along and starts supporting
> > that property with their own set of extra requirements, then we can call it
> > "standard". Then comes the effort cost: would it be easier to start with a vendor
> > property that only the vendor needs to support (and can submit patches into the
> > compositors to do so) and when the standard property gets added moves to that, or
> > should we start with a generic property that gets implemented by the compositors
> > (maybe, but then only one vendor supports it) and then later when we actually
> > standardise the property we will have to carry backwards compatibility code in the
> > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > the former option to be reflected in the documentation, but I would like to hear your
> > thoughts.
> 
> Just my 2c - if the mainline kernel isn't willing to commit to a feature
> for upstream userspace to use, why does that feature belong in the
> kernel at all? I don't see much value in exposing hardware for the sake
> of exposing it when, practically, Linux userspace /can't/ use it as-is.
> 
> Might these vendor properties be used on downstream Android userspaces?
> That's not generally an upstream goal to support.

I think the assumption is that we are willing to commit to supporting a feature for
userspace, just that (I personally) lack the confidence that I will be getting the
feature right on the first attempt and using only one vendor hardware. And that
supporting potential mistakes I might've made in the first version is harder if the
feature was deemed "standard".

I'm talking from my experience with the writeback connector. We almost committed the
feature twice before more people chipped in and asked us for changes, but that was lucky.

Best regards,
Liviu

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Daniel Vetter]

On Fri, Jun 11, 2021 at 09:53:19AM +0300, Tomi Valkeinen wrote:
> On 11/06/2021 08:54, Maxime Ripard wrote:
> > Hi,
> > 
> > On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
> > > On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
> > > > 
> > > > New KMS properties come with a bunch of requirements to avoid each
> > > > driver from running their own, inconsistent, set of properties,
> > > > eventually leading to issues like property conflicts, inconsistencies
> > > > between drivers and semantics, etc.
> > > > 
> > > > Let's document what we expect.
> > > > 
> > > > Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> > > > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > > > Cc: Alex Deucher <alexander.deucher@amd.com>
> > > > Cc: Alison Wang <alison.wang@nxp.com>
> > > > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > > > Cc: Andrew Jeffery <andrew@aj.id.au>
> > > > Cc: Andrzej Hajda <a.hajda@samsung.com>
> > > > Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> > > > Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> > > > Cc: Ben Skeggs <bskeggs@redhat.com>
> > > > Cc: Boris Brezillon <bbrezillon@kernel.org>
> > > > Cc: Brian Starkey <brian.starkey@arm.com>
> > > > Cc: Chen Feng <puck.chen@hisilicon.com>
> > > > Cc: Chen-Yu Tsai <wens@csie.org>
> > > > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> > > > Cc: Edmund Dea <edmund.j.dea@intel.com>
> > > > Cc: Eric Anholt <eric@anholt.net>
> > > > Cc: Fabio Estevam <festevam@gmail.com>
> > > > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > > > Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> > > > Cc: Hans de Goede <hdegoede@redhat.com>
> > > > Cc: "Heiko Stübner" <heiko@sntech.de>
> > > > Cc: Huang Rui <ray.huang@amd.com>
> > > > Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> > > > Cc: Inki Dae <inki.dae@samsung.com>
> > > > Cc: Jani Nikula <jani.nikula@linux.intel.com>
> > > > Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> > > > Cc: Jerome Brunet <jbrunet@baylibre.com>
> > > > Cc: Joel Stanley <joel@jms.id.au>
> > > > Cc: John Stultz <john.stultz@linaro.org>
> > > > Cc: Jonas Karlman <jonas@kwiboo.se>
> > > > Cc: Jonathan Hunter <jonathanh@nvidia.com>
> > > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > > Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> > > > Cc: Jyri Sarha <jyri.sarha@iki.fi>
> > > > Cc: Kevin Hilman <khilman@baylibre.com>
> > > > Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > > > Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> > > > Cc: Kyungmin Park <kyungmin.park@samsung.com>
> > > > Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> > > > Cc: Linus Walleij <linus.walleij@linaro.org>
> > > > Cc: Liviu Dudau <liviu.dudau@arm.com>
> > > > Cc: Lucas Stach <l.stach@pengutronix.de>
> > > > Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> > > > Cc: Marek Vasut <marex@denx.de>
> > > > Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> > > > Cc: Matthias Brugger <matthias.bgg@gmail.com>
> > > > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > > > Cc: Maxime Ripard <mripard@kernel.org>
> > > > Cc: Melissa Wen <melissa.srw@gmail.com>
> > > > Cc: Neil Armstrong <narmstrong@baylibre.com>
> > > > Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> > > > Cc: "Noralf Trønnes" <noralf@tronnes.org>
> > > > Cc: NXP Linux Team <linux-imx@nxp.com>
> > > > Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > > Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> > > > Cc: Paul Cercueil <paul@crapouillou.net>
> > > > Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> > > > Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> > > > Cc: Philipp Zabel <p.zabel@pengutronix.de>
> > > > Cc: Qiang Yu <yuq825@gmail.com>
> > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > Cc: Robert Foss <robert.foss@linaro.org>
> > > > Cc: Rob Herring <robh@kernel.org>
> > > > Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > Cc: Roland Scheidegger <sroland@vmware.com>
> > > > Cc: Russell King <linux@armlinux.org.uk>
> > > > Cc: Sam Ravnborg <sam@ravnborg.org>
> > > > Cc: Sandy Huang <hjc@rock-chips.com>
> > > > Cc: Sascha Hauer <s.hauer@pengutronix.de>
> > > > Cc: Sean Paul <sean@poorly.run>
> > > > Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> > > > Cc: Shawn Guo <shawnguo@kernel.org>
> > > > Cc: Stefan Agner <stefan@agner.ch>
> > > > Cc: Steven Price <steven.price@arm.com>
> > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > Cc: Thierry Reding <thierry.reding@gmail.com>
> > > > Cc: Tian Tao <tiantao6@hisilicon.com>
> > > > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > > Cc: Tomi Valkeinen <tomba@kernel.org>
> > > > Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> > > > Cc: Xinliang Liu <xinliang.liu@linaro.org>
> > > > Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> > > > Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> > > > Cc: Zack Rusin <zackr@vmware.com>
> > > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> > > > 
> > > > ---
> > > > 
> > > > Changes from v2:
> > > >    - Take into account the feedback from Laurent and Lidiu to no longer
> > > >      force generic properties, but prefix vendor-specific properties with
> > > >      the vendor name
> > > 
> > > I'm pretty sure my r-b was without this ...
> > 
> > Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
> > second look, but I shouldn't have kept it and caught you by surprise
> > indeed.
> > 
> > > Why exactly do we need this? KMS is meant to be fairly generic (bugs
> > > throw a wrench around here sometimes, and semantics can be tricky). If
> > > we open up the door to yolo vendor properties in upstream, then that
> > > goal is pretty much written off. And we've been there with vendor
> > > properties, it's a giantic mess.
> > > 
> > > Minimally drop my r-b, I'm definitely not in support of this idea.
> > 
> > So the argument Lidiu and Laurent made was that in some cases, getting a
> > generic property right with only a couple of users is hard. So they
> > advocated for the right to keep non-generic properties. I can get the
> > argument, and no-one else said that was wrong, so it felt like the
> > consensus was there.
> 
> I also think that (maybe mainly on embedded side) we may have 1) esoteric HW
> features which perhaps can't even be made generic, and 2) features which may
> or may not be generic, but for which support cannot be added to any common
> opensource userspace projects like X or Weston, as the only use cases for
> the features are specialized low level apps (often customer's closed-source
> apps).
> 
> While I agree with Daniel's "gigantic mess" problem, it would also be quite
> nice to have a way to support all the HW features upstream instead of
> carrying them in vendor trees.

So this means to be able to accomodate this "vendor properties are totally
fine, go wild" exception we also need to throw "open source userspace
user, fully reviewed and tested" into the drink?

At least my experience from what I've seen with funky vendor properties
isn't so much that we can't figure out a reasonable way to expose them.
The problem is that the userspace tends to be a (often closed source)
vendor fork of a random compositor somewhere. So if that requirement is
somehow a problem, we need to talk about _that_.

Not promising we'll totally merge some vendor properties without spelling
out what exactly this means. All that ensures is that people submit
patches and then get annoyed because they still can't be merged because
the userspace situation is all the same.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Daniel Vetter]

On Fri, Jun 11, 2021 at 09:53:19AM +0300, Tomi Valkeinen wrote:
> On 11/06/2021 08:54, Maxime Ripard wrote:
> > Hi,
> > 
> > On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
> > > On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
> > > > 
> > > > New KMS properties come with a bunch of requirements to avoid each
> > > > driver from running their own, inconsistent, set of properties,
> > > > eventually leading to issues like property conflicts, inconsistencies
> > > > between drivers and semantics, etc.
> > > > 
> > > > Let's document what we expect.
> > > > 
> > > > Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> > > > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > > > Cc: Alex Deucher <alexander.deucher@amd.com>
> > > > Cc: Alison Wang <alison.wang@nxp.com>
> > > > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > > > Cc: Andrew Jeffery <andrew@aj.id.au>
> > > > Cc: Andrzej Hajda <a.hajda@samsung.com>
> > > > Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> > > > Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> > > > Cc: Ben Skeggs <bskeggs@redhat.com>
> > > > Cc: Boris Brezillon <bbrezillon@kernel.org>
> > > > Cc: Brian Starkey <brian.starkey@arm.com>
> > > > Cc: Chen Feng <puck.chen@hisilicon.com>
> > > > Cc: Chen-Yu Tsai <wens@csie.org>
> > > > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> > > > Cc: Edmund Dea <edmund.j.dea@intel.com>
> > > > Cc: Eric Anholt <eric@anholt.net>
> > > > Cc: Fabio Estevam <festevam@gmail.com>
> > > > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > > > Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> > > > Cc: Hans de Goede <hdegoede@redhat.com>
> > > > Cc: "Heiko Stübner" <heiko@sntech.de>
> > > > Cc: Huang Rui <ray.huang@amd.com>
> > > > Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> > > > Cc: Inki Dae <inki.dae@samsung.com>
> > > > Cc: Jani Nikula <jani.nikula@linux.intel.com>
> > > > Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> > > > Cc: Jerome Brunet <jbrunet@baylibre.com>
> > > > Cc: Joel Stanley <joel@jms.id.au>
> > > > Cc: John Stultz <john.stultz@linaro.org>
> > > > Cc: Jonas Karlman <jonas@kwiboo.se>
> > > > Cc: Jonathan Hunter <jonathanh@nvidia.com>
> > > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > > Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> > > > Cc: Jyri Sarha <jyri.sarha@iki.fi>
> > > > Cc: Kevin Hilman <khilman@baylibre.com>
> > > > Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > > > Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> > > > Cc: Kyungmin Park <kyungmin.park@samsung.com>
> > > > Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> > > > Cc: Linus Walleij <linus.walleij@linaro.org>
> > > > Cc: Liviu Dudau <liviu.dudau@arm.com>
> > > > Cc: Lucas Stach <l.stach@pengutronix.de>
> > > > Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> > > > Cc: Marek Vasut <marex@denx.de>
> > > > Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> > > > Cc: Matthias Brugger <matthias.bgg@gmail.com>
> > > > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > > > Cc: Maxime Ripard <mripard@kernel.org>
> > > > Cc: Melissa Wen <melissa.srw@gmail.com>
> > > > Cc: Neil Armstrong <narmstrong@baylibre.com>
> > > > Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> > > > Cc: "Noralf Trønnes" <noralf@tronnes.org>
> > > > Cc: NXP Linux Team <linux-imx@nxp.com>
> > > > Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > > Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> > > > Cc: Paul Cercueil <paul@crapouillou.net>
> > > > Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> > > > Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> > > > Cc: Philipp Zabel <p.zabel@pengutronix.de>
> > > > Cc: Qiang Yu <yuq825@gmail.com>
> > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > Cc: Robert Foss <robert.foss@linaro.org>
> > > > Cc: Rob Herring <robh@kernel.org>
> > > > Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > Cc: Roland Scheidegger <sroland@vmware.com>
> > > > Cc: Russell King <linux@armlinux.org.uk>
> > > > Cc: Sam Ravnborg <sam@ravnborg.org>
> > > > Cc: Sandy Huang <hjc@rock-chips.com>
> > > > Cc: Sascha Hauer <s.hauer@pengutronix.de>
> > > > Cc: Sean Paul <sean@poorly.run>
> > > > Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> > > > Cc: Shawn Guo <shawnguo@kernel.org>
> > > > Cc: Stefan Agner <stefan@agner.ch>
> > > > Cc: Steven Price <steven.price@arm.com>
> > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > Cc: Thierry Reding <thierry.reding@gmail.com>
> > > > Cc: Tian Tao <tiantao6@hisilicon.com>
> > > > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > > Cc: Tomi Valkeinen <tomba@kernel.org>
> > > > Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> > > > Cc: Xinliang Liu <xinliang.liu@linaro.org>
> > > > Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> > > > Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> > > > Cc: Zack Rusin <zackr@vmware.com>
> > > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> > > > 
> > > > ---
> > > > 
> > > > Changes from v2:
> > > >    - Take into account the feedback from Laurent and Lidiu to no longer
> > > >      force generic properties, but prefix vendor-specific properties with
> > > >      the vendor name
> > > 
> > > I'm pretty sure my r-b was without this ...
> > 
> > Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
> > second look, but I shouldn't have kept it and caught you by surprise
> > indeed.
> > 
> > > Why exactly do we need this? KMS is meant to be fairly generic (bugs
> > > throw a wrench around here sometimes, and semantics can be tricky). If
> > > we open up the door to yolo vendor properties in upstream, then that
> > > goal is pretty much written off. And we've been there with vendor
> > > properties, it's a giantic mess.
> > > 
> > > Minimally drop my r-b, I'm definitely not in support of this idea.
> > 
> > So the argument Lidiu and Laurent made was that in some cases, getting a
> > generic property right with only a couple of users is hard. So they
> > advocated for the right to keep non-generic properties. I can get the
> > argument, and no-one else said that was wrong, so it felt like the
> > consensus was there.
> 
> I also think that (maybe mainly on embedded side) we may have 1) esoteric HW
> features which perhaps can't even be made generic, and 2) features which may
> or may not be generic, but for which support cannot be added to any common
> opensource userspace projects like X or Weston, as the only use cases for
> the features are specialized low level apps (often customer's closed-source
> apps).
> 
> While I agree with Daniel's "gigantic mess" problem, it would also be quite
> nice to have a way to support all the HW features upstream instead of
> carrying them in vendor trees.

So this means to be able to accomodate this "vendor properties are totally
fine, go wild" exception we also need to throw "open source userspace
user, fully reviewed and tested" into the drink?

At least my experience from what I've seen with funky vendor properties
isn't so much that we can't figure out a reasonable way to expose them.
The problem is that the userspace tends to be a (often closed source)
vendor fork of a random compositor somewhere. So if that requirement is
somehow a problem, we need to talk about _that_.

Not promising we'll totally merge some vendor properties without spelling
out what exactly this means. All that ensures is that people submit
patches and then get annoyed because they still can't be merged because
the userspace situation is all the same.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

On Fri, 11 Jun 2021 07:54:07 +0200
Maxime Ripard <maxime@cerno.tech> wrote:

> Hi,
> 
> On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
> > On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:  
> > >
> > > New KMS properties come with a bunch of requirements to avoid each
> > > driver from running their own, inconsistent, set of properties,
> > > eventually leading to issues like property conflicts, inconsistencies
> > > between drivers and semantics, etc.
> > >
> > > Let's document what we expect.

Sorry for intentionally losing the whole CC list, but I'm not
comfortable CC'ing such a huge list (of lists) with just this.


> > ltdr; I'd go back to v2. And then cc compositor folks on this to get
> > their ack.  
> 
> So, Pekka, Simon, is there anyone else I should Cc?

You can find some Wayland compositor contacts in the following. These
are not all compositor projects listed there though. They also signed
up for wayland-protocols rather than DRM UAPI discussions, but maybe
taking a few won't mind:
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/MEMBERS.md


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 5501 bytes --]

On Fri, 11 Jun 2021 13:03:09 +0100
Liviu Dudau <liviu.dudau@arm.com> wrote:

> On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >   
> > > If there's a strong consensus that we really need this then I'm not
> > > going to nack this, but this really needs a pile of acks from
> > > compositor folks that they're willing to live with the resulting
> > > fallout this will likely bring. Your cc list seems to have an absence
> > > of compositor folks, but instead every driver maintainer. That's
> > > backwards. We make uapi for userspace, not for kernel driver
> > > maintainers!  
> > 
> > In wlroots we have a policy of only allowing standard KMS properties to
> > be used. Any vendor-specific property is going to be less well-defined,
> > less widely useful, potentially have more design issues, potentially
> > overlap in functionality with other vendor-specific properties, likely
> > have some hardware-specific assumptions, etc.
> > 
> > What matters here is discussing with other driver & user-space folks to
> > make sure the new property's design is sound. Designing uAPI is hard.
> > 
> > If kernel folks are struggling with a user-space implementation, they
> > should discuss with user-space folks to see which project would be
> > interested. There's a chance a compositor will be interested in the new
> > property and will just do the user-space part for you, if not we can
> > suggest candidate projects.
> > 
> > tl;dr strong agree with Daniel here.  
> 
> I think the assumption you and Daniel are making is that the first implementation of
> a new KMS property can be made standard from day one and that it will work for any
> late comer driver as is, without having to make changes to its behaviour in a
> significant way. In my experience that is not the case.
> 
> I think we have moved from the times when we were trying to implement in the Linux
> world features that were available in the hardware but needed a kernel and userspace
> API. The set of properties that exist in KMS cover a lot of needed functionality and
> I don't expect to see new properties for stuff that is already supported by hardware.
> 
> What I'm expected to see in the future is new functionality that gets implemented by
> one hardware vendor and the kernel developers trying to enable that for userspace. It
> could be that the new property is generic, but there is no way of testing that on
> more than one implementation yet, so I'd say we are generous calling it "standard
> property". When the second or third hardware vendor comes along and starts supporting
> that property with their own set of extra requirements, then we can call it
> "standard".

I agree that is a problem with trying to make generic anything. But it
does not mean you should not even try. Maybe trying really hard saves a
couple revisions.

What I think should be planned for is revisions. How to add new
properties that do the same thing but better, while documenting that a
userspace KMS client can use only one revision at a time. You never
remove old revisions, unless maybe with a DRM client cap they
could disappear from that file description if that is necessary for
seeing the new revision.

While designing this, one also needs to take into account that KMS
clients need to be able to save and restore properties *they do not
understand*. So exposing two revisions of the same feature
simultaneously would break save/restore is that's an error.

> Then comes the effort cost: would it be easier to start with a vendor
> property that only the vendor needs to support (and can submit patches into the
> compositors to do so) and when the standard property gets added moves to that, or

But you can't move, you can only add? You can't delete the old property
in kernel if it was ever released with a kernel and anyone used it. In
the same sentence you also imply that there is a user of it, so
removing it will break that user. Then you'll have to track the
userspace lifetime to figure out which decade you can try removing it.

> should we start with a generic property that gets implemented by the compositors
> (maybe, but then only one vendor supports it) and then later when we actually
> standardise the property we will have to carry backwards compatibility code in the
> kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> the former option to be reflected in the documentation, but I would like to hear your
> thoughts.

You have to carry the backward compatibility in all cases, right?

Userspace OTOH can drop support for older less supported KMS properties
while taking advantage of a new revision. Userspace is not required to
support old kernels forever.


Here's a wild counter-proposal off a tangent:

How about we make "implemented in and testable with VKMS" the rule,
instead of "is generic" for new properties?

VKMS is what compositors (will) use in CI. I would feel hugely less bad
about using a property that only one hardware driver ever implements,
if also VKMS implements it in a way that compositor CI can observe it
working.

I don't expect this proposal to be accepted, but it's food for thought.
The major problem for compositor projects is testing as you usually
don't have the hardware, IMO. CI tends to not have any hardware.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 5501 bytes --]

On Fri, 11 Jun 2021 13:03:09 +0100
Liviu Dudau <liviu.dudau@arm.com> wrote:

> On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >   
> > > If there's a strong consensus that we really need this then I'm not
> > > going to nack this, but this really needs a pile of acks from
> > > compositor folks that they're willing to live with the resulting
> > > fallout this will likely bring. Your cc list seems to have an absence
> > > of compositor folks, but instead every driver maintainer. That's
> > > backwards. We make uapi for userspace, not for kernel driver
> > > maintainers!  
> > 
> > In wlroots we have a policy of only allowing standard KMS properties to
> > be used. Any vendor-specific property is going to be less well-defined,
> > less widely useful, potentially have more design issues, potentially
> > overlap in functionality with other vendor-specific properties, likely
> > have some hardware-specific assumptions, etc.
> > 
> > What matters here is discussing with other driver & user-space folks to
> > make sure the new property's design is sound. Designing uAPI is hard.
> > 
> > If kernel folks are struggling with a user-space implementation, they
> > should discuss with user-space folks to see which project would be
> > interested. There's a chance a compositor will be interested in the new
> > property and will just do the user-space part for you, if not we can
> > suggest candidate projects.
> > 
> > tl;dr strong agree with Daniel here.  
> 
> I think the assumption you and Daniel are making is that the first implementation of
> a new KMS property can be made standard from day one and that it will work for any
> late comer driver as is, without having to make changes to its behaviour in a
> significant way. In my experience that is not the case.
> 
> I think we have moved from the times when we were trying to implement in the Linux
> world features that were available in the hardware but needed a kernel and userspace
> API. The set of properties that exist in KMS cover a lot of needed functionality and
> I don't expect to see new properties for stuff that is already supported by hardware.
> 
> What I'm expected to see in the future is new functionality that gets implemented by
> one hardware vendor and the kernel developers trying to enable that for userspace. It
> could be that the new property is generic, but there is no way of testing that on
> more than one implementation yet, so I'd say we are generous calling it "standard
> property". When the second or third hardware vendor comes along and starts supporting
> that property with their own set of extra requirements, then we can call it
> "standard".

I agree that is a problem with trying to make generic anything. But it
does not mean you should not even try. Maybe trying really hard saves a
couple revisions.

What I think should be planned for is revisions. How to add new
properties that do the same thing but better, while documenting that a
userspace KMS client can use only one revision at a time. You never
remove old revisions, unless maybe with a DRM client cap they
could disappear from that file description if that is necessary for
seeing the new revision.

While designing this, one also needs to take into account that KMS
clients need to be able to save and restore properties *they do not
understand*. So exposing two revisions of the same feature
simultaneously would break save/restore is that's an error.

> Then comes the effort cost: would it be easier to start with a vendor
> property that only the vendor needs to support (and can submit patches into the
> compositors to do so) and when the standard property gets added moves to that, or

But you can't move, you can only add? You can't delete the old property
in kernel if it was ever released with a kernel and anyone used it. In
the same sentence you also imply that there is a user of it, so
removing it will break that user. Then you'll have to track the
userspace lifetime to figure out which decade you can try removing it.

> should we start with a generic property that gets implemented by the compositors
> (maybe, but then only one vendor supports it) and then later when we actually
> standardise the property we will have to carry backwards compatibility code in the
> kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> the former option to be reflected in the documentation, but I would like to hear your
> thoughts.

You have to carry the backward compatibility in all cases, right?

Userspace OTOH can drop support for older less supported KMS properties
while taking advantage of a new revision. Userspace is not required to
support old kernels forever.


Here's a wild counter-proposal off a tangent:

How about we make "implemented in and testable with VKMS" the rule,
instead of "is generic" for new properties?

VKMS is what compositors (will) use in CI. I would feel hugely less bad
about using a property that only one hardware driver ever implements,
if also VKMS implements it in a way that compositor CI can observe it
working.

I don't expect this proposal to be accepted, but it's food for thought.
The major problem for compositor projects is testing as you usually
don't have the hardware, IMO. CI tends to not have any hardware.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

On Mon, Jun 14, 2021 at 05:49:12PM +0300, Pekka Paalanen wrote:
> On Fri, 11 Jun 2021 13:03:09 +0100
> Liviu Dudau <liviu.dudau@arm.com> wrote:
> 
> > On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> > > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >   
> > > > If there's a strong consensus that we really need this then I'm not
> > > > going to nack this, but this really needs a pile of acks from
> > > > compositor folks that they're willing to live with the resulting
> > > > fallout this will likely bring. Your cc list seems to have an absence
> > > > of compositor folks, but instead every driver maintainer. That's
> > > > backwards. We make uapi for userspace, not for kernel driver
> > > > maintainers!  
> > > 
> > > In wlroots we have a policy of only allowing standard KMS properties to
> > > be used. Any vendor-specific property is going to be less well-defined,
> > > less widely useful, potentially have more design issues, potentially
> > > overlap in functionality with other vendor-specific properties, likely
> > > have some hardware-specific assumptions, etc.
> > > 
> > > What matters here is discussing with other driver & user-space folks to
> > > make sure the new property's design is sound. Designing uAPI is hard.
> > > 
> > > If kernel folks are struggling with a user-space implementation, they
> > > should discuss with user-space folks to see which project would be
> > > interested. There's a chance a compositor will be interested in the new
> > > property and will just do the user-space part for you, if not we can
> > > suggest candidate projects.
> > > 
> > > tl;dr strong agree with Daniel here.  
> > 
> > I think the assumption you and Daniel are making is that the first implementation of
> > a new KMS property can be made standard from day one and that it will work for any
> > late comer driver as is, without having to make changes to its behaviour in a
> > significant way. In my experience that is not the case.
> > 
> > I think we have moved from the times when we were trying to implement in the Linux
> > world features that were available in the hardware but needed a kernel and userspace
> > API. The set of properties that exist in KMS cover a lot of needed functionality and
> > I don't expect to see new properties for stuff that is already supported by hardware.
> > 
> > What I'm expected to see in the future is new functionality that gets implemented by
> > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > could be that the new property is generic, but there is no way of testing that on
> > more than one implementation yet, so I'd say we are generous calling it "standard
> > property". When the second or third hardware vendor comes along and starts supporting
> > that property with their own set of extra requirements, then we can call it
> > "standard".
> 
> I agree that is a problem with trying to make generic anything. But it
> does not mean you should not even try. Maybe trying really hard saves a
> couple revisions.

Agree.

> 
> What I think should be planned for is revisions. How to add new
> properties that do the same thing but better, while documenting that a
> userspace KMS client can use only one revision at a time. You never
> remove old revisions, unless maybe with a DRM client cap they
> could disappear from that file description if that is necessary for
> seeing the new revision.
> 
> While designing this, one also needs to take into account that KMS
> clients need to be able to save and restore properties *they do not
> understand*. So exposing two revisions of the same feature
> simultaneously would break save/restore is that's an error.

I quite like the idea of having versions for properties.

> 
> > Then comes the effort cost: would it be easier to start with a vendor
> > property that only the vendor needs to support (and can submit patches into the
> > compositors to do so) and when the standard property gets added moves to that, or
> 
> But you can't move, you can only add? You can't delete the old property
> in kernel if it was ever released with a kernel and anyone used it. In
> the same sentence you also imply that there is a user of it, so
> removing it will break that user. Then you'll have to track the
> userspace lifetime to figure out which decade you can try removing it.

Not that I am supporting the workflow, but I was trying to address the comments that
vendors are going to push their own userspace implementation for their vendor
properties. If that is the case, when they switch to the standard ones they can drop
the support in userspace for their changes. With the implied assumption that you will
have fewer vendor implementations hence easier to make changes, KMS properties can be
deleted if you know there is no user of them (e.g. the vendor has upgraded all their
software to the standard property).

> 
> > should we start with a generic property that gets implemented by the compositors
> > (maybe, but then only one vendor supports it) and then later when we actually
> > standardise the property we will have to carry backwards compatibility code in the
> > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > the former option to be reflected in the documentation, but I would like to hear your
> > thoughts.
> 
> You have to carry the backward compatibility in all cases, right?
> 
> Userspace OTOH can drop support for older less supported KMS properties
> while taking advantage of a new revision. Userspace is not required to
> support old kernels forever.
> 
> 
> Here's a wild counter-proposal off a tangent:
> 
> How about we make "implemented in and testable with VKMS" the rule,
> instead of "is generic" for new properties?
> 
> VKMS is what compositors (will) use in CI. I would feel hugely less bad
> about using a property that only one hardware driver ever implements,
> if also VKMS implements it in a way that compositor CI can observe it
> working.
> 
> I don't expect this proposal to be accepted, but it's food for thought.
> The major problem for compositor projects is testing as you usually
> don't have the hardware, IMO. CI tends to not have any hardware.

While I don't dislike the proposal (I think it is quite sensible), I am worried that
for some behaviours VKMS will implement them in a quirky way. To pick (again) the
example of writeback, real hardware will have a way to tell if the buffer has been
sent successfully to memory and it might take more than one refresh period, while
VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If
you code your compositor based on VKMS you might get unexpected artifacts on real
hardware.

Best regards,
Liviu


> 
> 
> Thanks,
> pq



-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Liviu Dudau]

On Mon, Jun 14, 2021 at 05:49:12PM +0300, Pekka Paalanen wrote:
> On Fri, 11 Jun 2021 13:03:09 +0100
> Liviu Dudau <liviu.dudau@arm.com> wrote:
> 
> > On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> > > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >   
> > > > If there's a strong consensus that we really need this then I'm not
> > > > going to nack this, but this really needs a pile of acks from
> > > > compositor folks that they're willing to live with the resulting
> > > > fallout this will likely bring. Your cc list seems to have an absence
> > > > of compositor folks, but instead every driver maintainer. That's
> > > > backwards. We make uapi for userspace, not for kernel driver
> > > > maintainers!  
> > > 
> > > In wlroots we have a policy of only allowing standard KMS properties to
> > > be used. Any vendor-specific property is going to be less well-defined,
> > > less widely useful, potentially have more design issues, potentially
> > > overlap in functionality with other vendor-specific properties, likely
> > > have some hardware-specific assumptions, etc.
> > > 
> > > What matters here is discussing with other driver & user-space folks to
> > > make sure the new property's design is sound. Designing uAPI is hard.
> > > 
> > > If kernel folks are struggling with a user-space implementation, they
> > > should discuss with user-space folks to see which project would be
> > > interested. There's a chance a compositor will be interested in the new
> > > property and will just do the user-space part for you, if not we can
> > > suggest candidate projects.
> > > 
> > > tl;dr strong agree with Daniel here.  
> > 
> > I think the assumption you and Daniel are making is that the first implementation of
> > a new KMS property can be made standard from day one and that it will work for any
> > late comer driver as is, without having to make changes to its behaviour in a
> > significant way. In my experience that is not the case.
> > 
> > I think we have moved from the times when we were trying to implement in the Linux
> > world features that were available in the hardware but needed a kernel and userspace
> > API. The set of properties that exist in KMS cover a lot of needed functionality and
> > I don't expect to see new properties for stuff that is already supported by hardware.
> > 
> > What I'm expected to see in the future is new functionality that gets implemented by
> > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > could be that the new property is generic, but there is no way of testing that on
> > more than one implementation yet, so I'd say we are generous calling it "standard
> > property". When the second or third hardware vendor comes along and starts supporting
> > that property with their own set of extra requirements, then we can call it
> > "standard".
> 
> I agree that is a problem with trying to make generic anything. But it
> does not mean you should not even try. Maybe trying really hard saves a
> couple revisions.

Agree.

> 
> What I think should be planned for is revisions. How to add new
> properties that do the same thing but better, while documenting that a
> userspace KMS client can use only one revision at a time. You never
> remove old revisions, unless maybe with a DRM client cap they
> could disappear from that file description if that is necessary for
> seeing the new revision.
> 
> While designing this, one also needs to take into account that KMS
> clients need to be able to save and restore properties *they do not
> understand*. So exposing two revisions of the same feature
> simultaneously would break save/restore is that's an error.

I quite like the idea of having versions for properties.

> 
> > Then comes the effort cost: would it be easier to start with a vendor
> > property that only the vendor needs to support (and can submit patches into the
> > compositors to do so) and when the standard property gets added moves to that, or
> 
> But you can't move, you can only add? You can't delete the old property
> in kernel if it was ever released with a kernel and anyone used it. In
> the same sentence you also imply that there is a user of it, so
> removing it will break that user. Then you'll have to track the
> userspace lifetime to figure out which decade you can try removing it.

Not that I am supporting the workflow, but I was trying to address the comments that
vendors are going to push their own userspace implementation for their vendor
properties. If that is the case, when they switch to the standard ones they can drop
the support in userspace for their changes. With the implied assumption that you will
have fewer vendor implementations hence easier to make changes, KMS properties can be
deleted if you know there is no user of them (e.g. the vendor has upgraded all their
software to the standard property).

> 
> > should we start with a generic property that gets implemented by the compositors
> > (maybe, but then only one vendor supports it) and then later when we actually
> > standardise the property we will have to carry backwards compatibility code in the
> > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > the former option to be reflected in the documentation, but I would like to hear your
> > thoughts.
> 
> You have to carry the backward compatibility in all cases, right?
> 
> Userspace OTOH can drop support for older less supported KMS properties
> while taking advantage of a new revision. Userspace is not required to
> support old kernels forever.
> 
> 
> Here's a wild counter-proposal off a tangent:
> 
> How about we make "implemented in and testable with VKMS" the rule,
> instead of "is generic" for new properties?
> 
> VKMS is what compositors (will) use in CI. I would feel hugely less bad
> about using a property that only one hardware driver ever implements,
> if also VKMS implements it in a way that compositor CI can observe it
> working.
> 
> I don't expect this proposal to be accepted, but it's food for thought.
> The major problem for compositor projects is testing as you usually
> don't have the hardware, IMO. CI tends to not have any hardware.

While I don't dislike the proposal (I think it is quite sensible), I am worried that
for some behaviours VKMS will implement them in a quirky way. To pick (again) the
example of writeback, real hardware will have a way to tell if the buffer has been
sent successfully to memory and it might take more than one refresh period, while
VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If
you code your compositor based on VKMS you might get unexpected artifacts on real
hardware.

Best regards,
Liviu


> 
> 
> Thanks,
> pq



-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Fri, Jun 11, 2021 at 07:54:07AM +0200, Maxime Ripard wrote:
> On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
> > On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
> > >
> > > New KMS properties come with a bunch of requirements to avoid each
> > > driver from running their own, inconsistent, set of properties,
> > > eventually leading to issues like property conflicts, inconsistencies
> > > between drivers and semantics, etc.
> > >
> > > Let's document what we expect.
> > >
> > > Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> > > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > > Cc: Alex Deucher <alexander.deucher@amd.com>
> > > Cc: Alison Wang <alison.wang@nxp.com>
> > > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > > Cc: Andrew Jeffery <andrew@aj.id.au>
> > > Cc: Andrzej Hajda <a.hajda@samsung.com>
> > > Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> > > Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> > > Cc: Ben Skeggs <bskeggs@redhat.com>
> > > Cc: Boris Brezillon <bbrezillon@kernel.org>
> > > Cc: Brian Starkey <brian.starkey@arm.com>
> > > Cc: Chen Feng <puck.chen@hisilicon.com>
> > > Cc: Chen-Yu Tsai <wens@csie.org>
> > > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > > Cc: "Christian König" <christian.koenig@amd.com>
> > > Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> > > Cc: Edmund Dea <edmund.j.dea@intel.com>
> > > Cc: Eric Anholt <eric@anholt.net>
> > > Cc: Fabio Estevam <festevam@gmail.com>
> > > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > > Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> > > Cc: Hans de Goede <hdegoede@redhat.com>
> > > Cc: "Heiko Stübner" <heiko@sntech.de>
> > > Cc: Huang Rui <ray.huang@amd.com>
> > > Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> > > Cc: Inki Dae <inki.dae@samsung.com>
> > > Cc: Jani Nikula <jani.nikula@linux.intel.com>
> > > Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> > > Cc: Jerome Brunet <jbrunet@baylibre.com>
> > > Cc: Joel Stanley <joel@jms.id.au>
> > > Cc: John Stultz <john.stultz@linaro.org>
> > > Cc: Jonas Karlman <jonas@kwiboo.se>
> > > Cc: Jonathan Hunter <jonathanh@nvidia.com>
> > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> > > Cc: Jyri Sarha <jyri.sarha@iki.fi>
> > > Cc: Kevin Hilman <khilman@baylibre.com>
> > > Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > > Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> > > Cc: Kyungmin Park <kyungmin.park@samsung.com>
> > > Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> > > Cc: Linus Walleij <linus.walleij@linaro.org>
> > > Cc: Liviu Dudau <liviu.dudau@arm.com>
> > > Cc: Lucas Stach <l.stach@pengutronix.de>
> > > Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> > > Cc: Marek Vasut <marex@denx.de>
> > > Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> > > Cc: Matthias Brugger <matthias.bgg@gmail.com>
> > > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > > Cc: Maxime Ripard <mripard@kernel.org>
> > > Cc: Melissa Wen <melissa.srw@gmail.com>
> > > Cc: Neil Armstrong <narmstrong@baylibre.com>
> > > Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> > > Cc: "Noralf Trønnes" <noralf@tronnes.org>
> > > Cc: NXP Linux Team <linux-imx@nxp.com>
> > > Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> > > Cc: Paul Cercueil <paul@crapouillou.net>
> > > Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> > > Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> > > Cc: Philipp Zabel <p.zabel@pengutronix.de>
> > > Cc: Qiang Yu <yuq825@gmail.com>
> > > Cc: Rob Clark <robdclark@gmail.com>
> > > Cc: Robert Foss <robert.foss@linaro.org>
> > > Cc: Rob Herring <robh@kernel.org>
> > > Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Cc: Roland Scheidegger <sroland@vmware.com>
> > > Cc: Russell King <linux@armlinux.org.uk>
> > > Cc: Sam Ravnborg <sam@ravnborg.org>
> > > Cc: Sandy Huang <hjc@rock-chips.com>
> > > Cc: Sascha Hauer <s.hauer@pengutronix.de>
> > > Cc: Sean Paul <sean@poorly.run>
> > > Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> > > Cc: Shawn Guo <shawnguo@kernel.org>
> > > Cc: Stefan Agner <stefan@agner.ch>
> > > Cc: Steven Price <steven.price@arm.com>
> > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > Cc: Thierry Reding <thierry.reding@gmail.com>
> > > Cc: Tian Tao <tiantao6@hisilicon.com>
> > > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > Cc: Tomi Valkeinen <tomba@kernel.org>
> > > Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> > > Cc: Xinliang Liu <xinliang.liu@linaro.org>
> > > Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> > > Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> > > Cc: Zack Rusin <zackr@vmware.com>
> > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> > >
> > > ---
> > >
> > > Changes from v2:
> > >   - Take into account the feedback from Laurent and Lidiu to no longer
> > >     force generic properties, but prefix vendor-specific properties with
> > >     the vendor name
> > 
> > I'm pretty sure my r-b was without this ...
> 
> Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
> second look, but I shouldn't have kept it and caught you by surprise
> indeed.
> 
> > Why exactly do we need this? KMS is meant to be fairly generic (bugs
> > throw a wrench around here sometimes, and semantics can be tricky). If
> > we open up the door to yolo vendor properties in upstream, then that
> > goal is pretty much written off. And we've been there with vendor
> > properties, it's a giantic mess.
> > 
> > Minimally drop my r-b, I'm definitely not in support of this idea.
> 
> So the argument Lidiu and Laurent made was that in some cases, getting a
> generic property right with only a couple of users is hard. So they
> advocated for the right to keep non-generic properties. I can get the
> argument, and no-one else said that was wrong, so it felt like the
> consensus was there.

My argument was two-fold. On one hand, there's the issue of
standardizing properties when we have a single example. I don't have a
good solution for that chicken-and-egg issue, when we tell vendor they
should standardize properties, and when they ask how, we tell them we
don't know as it hasn't been done before. The option for allowing a
staging playground for vendor properties here isn't something I like,
but we'll need something similar one way or another. Perhaps at the cost
of not guaranteeing userspace ABI for those properties, and converting
them later to something standard (it won't be received well by vendor of
course, and will make a push for mainline more difficult to sell, so
it's not a great solution).

On the other hand, there are esoteric vendor-specific features for which
standardization really doesn't make sense. For those, I think
vendor-specific properties are fine, as long as they're properly
documented, with a design documentation merged at the same time as the
property. A vendor prefix is really just a namespace clash handling tool
here.

> > If there's a strong consensus that we really need this then I'm not
> > going to nack this, but this really needs a pile of acks from
> > compositor folks that they're willing to live with the resulting
> > fallout this will likely bring. Your cc list seems to have an absence
> > of compositor folks, but instead every driver maintainer. That's
> > backwards. We make uapi for userspace, not for kernel driver
> > maintainers!
> 
> Right, but it's mostly about in-kernel rules though? And you're the one
> who mentionned CC'ing the driver maintainers in the first iteration?
> 
> > ltdr; I'd go back to v2. And then cc compositor folks on this to get
> > their ack.
> 
> So, Pekka, Simon, is there anyone else I should Cc?

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Fri, Jun 11, 2021 at 07:54:07AM +0200, Maxime Ripard wrote:
> On Thu, Jun 10, 2021 at 11:00:05PM +0200, Daniel Vetter wrote:
> > On Thu, Jun 10, 2021 at 7:47 PM Maxime Ripard <maxime@cerno.tech> wrote:
> > >
> > > New KMS properties come with a bunch of requirements to avoid each
> > > driver from running their own, inconsistent, set of properties,
> > > eventually leading to issues like property conflicts, inconsistencies
> > > between drivers and semantics, etc.
> > >
> > > Let's document what we expect.
> > >
> > > Cc: Alexandre Belloni <alexandre.belloni@bootlin.com>
> > > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > > Cc: Alex Deucher <alexander.deucher@amd.com>
> > > Cc: Alison Wang <alison.wang@nxp.com>
> > > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > > Cc: Andrew Jeffery <andrew@aj.id.au>
> > > Cc: Andrzej Hajda <a.hajda@samsung.com>
> > > Cc: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
> > > Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> > > Cc: Ben Skeggs <bskeggs@redhat.com>
> > > Cc: Boris Brezillon <bbrezillon@kernel.org>
> > > Cc: Brian Starkey <brian.starkey@arm.com>
> > > Cc: Chen Feng <puck.chen@hisilicon.com>
> > > Cc: Chen-Yu Tsai <wens@csie.org>
> > > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > > Cc: "Christian König" <christian.koenig@amd.com>
> > > Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>
> > > Cc: Edmund Dea <edmund.j.dea@intel.com>
> > > Cc: Eric Anholt <eric@anholt.net>
> > > Cc: Fabio Estevam <festevam@gmail.com>
> > > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > > Cc: Haneen Mohammed <hamohammed.sa@gmail.com>
> > > Cc: Hans de Goede <hdegoede@redhat.com>
> > > Cc: "Heiko Stübner" <heiko@sntech.de>
> > > Cc: Huang Rui <ray.huang@amd.com>
> > > Cc: Hyun Kwon <hyun.kwon@xilinx.com>
> > > Cc: Inki Dae <inki.dae@samsung.com>
> > > Cc: Jani Nikula <jani.nikula@linux.intel.com>
> > > Cc: Jernej Skrabec <jernej.skrabec@siol.net>
> > > Cc: Jerome Brunet <jbrunet@baylibre.com>
> > > Cc: Joel Stanley <joel@jms.id.au>
> > > Cc: John Stultz <john.stultz@linaro.org>
> > > Cc: Jonas Karlman <jonas@kwiboo.se>
> > > Cc: Jonathan Hunter <jonathanh@nvidia.com>
> > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > Cc: Joonyoung Shim <jy0922.shim@samsung.com>
> > > Cc: Jyri Sarha <jyri.sarha@iki.fi>
> > > Cc: Kevin Hilman <khilman@baylibre.com>
> > > Cc: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > > Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> > > Cc: Kyungmin Park <kyungmin.park@samsung.com>
> > > Cc: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
> > > Cc: Linus Walleij <linus.walleij@linaro.org>
> > > Cc: Liviu Dudau <liviu.dudau@arm.com>
> > > Cc: Lucas Stach <l.stach@pengutronix.de>
> > > Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
> > > Cc: Marek Vasut <marex@denx.de>
> > > Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
> > > Cc: Matthias Brugger <matthias.bgg@gmail.com>
> > > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > > Cc: Maxime Ripard <mripard@kernel.org>
> > > Cc: Melissa Wen <melissa.srw@gmail.com>
> > > Cc: Neil Armstrong <narmstrong@baylibre.com>
> > > Cc: Nicolas Ferre <nicolas.ferre@microchip.com>
> > > Cc: "Noralf Trønnes" <noralf@tronnes.org>
> > > Cc: NXP Linux Team <linux-imx@nxp.com>
> > > Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> > > Cc: Paul Cercueil <paul@crapouillou.net>
> > > Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
> > > Cc: Philippe Cornu <philippe.cornu@foss.st.com>
> > > Cc: Philipp Zabel <p.zabel@pengutronix.de>
> > > Cc: Qiang Yu <yuq825@gmail.com>
> > > Cc: Rob Clark <robdclark@gmail.com>
> > > Cc: Robert Foss <robert.foss@linaro.org>
> > > Cc: Rob Herring <robh@kernel.org>
> > > Cc: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Cc: Roland Scheidegger <sroland@vmware.com>
> > > Cc: Russell King <linux@armlinux.org.uk>
> > > Cc: Sam Ravnborg <sam@ravnborg.org>
> > > Cc: Sandy Huang <hjc@rock-chips.com>
> > > Cc: Sascha Hauer <s.hauer@pengutronix.de>
> > > Cc: Sean Paul <sean@poorly.run>
> > > Cc: Seung-Woo Kim <sw0312.kim@samsung.com>
> > > Cc: Shawn Guo <shawnguo@kernel.org>
> > > Cc: Stefan Agner <stefan@agner.ch>
> > > Cc: Steven Price <steven.price@arm.com>
> > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > Cc: Thierry Reding <thierry.reding@gmail.com>
> > > Cc: Tian Tao <tiantao6@hisilicon.com>
> > > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > Cc: Tomi Valkeinen <tomba@kernel.org>
> > > Cc: VMware Graphics <linux-graphics-maintainer@vmware.com>
> > > Cc: Xinliang Liu <xinliang.liu@linaro.org>
> > > Cc: Xinwei Kong <kong.kongxinwei@hisilicon.com>
> > > Cc: Yannick Fertre <yannick.fertre@foss.st.com>
> > > Cc: Zack Rusin <zackr@vmware.com>
> > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> > >
> > > ---
> > >
> > > Changes from v2:
> > >   - Take into account the feedback from Laurent and Lidiu to no longer
> > >     force generic properties, but prefix vendor-specific properties with
> > >     the vendor name
> > 
> > I'm pretty sure my r-b was without this ...
> 
> Yeah, sorry. I wanted to tell you on IRC that you wanted to have a
> second look, but I shouldn't have kept it and caught you by surprise
> indeed.
> 
> > Why exactly do we need this? KMS is meant to be fairly generic (bugs
> > throw a wrench around here sometimes, and semantics can be tricky). If
> > we open up the door to yolo vendor properties in upstream, then that
> > goal is pretty much written off. And we've been there with vendor
> > properties, it's a giantic mess.
> > 
> > Minimally drop my r-b, I'm definitely not in support of this idea.
> 
> So the argument Lidiu and Laurent made was that in some cases, getting a
> generic property right with only a couple of users is hard. So they
> advocated for the right to keep non-generic properties. I can get the
> argument, and no-one else said that was wrong, so it felt like the
> consensus was there.

My argument was two-fold. On one hand, there's the issue of
standardizing properties when we have a single example. I don't have a
good solution for that chicken-and-egg issue, when we tell vendor they
should standardize properties, and when they ask how, we tell them we
don't know as it hasn't been done before. The option for allowing a
staging playground for vendor properties here isn't something I like,
but we'll need something similar one way or another. Perhaps at the cost
of not guaranteeing userspace ABI for those properties, and converting
them later to something standard (it won't be received well by vendor of
course, and will make a push for mainline more difficult to sell, so
it's not a great solution).

On the other hand, there are esoteric vendor-specific features for which
standardization really doesn't make sense. For those, I think
vendor-specific properties are fine, as long as they're properly
documented, with a design documentation merged at the same time as the
property. A vendor prefix is really just a namespace clash handling tool
here.

> > If there's a strong consensus that we really need this then I'm not
> > going to nack this, but this really needs a pile of acks from
> > compositor folks that they're willing to live with the resulting
> > fallout this will likely bring. Your cc list seems to have an absence
> > of compositor folks, but instead every driver maintainer. That's
> > backwards. We make uapi for userspace, not for kernel driver
> > maintainers!
> 
> Right, but it's mostly about in-kernel rules though? And you're the one
> who mentionned CC'ing the driver maintainers in the first iteration?
> 
> > ltdr; I'd go back to v2. And then cc compositor folks on this to get
> > their ack.
> 
> So, Pekka, Simon, is there anyone else I should Cc?

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Fri, Jun 11, 2021 at 02:34:18PM +0100, Liviu Dudau wrote:
> On Fri, Jun 11, 2021 at 08:56:04AM -0400, Alyssa Rosenzweig wrote:
> > > What I'm expected to see in the future is new functionality that gets implemented by
> > > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > > could be that the new property is generic, but there is no way of testing that on
> > > more than one implementation yet, so I'd say we are generous calling it "standard
> > > property". When the second or third hardware vendor comes along and starts supporting
> > > that property with their own set of extra requirements, then we can call it
> > > "standard". Then comes the effort cost: would it be easier to start with a vendor
> > > property that only the vendor needs to support (and can submit patches into the
> > > compositors to do so) and when the standard property gets added moves to that, or
> > > should we start with a generic property that gets implemented by the compositors
> > > (maybe, but then only one vendor supports it) and then later when we actually
> > > standardise the property we will have to carry backwards compatibility code in the
> > > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > > the former option to be reflected in the documentation, but I would like to hear your
> > > thoughts.
> > 
> > Just my 2c - if the mainline kernel isn't willing to commit to a feature
> > for upstream userspace to use, why does that feature belong in the
> > kernel at all? I don't see much value in exposing hardware for the sake
> > of exposing it when, practically, Linux userspace /can't/ use it as-is.
> > 
> > Might these vendor properties be used on downstream Android userspaces?
> > That's not generally an upstream goal to support.
> 
> I think the assumption is that we are willing to commit to supporting a feature for
> userspace, just that (I personally) lack the confidence that I will be getting the
> feature right on the first attempt and using only one vendor hardware. And that
> supporting potential mistakes I might've made in the first version is harder if the
> feature was deemed "standard".

There's also the issue that the first developer to try to upstream a
standard property for a new feature often gets ignored as nobody else
has experience with that feature, and thus lack personal interest.
That's not a pure technical issue, there's a management problem there
too. The right solution may ne to figure out a good process to
standardize new property without making it so difficult that everybody
will give up and only use downstream kernels. I've heard too many times
already from vendors that upstream isn't something they can target as
the bar to entry is too high, and when I convince them to submit patches
to extend APIs, those patches get ignored (I'm not talking about DRM/KMS
only here).

> I'm talking from my experience with the writeback connector. We almost committed the
> feature twice before more people chipped in and asked us for changes, but that was lucky.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Fri, Jun 11, 2021 at 02:34:18PM +0100, Liviu Dudau wrote:
> On Fri, Jun 11, 2021 at 08:56:04AM -0400, Alyssa Rosenzweig wrote:
> > > What I'm expected to see in the future is new functionality that gets implemented by
> > > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > > could be that the new property is generic, but there is no way of testing that on
> > > more than one implementation yet, so I'd say we are generous calling it "standard
> > > property". When the second or third hardware vendor comes along and starts supporting
> > > that property with their own set of extra requirements, then we can call it
> > > "standard". Then comes the effort cost: would it be easier to start with a vendor
> > > property that only the vendor needs to support (and can submit patches into the
> > > compositors to do so) and when the standard property gets added moves to that, or
> > > should we start with a generic property that gets implemented by the compositors
> > > (maybe, but then only one vendor supports it) and then later when we actually
> > > standardise the property we will have to carry backwards compatibility code in the
> > > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > > the former option to be reflected in the documentation, but I would like to hear your
> > > thoughts.
> > 
> > Just my 2c - if the mainline kernel isn't willing to commit to a feature
> > for upstream userspace to use, why does that feature belong in the
> > kernel at all? I don't see much value in exposing hardware for the sake
> > of exposing it when, practically, Linux userspace /can't/ use it as-is.
> > 
> > Might these vendor properties be used on downstream Android userspaces?
> > That's not generally an upstream goal to support.
> 
> I think the assumption is that we are willing to commit to supporting a feature for
> userspace, just that (I personally) lack the confidence that I will be getting the
> feature right on the first attempt and using only one vendor hardware. And that
> supporting potential mistakes I might've made in the first version is harder if the
> feature was deemed "standard".

There's also the issue that the first developer to try to upstream a
standard property for a new feature often gets ignored as nobody else
has experience with that feature, and thus lack personal interest.
That's not a pure technical issue, there's a management problem there
too. The right solution may ne to figure out a good process to
standardize new property without making it so difficult that everybody
will give up and only use downstream kernels. I've heard too many times
already from vendors that upstream isn't something they can target as
the bar to entry is too high, and when I convince them to submit patches
to extend APIs, those patches get ignored (I'm not talking about DRM/KMS
only here).

> I'm talking from my experience with the writeback connector. We almost committed the
> feature twice before more people chipped in and asked us for changes, but that was lucky.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Mon, Jun 14, 2021 at 04:24:13PM +0100, Liviu Dudau wrote:
> On Mon, Jun 14, 2021 at 05:49:12PM +0300, Pekka Paalanen wrote:
> > On Fri, 11 Jun 2021 13:03:09 +0100
> > Liviu Dudau <liviu.dudau@arm.com> wrote:
> > 
> > > On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> > > > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > >   
> > > > > If there's a strong consensus that we really need this then I'm not
> > > > > going to nack this, but this really needs a pile of acks from
> > > > > compositor folks that they're willing to live with the resulting
> > > > > fallout this will likely bring. Your cc list seems to have an absence
> > > > > of compositor folks, but instead every driver maintainer. That's
> > > > > backwards. We make uapi for userspace, not for kernel driver
> > > > > maintainers!  
> > > > 
> > > > In wlroots we have a policy of only allowing standard KMS properties to
> > > > be used. Any vendor-specific property is going to be less well-defined,
> > > > less widely useful, potentially have more design issues, potentially
> > > > overlap in functionality with other vendor-specific properties, likely
> > > > have some hardware-specific assumptions, etc.
> > > > 
> > > > What matters here is discussing with other driver & user-space folks to
> > > > make sure the new property's design is sound. Designing uAPI is hard.
> > > > 
> > > > If kernel folks are struggling with a user-space implementation, they
> > > > should discuss with user-space folks to see which project would be
> > > > interested. There's a chance a compositor will be interested in the new
> > > > property and will just do the user-space part for you, if not we can
> > > > suggest candidate projects.
> > > > 
> > > > tl;dr strong agree with Daniel here.  
> > > 
> > > I think the assumption you and Daniel are making is that the first implementation of
> > > a new KMS property can be made standard from day one and that it will work for any
> > > late comer driver as is, without having to make changes to its behaviour in a
> > > significant way. In my experience that is not the case.
> > > 
> > > I think we have moved from the times when we were trying to implement in the Linux
> > > world features that were available in the hardware but needed a kernel and userspace
> > > API. The set of properties that exist in KMS cover a lot of needed functionality and
> > > I don't expect to see new properties for stuff that is already supported by hardware.
> > > 
> > > What I'm expected to see in the future is new functionality that gets implemented by
> > > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > > could be that the new property is generic, but there is no way of testing that on
> > > more than one implementation yet, so I'd say we are generous calling it "standard
> > > property". When the second or third hardware vendor comes along and starts supporting
> > > that property with their own set of extra requirements, then we can call it
> > > "standard".
> > 
> > I agree that is a problem with trying to make generic anything. But it
> > does not mean you should not even try. Maybe trying really hard saves a
> > couple revisions.
> 
> Agree.
> 
> > What I think should be planned for is revisions. How to add new
> > properties that do the same thing but better, while documenting that a
> > userspace KMS client can use only one revision at a time. You never
> > remove old revisions, unless maybe with a DRM client cap they
> > could disappear from that file description if that is necessary for
> > seeing the new revision.
> > 
> > While designing this, one also needs to take into account that KMS
> > clients need to be able to save and restore properties *they do not
> > understand*. So exposing two revisions of the same feature
> > simultaneously would break save/restore is that's an error.
> 
> I quite like the idea of having versions for properties.

It's an interesting idea. We'll need to consider bundles of properties
in that case I think, as in a v1 a feature could be implemented by
properties A, B and C, while in v2 C could be replaced by D and E
(beside A and B themselves also being changed in v2).

> > > Then comes the effort cost: would it be easier to start with a vendor
> > > property that only the vendor needs to support (and can submit patches into the
> > > compositors to do so) and when the standard property gets added moves to that, or
> > 
> > But you can't move, you can only add? You can't delete the old property
> > in kernel if it was ever released with a kernel and anyone used it. In
> > the same sentence you also imply that there is a user of it, so
> > removing it will break that user. Then you'll have to track the
> > userspace lifetime to figure out which decade you can try removing it.
> 
> Not that I am supporting the workflow, but I was trying to address the comments that
> vendors are going to push their own userspace implementation for their vendor
> properties. If that is the case, when they switch to the standard ones they can drop
> the support in userspace for their changes. With the implied assumption that you will
> have fewer vendor implementations hence easier to make changes, KMS properties can be
> deleted if you know there is no user of them (e.g. the vendor has upgraded all their
> software to the standard property).
> 
> > > should we start with a generic property that gets implemented by the compositors
> > > (maybe, but then only one vendor supports it) and then later when we actually
> > > standardise the property we will have to carry backwards compatibility code in the
> > > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > > the former option to be reflected in the documentation, but I would like to hear your
> > > thoughts.
> > 
> > You have to carry the backward compatibility in all cases, right?
> > 
> > Userspace OTOH can drop support for older less supported KMS properties
> > while taking advantage of a new revision. Userspace is not required to
> > support old kernels forever.
> > 
> > 
> > Here's a wild counter-proposal off a tangent:
> > 
> > How about we make "implemented in and testable with VKMS" the rule,
> > instead of "is generic" for new properties?
> > 
> > VKMS is what compositors (will) use in CI. I would feel hugely less bad
> > about using a property that only one hardware driver ever implements,
> > if also VKMS implements it in a way that compositor CI can observe it
> > working.
> > 
> > I don't expect this proposal to be accepted, but it's food for thought.
> > The major problem for compositor projects is testing as you usually
> > don't have the hardware, IMO. CI tends to not have any hardware.
> 
> While I don't dislike the proposal (I think it is quite sensible), I am worried that
> for some behaviours VKMS will implement them in a quirky way. To pick (again) the
> example of writeback, real hardware will have a way to tell if the buffer has been
> sent successfully to memory and it might take more than one refresh period, while
> VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If
> you code your compositor based on VKMS you might get unexpected artifacts on real
> hardware.

I'm also worried about properties that related to hardware processing of
frames, for instance related to colour spaces, or even scaling. VKMS has
a software composer implemented in the kernel, it may get fairly
complicated if we need to implement all kind of processing.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Mon, Jun 14, 2021 at 04:24:13PM +0100, Liviu Dudau wrote:
> On Mon, Jun 14, 2021 at 05:49:12PM +0300, Pekka Paalanen wrote:
> > On Fri, 11 Jun 2021 13:03:09 +0100
> > Liviu Dudau <liviu.dudau@arm.com> wrote:
> > 
> > > On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:
> > > > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > >   
> > > > > If there's a strong consensus that we really need this then I'm not
> > > > > going to nack this, but this really needs a pile of acks from
> > > > > compositor folks that they're willing to live with the resulting
> > > > > fallout this will likely bring. Your cc list seems to have an absence
> > > > > of compositor folks, but instead every driver maintainer. That's
> > > > > backwards. We make uapi for userspace, not for kernel driver
> > > > > maintainers!  
> > > > 
> > > > In wlroots we have a policy of only allowing standard KMS properties to
> > > > be used. Any vendor-specific property is going to be less well-defined,
> > > > less widely useful, potentially have more design issues, potentially
> > > > overlap in functionality with other vendor-specific properties, likely
> > > > have some hardware-specific assumptions, etc.
> > > > 
> > > > What matters here is discussing with other driver & user-space folks to
> > > > make sure the new property's design is sound. Designing uAPI is hard.
> > > > 
> > > > If kernel folks are struggling with a user-space implementation, they
> > > > should discuss with user-space folks to see which project would be
> > > > interested. There's a chance a compositor will be interested in the new
> > > > property and will just do the user-space part for you, if not we can
> > > > suggest candidate projects.
> > > > 
> > > > tl;dr strong agree with Daniel here.  
> > > 
> > > I think the assumption you and Daniel are making is that the first implementation of
> > > a new KMS property can be made standard from day one and that it will work for any
> > > late comer driver as is, without having to make changes to its behaviour in a
> > > significant way. In my experience that is not the case.
> > > 
> > > I think we have moved from the times when we were trying to implement in the Linux
> > > world features that were available in the hardware but needed a kernel and userspace
> > > API. The set of properties that exist in KMS cover a lot of needed functionality and
> > > I don't expect to see new properties for stuff that is already supported by hardware.
> > > 
> > > What I'm expected to see in the future is new functionality that gets implemented by
> > > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > > could be that the new property is generic, but there is no way of testing that on
> > > more than one implementation yet, so I'd say we are generous calling it "standard
> > > property". When the second or third hardware vendor comes along and starts supporting
> > > that property with their own set of extra requirements, then we can call it
> > > "standard".
> > 
> > I agree that is a problem with trying to make generic anything. But it
> > does not mean you should not even try. Maybe trying really hard saves a
> > couple revisions.
> 
> Agree.
> 
> > What I think should be planned for is revisions. How to add new
> > properties that do the same thing but better, while documenting that a
> > userspace KMS client can use only one revision at a time. You never
> > remove old revisions, unless maybe with a DRM client cap they
> > could disappear from that file description if that is necessary for
> > seeing the new revision.
> > 
> > While designing this, one also needs to take into account that KMS
> > clients need to be able to save and restore properties *they do not
> > understand*. So exposing two revisions of the same feature
> > simultaneously would break save/restore is that's an error.
> 
> I quite like the idea of having versions for properties.

It's an interesting idea. We'll need to consider bundles of properties
in that case I think, as in a v1 a feature could be implemented by
properties A, B and C, while in v2 C could be replaced by D and E
(beside A and B themselves also being changed in v2).

> > > Then comes the effort cost: would it be easier to start with a vendor
> > > property that only the vendor needs to support (and can submit patches into the
> > > compositors to do so) and when the standard property gets added moves to that, or
> > 
> > But you can't move, you can only add? You can't delete the old property
> > in kernel if it was ever released with a kernel and anyone used it. In
> > the same sentence you also imply that there is a user of it, so
> > removing it will break that user. Then you'll have to track the
> > userspace lifetime to figure out which decade you can try removing it.
> 
> Not that I am supporting the workflow, but I was trying to address the comments that
> vendors are going to push their own userspace implementation for their vendor
> properties. If that is the case, when they switch to the standard ones they can drop
> the support in userspace for their changes. With the implied assumption that you will
> have fewer vendor implementations hence easier to make changes, KMS properties can be
> deleted if you know there is no user of them (e.g. the vendor has upgraded all their
> software to the standard property).
> 
> > > should we start with a generic property that gets implemented by the compositors
> > > (maybe, but then only one vendor supports it) and then later when we actually
> > > standardise the property we will have to carry backwards compatibility code in the
> > > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > > the former option to be reflected in the documentation, but I would like to hear your
> > > thoughts.
> > 
> > You have to carry the backward compatibility in all cases, right?
> > 
> > Userspace OTOH can drop support for older less supported KMS properties
> > while taking advantage of a new revision. Userspace is not required to
> > support old kernels forever.
> > 
> > 
> > Here's a wild counter-proposal off a tangent:
> > 
> > How about we make "implemented in and testable with VKMS" the rule,
> > instead of "is generic" for new properties?
> > 
> > VKMS is what compositors (will) use in CI. I would feel hugely less bad
> > about using a property that only one hardware driver ever implements,
> > if also VKMS implements it in a way that compositor CI can observe it
> > working.
> > 
> > I don't expect this proposal to be accepted, but it's food for thought.
> > The major problem for compositor projects is testing as you usually
> > don't have the hardware, IMO. CI tends to not have any hardware.
> 
> While I don't dislike the proposal (I think it is quite sensible), I am worried that
> for some behaviours VKMS will implement them in a quirky way. To pick (again) the
> example of writeback, real hardware will have a way to tell if the buffer has been
> sent successfully to memory and it might take more than one refresh period, while
> VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If
> you code your compositor based on VKMS you might get unexpected artifacts on real
> hardware.

I'm also worried about properties that related to hardware processing of
frames, for instance related to colour spaces, or even scaling. VKMS has
a software composer implemented in the kernel, it may get fairly
complicated if we need to implement all kind of processing.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 9583 bytes --]

On Mon, 14 Jun 2021 19:33:47 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> On Mon, Jun 14, 2021 at 04:24:13PM +0100, Liviu Dudau wrote:
> > On Mon, Jun 14, 2021 at 05:49:12PM +0300, Pekka Paalanen wrote:  
> > > On Fri, 11 Jun 2021 13:03:09 +0100
> > > Liviu Dudau <liviu.dudau@arm.com> wrote:
> > >   
> > > > On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:  
> > > > > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > >     
> > > > > > If there's a strong consensus that we really need this then I'm not
> > > > > > going to nack this, but this really needs a pile of acks from
> > > > > > compositor folks that they're willing to live with the resulting
> > > > > > fallout this will likely bring. Your cc list seems to have an absence
> > > > > > of compositor folks, but instead every driver maintainer. That's
> > > > > > backwards. We make uapi for userspace, not for kernel driver
> > > > > > maintainers!    
> > > > > 
> > > > > In wlroots we have a policy of only allowing standard KMS properties to
> > > > > be used. Any vendor-specific property is going to be less well-defined,
> > > > > less widely useful, potentially have more design issues, potentially
> > > > > overlap in functionality with other vendor-specific properties, likely
> > > > > have some hardware-specific assumptions, etc.
> > > > > 
> > > > > What matters here is discussing with other driver & user-space folks to
> > > > > make sure the new property's design is sound. Designing uAPI is hard.
> > > > > 
> > > > > If kernel folks are struggling with a user-space implementation, they
> > > > > should discuss with user-space folks to see which project would be
> > > > > interested. There's a chance a compositor will be interested in the new
> > > > > property and will just do the user-space part for you, if not we can
> > > > > suggest candidate projects.
> > > > > 
> > > > > tl;dr strong agree with Daniel here.    
> > > > 
> > > > I think the assumption you and Daniel are making is that the first implementation of
> > > > a new KMS property can be made standard from day one and that it will work for any
> > > > late comer driver as is, without having to make changes to its behaviour in a
> > > > significant way. In my experience that is not the case.
> > > > 
> > > > I think we have moved from the times when we were trying to implement in the Linux
> > > > world features that were available in the hardware but needed a kernel and userspace
> > > > API. The set of properties that exist in KMS cover a lot of needed functionality and
> > > > I don't expect to see new properties for stuff that is already supported by hardware.
> > > > 
> > > > What I'm expected to see in the future is new functionality that gets implemented by
> > > > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > > > could be that the new property is generic, but there is no way of testing that on
> > > > more than one implementation yet, so I'd say we are generous calling it "standard
> > > > property". When the second or third hardware vendor comes along and starts supporting
> > > > that property with their own set of extra requirements, then we can call it
> > > > "standard".  
> > > 
> > > I agree that is a problem with trying to make generic anything. But it
> > > does not mean you should not even try. Maybe trying really hard saves a
> > > couple revisions.  
> > 
> > Agree.
> >   
> > > What I think should be planned for is revisions. How to add new
> > > properties that do the same thing but better, while documenting that a
> > > userspace KMS client can use only one revision at a time. You never
> > > remove old revisions, unless maybe with a DRM client cap they
> > > could disappear from that file description if that is necessary for
> > > seeing the new revision.
> > > 
> > > While designing this, one also needs to take into account that KMS
> > > clients need to be able to save and restore properties *they do not
> > > understand*. So exposing two revisions of the same feature
> > > simultaneously would break save/restore is that's an error.  
> > 
> > I quite like the idea of having versions for properties.  
> 
> It's an interesting idea. We'll need to consider bundles of properties
> in that case I think, as in a v1 a feature could be implemented by
> properties A, B and C, while in v2 C could be replaced by D and E
> (beside A and B themselves also being changed in v2).
> 
> > > > Then comes the effort cost: would it be easier to start with a vendor
> > > > property that only the vendor needs to support (and can submit patches into the
> > > > compositors to do so) and when the standard property gets added moves to that, or  
> > > 
> > > But you can't move, you can only add? You can't delete the old property
> > > in kernel if it was ever released with a kernel and anyone used it. In
> > > the same sentence you also imply that there is a user of it, so
> > > removing it will break that user. Then you'll have to track the
> > > userspace lifetime to figure out which decade you can try removing it.  
> > 
> > Not that I am supporting the workflow, but I was trying to address the comments that
> > vendors are going to push their own userspace implementation for their vendor
> > properties. If that is the case, when they switch to the standard ones they can drop
> > the support in userspace for their changes. With the implied assumption that you will
> > have fewer vendor implementations hence easier to make changes, KMS properties can be
> > deleted if you know there is no user of them (e.g. the vendor has upgraded all their
> > software to the standard property).
> >   
> > > > should we start with a generic property that gets implemented by the compositors
> > > > (maybe, but then only one vendor supports it) and then later when we actually
> > > > standardise the property we will have to carry backwards compatibility code in the
> > > > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > > > the former option to be reflected in the documentation, but I would like to hear your
> > > > thoughts.  
> > > 
> > > You have to carry the backward compatibility in all cases, right?
> > > 
> > > Userspace OTOH can drop support for older less supported KMS properties
> > > while taking advantage of a new revision. Userspace is not required to
> > > support old kernels forever.
> > > 
> > > 
> > > Here's a wild counter-proposal off a tangent:
> > > 
> > > How about we make "implemented in and testable with VKMS" the rule,
> > > instead of "is generic" for new properties?
> > > 
> > > VKMS is what compositors (will) use in CI. I would feel hugely less bad
> > > about using a property that only one hardware driver ever implements,
> > > if also VKMS implements it in a way that compositor CI can observe it
> > > working.
> > > 
> > > I don't expect this proposal to be accepted, but it's food for thought.
> > > The major problem for compositor projects is testing as you usually
> > > don't have the hardware, IMO. CI tends to not have any hardware.  
> > 
> > While I don't dislike the proposal (I think it is quite sensible), I am worried that
> > for some behaviours VKMS will implement them in a quirky way. To pick (again) the
> > example of writeback, real hardware will have a way to tell if the buffer has been
> > sent successfully to memory and it might take more than one refresh period, while
> > VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If
> > you code your compositor based on VKMS you might get unexpected artifacts on real
> > hardware.  
> 
> I'm also worried about properties that related to hardware processing of
> frames, for instance related to colour spaces, or even scaling. VKMS has
> a software composer implemented in the kernel, it may get fairly
> complicated if we need to implement all kind of processing.

Hi,

indeed it will, but what else could one do to test userspace KMS
clients in generic CI where all you can have is virtual hardware? Maybe
in the long run VKMS needs to loop back to a userspace daemon that
implements all the complex processing and returns the writeback result
via VKMS again? That daemon would then need a single upstream, like the
kernel, where it is maintained and correctness verified.

Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
stuff in userspace? Didn't someone already have something like that?
It would need to be lifted to be a required part of kernel UAPI
submissions, I suppose like IGT is nowadays.

Requiring a software implementation for testing has another huge
benefit (or disadvantage, depending on who you ask): "hardware does
secret magic" would not fly anymore. The software implementation would
need to deliver the same results, or at least one would need to define
the tolerances for e.g. color conversions. Otherwise userspace
compositor test suites might pass on the software implementation and
fail on hardware implementations. For compositor developers like me
knowing the exact formulas would be a huge benefit as it would allow me
to use KMS to off-load precision-sensitive operations (e.g.
professional color management). Otherwise, compositors probably need a
switch: "high quality color management? Then do not use KMS features."


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 9583 bytes --]

On Mon, 14 Jun 2021 19:33:47 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> On Mon, Jun 14, 2021 at 04:24:13PM +0100, Liviu Dudau wrote:
> > On Mon, Jun 14, 2021 at 05:49:12PM +0300, Pekka Paalanen wrote:  
> > > On Fri, 11 Jun 2021 13:03:09 +0100
> > > Liviu Dudau <liviu.dudau@arm.com> wrote:
> > >   
> > > > On Fri, Jun 11, 2021 at 08:14:59AM +0000, Simon Ser wrote:  
> > > > > On Thursday, June 10th, 2021 at 23:00, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > >     
> > > > > > If there's a strong consensus that we really need this then I'm not
> > > > > > going to nack this, but this really needs a pile of acks from
> > > > > > compositor folks that they're willing to live with the resulting
> > > > > > fallout this will likely bring. Your cc list seems to have an absence
> > > > > > of compositor folks, but instead every driver maintainer. That's
> > > > > > backwards. We make uapi for userspace, not for kernel driver
> > > > > > maintainers!    
> > > > > 
> > > > > In wlroots we have a policy of only allowing standard KMS properties to
> > > > > be used. Any vendor-specific property is going to be less well-defined,
> > > > > less widely useful, potentially have more design issues, potentially
> > > > > overlap in functionality with other vendor-specific properties, likely
> > > > > have some hardware-specific assumptions, etc.
> > > > > 
> > > > > What matters here is discussing with other driver & user-space folks to
> > > > > make sure the new property's design is sound. Designing uAPI is hard.
> > > > > 
> > > > > If kernel folks are struggling with a user-space implementation, they
> > > > > should discuss with user-space folks to see which project would be
> > > > > interested. There's a chance a compositor will be interested in the new
> > > > > property and will just do the user-space part for you, if not we can
> > > > > suggest candidate projects.
> > > > > 
> > > > > tl;dr strong agree with Daniel here.    
> > > > 
> > > > I think the assumption you and Daniel are making is that the first implementation of
> > > > a new KMS property can be made standard from day one and that it will work for any
> > > > late comer driver as is, without having to make changes to its behaviour in a
> > > > significant way. In my experience that is not the case.
> > > > 
> > > > I think we have moved from the times when we were trying to implement in the Linux
> > > > world features that were available in the hardware but needed a kernel and userspace
> > > > API. The set of properties that exist in KMS cover a lot of needed functionality and
> > > > I don't expect to see new properties for stuff that is already supported by hardware.
> > > > 
> > > > What I'm expected to see in the future is new functionality that gets implemented by
> > > > one hardware vendor and the kernel developers trying to enable that for userspace. It
> > > > could be that the new property is generic, but there is no way of testing that on
> > > > more than one implementation yet, so I'd say we are generous calling it "standard
> > > > property". When the second or third hardware vendor comes along and starts supporting
> > > > that property with their own set of extra requirements, then we can call it
> > > > "standard".  
> > > 
> > > I agree that is a problem with trying to make generic anything. But it
> > > does not mean you should not even try. Maybe trying really hard saves a
> > > couple revisions.  
> > 
> > Agree.
> >   
> > > What I think should be planned for is revisions. How to add new
> > > properties that do the same thing but better, while documenting that a
> > > userspace KMS client can use only one revision at a time. You never
> > > remove old revisions, unless maybe with a DRM client cap they
> > > could disappear from that file description if that is necessary for
> > > seeing the new revision.
> > > 
> > > While designing this, one also needs to take into account that KMS
> > > clients need to be able to save and restore properties *they do not
> > > understand*. So exposing two revisions of the same feature
> > > simultaneously would break save/restore is that's an error.  
> > 
> > I quite like the idea of having versions for properties.  
> 
> It's an interesting idea. We'll need to consider bundles of properties
> in that case I think, as in a v1 a feature could be implemented by
> properties A, B and C, while in v2 C could be replaced by D and E
> (beside A and B themselves also being changed in v2).
> 
> > > > Then comes the effort cost: would it be easier to start with a vendor
> > > > property that only the vendor needs to support (and can submit patches into the
> > > > compositors to do so) and when the standard property gets added moves to that, or  
> > > 
> > > But you can't move, you can only add? You can't delete the old property
> > > in kernel if it was ever released with a kernel and anyone used it. In
> > > the same sentence you also imply that there is a user of it, so
> > > removing it will break that user. Then you'll have to track the
> > > userspace lifetime to figure out which decade you can try removing it.  
> > 
> > Not that I am supporting the workflow, but I was trying to address the comments that
> > vendors are going to push their own userspace implementation for their vendor
> > properties. If that is the case, when they switch to the standard ones they can drop
> > the support in userspace for their changes. With the implied assumption that you will
> > have fewer vendor implementations hence easier to make changes, KMS properties can be
> > deleted if you know there is no user of them (e.g. the vendor has upgraded all their
> > software to the standard property).
> >   
> > > > should we start with a generic property that gets implemented by the compositors
> > > > (maybe, but then only one vendor supports it) and then later when we actually
> > > > standardise the property we will have to carry backwards compatibility code in the
> > > > kernel to handle the old behaviour for old userspace? My proposal to Maxime was for
> > > > the former option to be reflected in the documentation, but I would like to hear your
> > > > thoughts.  
> > > 
> > > You have to carry the backward compatibility in all cases, right?
> > > 
> > > Userspace OTOH can drop support for older less supported KMS properties
> > > while taking advantage of a new revision. Userspace is not required to
> > > support old kernels forever.
> > > 
> > > 
> > > Here's a wild counter-proposal off a tangent:
> > > 
> > > How about we make "implemented in and testable with VKMS" the rule,
> > > instead of "is generic" for new properties?
> > > 
> > > VKMS is what compositors (will) use in CI. I would feel hugely less bad
> > > about using a property that only one hardware driver ever implements,
> > > if also VKMS implements it in a way that compositor CI can observe it
> > > working.
> > > 
> > > I don't expect this proposal to be accepted, but it's food for thought.
> > > The major problem for compositor projects is testing as you usually
> > > don't have the hardware, IMO. CI tends to not have any hardware.  
> > 
> > While I don't dislike the proposal (I think it is quite sensible), I am worried that
> > for some behaviours VKMS will implement them in a quirky way. To pick (again) the
> > example of writeback, real hardware will have a way to tell if the buffer has been
> > sent successfully to memory and it might take more than one refresh period, while
> > VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If
> > you code your compositor based on VKMS you might get unexpected artifacts on real
> > hardware.  
> 
> I'm also worried about properties that related to hardware processing of
> frames, for instance related to colour spaces, or even scaling. VKMS has
> a software composer implemented in the kernel, it may get fairly
> complicated if we need to implement all kind of processing.

Hi,

indeed it will, but what else could one do to test userspace KMS
clients in generic CI where all you can have is virtual hardware? Maybe
in the long run VKMS needs to loop back to a userspace daemon that
implements all the complex processing and returns the writeback result
via VKMS again? That daemon would then need a single upstream, like the
kernel, where it is maintained and correctness verified.

Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
stuff in userspace? Didn't someone already have something like that?
It would need to be lifted to be a required part of kernel UAPI
submissions, I suppose like IGT is nowadays.

Requiring a software implementation for testing has another huge
benefit (or disadvantage, depending on who you ask): "hardware does
secret magic" would not fly anymore. The software implementation would
need to deliver the same results, or at least one would need to define
the tolerances for e.g. color conversions. Otherwise userspace
compositor test suites might pass on the software implementation and
fail on hardware implementations. For compositor developers like me
knowing the exact formulas would be a huge benefit as it would allow me
to use KMS to off-load precision-sensitive operations (e.g.
professional color management). Otherwise, compositors probably need a
switch: "high quality color management? Then do not use KMS features."


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Simon Ser]

On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen <ppaalanen@gmail.com> wrote:

> indeed it will, but what else could one do to test userspace KMS
> clients in generic CI where all you can have is virtual hardware? Maybe
> in the long run VKMS needs to loop back to a userspace daemon that
> implements all the complex processing and returns the writeback result
> via VKMS again? That daemon would then need a single upstream, like the
> kernel, where it is maintained and correctness verified.

The complex processing must be implemented even without write-back, because
user-space can ask for CRCs of the CRTC.

> Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> stuff in userspace? Didn't someone already have something like that?
> It would need to be lifted to be a required part of kernel UAPI
> submissions, I suppose like IGT is nowadays.

FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
software implementation with write-back connectors, but allows to expose
virtual planes and check atomic commits in CI.

[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c

> For compositor developers like me knowing the exact formulas would be a huge
> benefit as it would allow me to use KMS to off-load precision-sensitive
> operations (e.g.  professional color management). Otherwise, compositors
> probably need a switch: "high quality color management? Then do not use KMS
> features."

I think for alpha blending there are already rounding issues depending on the
hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
exact same formulae for color management stuff.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Simon Ser]

On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen <ppaalanen@gmail.com> wrote:

> indeed it will, but what else could one do to test userspace KMS
> clients in generic CI where all you can have is virtual hardware? Maybe
> in the long run VKMS needs to loop back to a userspace daemon that
> implements all the complex processing and returns the writeback result
> via VKMS again? That daemon would then need a single upstream, like the
> kernel, where it is maintained and correctness verified.

The complex processing must be implemented even without write-back, because
user-space can ask for CRCs of the CRTC.

> Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> stuff in userspace? Didn't someone already have something like that?
> It would need to be lifted to be a required part of kernel UAPI
> submissions, I suppose like IGT is nowadays.

FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
software implementation with write-back connectors, but allows to expose
virtual planes and check atomic commits in CI.

[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c

> For compositor developers like me knowing the exact formulas would be a huge
> benefit as it would allow me to use KMS to off-load precision-sensitive
> operations (e.g.  professional color management). Otherwise, compositors
> probably need a switch: "high quality color management? Then do not use KMS
> features."

I think for alpha blending there are already rounding issues depending on the
hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
exact same formulae for color management stuff.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:
> On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen <ppaalanen@gmail.com> wrote:
> 
> > indeed it will, but what else could one do to test userspace KMS
> > clients in generic CI where all you can have is virtual hardware? Maybe
> > in the long run VKMS needs to loop back to a userspace daemon that
> > implements all the complex processing and returns the writeback result
> > via VKMS again? That daemon would then need a single upstream, like the
> > kernel, where it is maintained and correctness verified.
> 
> The complex processing must be implemented even without write-back, because
> user-space can ask for CRCs of the CRTC.
> 
> > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > stuff in userspace? Didn't someone already have something like that?
> > It would need to be lifted to be a required part of kernel UAPI
> > submissions, I suppose like IGT is nowadays.
> 
> FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> software implementation with write-back connectors, but allows to expose
> virtual planes and check atomic commits in CI.
> 
> [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> 
> > For compositor developers like me knowing the exact formulas would be a huge
> > benefit as it would allow me to use KMS to off-load precision-sensitive
> > operations (e.g.  professional color management). Otherwise, compositors
> > probably need a switch: "high quality color management? Then do not use KMS
> > features."
> 
> I think for alpha blending there are already rounding issues depending on the
> hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> exact same formulae for color management stuff.

Good, because otherwise you would be very quickly disappointed :-)

For scaling we would also need to replicate the exact same filter taps,
which are often not documented.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:
> On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen <ppaalanen@gmail.com> wrote:
> 
> > indeed it will, but what else could one do to test userspace KMS
> > clients in generic CI where all you can have is virtual hardware? Maybe
> > in the long run VKMS needs to loop back to a userspace daemon that
> > implements all the complex processing and returns the writeback result
> > via VKMS again? That daemon would then need a single upstream, like the
> > kernel, where it is maintained and correctness verified.
> 
> The complex processing must be implemented even without write-back, because
> user-space can ask for CRCs of the CRTC.
> 
> > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > stuff in userspace? Didn't someone already have something like that?
> > It would need to be lifted to be a required part of kernel UAPI
> > submissions, I suppose like IGT is nowadays.
> 
> FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> software implementation with write-back connectors, but allows to expose
> virtual planes and check atomic commits in CI.
> 
> [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> 
> > For compositor developers like me knowing the exact formulas would be a huge
> > benefit as it would allow me to use KMS to off-load precision-sensitive
> > operations (e.g.  professional color management). Otherwise, compositors
> > probably need a switch: "high quality color management? Then do not use KMS
> > features."
> 
> I think for alpha blending there are already rounding issues depending on the
> hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> exact same formulae for color management stuff.

Good, because otherwise you would be very quickly disappointed :-)

For scaling we would also need to replicate the exact same filter taps,
which are often not documented.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 2521 bytes --]

On Tue, 15 Jun 2021 12:45:57 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:
> > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen <ppaalanen@gmail.com> wrote:
> >   
> > > indeed it will, but what else could one do to test userspace KMS
> > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > in the long run VKMS needs to loop back to a userspace daemon that
> > > implements all the complex processing and returns the writeback result
> > > via VKMS again? That daemon would then need a single upstream, like the
> > > kernel, where it is maintained and correctness verified.  
> > 
> > The complex processing must be implemented even without write-back, because
> > user-space can ask for CRCs of the CRTC.
> >   
> > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > stuff in userspace? Didn't someone already have something like that?
> > > It would need to be lifted to be a required part of kernel UAPI
> > > submissions, I suppose like IGT is nowadays.  
> > 
> > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > software implementation with write-back connectors, but allows to expose
> > virtual planes and check atomic commits in CI.
> > 
> > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> >   
> > > For compositor developers like me knowing the exact formulas would be a huge
> > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > operations (e.g.  professional color management). Otherwise, compositors
> > > probably need a switch: "high quality color management? Then do not use KMS
> > > features."  
> > 
> > I think for alpha blending there are already rounding issues depending on the
> > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > exact same formulae for color management stuff.  
> 
> Good, because otherwise you would be very quickly disappointed :-)
> 
> For scaling we would also need to replicate the exact same filter taps,
> which are often not documented.

That is where the documented tolerances come into play.

Userspace projects need screenshot-based testing, and we need to know
how much tolerance we should allow or expect.

Good reminder about CRCs. CRCs have zero tolerance, so they are not
useful for testing properties that have any leeway, are they?


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 2521 bytes --]

On Tue, 15 Jun 2021 12:45:57 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:
> > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen <ppaalanen@gmail.com> wrote:
> >   
> > > indeed it will, but what else could one do to test userspace KMS
> > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > in the long run VKMS needs to loop back to a userspace daemon that
> > > implements all the complex processing and returns the writeback result
> > > via VKMS again? That daemon would then need a single upstream, like the
> > > kernel, where it is maintained and correctness verified.  
> > 
> > The complex processing must be implemented even without write-back, because
> > user-space can ask for CRCs of the CRTC.
> >   
> > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > stuff in userspace? Didn't someone already have something like that?
> > > It would need to be lifted to be a required part of kernel UAPI
> > > submissions, I suppose like IGT is nowadays.  
> > 
> > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > software implementation with write-back connectors, but allows to expose
> > virtual planes and check atomic commits in CI.
> > 
> > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> >   
> > > For compositor developers like me knowing the exact formulas would be a huge
> > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > operations (e.g.  professional color management). Otherwise, compositors
> > > probably need a switch: "high quality color management? Then do not use KMS
> > > features."  
> > 
> > I think for alpha blending there are already rounding issues depending on the
> > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > exact same formulae for color management stuff.  
> 
> Good, because otherwise you would be very quickly disappointed :-)
> 
> For scaling we would also need to replicate the exact same filter taps,
> which are often not documented.

That is where the documented tolerances come into play.

Userspace projects need screenshot-based testing, and we need to know
how much tolerance we should allow or expect.

Good reminder about CRCs. CRCs have zero tolerance, so they are not
useful for testing properties that have any leeway, are they?


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Simon Ser]

On Tuesday, June 15th, 2021 at 12:16, Pekka Paalanen <ppaalanen@gmail.com> wrote:

> Good reminder about CRCs. CRCs have zero tolerance, so they are not
> useful for testing properties that have any leeway, are they?

IIRC, IGT's alpha blending test currently computes the CRC for all
possible roundings, then checks that the hw returns one of the
acceptable CRCs.

With more complex color management properties, this approach might not
be possible and write-back support in hw drivers would really help.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Simon Ser]

On Tuesday, June 15th, 2021 at 12:16, Pekka Paalanen <ppaalanen@gmail.com> wrote:

> Good reminder about CRCs. CRCs have zero tolerance, so they are not
> useful for testing properties that have any leeway, are they?

IIRC, IGT's alpha blending test currently computes the CRC for all
possible roundings, then checks that the hw returns one of the
acceptable CRCs.

With more complex color management properties, this approach might not
be possible and write-back support in hw drivers would really help.

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Daniel Vetter]

On Tue, Jun 15, 2021 at 2:09 PM Simon Ser <contact@emersion.fr> wrote:
>
> On Tuesday, June 15th, 2021 at 12:16, Pekka Paalanen <ppaalanen@gmail.com> wrote:
>
> > Good reminder about CRCs. CRCs have zero tolerance, so they are not
> > useful for testing properties that have any leeway, are they?
>
> IIRC, IGT's alpha blending test currently computes the CRC for all
> possible roundings, then checks that the hw returns one of the
> acceptable CRCs.
>
> With more complex color management properties, this approach might not
> be possible and write-back support in hw drivers would really help.

Yeah CRC based tests have severe limits, and even if you only try to
test the extreme stuff there's enough busted hw out there that they
will fail. E.g. when scaling I'm sure there's hw that bleeds in pixels
from outside the bounding box and can't be fixed, or there's some
intel hw where the alpha blending gets the mapping between [0, 0xff]
<-> [0.0, 1.0] and you get a nice faint ghost instead of full
transparency or opaqueness.

I think for those broken hw is just broken, nothing we can do.

Also writeback isn't supported by enough display hw, and not everyone
has access to chamelium or similar.

I think best we can do is that relevant igt have an interactive mode
(built-in for crc tests, so you can pause and look at the screen) and
then perhaps compare with the vkms reference implementation manually.
That's not great, and vkms is also not anywhere near close enough for
this either (and like currently you can't even look at it, we'd first
need a v4l output mode).
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Daniel Vetter]

On Tue, Jun 15, 2021 at 2:09 PM Simon Ser <contact@emersion.fr> wrote:
>
> On Tuesday, June 15th, 2021 at 12:16, Pekka Paalanen <ppaalanen@gmail.com> wrote:
>
> > Good reminder about CRCs. CRCs have zero tolerance, so they are not
> > useful for testing properties that have any leeway, are they?
>
> IIRC, IGT's alpha blending test currently computes the CRC for all
> possible roundings, then checks that the hw returns one of the
> acceptable CRCs.
>
> With more complex color management properties, this approach might not
> be possible and write-back support in hw drivers would really help.

Yeah CRC based tests have severe limits, and even if you only try to
test the extreme stuff there's enough busted hw out there that they
will fail. E.g. when scaling I'm sure there's hw that bleeds in pixels
from outside the bounding box and can't be fixed, or there's some
intel hw where the alpha blending gets the mapping between [0, 0xff]
<-> [0.0, 1.0] and you get a nice faint ghost instead of full
transparency or opaqueness.

I think for those broken hw is just broken, nothing we can do.

Also writeback isn't supported by enough display hw, and not everyone
has access to chamelium or similar.

I think best we can do is that relevant igt have an interactive mode
(built-in for crc tests, so you can pause and look at the screen) and
then perhaps compare with the vkms reference implementation manually.
That's not great, and vkms is also not anywhere near close enough for
this either (and like currently you can't even look at it, we'd first
need a v4l output mode).
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
> On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:
> > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:
> > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > >   
> > > > indeed it will, but what else could one do to test userspace KMS
> > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > implements all the complex processing and returns the writeback result
> > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > kernel, where it is maintained and correctness verified.  
> > > 
> > > The complex processing must be implemented even without write-back, because
> > > user-space can ask for CRCs of the CRTC.
> > >   
> > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > stuff in userspace? Didn't someone already have something like that?
> > > > It would need to be lifted to be a required part of kernel UAPI
> > > > submissions, I suppose like IGT is nowadays.  
> > > 
> > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > software implementation with write-back connectors, but allows to expose
> > > virtual planes and check atomic commits in CI.
> > > 
> > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > >   
> > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > features."  
> > > 
> > > I think for alpha blending there are already rounding issues depending on the
> > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > exact same formulae for color management stuff.  
> > 
> > Good, because otherwise you would be very quickly disappointed :-)
> > 
> > For scaling we would also need to replicate the exact same filter taps,
> > which are often not documented.
> 
> That is where the documented tolerances come into play.

This is something I've experimented with a while ago, when developing
automated tests for the rcar-du driver. When playing with different
input images we had to constantly increases tolerances, up to a point
where the tests started to miss real problems :-(

> Userspace projects need screenshot-based testing, and we need to know
> how much tolerance we should allow or expect.
> 
> Good reminder about CRCs. CRCs have zero tolerance, so they are not
> useful for testing properties that have any leeway, are they?

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
> On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:
> > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:
> > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > >   
> > > > indeed it will, but what else could one do to test userspace KMS
> > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > implements all the complex processing and returns the writeback result
> > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > kernel, where it is maintained and correctness verified.  
> > > 
> > > The complex processing must be implemented even without write-back, because
> > > user-space can ask for CRCs of the CRTC.
> > >   
> > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > stuff in userspace? Didn't someone already have something like that?
> > > > It would need to be lifted to be a required part of kernel UAPI
> > > > submissions, I suppose like IGT is nowadays.  
> > > 
> > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > software implementation with write-back connectors, but allows to expose
> > > virtual planes and check atomic commits in CI.
> > > 
> > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > >   
> > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > features."  
> > > 
> > > I think for alpha blending there are already rounding issues depending on the
> > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > exact same formulae for color management stuff.  
> > 
> > Good, because otherwise you would be very quickly disappointed :-)
> > 
> > For scaling we would also need to replicate the exact same filter taps,
> > which are often not documented.
> 
> That is where the documented tolerances come into play.

This is something I've experimented with a while ago, when developing
automated tests for the rcar-du driver. When playing with different
input images we had to constantly increases tolerances, up to a point
where the tests started to miss real problems :-(

> Userspace projects need screenshot-based testing, and we need to know
> how much tolerance we should allow or expect.
> 
> Good reminder about CRCs. CRCs have zero tolerance, so they are not
> useful for testing properties that have any leeway, are they?

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 4509 bytes --]

On Thu, 17 Jun 2021 00:05:24 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
> > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:  
> > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:  
> > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > >     
> > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > implements all the complex processing and returns the writeback result
> > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > kernel, where it is maintained and correctness verified.    
> > > > 
> > > > The complex processing must be implemented even without write-back, because
> > > > user-space can ask for CRCs of the CRTC.
> > > >     
> > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > submissions, I suppose like IGT is nowadays.    
> > > > 
> > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > software implementation with write-back connectors, but allows to expose
> > > > virtual planes and check atomic commits in CI.
> > > > 
> > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > >     
> > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > features."    
> > > > 
> > > > I think for alpha blending there are already rounding issues depending on the
> > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > exact same formulae for color management stuff.    
> > > 
> > > Good, because otherwise you would be very quickly disappointed :-)
> > > 
> > > For scaling we would also need to replicate the exact same filter taps,
> > > which are often not documented.  
> > 
> > That is where the documented tolerances come into play.  
> 
> This is something I've experimented with a while ago, when developing
> automated tests for the rcar-du driver. When playing with different
> input images we had to constantly increases tolerances, up to a point
> where the tests started to miss real problems :-(

What should we infer from that? That the hardware is broken and
exposing those KMS properties is a false promise?

If a driver on certain hardware cannot correctly implement a KMS
property over the full domain of the input space, should that driver
then simply not expose the KMS property at all?

But I would assume that the vendor still wants to expose the features
in upstream kernels, yet they cannot use the standard KMS properties
for that. Should the driver then expose vendor-specific properties with
the disclaimer that the result is not always what one would expect, so
that userspace written and tested explicitly for that hardware can
still work?

That is, a sufficient justification for a vendor-specific KMS property
would be that a standard property already exists, but the hardware is
too buggy to make it work. IOW, give up trying to make sense.

I would like to move towards a direction where *hardware* design and
testing is eventually guided by Linux KMS property definitions and
their tests. If we could have a rule that if a driver cannot correctly
implement a property then it must not expose the property, maybe in the
long term that might start having an effect?

My underlying assumption is that generic userspace will not use
vendor-specific properties.

Or, since we have atomic commits with TEST_ONLY, should it be driver's
responsibility to carefully inspect the full state and reject the
commit if the hardware is incapable of implementing it correctly?
Vendor-specific userspace would know to avoid failing configurations to
begin with. I suppose that might put an endless whack-a-mole game on
drivers though.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 4509 bytes --]

On Thu, 17 Jun 2021 00:05:24 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
> > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:  
> > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:  
> > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > >     
> > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > implements all the complex processing and returns the writeback result
> > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > kernel, where it is maintained and correctness verified.    
> > > > 
> > > > The complex processing must be implemented even without write-back, because
> > > > user-space can ask for CRCs of the CRTC.
> > > >     
> > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > submissions, I suppose like IGT is nowadays.    
> > > > 
> > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > software implementation with write-back connectors, but allows to expose
> > > > virtual planes and check atomic commits in CI.
> > > > 
> > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > >     
> > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > features."    
> > > > 
> > > > I think for alpha blending there are already rounding issues depending on the
> > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > exact same formulae for color management stuff.    
> > > 
> > > Good, because otherwise you would be very quickly disappointed :-)
> > > 
> > > For scaling we would also need to replicate the exact same filter taps,
> > > which are often not documented.  
> > 
> > That is where the documented tolerances come into play.  
> 
> This is something I've experimented with a while ago, when developing
> automated tests for the rcar-du driver. When playing with different
> input images we had to constantly increases tolerances, up to a point
> where the tests started to miss real problems :-(

What should we infer from that? That the hardware is broken and
exposing those KMS properties is a false promise?

If a driver on certain hardware cannot correctly implement a KMS
property over the full domain of the input space, should that driver
then simply not expose the KMS property at all?

But I would assume that the vendor still wants to expose the features
in upstream kernels, yet they cannot use the standard KMS properties
for that. Should the driver then expose vendor-specific properties with
the disclaimer that the result is not always what one would expect, so
that userspace written and tested explicitly for that hardware can
still work?

That is, a sufficient justification for a vendor-specific KMS property
would be that a standard property already exists, but the hardware is
too buggy to make it work. IOW, give up trying to make sense.

I would like to move towards a direction where *hardware* design and
testing is eventually guided by Linux KMS property definitions and
their tests. If we could have a rule that if a driver cannot correctly
implement a property then it must not expose the property, maybe in the
long term that might start having an effect?

My underlying assumption is that generic userspace will not use
vendor-specific properties.

Or, since we have atomic commits with TEST_ONLY, should it be driver's
responsibility to carefully inspect the full state and reject the
commit if the hardware is incapable of implementing it correctly?
Vendor-specific userspace would know to avoid failing configurations to
begin with. I suppose that might put an endless whack-a-mole game on
drivers though.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:
> On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
> > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
> > > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:  
> > > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:  
> > > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > > >     
> > > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > > implements all the complex processing and returns the writeback result
> > > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > > kernel, where it is maintained and correctness verified.    
> > > > > 
> > > > > The complex processing must be implemented even without write-back, because
> > > > > user-space can ask for CRCs of the CRTC.
> > > > >     
> > > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > > submissions, I suppose like IGT is nowadays.    
> > > > > 
> > > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > > software implementation with write-back connectors, but allows to expose
> > > > > virtual planes and check atomic commits in CI.
> > > > > 
> > > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > > >     
> > > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > > features."    
> > > > > 
> > > > > I think for alpha blending there are already rounding issues depending on the
> > > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > > exact same formulae for color management stuff.    
> > > > 
> > > > Good, because otherwise you would be very quickly disappointed :-)
> > > > 
> > > > For scaling we would also need to replicate the exact same filter taps,
> > > > which are often not documented.  
> > > 
> > > That is where the documented tolerances come into play.  
> > 
> > This is something I've experimented with a while ago, when developing
> > automated tests for the rcar-du driver. When playing with different
> > input images we had to constantly increases tolerances, up to a point
> > where the tests started to miss real problems :-(
> 
> What should we infer from that? That the hardware is broken and
> exposing those KMS properties is a false promise?

No, just that the scaler doesn't document the internal hardware
implementation (number of taps in the filters, coefficients, rounding,
...). That's the rule, not the exception, and it doesn't prevent correct
operation, images get scaled in a reproducible way (the same input
produces the same output).

> If a driver on certain hardware cannot correctly implement a KMS
> property over the full domain of the input space, should that driver
> then simply not expose the KMS property at all?

The properties involved here would the the SRC and CRTC rectangles for
the planes. They don't document pixel-perfect scaling :-)

> But I would assume that the vendor still wants to expose the features
> in upstream kernels, yet they cannot use the standard KMS properties
> for that. Should the driver then expose vendor-specific properties with
> the disclaimer that the result is not always what one would expect, so
> that userspace written and tested explicitly for that hardware can
> still work?
> 
> That is, a sufficient justification for a vendor-specific KMS property
> would be that a standard property already exists, but the hardware is
> too buggy to make it work. IOW, give up trying to make sense.

It's not just about buggy hardware, it's also about implementation
specificities, such as rounding, filters, order of operations in the
color management pipeline (it's relatively easy when you only have two
LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
features in the mix, not all hardware will implement the same pipeline),
or various types of image compression (this device implements a
"near-lossless" compression scheme that reduces the memory bandwidth by
50% for instance).

> I would like to move towards a direction where *hardware* design and
> testing is eventually guided by Linux KMS property definitions and
> their tests. If we could have a rule that if a driver cannot correctly
> implement a property then it must not expose the property, maybe in the
> long term that might start having an effect?

I didn't even know a 3D LUT existed before getting the datasheet for
this device. It's a long shot before the community will acquire enough
knowledge to guide hardware design, especially if you consider the
feature of tomorrow's devices that nobody knows about yet. A worthwhile
goal, but not something we can fully achieve, and not in a short time
frame.

> My underlying assumption is that generic userspace will not use
> vendor-specific properties.

I expect some amount of device-specific code in userspace, yes. 

There are usually large variations in how the hardware exposes access to
a given feature, which leads to code having to convert the standard API
parameters to hardware parameters. To a large extend this can be done in
drivers, but for some more complex features, it may put too much burden
on the kernel. There's a reason mesa is a userspace stack :-)

> Or, since we have atomic commits with TEST_ONLY, should it be driver's
> responsibility to carefully inspect the full state and reject the
> commit if the hardware is incapable of implementing it correctly?
> Vendor-specific userspace would know to avoid failing configurations to
> begin with. I suppose that might put an endless whack-a-mole game on
> drivers though.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:
> On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
> > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
> > > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:  
> > > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:  
> > > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > > >     
> > > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > > implements all the complex processing and returns the writeback result
> > > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > > kernel, where it is maintained and correctness verified.    
> > > > > 
> > > > > The complex processing must be implemented even without write-back, because
> > > > > user-space can ask for CRCs of the CRTC.
> > > > >     
> > > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > > submissions, I suppose like IGT is nowadays.    
> > > > > 
> > > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > > software implementation with write-back connectors, but allows to expose
> > > > > virtual planes and check atomic commits in CI.
> > > > > 
> > > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > > >     
> > > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > > features."    
> > > > > 
> > > > > I think for alpha blending there are already rounding issues depending on the
> > > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > > exact same formulae for color management stuff.    
> > > > 
> > > > Good, because otherwise you would be very quickly disappointed :-)
> > > > 
> > > > For scaling we would also need to replicate the exact same filter taps,
> > > > which are often not documented.  
> > > 
> > > That is where the documented tolerances come into play.  
> > 
> > This is something I've experimented with a while ago, when developing
> > automated tests for the rcar-du driver. When playing with different
> > input images we had to constantly increases tolerances, up to a point
> > where the tests started to miss real problems :-(
> 
> What should we infer from that? That the hardware is broken and
> exposing those KMS properties is a false promise?

No, just that the scaler doesn't document the internal hardware
implementation (number of taps in the filters, coefficients, rounding,
...). That's the rule, not the exception, and it doesn't prevent correct
operation, images get scaled in a reproducible way (the same input
produces the same output).

> If a driver on certain hardware cannot correctly implement a KMS
> property over the full domain of the input space, should that driver
> then simply not expose the KMS property at all?

The properties involved here would the the SRC and CRTC rectangles for
the planes. They don't document pixel-perfect scaling :-)

> But I would assume that the vendor still wants to expose the features
> in upstream kernels, yet they cannot use the standard KMS properties
> for that. Should the driver then expose vendor-specific properties with
> the disclaimer that the result is not always what one would expect, so
> that userspace written and tested explicitly for that hardware can
> still work?
> 
> That is, a sufficient justification for a vendor-specific KMS property
> would be that a standard property already exists, but the hardware is
> too buggy to make it work. IOW, give up trying to make sense.

It's not just about buggy hardware, it's also about implementation
specificities, such as rounding, filters, order of operations in the
color management pipeline (it's relatively easy when you only have two
LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
features in the mix, not all hardware will implement the same pipeline),
or various types of image compression (this device implements a
"near-lossless" compression scheme that reduces the memory bandwidth by
50% for instance).

> I would like to move towards a direction where *hardware* design and
> testing is eventually guided by Linux KMS property definitions and
> their tests. If we could have a rule that if a driver cannot correctly
> implement a property then it must not expose the property, maybe in the
> long term that might start having an effect?

I didn't even know a 3D LUT existed before getting the datasheet for
this device. It's a long shot before the community will acquire enough
knowledge to guide hardware design, especially if you consider the
feature of tomorrow's devices that nobody knows about yet. A worthwhile
goal, but not something we can fully achieve, and not in a short time
frame.

> My underlying assumption is that generic userspace will not use
> vendor-specific properties.

I expect some amount of device-specific code in userspace, yes. 

There are usually large variations in how the hardware exposes access to
a given feature, which leads to code having to convert the standard API
parameters to hardware parameters. To a large extend this can be done in
drivers, but for some more complex features, it may put too much burden
on the kernel. There's a reason mesa is a userspace stack :-)

> Or, since we have atomic commits with TEST_ONLY, should it be driver's
> responsibility to carefully inspect the full state and reject the
> commit if the hardware is incapable of implementing it correctly?
> Vendor-specific userspace would know to avoid failing configurations to
> begin with. I suppose that might put an endless whack-a-mole game on
> drivers though.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 7764 bytes --]

On Thu, 17 Jun 2021 13:29:48 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:
> > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:  
> > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:  
> > > > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:    
> > > > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:    
> > > > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > > > >       
> > > > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > > > implements all the complex processing and returns the writeback result
> > > > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > > > kernel, where it is maintained and correctness verified.      
> > > > > > 
> > > > > > The complex processing must be implemented even without write-back, because
> > > > > > user-space can ask for CRCs of the CRTC.
> > > > > >       
> > > > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > > > submissions, I suppose like IGT is nowadays.      
> > > > > > 
> > > > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > > > software implementation with write-back connectors, but allows to expose
> > > > > > virtual planes and check atomic commits in CI.
> > > > > > 
> > > > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > > > >       
> > > > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > > > features."      
> > > > > > 
> > > > > > I think for alpha blending there are already rounding issues depending on the
> > > > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > > > exact same formulae for color management stuff.      
> > > > > 
> > > > > Good, because otherwise you would be very quickly disappointed :-)
> > > > > 
> > > > > For scaling we would also need to replicate the exact same filter taps,
> > > > > which are often not documented.    
> > > > 
> > > > That is where the documented tolerances come into play.    
> > > 
> > > This is something I've experimented with a while ago, when developing
> > > automated tests for the rcar-du driver. When playing with different
> > > input images we had to constantly increases tolerances, up to a point
> > > where the tests started to miss real problems :-(  
> > 
> > What should we infer from that? That the hardware is broken and
> > exposing those KMS properties is a false promise?  
> 
> No, just that the scaler doesn't document the internal hardware
> implementation (number of taps in the filters, coefficients, rounding,
> ...). That's the rule, not the exception, and it doesn't prevent correct
> operation, images get scaled in a reproducible way (the same input
> produces the same output).
> 
> > If a driver on certain hardware cannot correctly implement a KMS
> > property over the full domain of the input space, should that driver
> > then simply not expose the KMS property at all?  
> 
> The properties involved here would the the SRC and CRTC rectangles for
> the planes. They don't document pixel-perfect scaling :-)
> 
> > But I would assume that the vendor still wants to expose the features
> > in upstream kernels, yet they cannot use the standard KMS properties
> > for that. Should the driver then expose vendor-specific properties with
> > the disclaimer that the result is not always what one would expect, so
> > that userspace written and tested explicitly for that hardware can
> > still work?
> > 
> > That is, a sufficient justification for a vendor-specific KMS property
> > would be that a standard property already exists, but the hardware is
> > too buggy to make it work. IOW, give up trying to make sense.  
> 
> It's not just about buggy hardware, it's also about implementation
> specificities, such as rounding, filters, order of operations in the
> color management pipeline (it's relatively easy when you only have two
> LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> features in the mix, not all hardware will implement the same pipeline),
> or various types of image compression (this device implements a
> "near-lossless" compression scheme that reduces the memory bandwidth by
> 50% for instance).

Rounding shouldn't result in needing wide tolerances.

Filters are more difficult, but at least we can factor them out when
testing other things. Filters could be tested in isolation with some
image difference metrics rather than per-pixel independent comparisons.

The order of operations in the color management pipeline is very
important. We can't work with "whatever". All the variability in
hardware is exactly why I have been calling out for defined abstract
color pipeline in the DRM UAPI docs, so that drivers would know which
properties to map to their elements, so that userspace can have any
possibility of using them correctly. If the hardware has a block
that doesn't fit in the abstract pipeline, you get to add things to the
abstract pipeline, or invent a whole another abstract pipeline and
document that.

Lossy compression needs its own KMS properties to ensure it can be
disabled when necessary. Near-lossless is not lossless, if a difference
can be measured. The driver or hardware cannot guess if the end user is
ok with "near-lossless" or not, so you have to be conservative and
assume not ok, offering an opt-in for lossy.

...

> > My underlying assumption is that generic userspace will not use
> > vendor-specific properties.  
> 
> I expect some amount of device-specific code in userspace, yes. 

If we had a reliable way to test device-specific code without the
hardware and automatically in CI, then maybe.

> There are usually large variations in how the hardware exposes access to
> a given feature, which leads to code having to convert the standard API
> parameters to hardware parameters. To a large extend this can be done in
> drivers, but for some more complex features, it may put too much burden
> on the kernel. There's a reason mesa is a userspace stack :-)

If we get a Khronos standardised 2D composition API... oh wait.

Nothing wrong with userspace libraries, but it does mean that driver
developers need to contribute to them, like they do to Mesa. Is there
any of that going on for KMS?

In the mean time, DRM UAPI basically must define a 2D composition API,
or the new KMS properties will not see use outside of vendor trees.

> > Or, since we have atomic commits with TEST_ONLY, should it be driver's
> > responsibility to carefully inspect the full state and reject the
> > commit if the hardware is incapable of implementing it correctly?
> > Vendor-specific userspace would know to avoid failing configurations to
> > begin with. I suppose that might put an endless whack-a-mole game on
> > drivers though.  


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 7764 bytes --]

On Thu, 17 Jun 2021 13:29:48 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:
> > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:  
> > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:  
> > > > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:    
> > > > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:    
> > > > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > > > >       
> > > > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > > > implements all the complex processing and returns the writeback result
> > > > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > > > kernel, where it is maintained and correctness verified.      
> > > > > > 
> > > > > > The complex processing must be implemented even without write-back, because
> > > > > > user-space can ask for CRCs of the CRTC.
> > > > > >       
> > > > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > > > submissions, I suppose like IGT is nowadays.      
> > > > > > 
> > > > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > > > software implementation with write-back connectors, but allows to expose
> > > > > > virtual planes and check atomic commits in CI.
> > > > > > 
> > > > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > > > >       
> > > > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > > > features."      
> > > > > > 
> > > > > > I think for alpha blending there are already rounding issues depending on the
> > > > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > > > exact same formulae for color management stuff.      
> > > > > 
> > > > > Good, because otherwise you would be very quickly disappointed :-)
> > > > > 
> > > > > For scaling we would also need to replicate the exact same filter taps,
> > > > > which are often not documented.    
> > > > 
> > > > That is where the documented tolerances come into play.    
> > > 
> > > This is something I've experimented with a while ago, when developing
> > > automated tests for the rcar-du driver. When playing with different
> > > input images we had to constantly increases tolerances, up to a point
> > > where the tests started to miss real problems :-(  
> > 
> > What should we infer from that? That the hardware is broken and
> > exposing those KMS properties is a false promise?  
> 
> No, just that the scaler doesn't document the internal hardware
> implementation (number of taps in the filters, coefficients, rounding,
> ...). That's the rule, not the exception, and it doesn't prevent correct
> operation, images get scaled in a reproducible way (the same input
> produces the same output).
> 
> > If a driver on certain hardware cannot correctly implement a KMS
> > property over the full domain of the input space, should that driver
> > then simply not expose the KMS property at all?  
> 
> The properties involved here would the the SRC and CRTC rectangles for
> the planes. They don't document pixel-perfect scaling :-)
> 
> > But I would assume that the vendor still wants to expose the features
> > in upstream kernels, yet they cannot use the standard KMS properties
> > for that. Should the driver then expose vendor-specific properties with
> > the disclaimer that the result is not always what one would expect, so
> > that userspace written and tested explicitly for that hardware can
> > still work?
> > 
> > That is, a sufficient justification for a vendor-specific KMS property
> > would be that a standard property already exists, but the hardware is
> > too buggy to make it work. IOW, give up trying to make sense.  
> 
> It's not just about buggy hardware, it's also about implementation
> specificities, such as rounding, filters, order of operations in the
> color management pipeline (it's relatively easy when you only have two
> LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> features in the mix, not all hardware will implement the same pipeline),
> or various types of image compression (this device implements a
> "near-lossless" compression scheme that reduces the memory bandwidth by
> 50% for instance).

Rounding shouldn't result in needing wide tolerances.

Filters are more difficult, but at least we can factor them out when
testing other things. Filters could be tested in isolation with some
image difference metrics rather than per-pixel independent comparisons.

The order of operations in the color management pipeline is very
important. We can't work with "whatever". All the variability in
hardware is exactly why I have been calling out for defined abstract
color pipeline in the DRM UAPI docs, so that drivers would know which
properties to map to their elements, so that userspace can have any
possibility of using them correctly. If the hardware has a block
that doesn't fit in the abstract pipeline, you get to add things to the
abstract pipeline, or invent a whole another abstract pipeline and
document that.

Lossy compression needs its own KMS properties to ensure it can be
disabled when necessary. Near-lossless is not lossless, if a difference
can be measured. The driver or hardware cannot guess if the end user is
ok with "near-lossless" or not, so you have to be conservative and
assume not ok, offering an opt-in for lossy.

...

> > My underlying assumption is that generic userspace will not use
> > vendor-specific properties.  
> 
> I expect some amount of device-specific code in userspace, yes. 

If we had a reliable way to test device-specific code without the
hardware and automatically in CI, then maybe.

> There are usually large variations in how the hardware exposes access to
> a given feature, which leads to code having to convert the standard API
> parameters to hardware parameters. To a large extend this can be done in
> drivers, but for some more complex features, it may put too much burden
> on the kernel. There's a reason mesa is a userspace stack :-)

If we get a Khronos standardised 2D composition API... oh wait.

Nothing wrong with userspace libraries, but it does mean that driver
developers need to contribute to them, like they do to Mesa. Is there
any of that going on for KMS?

In the mean time, DRM UAPI basically must define a 2D composition API,
or the new KMS properties will not see use outside of vendor trees.

> > Or, since we have atomic commits with TEST_ONLY, should it be driver's
> > responsibility to carefully inspect the full state and reject the
> > commit if the hardware is incapable of implementing it correctly?
> > Vendor-specific userspace would know to avoid failing configurations to
> > begin with. I suppose that might put an endless whack-a-mole game on
> > drivers though.  


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:
> On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
> > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:
> > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:  
> > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:  
> > > > > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:    
> > > > > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:    
> > > > > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > > > > >       
> > > > > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > > > > implements all the complex processing and returns the writeback result
> > > > > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > > > > kernel, where it is maintained and correctness verified.      
> > > > > > > 
> > > > > > > The complex processing must be implemented even without write-back, because
> > > > > > > user-space can ask for CRCs of the CRTC.
> > > > > > >       
> > > > > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > > > > submissions, I suppose like IGT is nowadays.      
> > > > > > > 
> > > > > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > > > > software implementation with write-back connectors, but allows to expose
> > > > > > > virtual planes and check atomic commits in CI.
> > > > > > > 
> > > > > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > > > > >       
> > > > > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > > > > features."      
> > > > > > > 
> > > > > > > I think for alpha blending there are already rounding issues depending on the
> > > > > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > > > > exact same formulae for color management stuff.      
> > > > > > 
> > > > > > Good, because otherwise you would be very quickly disappointed :-)
> > > > > > 
> > > > > > For scaling we would also need to replicate the exact same filter taps,
> > > > > > which are often not documented.    
> > > > > 
> > > > > That is where the documented tolerances come into play.    
> > > > 
> > > > This is something I've experimented with a while ago, when developing
> > > > automated tests for the rcar-du driver. When playing with different
> > > > input images we had to constantly increases tolerances, up to a point
> > > > where the tests started to miss real problems :-(  
> > > 
> > > What should we infer from that? That the hardware is broken and
> > > exposing those KMS properties is a false promise?  
> > 
> > No, just that the scaler doesn't document the internal hardware
> > implementation (number of taps in the filters, coefficients, rounding,
> > ...). That's the rule, not the exception, and it doesn't prevent correct
> > operation, images get scaled in a reproducible way (the same input
> > produces the same output).
> > 
> > > If a driver on certain hardware cannot correctly implement a KMS
> > > property over the full domain of the input space, should that driver
> > > then simply not expose the KMS property at all?  
> > 
> > The properties involved here would the the SRC and CRTC rectangles for
> > the planes. They don't document pixel-perfect scaling :-)
> > 
> > > But I would assume that the vendor still wants to expose the features
> > > in upstream kernels, yet they cannot use the standard KMS properties
> > > for that. Should the driver then expose vendor-specific properties with
> > > the disclaimer that the result is not always what one would expect, so
> > > that userspace written and tested explicitly for that hardware can
> > > still work?
> > > 
> > > That is, a sufficient justification for a vendor-specific KMS property
> > > would be that a standard property already exists, but the hardware is
> > > too buggy to make it work. IOW, give up trying to make sense.  
> > 
> > It's not just about buggy hardware, it's also about implementation
> > specificities, such as rounding, filters, order of operations in the
> > color management pipeline (it's relatively easy when you only have two
> > LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> > features in the mix, not all hardware will implement the same pipeline),
> > or various types of image compression (this device implements a
> > "near-lossless" compression scheme that reduces the memory bandwidth by
> > 50% for instance).
> 
> Rounding shouldn't result in needing wide tolerances.
> 
> Filters are more difficult, but at least we can factor them out when
> testing other things. Filters could be tested in isolation with some
> image difference metrics rather than per-pixel independent comparisons.

The metrics I was using had both a tolerance on the pixel value and on
the number of pixels accepted outside of the value tolerance. I'm sure
we can improve it (perhaps taking locality into account), but that's
heuristics, and keeping heuristics working across a wide variety of use
cases is hard.

The filter I mentioned, by the way, is the scaler filter. Out of
curiosity, do any of the devices you work on document with pixel-perfect
precision how the hardware scaler is implemented ?

> The order of operations in the color management pipeline is very
> important. We can't work with "whatever". All the variability in
> hardware is exactly why I have been calling out for defined abstract
> color pipeline in the DRM UAPI docs, so that drivers would know which
> properties to map to their elements, so that userspace can have any
> possibility of using them correctly. If the hardware has a block
> that doesn't fit in the abstract pipeline, you get to add things to the
> abstract pipeline, or invent a whole another abstract pipeline and
> document that.

One very typical difference between devices is the order of the
processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
gamma, we can accommodate hardware that have any combination of
[1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
different points in the pipeline depending on the device, and it will
start getting complicated, even if the use case is quite simple and
common. This is getting a bit out of topic, but how would you solve this
one in particular ?

> Lossy compression needs its own KMS properties to ensure it can be
> disabled when necessary. Near-lossless is not lossless, if a difference
> can be measured. The driver or hardware cannot guess if the end user is
> ok with "near-lossless" or not, so you have to be conservative and
> assume not ok, offering an opt-in for lossy.

Sure, but what would be the barrier to entry for such a property that
would enable the compression (it could actually be a pixel format
modifier) ? Would it only need to be documented ? Would we need a
software implementation in VKMS and/or in IGT ? The compression
algorithm is proprietary and not documented, so the latter can't be
done.

> ...
> 
> > > My underlying assumption is that generic userspace will not use
> > > vendor-specific properties.  
> > 
> > I expect some amount of device-specific code in userspace, yes. 
> 
> If we had a reliable way to test device-specific code without the
> hardware and automatically in CI, then maybe.
> 
> > There are usually large variations in how the hardware exposes access to
> > a given feature, which leads to code having to convert the standard API
> > parameters to hardware parameters. To a large extend this can be done in
> > drivers, but for some more complex features, it may put too much burden
> > on the kernel. There's a reason mesa is a userspace stack :-)
> 
> If we get a Khronos standardised 2D composition API... oh wait.
> 
> Nothing wrong with userspace libraries, but it does mean that driver
> developers need to contribute to them, like they do to Mesa. Is there
> any of that going on for KMS?

Not that I'm aware of, but I think that's a direction we can consider
seriously.

> In the mean time, DRM UAPI basically must define a 2D composition API,
> or the new KMS properties will not see use outside of vendor trees.
> 
> > > Or, since we have atomic commits with TEST_ONLY, should it be driver's
> > > responsibility to carefully inspect the full state and reject the
> > > commit if the hardware is incapable of implementing it correctly?
> > > Vendor-specific userspace would know to avoid failing configurations to
> > > begin with. I suppose that might put an endless whack-a-mole game on
> > > drivers though.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:
> On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
> > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:
> > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:  
> > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:  
> > > > > On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:    
> > > > > > On Tue, Jun 15, 2021 at 07:15:18AM +0000, Simon Ser wrote:    
> > > > > > > On Tuesday, June 15th, 2021 at 09:03, Pekka Paalanen wrote:
> > > > > > >       
> > > > > > > > indeed it will, but what else could one do to test userspace KMS
> > > > > > > > clients in generic CI where all you can have is virtual hardware? Maybe
> > > > > > > > in the long run VKMS needs to loop back to a userspace daemon that
> > > > > > > > implements all the complex processing and returns the writeback result
> > > > > > > > via VKMS again? That daemon would then need a single upstream, like the
> > > > > > > > kernel, where it is maintained and correctness verified.      
> > > > > > > 
> > > > > > > The complex processing must be implemented even without write-back, because
> > > > > > > user-space can ask for CRCs of the CRTC.
> > > > > > >       
> > > > > > > > Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual
> > > > > > > > stuff in userspace? Didn't someone already have something like that?
> > > > > > > > It would need to be lifted to be a required part of kernel UAPI
> > > > > > > > submissions, I suppose like IGT is nowadays.      
> > > > > > > 
> > > > > > > FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full
> > > > > > > software implementation with write-back connectors, but allows to expose
> > > > > > > virtual planes and check atomic commits in CI.
> > > > > > > 
> > > > > > > [1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
> > > > > > >       
> > > > > > > > For compositor developers like me knowing the exact formulas would be a huge
> > > > > > > > benefit as it would allow me to use KMS to off-load precision-sensitive
> > > > > > > > operations (e.g.  professional color management). Otherwise, compositors
> > > > > > > > probably need a switch: "high quality color management? Then do not use KMS
> > > > > > > > features."      
> > > > > > > 
> > > > > > > I think for alpha blending there are already rounding issues depending on the
> > > > > > > hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the
> > > > > > > exact same formulae for color management stuff.      
> > > > > > 
> > > > > > Good, because otherwise you would be very quickly disappointed :-)
> > > > > > 
> > > > > > For scaling we would also need to replicate the exact same filter taps,
> > > > > > which are often not documented.    
> > > > > 
> > > > > That is where the documented tolerances come into play.    
> > > > 
> > > > This is something I've experimented with a while ago, when developing
> > > > automated tests for the rcar-du driver. When playing with different
> > > > input images we had to constantly increases tolerances, up to a point
> > > > where the tests started to miss real problems :-(  
> > > 
> > > What should we infer from that? That the hardware is broken and
> > > exposing those KMS properties is a false promise?  
> > 
> > No, just that the scaler doesn't document the internal hardware
> > implementation (number of taps in the filters, coefficients, rounding,
> > ...). That's the rule, not the exception, and it doesn't prevent correct
> > operation, images get scaled in a reproducible way (the same input
> > produces the same output).
> > 
> > > If a driver on certain hardware cannot correctly implement a KMS
> > > property over the full domain of the input space, should that driver
> > > then simply not expose the KMS property at all?  
> > 
> > The properties involved here would the the SRC and CRTC rectangles for
> > the planes. They don't document pixel-perfect scaling :-)
> > 
> > > But I would assume that the vendor still wants to expose the features
> > > in upstream kernels, yet they cannot use the standard KMS properties
> > > for that. Should the driver then expose vendor-specific properties with
> > > the disclaimer that the result is not always what one would expect, so
> > > that userspace written and tested explicitly for that hardware can
> > > still work?
> > > 
> > > That is, a sufficient justification for a vendor-specific KMS property
> > > would be that a standard property already exists, but the hardware is
> > > too buggy to make it work. IOW, give up trying to make sense.  
> > 
> > It's not just about buggy hardware, it's also about implementation
> > specificities, such as rounding, filters, order of operations in the
> > color management pipeline (it's relatively easy when you only have two
> > LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> > features in the mix, not all hardware will implement the same pipeline),
> > or various types of image compression (this device implements a
> > "near-lossless" compression scheme that reduces the memory bandwidth by
> > 50% for instance).
> 
> Rounding shouldn't result in needing wide tolerances.
> 
> Filters are more difficult, but at least we can factor them out when
> testing other things. Filters could be tested in isolation with some
> image difference metrics rather than per-pixel independent comparisons.

The metrics I was using had both a tolerance on the pixel value and on
the number of pixels accepted outside of the value tolerance. I'm sure
we can improve it (perhaps taking locality into account), but that's
heuristics, and keeping heuristics working across a wide variety of use
cases is hard.

The filter I mentioned, by the way, is the scaler filter. Out of
curiosity, do any of the devices you work on document with pixel-perfect
precision how the hardware scaler is implemented ?

> The order of operations in the color management pipeline is very
> important. We can't work with "whatever". All the variability in
> hardware is exactly why I have been calling out for defined abstract
> color pipeline in the DRM UAPI docs, so that drivers would know which
> properties to map to their elements, so that userspace can have any
> possibility of using them correctly. If the hardware has a block
> that doesn't fit in the abstract pipeline, you get to add things to the
> abstract pipeline, or invent a whole another abstract pipeline and
> document that.

One very typical difference between devices is the order of the
processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
gamma, we can accommodate hardware that have any combination of
[1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
different points in the pipeline depending on the device, and it will
start getting complicated, even if the use case is quite simple and
common. This is getting a bit out of topic, but how would you solve this
one in particular ?

> Lossy compression needs its own KMS properties to ensure it can be
> disabled when necessary. Near-lossless is not lossless, if a difference
> can be measured. The driver or hardware cannot guess if the end user is
> ok with "near-lossless" or not, so you have to be conservative and
> assume not ok, offering an opt-in for lossy.

Sure, but what would be the barrier to entry for such a property that
would enable the compression (it could actually be a pixel format
modifier) ? Would it only need to be documented ? Would we need a
software implementation in VKMS and/or in IGT ? The compression
algorithm is proprietary and not documented, so the latter can't be
done.

> ...
> 
> > > My underlying assumption is that generic userspace will not use
> > > vendor-specific properties.  
> > 
> > I expect some amount of device-specific code in userspace, yes. 
> 
> If we had a reliable way to test device-specific code without the
> hardware and automatically in CI, then maybe.
> 
> > There are usually large variations in how the hardware exposes access to
> > a given feature, which leads to code having to convert the standard API
> > parameters to hardware parameters. To a large extend this can be done in
> > drivers, but for some more complex features, it may put too much burden
> > on the kernel. There's a reason mesa is a userspace stack :-)
> 
> If we get a Khronos standardised 2D composition API... oh wait.
> 
> Nothing wrong with userspace libraries, but it does mean that driver
> developers need to contribute to them, like they do to Mesa. Is there
> any of that going on for KMS?

Not that I'm aware of, but I think that's a direction we can consider
seriously.

> In the mean time, DRM UAPI basically must define a 2D composition API,
> or the new KMS properties will not see use outside of vendor trees.
> 
> > > Or, since we have atomic commits with TEST_ONLY, should it be driver's
> > > responsibility to carefully inspect the full state and reject the
> > > commit if the hardware is incapable of implementing it correctly?
> > > Vendor-specific userspace would know to avoid failing configurations to
> > > begin with. I suppose that might put an endless whack-a-mole game on
> > > drivers though.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 10776 bytes --]

On Thu, 17 Jun 2021 16:37:14 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:
> > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:  
> > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:  
> > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:    
> > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:    

...

> > > > > > That is where the documented tolerances come into play.      
> > > > > 
> > > > > This is something I've experimented with a while ago, when developing
> > > > > automated tests for the rcar-du driver. When playing with different
> > > > > input images we had to constantly increases tolerances, up to a point
> > > > > where the tests started to miss real problems :-(    
> > > > 
> > > > What should we infer from that? That the hardware is broken and
> > > > exposing those KMS properties is a false promise?    
> > > 
> > > No, just that the scaler doesn't document the internal hardware
> > > implementation (number of taps in the filters, coefficients, rounding,
> > > ...). That's the rule, not the exception, and it doesn't prevent correct
> > > operation, images get scaled in a reproducible way (the same input
> > > produces the same output).
> > >   
> > > > If a driver on certain hardware cannot correctly implement a KMS
> > > > property over the full domain of the input space, should that driver
> > > > then simply not expose the KMS property at all?    
> > > 
> > > The properties involved here would the the SRC and CRTC rectangles for
> > > the planes. They don't document pixel-perfect scaling :-)
> > >   
> > > > But I would assume that the vendor still wants to expose the features
> > > > in upstream kernels, yet they cannot use the standard KMS properties
> > > > for that. Should the driver then expose vendor-specific properties with
> > > > the disclaimer that the result is not always what one would expect, so
> > > > that userspace written and tested explicitly for that hardware can
> > > > still work?
> > > > 
> > > > That is, a sufficient justification for a vendor-specific KMS property
> > > > would be that a standard property already exists, but the hardware is
> > > > too buggy to make it work. IOW, give up trying to make sense.    
> > > 
> > > It's not just about buggy hardware, it's also about implementation
> > > specificities, such as rounding, filters, order of operations in the
> > > color management pipeline (it's relatively easy when you only have two
> > > LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> > > features in the mix, not all hardware will implement the same pipeline),
> > > or various types of image compression (this device implements a
> > > "near-lossless" compression scheme that reduces the memory bandwidth by
> > > 50% for instance).  
> > 
> > Rounding shouldn't result in needing wide tolerances.
> > 
> > Filters are more difficult, but at least we can factor them out when
> > testing other things. Filters could be tested in isolation with some
> > image difference metrics rather than per-pixel independent comparisons.  
> 
> The metrics I was using had both a tolerance on the pixel value and on
> the number of pixels accepted outside of the value tolerance. I'm sure
> we can improve it (perhaps taking locality into account), but that's
> heuristics, and keeping heuristics working across a wide variety of use
> cases is hard.

Hi Laurent,

I was thinking of using a more, um, scientific error measures, e.g.
sum or squared errors (SSE) or average SSE over the whole
re-scaled/filtered image result, ignoring pixels outside of that.

What one normally uses when matching images in computer vision, for
example. Could even add a threshold such that simple rounding-level
errors would not even register in SSE. I'm sure there is plenty of
literature on that, but it may be behind a paywall like IEEE.

SSE may or may not need to computed from light-linear pixel values,
too. If one wanted to go even further, I'm sure there are computational
models about human color and brightness difference sensitivity that
could be used to weigh the errors.


> The filter I mentioned, by the way, is the scaler filter. Out of
> curiosity, do any of the devices you work on document with pixel-perfect
> precision how the hardware scaler is implemented ?

I don't work on drivers, so wouldn't even look for hardware docs. I go
by what KMS UAPI documents because that is the API I work with and
nothing else. And yes, I ignore all the scaling filter issues for now.

Because I don't have a way to get feedback (writeback connectors maybe
not existing and not hooked up in Weston quite yet), testing
scaling/filtering precision has not been on-topic yet. Right now I'm
interested in color correctness rather than geometrical filtering. For
traditional color management, the expected pixel values are quite
precise.

> > The order of operations in the color management pipeline is very
> > important. We can't work with "whatever". All the variability in
> > hardware is exactly why I have been calling out for defined abstract
> > color pipeline in the DRM UAPI docs, so that drivers would know which
> > properties to map to their elements, so that userspace can have any
> > possibility of using them correctly. If the hardware has a block
> > that doesn't fit in the abstract pipeline, you get to add things to the
> > abstract pipeline, or invent a whole another abstract pipeline and
> > document that.  
> 
> One very typical difference between devices is the order of the
> processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> gamma, we can accommodate hardware that have any combination of
> [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at

But you cannot represent pipelines like
1D LUT -> 1D LUT -> CCM
because the abstract pipeline just doesn't have the elements for that.
OTOH, maybe that ordering does not even make sense to have in hardware?
So maybe not all combinations are actually needed.

> different points in the pipeline depending on the device, and it will
> start getting complicated, even if the use case is quite simple and
> common. This is getting a bit out of topic, but how would you solve this
> one in particular ?

By defining all the points in the abstract color pipeline where a 3D
LUT could exist. Then each point would probably need its own KMS
property.

We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
drivers need to respect that order.

If the combinatorial explosion gets out of hand, maybe we need a KMS
property to switch to whole another abstract pipeline which defines a
different ordering on the same and/or different KMS properties.

From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
on each side of it for memory vs. precision optimization. And after the
degamma -> ctm -> gamma pipeline you may want one more ctm for
RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
all actually implemented hardware features might not go totally out of
hand.

> > Lossy compression needs its own KMS properties to ensure it can be
> > disabled when necessary. Near-lossless is not lossless, if a difference
> > can be measured. The driver or hardware cannot guess if the end user is
> > ok with "near-lossless" or not, so you have to be conservative and
> > assume not ok, offering an opt-in for lossy.  
> 
> Sure, but what would be the barrier to entry for such a property that
> would enable the compression (it could actually be a pixel format
> modifier) ? Would it only need to be documented ? Would we need a
> software implementation in VKMS and/or in IGT ? The compression
> algorithm is proprietary and not documented, so the latter can't be
> done.

Good questions. Shows that the idea of strictly requiring a VKMS
implementation won't fly, which is what I expected.

Saying it could be a pixel format modifier is a really good point. A
modifier cannot be the only thing to control it. Userspace does not
decode modifiers, so it cannot filter it out when it wants lossless
pixels. There must be something else to control it.

As a userspace dev, I would be ok with documenting a KMS property as
"improves blah blah, but also does significant violence to your
pixels", so I would know that this is something I need to consider.

You could argue that all KMS properties do violence to pixels, but
that's not a useful definition. It would just mean that in some use
cases I would never off-load anything to KMS. Depending on the use case
that might still be true even if the errors were limited to reasonable
rounding errors. I need an idea of how much error does KMS processing
do, and ultimately I expect compositors to also need a last resort
button for "do not trust KMS processing at all" which then makes the
display server always use the exact same simplest possible KMS
configuration and let the end user deal with the KMS errors via
color-profiling his monitor.

It's quite a different thing to have color processing elements in an
unexpected order in the pipeline than it is to have a scaling filter
doing slightly unknown operations.

> > ...
> >   
> > > > My underlying assumption is that generic userspace will not use
> > > > vendor-specific properties.    
> > > 
> > > I expect some amount of device-specific code in userspace, yes.   
> > 
> > If we had a reliable way to test device-specific code without the
> > hardware and automatically in CI, then maybe.
> >   
> > > There are usually large variations in how the hardware exposes access to
> > > a given feature, which leads to code having to convert the standard API
> > > parameters to hardware parameters. To a large extend this can be done in
> > > drivers, but for some more complex features, it may put too much burden
> > > on the kernel. There's a reason mesa is a userspace stack :-)  
> > 
> > If we get a Khronos standardised 2D composition API... oh wait.
> > 
> > Nothing wrong with userspace libraries, but it does mean that driver
> > developers need to contribute to them, like they do to Mesa. Is there
> > any of that going on for KMS?  
> 
> Not that I'm aware of, but I think that's a direction we can consider
> seriously.

That would be awesome if the API is generic.

> > In the mean time, DRM UAPI basically must define a 2D composition API,
> > or the new KMS properties will not see use outside of vendor trees.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 10776 bytes --]

On Thu, 17 Jun 2021 16:37:14 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:
> > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:  
> > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:  
> > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:    
> > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:    

...

> > > > > > That is where the documented tolerances come into play.      
> > > > > 
> > > > > This is something I've experimented with a while ago, when developing
> > > > > automated tests for the rcar-du driver. When playing with different
> > > > > input images we had to constantly increases tolerances, up to a point
> > > > > where the tests started to miss real problems :-(    
> > > > 
> > > > What should we infer from that? That the hardware is broken and
> > > > exposing those KMS properties is a false promise?    
> > > 
> > > No, just that the scaler doesn't document the internal hardware
> > > implementation (number of taps in the filters, coefficients, rounding,
> > > ...). That's the rule, not the exception, and it doesn't prevent correct
> > > operation, images get scaled in a reproducible way (the same input
> > > produces the same output).
> > >   
> > > > If a driver on certain hardware cannot correctly implement a KMS
> > > > property over the full domain of the input space, should that driver
> > > > then simply not expose the KMS property at all?    
> > > 
> > > The properties involved here would the the SRC and CRTC rectangles for
> > > the planes. They don't document pixel-perfect scaling :-)
> > >   
> > > > But I would assume that the vendor still wants to expose the features
> > > > in upstream kernels, yet they cannot use the standard KMS properties
> > > > for that. Should the driver then expose vendor-specific properties with
> > > > the disclaimer that the result is not always what one would expect, so
> > > > that userspace written and tested explicitly for that hardware can
> > > > still work?
> > > > 
> > > > That is, a sufficient justification for a vendor-specific KMS property
> > > > would be that a standard property already exists, but the hardware is
> > > > too buggy to make it work. IOW, give up trying to make sense.    
> > > 
> > > It's not just about buggy hardware, it's also about implementation
> > > specificities, such as rounding, filters, order of operations in the
> > > color management pipeline (it's relatively easy when you only have two
> > > LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> > > features in the mix, not all hardware will implement the same pipeline),
> > > or various types of image compression (this device implements a
> > > "near-lossless" compression scheme that reduces the memory bandwidth by
> > > 50% for instance).  
> > 
> > Rounding shouldn't result in needing wide tolerances.
> > 
> > Filters are more difficult, but at least we can factor them out when
> > testing other things. Filters could be tested in isolation with some
> > image difference metrics rather than per-pixel independent comparisons.  
> 
> The metrics I was using had both a tolerance on the pixel value and on
> the number of pixels accepted outside of the value tolerance. I'm sure
> we can improve it (perhaps taking locality into account), but that's
> heuristics, and keeping heuristics working across a wide variety of use
> cases is hard.

Hi Laurent,

I was thinking of using a more, um, scientific error measures, e.g.
sum or squared errors (SSE) or average SSE over the whole
re-scaled/filtered image result, ignoring pixels outside of that.

What one normally uses when matching images in computer vision, for
example. Could even add a threshold such that simple rounding-level
errors would not even register in SSE. I'm sure there is plenty of
literature on that, but it may be behind a paywall like IEEE.

SSE may or may not need to computed from light-linear pixel values,
too. If one wanted to go even further, I'm sure there are computational
models about human color and brightness difference sensitivity that
could be used to weigh the errors.


> The filter I mentioned, by the way, is the scaler filter. Out of
> curiosity, do any of the devices you work on document with pixel-perfect
> precision how the hardware scaler is implemented ?

I don't work on drivers, so wouldn't even look for hardware docs. I go
by what KMS UAPI documents because that is the API I work with and
nothing else. And yes, I ignore all the scaling filter issues for now.

Because I don't have a way to get feedback (writeback connectors maybe
not existing and not hooked up in Weston quite yet), testing
scaling/filtering precision has not been on-topic yet. Right now I'm
interested in color correctness rather than geometrical filtering. For
traditional color management, the expected pixel values are quite
precise.

> > The order of operations in the color management pipeline is very
> > important. We can't work with "whatever". All the variability in
> > hardware is exactly why I have been calling out for defined abstract
> > color pipeline in the DRM UAPI docs, so that drivers would know which
> > properties to map to their elements, so that userspace can have any
> > possibility of using them correctly. If the hardware has a block
> > that doesn't fit in the abstract pipeline, you get to add things to the
> > abstract pipeline, or invent a whole another abstract pipeline and
> > document that.  
> 
> One very typical difference between devices is the order of the
> processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> gamma, we can accommodate hardware that have any combination of
> [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at

But you cannot represent pipelines like
1D LUT -> 1D LUT -> CCM
because the abstract pipeline just doesn't have the elements for that.
OTOH, maybe that ordering does not even make sense to have in hardware?
So maybe not all combinations are actually needed.

> different points in the pipeline depending on the device, and it will
> start getting complicated, even if the use case is quite simple and
> common. This is getting a bit out of topic, but how would you solve this
> one in particular ?

By defining all the points in the abstract color pipeline where a 3D
LUT could exist. Then each point would probably need its own KMS
property.

We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
drivers need to respect that order.

If the combinatorial explosion gets out of hand, maybe we need a KMS
property to switch to whole another abstract pipeline which defines a
different ordering on the same and/or different KMS properties.

From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
on each side of it for memory vs. precision optimization. And after the
degamma -> ctm -> gamma pipeline you may want one more ctm for
RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
all actually implemented hardware features might not go totally out of
hand.

> > Lossy compression needs its own KMS properties to ensure it can be
> > disabled when necessary. Near-lossless is not lossless, if a difference
> > can be measured. The driver or hardware cannot guess if the end user is
> > ok with "near-lossless" or not, so you have to be conservative and
> > assume not ok, offering an opt-in for lossy.  
> 
> Sure, but what would be the barrier to entry for such a property that
> would enable the compression (it could actually be a pixel format
> modifier) ? Would it only need to be documented ? Would we need a
> software implementation in VKMS and/or in IGT ? The compression
> algorithm is proprietary and not documented, so the latter can't be
> done.

Good questions. Shows that the idea of strictly requiring a VKMS
implementation won't fly, which is what I expected.

Saying it could be a pixel format modifier is a really good point. A
modifier cannot be the only thing to control it. Userspace does not
decode modifiers, so it cannot filter it out when it wants lossless
pixels. There must be something else to control it.

As a userspace dev, I would be ok with documenting a KMS property as
"improves blah blah, but also does significant violence to your
pixels", so I would know that this is something I need to consider.

You could argue that all KMS properties do violence to pixels, but
that's not a useful definition. It would just mean that in some use
cases I would never off-load anything to KMS. Depending on the use case
that might still be true even if the errors were limited to reasonable
rounding errors. I need an idea of how much error does KMS processing
do, and ultimately I expect compositors to also need a last resort
button for "do not trust KMS processing at all" which then makes the
display server always use the exact same simplest possible KMS
configuration and let the end user deal with the KMS errors via
color-profiling his monitor.

It's quite a different thing to have color processing elements in an
unexpected order in the pipeline than it is to have a scaling filter
doing slightly unknown operations.

> > ...
> >   
> > > > My underlying assumption is that generic userspace will not use
> > > > vendor-specific properties.    
> > > 
> > > I expect some amount of device-specific code in userspace, yes.   
> > 
> > If we had a reliable way to test device-specific code without the
> > hardware and automatically in CI, then maybe.
> >   
> > > There are usually large variations in how the hardware exposes access to
> > > a given feature, which leads to code having to convert the standard API
> > > parameters to hardware parameters. To a large extend this can be done in
> > > drivers, but for some more complex features, it may put too much burden
> > > on the kernel. There's a reason mesa is a userspace stack :-)  
> > 
> > If we get a Khronos standardised 2D composition API... oh wait.
> > 
> > Nothing wrong with userspace libraries, but it does mean that driver
> > developers need to contribute to them, like they do to Mesa. Is there
> > any of that going on for KMS?  
> 
> Not that I'm aware of, but I think that's a direction we can consider
> seriously.

That would be awesome if the API is generic.

> > In the mean time, DRM UAPI basically must define a 2D composition API,
> > or the new KMS properties will not see use outside of vendor trees.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:
> On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:
> > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:
> > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:  
> > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:  
> > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:    
> > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:    
> 
> ...
> 
> > > > > > > That is where the documented tolerances come into play.      
> > > > > > 
> > > > > > This is something I've experimented with a while ago, when developing
> > > > > > automated tests for the rcar-du driver. When playing with different
> > > > > > input images we had to constantly increases tolerances, up to a point
> > > > > > where the tests started to miss real problems :-(    
> > > > > 
> > > > > What should we infer from that? That the hardware is broken and
> > > > > exposing those KMS properties is a false promise?    
> > > > 
> > > > No, just that the scaler doesn't document the internal hardware
> > > > implementation (number of taps in the filters, coefficients, rounding,
> > > > ...). That's the rule, not the exception, and it doesn't prevent correct
> > > > operation, images get scaled in a reproducible way (the same input
> > > > produces the same output).
> > > >   
> > > > > If a driver on certain hardware cannot correctly implement a KMS
> > > > > property over the full domain of the input space, should that driver
> > > > > then simply not expose the KMS property at all?    
> > > > 
> > > > The properties involved here would the the SRC and CRTC rectangles for
> > > > the planes. They don't document pixel-perfect scaling :-)
> > > >   
> > > > > But I would assume that the vendor still wants to expose the features
> > > > > in upstream kernels, yet they cannot use the standard KMS properties
> > > > > for that. Should the driver then expose vendor-specific properties with
> > > > > the disclaimer that the result is not always what one would expect, so
> > > > > that userspace written and tested explicitly for that hardware can
> > > > > still work?
> > > > > 
> > > > > That is, a sufficient justification for a vendor-specific KMS property
> > > > > would be that a standard property already exists, but the hardware is
> > > > > too buggy to make it work. IOW, give up trying to make sense.    
> > > > 
> > > > It's not just about buggy hardware, it's also about implementation
> > > > specificities, such as rounding, filters, order of operations in the
> > > > color management pipeline (it's relatively easy when you only have two
> > > > LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> > > > features in the mix, not all hardware will implement the same pipeline),
> > > > or various types of image compression (this device implements a
> > > > "near-lossless" compression scheme that reduces the memory bandwidth by
> > > > 50% for instance).  
> > > 
> > > Rounding shouldn't result in needing wide tolerances.
> > > 
> > > Filters are more difficult, but at least we can factor them out when
> > > testing other things. Filters could be tested in isolation with some
> > > image difference metrics rather than per-pixel independent comparisons.  
> > 
> > The metrics I was using had both a tolerance on the pixel value and on
> > the number of pixels accepted outside of the value tolerance. I'm sure
> > we can improve it (perhaps taking locality into account), but that's
> > heuristics, and keeping heuristics working across a wide variety of use
> > cases is hard.
> 
> Hi Laurent,
> 
> I was thinking of using a more, um, scientific error measures, e.g.
> sum or squared errors (SSE) or average SSE over the whole
> re-scaled/filtered image result, ignoring pixels outside of that.

There are quite a few matching algorithms implement in imagemagick, we
have tried them all, and they all suffered for the issue I initially
described; we had to push the thresholds high enough to avoid false
positives, and we then saw false negatives.

> What one normally uses when matching images in computer vision, for
> example. Could even add a threshold such that simple rounding-level
> errors would not even register in SSE. I'm sure there is plenty of
> literature on that, but it may be behind a paywall like IEEE.
> 
> SSE may or may not need to computed from light-linear pixel values,
> too. If one wanted to go even further, I'm sure there are computational
> models about human color and brightness difference sensitivity that
> could be used to weigh the errors.

This isn't about human perception, but about the ability to check if the
driver correctly configures the device :-) For driver testing, I don't
care if the colours look right or not if userspace misconfigures the
colour pipeline. What I need is to catch regressions when my driver
messes up the hardware configuration. This particular device processes
the images in stripes, and it's very easy for a stripe to be one pixel
off (or actually a fraction of a pixel when the scaler is involved).

Testing if a device correctly implements colour processing is of course
also useful, but is a distinct goal. Both need to be considered, when
stating requirements such as fully documented properties with a VKMS
software implementation and pixel-perfect tests in IGT.

> > The filter I mentioned, by the way, is the scaler filter. Out of
> > curiosity, do any of the devices you work on document with pixel-perfect
> > precision how the hardware scaler is implemented ?
> 
> I don't work on drivers, so wouldn't even look for hardware docs. I go
> by what KMS UAPI documents because that is the API I work with and
> nothing else. And yes, I ignore all the scaling filter issues for now.
> 
> Because I don't have a way to get feedback (writeback connectors maybe
> not existing and not hooked up in Weston quite yet), testing
> scaling/filtering precision has not been on-topic yet. Right now I'm
> interested in color correctness rather than geometrical filtering. For
> traditional color management, the expected pixel values are quite
> precise.
> 
> > > The order of operations in the color management pipeline is very
> > > important. We can't work with "whatever". All the variability in
> > > hardware is exactly why I have been calling out for defined abstract
> > > color pipeline in the DRM UAPI docs, so that drivers would know which
> > > properties to map to their elements, so that userspace can have any
> > > possibility of using them correctly. If the hardware has a block
> > > that doesn't fit in the abstract pipeline, you get to add things to the
> > > abstract pipeline, or invent a whole another abstract pipeline and
> > > document that.  
> > 
> > One very typical difference between devices is the order of the
> > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > gamma, we can accommodate hardware that have any combination of
> > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
> 
> But you cannot represent pipelines like
> 1D LUT -> 1D LUT -> CCM
> because the abstract pipeline just doesn't have the elements for that.
> OTOH, maybe that ordering does not even make sense to have in hardware?
> So maybe not all combinations are actually needed.

If userspace wanted such a pipeline (I'm not sure why it would), then it
could just combine the two LUTs in one.

> > different points in the pipeline depending on the device, and it will
> > start getting complicated, even if the use case is quite simple and
> > common. This is getting a bit out of topic, but how would you solve this
> > one in particular ?
> 
> By defining all the points in the abstract color pipeline where a 3D
> LUT could exist. Then each point would probably need its own KMS
> property.

And when we'll add the next object ? This can't scale I'm afraid. You'll
have a spec that tells you that things can be in any order, and may well
be able to expose the order to userspace, but you won't be able to
implement a generic userspace that makes use of it.

BTW, if we want to expose a finer-grained topology of processing blocks,
I'd recommend looking at the Media Controller API, it was designed just
for that purpose.

> We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
> drivers need to respect that order.
> 
> If the combinatorial explosion gets out of hand, maybe we need a KMS
> property to switch to whole another abstract pipeline which defines a
> different ordering on the same and/or different KMS properties.
> 
> From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
> on each side of it for memory vs. precision optimization. And after the
> degamma -> ctm -> gamma pipeline you may want one more ctm for
> RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
> all actually implemented hardware features might not go totally out of
> hand.

On the output of the CRTC, I have, in this order

- RGB to YUV conversion (can only use presets for BT.601 and BT.709, with
  limited or full range)
- 3D LUT
- 1D LUT
- YUV to RGB conversion (presets only)

The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.

There's also a histogram engine, to allow implementation of dynamic
gamma correction. This would require userspace to read the histogram
data for every frame, and update the LUTs accordingly.

> > > Lossy compression needs its own KMS properties to ensure it can be
> > > disabled when necessary. Near-lossless is not lossless, if a difference
> > > can be measured. The driver or hardware cannot guess if the end user is
> > > ok with "near-lossless" or not, so you have to be conservative and
> > > assume not ok, offering an opt-in for lossy.  
> > 
> > Sure, but what would be the barrier to entry for such a property that
> > would enable the compression (it could actually be a pixel format
> > modifier) ? Would it only need to be documented ? Would we need a
> > software implementation in VKMS and/or in IGT ? The compression
> > algorithm is proprietary and not documented, so the latter can't be
> > done.
> 
> Good questions. Shows that the idea of strictly requiring a VKMS
> implementation won't fly, which is what I expected.
> 
> Saying it could be a pixel format modifier is a really good point. A
> modifier cannot be the only thing to control it. Userspace does not
> decode modifiers, so it cannot filter it out when it wants lossless
> pixels. There must be something else to control it.

The hardware implementation is such that the decompression is
implemented ahead of the display DMA engine, which means that it depends
on how the framebuffer memory is mapped. The display controller itself
isn't aware of the compression. The driver needs to map framebuffers to
the device through a specific memory window to decompress them.

> As a userspace dev, I would be ok with documenting a KMS property as
> "improves blah blah, but also does significant violence to your
> pixels", so I would know that this is something I need to consider.
> 
> You could argue that all KMS properties do violence to pixels, but
> that's not a useful definition. It would just mean that in some use
> cases I would never off-load anything to KMS. Depending on the use case
> that might still be true even if the errors were limited to reasonable
> rounding errors. I need an idea of how much error does KMS processing
> do, and ultimately I expect compositors to also need a last resort
> button for "do not trust KMS processing at all" which then makes the
> display server always use the exact same simplest possible KMS
> configuration and let the end user deal with the KMS errors via
> color-profiling his monitor.

I really think we'll end up needing device-specific userspace
components. Given the example above, with histogram calculation and
dynamic gamma correction, there's no way we'll be able to have a single
userspace implementation that would support these kind of features for
all devices. It doesn't have to be solved now, but we probably want to
start thinking about an API to plug device-specific components in
compositors. This is the kind of problem that the libcamera project is
tackling on the camera side, I believe a similar approach will be needed
for displays too.

> It's quite a different thing to have color processing elements in an
> unexpected order in the pipeline than it is to have a scaling filter
> doing slightly unknown operations.

That I agree with, those are two different issues. We all have our cross
to bear :-)

> > > ...
> > >   
> > > > > My underlying assumption is that generic userspace will not use
> > > > > vendor-specific properties.    
> > > > 
> > > > I expect some amount of device-specific code in userspace, yes.   
> > > 
> > > If we had a reliable way to test device-specific code without the
> > > hardware and automatically in CI, then maybe.
> > >   
> > > > There are usually large variations in how the hardware exposes access to
> > > > a given feature, which leads to code having to convert the standard API
> > > > parameters to hardware parameters. To a large extend this can be done in
> > > > drivers, but for some more complex features, it may put too much burden
> > > > on the kernel. There's a reason mesa is a userspace stack :-)  
> > > 
> > > If we get a Khronos standardised 2D composition API... oh wait.
> > > 
> > > Nothing wrong with userspace libraries, but it does mean that driver
> > > developers need to contribute to them, like they do to Mesa. Is there
> > > any of that going on for KMS?  
> > 
> > Not that I'm aware of, but I think that's a direction we can consider
> > seriously.
> 
> That would be awesome if the API is generic.
> 
> > > In the mean time, DRM UAPI basically must define a 2D composition API,
> > > or the new KMS properties will not see use outside of vendor trees.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:
> On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:
> > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:
> > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:  
> > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:  
> > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:    
> > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:    
> 
> ...
> 
> > > > > > > That is where the documented tolerances come into play.      
> > > > > > 
> > > > > > This is something I've experimented with a while ago, when developing
> > > > > > automated tests for the rcar-du driver. When playing with different
> > > > > > input images we had to constantly increases tolerances, up to a point
> > > > > > where the tests started to miss real problems :-(    
> > > > > 
> > > > > What should we infer from that? That the hardware is broken and
> > > > > exposing those KMS properties is a false promise?    
> > > > 
> > > > No, just that the scaler doesn't document the internal hardware
> > > > implementation (number of taps in the filters, coefficients, rounding,
> > > > ...). That's the rule, not the exception, and it doesn't prevent correct
> > > > operation, images get scaled in a reproducible way (the same input
> > > > produces the same output).
> > > >   
> > > > > If a driver on certain hardware cannot correctly implement a KMS
> > > > > property over the full domain of the input space, should that driver
> > > > > then simply not expose the KMS property at all?    
> > > > 
> > > > The properties involved here would the the SRC and CRTC rectangles for
> > > > the planes. They don't document pixel-perfect scaling :-)
> > > >   
> > > > > But I would assume that the vendor still wants to expose the features
> > > > > in upstream kernels, yet they cannot use the standard KMS properties
> > > > > for that. Should the driver then expose vendor-specific properties with
> > > > > the disclaimer that the result is not always what one would expect, so
> > > > > that userspace written and tested explicitly for that hardware can
> > > > > still work?
> > > > > 
> > > > > That is, a sufficient justification for a vendor-specific KMS property
> > > > > would be that a standard property already exists, but the hardware is
> > > > > too buggy to make it work. IOW, give up trying to make sense.    
> > > > 
> > > > It's not just about buggy hardware, it's also about implementation
> > > > specificities, such as rounding, filters, order of operations in the
> > > > color management pipeline (it's relatively easy when you only have two
> > > > LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping
> > > > features in the mix, not all hardware will implement the same pipeline),
> > > > or various types of image compression (this device implements a
> > > > "near-lossless" compression scheme that reduces the memory bandwidth by
> > > > 50% for instance).  
> > > 
> > > Rounding shouldn't result in needing wide tolerances.
> > > 
> > > Filters are more difficult, but at least we can factor them out when
> > > testing other things. Filters could be tested in isolation with some
> > > image difference metrics rather than per-pixel independent comparisons.  
> > 
> > The metrics I was using had both a tolerance on the pixel value and on
> > the number of pixels accepted outside of the value tolerance. I'm sure
> > we can improve it (perhaps taking locality into account), but that's
> > heuristics, and keeping heuristics working across a wide variety of use
> > cases is hard.
> 
> Hi Laurent,
> 
> I was thinking of using a more, um, scientific error measures, e.g.
> sum or squared errors (SSE) or average SSE over the whole
> re-scaled/filtered image result, ignoring pixels outside of that.

There are quite a few matching algorithms implement in imagemagick, we
have tried them all, and they all suffered for the issue I initially
described; we had to push the thresholds high enough to avoid false
positives, and we then saw false negatives.

> What one normally uses when matching images in computer vision, for
> example. Could even add a threshold such that simple rounding-level
> errors would not even register in SSE. I'm sure there is plenty of
> literature on that, but it may be behind a paywall like IEEE.
> 
> SSE may or may not need to computed from light-linear pixel values,
> too. If one wanted to go even further, I'm sure there are computational
> models about human color and brightness difference sensitivity that
> could be used to weigh the errors.

This isn't about human perception, but about the ability to check if the
driver correctly configures the device :-) For driver testing, I don't
care if the colours look right or not if userspace misconfigures the
colour pipeline. What I need is to catch regressions when my driver
messes up the hardware configuration. This particular device processes
the images in stripes, and it's very easy for a stripe to be one pixel
off (or actually a fraction of a pixel when the scaler is involved).

Testing if a device correctly implements colour processing is of course
also useful, but is a distinct goal. Both need to be considered, when
stating requirements such as fully documented properties with a VKMS
software implementation and pixel-perfect tests in IGT.

> > The filter I mentioned, by the way, is the scaler filter. Out of
> > curiosity, do any of the devices you work on document with pixel-perfect
> > precision how the hardware scaler is implemented ?
> 
> I don't work on drivers, so wouldn't even look for hardware docs. I go
> by what KMS UAPI documents because that is the API I work with and
> nothing else. And yes, I ignore all the scaling filter issues for now.
> 
> Because I don't have a way to get feedback (writeback connectors maybe
> not existing and not hooked up in Weston quite yet), testing
> scaling/filtering precision has not been on-topic yet. Right now I'm
> interested in color correctness rather than geometrical filtering. For
> traditional color management, the expected pixel values are quite
> precise.
> 
> > > The order of operations in the color management pipeline is very
> > > important. We can't work with "whatever". All the variability in
> > > hardware is exactly why I have been calling out for defined abstract
> > > color pipeline in the DRM UAPI docs, so that drivers would know which
> > > properties to map to their elements, so that userspace can have any
> > > possibility of using them correctly. If the hardware has a block
> > > that doesn't fit in the abstract pipeline, you get to add things to the
> > > abstract pipeline, or invent a whole another abstract pipeline and
> > > document that.  
> > 
> > One very typical difference between devices is the order of the
> > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > gamma, we can accommodate hardware that have any combination of
> > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
> 
> But you cannot represent pipelines like
> 1D LUT -> 1D LUT -> CCM
> because the abstract pipeline just doesn't have the elements for that.
> OTOH, maybe that ordering does not even make sense to have in hardware?
> So maybe not all combinations are actually needed.

If userspace wanted such a pipeline (I'm not sure why it would), then it
could just combine the two LUTs in one.

> > different points in the pipeline depending on the device, and it will
> > start getting complicated, even if the use case is quite simple and
> > common. This is getting a bit out of topic, but how would you solve this
> > one in particular ?
> 
> By defining all the points in the abstract color pipeline where a 3D
> LUT could exist. Then each point would probably need its own KMS
> property.

And when we'll add the next object ? This can't scale I'm afraid. You'll
have a spec that tells you that things can be in any order, and may well
be able to expose the order to userspace, but you won't be able to
implement a generic userspace that makes use of it.

BTW, if we want to expose a finer-grained topology of processing blocks,
I'd recommend looking at the Media Controller API, it was designed just
for that purpose.

> We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
> drivers need to respect that order.
> 
> If the combinatorial explosion gets out of hand, maybe we need a KMS
> property to switch to whole another abstract pipeline which defines a
> different ordering on the same and/or different KMS properties.
> 
> From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
> on each side of it for memory vs. precision optimization. And after the
> degamma -> ctm -> gamma pipeline you may want one more ctm for
> RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
> all actually implemented hardware features might not go totally out of
> hand.

On the output of the CRTC, I have, in this order

- RGB to YUV conversion (can only use presets for BT.601 and BT.709, with
  limited or full range)
- 3D LUT
- 1D LUT
- YUV to RGB conversion (presets only)

The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.

There's also a histogram engine, to allow implementation of dynamic
gamma correction. This would require userspace to read the histogram
data for every frame, and update the LUTs accordingly.

> > > Lossy compression needs its own KMS properties to ensure it can be
> > > disabled when necessary. Near-lossless is not lossless, if a difference
> > > can be measured. The driver or hardware cannot guess if the end user is
> > > ok with "near-lossless" or not, so you have to be conservative and
> > > assume not ok, offering an opt-in for lossy.  
> > 
> > Sure, but what would be the barrier to entry for such a property that
> > would enable the compression (it could actually be a pixel format
> > modifier) ? Would it only need to be documented ? Would we need a
> > software implementation in VKMS and/or in IGT ? The compression
> > algorithm is proprietary and not documented, so the latter can't be
> > done.
> 
> Good questions. Shows that the idea of strictly requiring a VKMS
> implementation won't fly, which is what I expected.
> 
> Saying it could be a pixel format modifier is a really good point. A
> modifier cannot be the only thing to control it. Userspace does not
> decode modifiers, so it cannot filter it out when it wants lossless
> pixels. There must be something else to control it.

The hardware implementation is such that the decompression is
implemented ahead of the display DMA engine, which means that it depends
on how the framebuffer memory is mapped. The display controller itself
isn't aware of the compression. The driver needs to map framebuffers to
the device through a specific memory window to decompress them.

> As a userspace dev, I would be ok with documenting a KMS property as
> "improves blah blah, but also does significant violence to your
> pixels", so I would know that this is something I need to consider.
> 
> You could argue that all KMS properties do violence to pixels, but
> that's not a useful definition. It would just mean that in some use
> cases I would never off-load anything to KMS. Depending on the use case
> that might still be true even if the errors were limited to reasonable
> rounding errors. I need an idea of how much error does KMS processing
> do, and ultimately I expect compositors to also need a last resort
> button for "do not trust KMS processing at all" which then makes the
> display server always use the exact same simplest possible KMS
> configuration and let the end user deal with the KMS errors via
> color-profiling his monitor.

I really think we'll end up needing device-specific userspace
components. Given the example above, with histogram calculation and
dynamic gamma correction, there's no way we'll be able to have a single
userspace implementation that would support these kind of features for
all devices. It doesn't have to be solved now, but we probably want to
start thinking about an API to plug device-specific components in
compositors. This is the kind of problem that the libcamera project is
tackling on the camera side, I believe a similar approach will be needed
for displays too.

> It's quite a different thing to have color processing elements in an
> unexpected order in the pipeline than it is to have a scaling filter
> doing slightly unknown operations.

That I agree with, those are two different issues. We all have our cross
to bear :-)

> > > ...
> > >   
> > > > > My underlying assumption is that generic userspace will not use
> > > > > vendor-specific properties.    
> > > > 
> > > > I expect some amount of device-specific code in userspace, yes.   
> > > 
> > > If we had a reliable way to test device-specific code without the
> > > hardware and automatically in CI, then maybe.
> > >   
> > > > There are usually large variations in how the hardware exposes access to
> > > > a given feature, which leads to code having to convert the standard API
> > > > parameters to hardware parameters. To a large extend this can be done in
> > > > drivers, but for some more complex features, it may put too much burden
> > > > on the kernel. There's a reason mesa is a userspace stack :-)  
> > > 
> > > If we get a Khronos standardised 2D composition API... oh wait.
> > > 
> > > Nothing wrong with userspace libraries, but it does mean that driver
> > > developers need to contribute to them, like they do to Mesa. Is there
> > > any of that going on for KMS?  
> > 
> > Not that I'm aware of, but I think that's a direction we can consider
> > seriously.
> 
> That would be awesome if the API is generic.
> 
> > > In the mean time, DRM UAPI basically must define a 2D composition API,
> > > or the new KMS properties will not see use outside of vendor trees.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 6996 bytes --]

On Fri, 18 Jun 2021 12:58:49 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:
> > On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:  
> > > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:  
> > > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:    
> > > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:    
> > > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:      
> > > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:      

...

> This isn't about human perception, but about the ability to check if the
> driver correctly configures the device :-) For driver testing, I don't
> care if the colours look right or not if userspace misconfigures the
> colour pipeline. What I need is to catch regressions when my driver
> messes up the hardware configuration. This particular device processes
> the images in stripes, and it's very easy for a stripe to be one pixel
> off (or actually a fraction of a pixel when the scaler is involved).
> 
> Testing if a device correctly implements colour processing is of course
> also useful, but is a distinct goal. Both need to be considered, when
> stating requirements such as fully documented properties with a VKMS
> software implementation and pixel-perfect tests in IGT.

Right. Your driver testing requirements are different. You know how your
hardware works, so you can build your tests to suit.

What this thread is about is KMS UAPI, and verifying that a KMS
implementation matches the KMS UAPI specification. The UAPI
specification is the only thing I can look at when writing userspace.
As a userspace and compositor developer, I am specifically concerned
about how humans will perceive the final image.

If you use your hardware-specific tests to ensure that your driver
programs the hardware correctly, and IGT tests ensure that the KMS UAPI
is implemented to the KMS UAPI spec, then I would hope that everything
is fine. By reading the KMS UAPI spec I can decide to use or not use
specific KMS features.

...in an ideal world.


> > > One very typical difference between devices is the order of the
> > > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > > gamma, we can accommodate hardware that have any combination of
> > > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at  
> > 
> > But you cannot represent pipelines like
> > 1D LUT -> 1D LUT -> CCM
> > because the abstract pipeline just doesn't have the elements for that.
> > OTOH, maybe that ordering does not even make sense to have in hardware?
> > So maybe not all combinations are actually needed.  
> 
> If userspace wanted such a pipeline (I'm not sure why it would), then it
> could just combine the two LUTs in one.

Maybe? You can also combine the 1D LUTs into the 3D LUT in the
middle, too, but the result is not generally the same as using them
separately when the 3D LUT size is limited.

> > > different points in the pipeline depending on the device, and it will
> > > start getting complicated, even if the use case is quite simple and
> > > common. This is getting a bit out of topic, but how would you solve this
> > > one in particular ?  
> > 
> > By defining all the points in the abstract color pipeline where a 3D
> > LUT could exist. Then each point would probably need its own KMS
> > property.  
> 
> And when we'll add the next object ? This can't scale I'm afraid. You'll
> have a spec that tells you that things can be in any order, and may well
> be able to expose the order to userspace, but you won't be able to
> implement a generic userspace that makes use of it.

Are you saying that generic userspace cannot happen? On top of KMS
UAPI, that is.

Why would a generic userspace library API be a more feasible effort?

> BTW, if we want to expose a finer-grained topology of processing blocks,
> I'd recommend looking at the Media Controller API, it was designed just
> for that purpose.

Sure. However, right now we have KMS properties instead. Or maybe they
should just go all unused?

Maybe the rule should be to not add any more KMS properties and tell
people to design something completely new based on Media Controller API
instead?

I've never looked at Media Controller API, so I have no idea how it
would suit display servers.

I very much think that I cannot use processing blocks exposed by
KMS if I cannot have guarantees that they work the same across all
Linux drivers.

> > We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
> > drivers need to respect that order.
> > 
> > If the combinatorial explosion gets out of hand, maybe we need a KMS
> > property to switch to whole another abstract pipeline which defines a
> > different ordering on the same and/or different KMS properties.
> > 
> > From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
> > on each side of it for memory vs. precision optimization. And after the
> > degamma -> ctm -> gamma pipeline you may want one more ctm for
> > RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
> > all actually implemented hardware features might not go totally out of
> > hand.  
> 
> On the output of the CRTC, I have, in this order
> 
> - RGB to YUV conversion (can only use presets for BT.601 and BT.709, with
>   limited or full range)
> - 3D LUT
> - 1D LUT
> - YUV to RGB conversion (presets only)
> 
> The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.
> 
> There's also a histogram engine, to allow implementation of dynamic
> gamma correction. This would require userspace to read the histogram
> data for every frame, and update the LUTs accordingly.

Interesting, but almost none of that would be used by the color
management pipeline model I'm familiar with. I wonder what kind of use
cases doing LUTs in YUV have been designed for, and *after* blending,
even.

...

> I really think we'll end up needing device-specific userspace
> components. Given the example above, with histogram calculation and
> dynamic gamma correction, there's no way we'll be able to have a single
> userspace implementation that would support these kind of features for
> all devices. It doesn't have to be solved now, but we probably want to
> start thinking about an API to plug device-specific components in
> compositors. This is the kind of problem that the libcamera project is
> tackling on the camera side, I believe a similar approach will be needed
> for displays too.

To be able to plug device-specific components in compositors, we need a
driver-agnostic API through which compositors can use those device
specific components. Still the same problem, now just in userspace than
at the UAPI level.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 6996 bytes --]

On Fri, 18 Jun 2021 12:58:49 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:
> > On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:  
> > > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:  
> > > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:    
> > > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:    
> > > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:      
> > > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:      

...

> This isn't about human perception, but about the ability to check if the
> driver correctly configures the device :-) For driver testing, I don't
> care if the colours look right or not if userspace misconfigures the
> colour pipeline. What I need is to catch regressions when my driver
> messes up the hardware configuration. This particular device processes
> the images in stripes, and it's very easy for a stripe to be one pixel
> off (or actually a fraction of a pixel when the scaler is involved).
> 
> Testing if a device correctly implements colour processing is of course
> also useful, but is a distinct goal. Both need to be considered, when
> stating requirements such as fully documented properties with a VKMS
> software implementation and pixel-perfect tests in IGT.

Right. Your driver testing requirements are different. You know how your
hardware works, so you can build your tests to suit.

What this thread is about is KMS UAPI, and verifying that a KMS
implementation matches the KMS UAPI specification. The UAPI
specification is the only thing I can look at when writing userspace.
As a userspace and compositor developer, I am specifically concerned
about how humans will perceive the final image.

If you use your hardware-specific tests to ensure that your driver
programs the hardware correctly, and IGT tests ensure that the KMS UAPI
is implemented to the KMS UAPI spec, then I would hope that everything
is fine. By reading the KMS UAPI spec I can decide to use or not use
specific KMS features.

...in an ideal world.


> > > One very typical difference between devices is the order of the
> > > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > > gamma, we can accommodate hardware that have any combination of
> > > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at  
> > 
> > But you cannot represent pipelines like
> > 1D LUT -> 1D LUT -> CCM
> > because the abstract pipeline just doesn't have the elements for that.
> > OTOH, maybe that ordering does not even make sense to have in hardware?
> > So maybe not all combinations are actually needed.  
> 
> If userspace wanted such a pipeline (I'm not sure why it would), then it
> could just combine the two LUTs in one.

Maybe? You can also combine the 1D LUTs into the 3D LUT in the
middle, too, but the result is not generally the same as using them
separately when the 3D LUT size is limited.

> > > different points in the pipeline depending on the device, and it will
> > > start getting complicated, even if the use case is quite simple and
> > > common. This is getting a bit out of topic, but how would you solve this
> > > one in particular ?  
> > 
> > By defining all the points in the abstract color pipeline where a 3D
> > LUT could exist. Then each point would probably need its own KMS
> > property.  
> 
> And when we'll add the next object ? This can't scale I'm afraid. You'll
> have a spec that tells you that things can be in any order, and may well
> be able to expose the order to userspace, but you won't be able to
> implement a generic userspace that makes use of it.

Are you saying that generic userspace cannot happen? On top of KMS
UAPI, that is.

Why would a generic userspace library API be a more feasible effort?

> BTW, if we want to expose a finer-grained topology of processing blocks,
> I'd recommend looking at the Media Controller API, it was designed just
> for that purpose.

Sure. However, right now we have KMS properties instead. Or maybe they
should just go all unused?

Maybe the rule should be to not add any more KMS properties and tell
people to design something completely new based on Media Controller API
instead?

I've never looked at Media Controller API, so I have no idea how it
would suit display servers.

I very much think that I cannot use processing blocks exposed by
KMS if I cannot have guarantees that they work the same across all
Linux drivers.

> > We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
> > drivers need to respect that order.
> > 
> > If the combinatorial explosion gets out of hand, maybe we need a KMS
> > property to switch to whole another abstract pipeline which defines a
> > different ordering on the same and/or different KMS properties.
> > 
> > From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
> > on each side of it for memory vs. precision optimization. And after the
> > degamma -> ctm -> gamma pipeline you may want one more ctm for
> > RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
> > all actually implemented hardware features might not go totally out of
> > hand.  
> 
> On the output of the CRTC, I have, in this order
> 
> - RGB to YUV conversion (can only use presets for BT.601 and BT.709, with
>   limited or full range)
> - 3D LUT
> - 1D LUT
> - YUV to RGB conversion (presets only)
> 
> The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.
> 
> There's also a histogram engine, to allow implementation of dynamic
> gamma correction. This would require userspace to read the histogram
> data for every frame, and update the LUTs accordingly.

Interesting, but almost none of that would be used by the color
management pipeline model I'm familiar with. I wonder what kind of use
cases doing LUTs in YUV have been designed for, and *after* blending,
even.

...

> I really think we'll end up needing device-specific userspace
> components. Given the example above, with histogram calculation and
> dynamic gamma correction, there's no way we'll be able to have a single
> userspace implementation that would support these kind of features for
> all devices. It doesn't have to be solved now, but we probably want to
> start thinking about an API to plug device-specific components in
> compositors. This is the kind of problem that the libcamera project is
> tackling on the camera side, I believe a similar approach will be needed
> for displays too.

To be able to plug device-specific components in compositors, we need a
driver-agnostic API through which compositors can use those device
specific components. Still the same problem, now just in userspace than
at the UAPI level.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Fri, Jun 18, 2021 at 02:32:00PM +0300, Pekka Paalanen wrote:
> On Fri, 18 Jun 2021 12:58:49 +0300 Laurent Pinchart wrote:
> > On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:
> > > On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:  
> > > > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:  
> > > > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:    
> > > > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:    
> > > > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:      
> > > > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:      
> 
> ...
> 
> > This isn't about human perception, but about the ability to check if the
> > driver correctly configures the device :-) For driver testing, I don't
> > care if the colours look right or not if userspace misconfigures the
> > colour pipeline. What I need is to catch regressions when my driver
> > messes up the hardware configuration. This particular device processes
> > the images in stripes, and it's very easy for a stripe to be one pixel
> > off (or actually a fraction of a pixel when the scaler is involved).
> > 
> > Testing if a device correctly implements colour processing is of course
> > also useful, but is a distinct goal. Both need to be considered, when
> > stating requirements such as fully documented properties with a VKMS
> > software implementation and pixel-perfect tests in IGT.
> 
> Right. Your driver testing requirements are different. You know how your
> hardware works, so you can build your tests to suit.
> 
> What this thread is about is KMS UAPI, and verifying that a KMS
> implementation matches the KMS UAPI specification. The UAPI
> specification is the only thing I can look at when writing userspace.
> As a userspace and compositor developer, I am specifically concerned
> about how humans will perceive the final image.

Yes, the thread has gone in a few different directions.

> If you use your hardware-specific tests to ensure that your driver
> programs the hardware correctly, and IGT tests ensure that the KMS UAPI
> is implemented to the KMS UAPI spec, then I would hope that everything
> is fine. By reading the KMS UAPI spec I can decide to use or not use
> specific KMS features.
> 
> ...in an ideal world.

I wish I knew where that world was ;-)

I do agree with you though, you need predictability in userspace,
otherwise it's quite useless (or not very useful). This brings back many
memories (so unfortunately not too distant) with cameras. No latter than
yesterday, I heard from a person with lots of involvement in GPU API
design that GPUs seem easy compared to cameras. Hopefully this means
we'll be able to solve this on the display side :-)

> > > > One very typical difference between devices is the order of the
> > > > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > > > gamma, we can accommodate hardware that have any combination of
> > > > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at  
> > > 
> > > But you cannot represent pipelines like
> > > 1D LUT -> 1D LUT -> CCM
> > > because the abstract pipeline just doesn't have the elements for that.
> > > OTOH, maybe that ordering does not even make sense to have in hardware?
> > > So maybe not all combinations are actually needed.  
> > 
> > If userspace wanted such a pipeline (I'm not sure why it would), then it
> > could just combine the two LUTs in one.
> 
> Maybe? You can also combine the 1D LUTs into the 3D LUT in the
> middle, too, but the result is not generally the same as using them
> separately when the 3D LUT size is limited.

Yes, my 3D LUT has 17 points, I've heard about LUTs up to 33 points
(with bilinear interpolation), that's quite little if used as a 1D LUT.
There's also the fact that interpolation will result in cross-talk
between the three colour channels.

> > > > different points in the pipeline depending on the device, and it will
> > > > start getting complicated, even if the use case is quite simple and
> > > > common. This is getting a bit out of topic, but how would you solve this
> > > > one in particular ?  
> > > 
> > > By defining all the points in the abstract color pipeline where a 3D
> > > LUT could exist. Then each point would probably need its own KMS
> > > property.  
> > 
> > And when we'll add the next object ? This can't scale I'm afraid. You'll
> > have a spec that tells you that things can be in any order, and may well
> > be able to expose the order to userspace, but you won't be able to
> > implement a generic userspace that makes use of it.
> 
> Are you saying that generic userspace cannot happen? On top of KMS
> UAPI, that is.
> 
> Why would a generic userspace library API be a more feasible effort?

I'll make a parallel with the camera world again, because I think
display hardware is taking a similar path, just later.

We used to have capture devices that offered a fairly high-level
interface at the hardware or firmware level, with a set of features that
were more or less standard. Devices could be exposed with an abstract
model, as done in KMS today, with a set of standardized controls (V4L2
uses the word control when KMS uses property). We had way less automated
testing, and didn't enough attention to consistency of the userspace API
(we're talking about code from the beginning of the century, if not
earlier, the world was different), so different drivers were not
consistent in how the API was implemented (for instance the API didn't
specify gain units, so drivers would use different units), but more or
less, you could write a standard V4L2 application that would work with
most devices, with a few quirks here and there.

Then the world changed. Cameras become more complex, with the raw sensor
and the ISP exposed to Linux in the OMAP3 that Nokia used in the N900
and N9 phones. For a while we tried to stick with V4L2, and implement
ISP control within the kernel, with the help of a userspace daemon
behind the scenes to which the kernel driver called back. The daemon was
required, as control of the camera required complicated computation that
really benefited from floating point math. It was horrible, and the
daemon was closed-source, but from an application point of view, it was
still high-level V4L2 as before. This hit a big wall though, to expose a
high-level API on top of a low-level hardware interface, we had to
bridge the gap in the kernel. This meant hardcoding use cases on the
kernel side, which clearly wouldn't scale, as not everybody had the same
use cases (an industrial vision application and a smartphone camera
application really don't see the world the same way).  We were facing a
dead end.

We then decided to expose a lower-level API from kernel to userspace,
giving userspace the ability to control everything. That's how the Media
Controller API was born (its purpose is to expose the internal topology
of the device as a graph of connected entities, that's about it), as
well as the V4L2 subdev userspace API (in KMS term, that would mean more
or less exposing DRM bridges to userspace, with the ability to control
them individually). This simplified the kernel implementation (no more
hardcoded use cases, no more daemon behind the scenes), but at the same
time resulted in an API that wasn't a good fit for applications to
consume directly. One step closer to the hardware, one step further from
the user. You can think of this as the API exposed by GPU kernel
drivers, it's not enough, a userspace component to translate to a
higher-level API is needed.

This was followed by 10 years of limbo as Nokia pulled the plug from
Linux (and then smartphone) development. Fast forward, at the end of
2018, we created libcamera as the mesa of the camera world to solve this
issue. It offers a standard camera stack with a high-level API, where
device-specific code can be added (and kept to the minimum, as a
centralized framework means we can share code that is common between
devices).

I see lots of parallel with display pipelines getting more complex. It
wouldn't be fair to expose a very low-level API and ask every single
compositor to know about all the devices out there, but at the same
time, we're reaching a level of hardware complexity that I believe makes
it a dead end to try to abstract and standardize everything at the
kernel level. This doesn't mean we should allow all vendors to throw in
any kind of crappy properties or other API extensions, we can still
enforce some ground rules, such as proper documentation, reasonable
effort for standardization, and reference open-source userspace
implementation in a project we would consider to be the equalivant of
mesa for KMS.

> > BTW, if we want to expose a finer-grained topology of processing blocks,
> > I'd recommend looking at the Media Controller API, it was designed just
> > for that purpose.
> 
> Sure. However, right now we have KMS properties instead. Or maybe they
> should just go all unused?
> 
> Maybe the rule should be to not add any more KMS properties and tell
> people to design something completely new based on Media Controller API
> instead?

No, I think that would be too harsh, my point was that if we want to
expose how the building blocks are layed out, the MC API is meant to
expose a graph of connected entities, so it's a good match. Controlling
the device should still go through KMS properties I believe.

> I've never looked at Media Controller API, so I have no idea how it
> would suit display servers.
> 
> I very much think that I cannot use processing blocks exposed by
> KMS if I cannot have guarantees that they work the same across all
> Linux drivers.
>
> > > We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
> > > drivers need to respect that order.
> > > 
> > > If the combinatorial explosion gets out of hand, maybe we need a KMS
> > > property to switch to whole another abstract pipeline which defines a
> > > different ordering on the same and/or different KMS properties.
> > > 
> > > From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
> > > on each side of it for memory vs. precision optimization. And after the
> > > degamma -> ctm -> gamma pipeline you may want one more ctm for
> > > RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
> > > all actually implemented hardware features might not go totally out of
> > > hand.  
> > 
> > On the output of the CRTC, I have, in this order
> > 
> > - RGB to YUV conversion (can only use presets for BT.601 and BT.709, with
> >   limited or full range)
> > - 3D LUT
> > - 1D LUT
> > - YUV to RGB conversion (presets only)
> > 
> > The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.
> > 
> > There's also a histogram engine, to allow implementation of dynamic
> > gamma correction. This would require userspace to read the histogram
> > data for every frame, and update the LUTs accordingly.
> 
> Interesting, but almost none of that would be used by the color
> management pipeline model I'm familiar with. I wonder what kind of use
> cases doing LUTs in YUV have been designed for, and *after* blending,
> even.

I'm afraid I can't tell as I don't know. This reaches the limit of my
display colour management knowledge.

> ...
> 
> > I really think we'll end up needing device-specific userspace
> > components. Given the example above, with histogram calculation and
> > dynamic gamma correction, there's no way we'll be able to have a single
> > userspace implementation that would support these kind of features for
> > all devices. It doesn't have to be solved now, but we probably want to
> > start thinking about an API to plug device-specific components in
> > compositors. This is the kind of problem that the libcamera project is
> > tackling on the camera side, I believe a similar approach will be needed
> > for displays too.
> 
> To be able to plug device-specific components in compositors, we need a
> driver-agnostic API through which compositors can use those device
> specific components. Still the same problem, now just in userspace than
> at the UAPI level.

Agreed, it pushes the problem to a different place :-) I however think
it may be easier to solve it in userspace, as it's a better place to
hardcode use cases. A general-purpose compositor may not want to make
use of all hardware features, while a specific-purpose compositor may
want to exercise the hardware features differently. This however opens
the door to introducing badly-designed and un(der-)documented
properties, which is not something I want, so we would still need to set
ground rules, such as documentation and reference implementation in a
given userspace compositor/stack.

On that note, I think we need to do a bit more hand-holding. I'm sitting
on a patch series that is several years old to add 3D LUT support to my
driver, and while I believe I'm quite experienced with KMS development,
adding support for this in a reference compositor in userspace is a
really big task that I don't know how to tackle. I also know very little
about colour management, and have nobody in the team who has that
knowledge and could help me. I'm thus facing a bit of a dead-end, not
because I don't want to spend the time required for this, but because
the bar is so high that I just can't do it. I would expect many
developers to be facing the same challenges. If the community doesn't
make an effort in fostering collaboration between developers (most
kernel developers won't even know who to ask for help if they have to
step out of the kernel to userspace) and providing good documentation,
we'll just scare developers away. It took me weeks to figure out how to
run Weston on this device, as it has no GPU that I can use, and falling
back to a software compositor required finding out that lots of
documentation is outdated and referenced command line options that were
not accurate anymore (or I could mention painfully finding out that an
issue I was blocked with for days was solved by plugging in a USB mouse
to the board, as Weston didn't want to start without an input device).
Another developer from my team tried before me and gave up, thinking it
was impossible. That's the reality that many kernel developers are
facing, the same way I suppose than me saying saying "you just need to
recompile the kernel with this option enabled" is not very helpful for
lots of userspace developers.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Laurent Pinchart]

Hi Pekka,

On Fri, Jun 18, 2021 at 02:32:00PM +0300, Pekka Paalanen wrote:
> On Fri, 18 Jun 2021 12:58:49 +0300 Laurent Pinchart wrote:
> > On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:
> > > On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:  
> > > > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:  
> > > > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:    
> > > > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:    
> > > > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:      
> > > > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:      
> 
> ...
> 
> > This isn't about human perception, but about the ability to check if the
> > driver correctly configures the device :-) For driver testing, I don't
> > care if the colours look right or not if userspace misconfigures the
> > colour pipeline. What I need is to catch regressions when my driver
> > messes up the hardware configuration. This particular device processes
> > the images in stripes, and it's very easy for a stripe to be one pixel
> > off (or actually a fraction of a pixel when the scaler is involved).
> > 
> > Testing if a device correctly implements colour processing is of course
> > also useful, but is a distinct goal. Both need to be considered, when
> > stating requirements such as fully documented properties with a VKMS
> > software implementation and pixel-perfect tests in IGT.
> 
> Right. Your driver testing requirements are different. You know how your
> hardware works, so you can build your tests to suit.
> 
> What this thread is about is KMS UAPI, and verifying that a KMS
> implementation matches the KMS UAPI specification. The UAPI
> specification is the only thing I can look at when writing userspace.
> As a userspace and compositor developer, I am specifically concerned
> about how humans will perceive the final image.

Yes, the thread has gone in a few different directions.

> If you use your hardware-specific tests to ensure that your driver
> programs the hardware correctly, and IGT tests ensure that the KMS UAPI
> is implemented to the KMS UAPI spec, then I would hope that everything
> is fine. By reading the KMS UAPI spec I can decide to use or not use
> specific KMS features.
> 
> ...in an ideal world.

I wish I knew where that world was ;-)

I do agree with you though, you need predictability in userspace,
otherwise it's quite useless (or not very useful). This brings back many
memories (so unfortunately not too distant) with cameras. No latter than
yesterday, I heard from a person with lots of involvement in GPU API
design that GPUs seem easy compared to cameras. Hopefully this means
we'll be able to solve this on the display side :-)

> > > > One very typical difference between devices is the order of the
> > > > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > > > gamma, we can accommodate hardware that have any combination of
> > > > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at  
> > > 
> > > But you cannot represent pipelines like
> > > 1D LUT -> 1D LUT -> CCM
> > > because the abstract pipeline just doesn't have the elements for that.
> > > OTOH, maybe that ordering does not even make sense to have in hardware?
> > > So maybe not all combinations are actually needed.  
> > 
> > If userspace wanted such a pipeline (I'm not sure why it would), then it
> > could just combine the two LUTs in one.
> 
> Maybe? You can also combine the 1D LUTs into the 3D LUT in the
> middle, too, but the result is not generally the same as using them
> separately when the 3D LUT size is limited.

Yes, my 3D LUT has 17 points, I've heard about LUTs up to 33 points
(with bilinear interpolation), that's quite little if used as a 1D LUT.
There's also the fact that interpolation will result in cross-talk
between the three colour channels.

> > > > different points in the pipeline depending on the device, and it will
> > > > start getting complicated, even if the use case is quite simple and
> > > > common. This is getting a bit out of topic, but how would you solve this
> > > > one in particular ?  
> > > 
> > > By defining all the points in the abstract color pipeline where a 3D
> > > LUT could exist. Then each point would probably need its own KMS
> > > property.  
> > 
> > And when we'll add the next object ? This can't scale I'm afraid. You'll
> > have a spec that tells you that things can be in any order, and may well
> > be able to expose the order to userspace, but you won't be able to
> > implement a generic userspace that makes use of it.
> 
> Are you saying that generic userspace cannot happen? On top of KMS
> UAPI, that is.
> 
> Why would a generic userspace library API be a more feasible effort?

I'll make a parallel with the camera world again, because I think
display hardware is taking a similar path, just later.

We used to have capture devices that offered a fairly high-level
interface at the hardware or firmware level, with a set of features that
were more or less standard. Devices could be exposed with an abstract
model, as done in KMS today, with a set of standardized controls (V4L2
uses the word control when KMS uses property). We had way less automated
testing, and didn't enough attention to consistency of the userspace API
(we're talking about code from the beginning of the century, if not
earlier, the world was different), so different drivers were not
consistent in how the API was implemented (for instance the API didn't
specify gain units, so drivers would use different units), but more or
less, you could write a standard V4L2 application that would work with
most devices, with a few quirks here and there.

Then the world changed. Cameras become more complex, with the raw sensor
and the ISP exposed to Linux in the OMAP3 that Nokia used in the N900
and N9 phones. For a while we tried to stick with V4L2, and implement
ISP control within the kernel, with the help of a userspace daemon
behind the scenes to which the kernel driver called back. The daemon was
required, as control of the camera required complicated computation that
really benefited from floating point math. It was horrible, and the
daemon was closed-source, but from an application point of view, it was
still high-level V4L2 as before. This hit a big wall though, to expose a
high-level API on top of a low-level hardware interface, we had to
bridge the gap in the kernel. This meant hardcoding use cases on the
kernel side, which clearly wouldn't scale, as not everybody had the same
use cases (an industrial vision application and a smartphone camera
application really don't see the world the same way).  We were facing a
dead end.

We then decided to expose a lower-level API from kernel to userspace,
giving userspace the ability to control everything. That's how the Media
Controller API was born (its purpose is to expose the internal topology
of the device as a graph of connected entities, that's about it), as
well as the V4L2 subdev userspace API (in KMS term, that would mean more
or less exposing DRM bridges to userspace, with the ability to control
them individually). This simplified the kernel implementation (no more
hardcoded use cases, no more daemon behind the scenes), but at the same
time resulted in an API that wasn't a good fit for applications to
consume directly. One step closer to the hardware, one step further from
the user. You can think of this as the API exposed by GPU kernel
drivers, it's not enough, a userspace component to translate to a
higher-level API is needed.

This was followed by 10 years of limbo as Nokia pulled the plug from
Linux (and then smartphone) development. Fast forward, at the end of
2018, we created libcamera as the mesa of the camera world to solve this
issue. It offers a standard camera stack with a high-level API, where
device-specific code can be added (and kept to the minimum, as a
centralized framework means we can share code that is common between
devices).

I see lots of parallel with display pipelines getting more complex. It
wouldn't be fair to expose a very low-level API and ask every single
compositor to know about all the devices out there, but at the same
time, we're reaching a level of hardware complexity that I believe makes
it a dead end to try to abstract and standardize everything at the
kernel level. This doesn't mean we should allow all vendors to throw in
any kind of crappy properties or other API extensions, we can still
enforce some ground rules, such as proper documentation, reasonable
effort for standardization, and reference open-source userspace
implementation in a project we would consider to be the equalivant of
mesa for KMS.

> > BTW, if we want to expose a finer-grained topology of processing blocks,
> > I'd recommend looking at the Media Controller API, it was designed just
> > for that purpose.
> 
> Sure. However, right now we have KMS properties instead. Or maybe they
> should just go all unused?
> 
> Maybe the rule should be to not add any more KMS properties and tell
> people to design something completely new based on Media Controller API
> instead?

No, I think that would be too harsh, my point was that if we want to
expose how the building blocks are layed out, the MC API is meant to
expose a graph of connected entities, so it's a good match. Controlling
the device should still go through KMS properties I believe.

> I've never looked at Media Controller API, so I have no idea how it
> would suit display servers.
> 
> I very much think that I cannot use processing blocks exposed by
> KMS if I cannot have guarantees that they work the same across all
> Linux drivers.
>
> > > We already have the KMS pipeline exactly as degamma -> ctm -> gamma and
> > > drivers need to respect that order.
> > > 
> > > If the combinatorial explosion gets out of hand, maybe we need a KMS
> > > property to switch to whole another abstract pipeline which defines a
> > > different ordering on the same and/or different KMS properties.
> > > 
> > > From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT
> > > on each side of it for memory vs. precision optimization. And after the
> > > degamma -> ctm -> gamma pipeline you may want one more ctm for
> > > RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with
> > > all actually implemented hardware features might not go totally out of
> > > hand.  
> > 
> > On the output of the CRTC, I have, in this order
> > 
> > - RGB to YUV conversion (can only use presets for BT.601 and BT.709, with
> >   limited or full range)
> > - 3D LUT
> > - 1D LUT
> > - YUV to RGB conversion (presets only)
> > 
> > The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.
> > 
> > There's also a histogram engine, to allow implementation of dynamic
> > gamma correction. This would require userspace to read the histogram
> > data for every frame, and update the LUTs accordingly.
> 
> Interesting, but almost none of that would be used by the color
> management pipeline model I'm familiar with. I wonder what kind of use
> cases doing LUTs in YUV have been designed for, and *after* blending,
> even.

I'm afraid I can't tell as I don't know. This reaches the limit of my
display colour management knowledge.

> ...
> 
> > I really think we'll end up needing device-specific userspace
> > components. Given the example above, with histogram calculation and
> > dynamic gamma correction, there's no way we'll be able to have a single
> > userspace implementation that would support these kind of features for
> > all devices. It doesn't have to be solved now, but we probably want to
> > start thinking about an API to plug device-specific components in
> > compositors. This is the kind of problem that the libcamera project is
> > tackling on the camera side, I believe a similar approach will be needed
> > for displays too.
> 
> To be able to plug device-specific components in compositors, we need a
> driver-agnostic API through which compositors can use those device
> specific components. Still the same problem, now just in userspace than
> at the UAPI level.

Agreed, it pushes the problem to a different place :-) I however think
it may be easier to solve it in userspace, as it's a better place to
hardcode use cases. A general-purpose compositor may not want to make
use of all hardware features, while a specific-purpose compositor may
want to exercise the hardware features differently. This however opens
the door to introducing badly-designed and un(der-)documented
properties, which is not something I want, so we would still need to set
ground rules, such as documentation and reference implementation in a
given userspace compositor/stack.

On that note, I think we need to do a bit more hand-holding. I'm sitting
on a patch series that is several years old to add 3D LUT support to my
driver, and while I believe I'm quite experienced with KMS development,
adding support for this in a reference compositor in userspace is a
really big task that I don't know how to tackle. I also know very little
about colour management, and have nobody in the team who has that
knowledge and could help me. I'm thus facing a bit of a dead-end, not
because I don't want to spend the time required for this, but because
the bar is so high that I just can't do it. I would expect many
developers to be facing the same challenges. If the community doesn't
make an effort in fostering collaboration between developers (most
kernel developers won't even know who to ask for help if they have to
step out of the kernel to userspace) and providing good documentation,
we'll just scare developers away. It took me weeks to figure out how to
run Weston on this device, as it has no GPU that I can use, and falling
back to a software compositor required finding out that lots of
documentation is outdated and referenced command line options that were
not accurate anymore (or I could mention painfully finding out that an
issue I was blocked with for days was solved by plugging in a USB mouse
to the board, as Weston didn't want to start without an input device).
Another developer from my team tried before me and gave up, thinking it
was impossible. That's the reality that many kernel developers are
facing, the same way I suppose than me saying saying "you just need to
recompile the kernel with this option enabled" is not very helpful for
lots of userspace developers.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 12248 bytes --]

On Fri, 18 Jun 2021 15:19:15 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Fri, Jun 18, 2021 at 02:32:00PM +0300, Pekka Paalanen wrote:
> > On Fri, 18 Jun 2021 12:58:49 +0300 Laurent Pinchart wrote:  
> > > On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:  
> > > > On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:    
> > > > > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:    
> > > > > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:      
> > > > > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:      
> > > > > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:        
> > > > > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:        

...

> > > > > One very typical difference between devices is the order of the
> > > > > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > > > > gamma, we can accommodate hardware that have any combination of
> > > > > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at    
> > > > 
> > > > But you cannot represent pipelines like
> > > > 1D LUT -> 1D LUT -> CCM
> > > > because the abstract pipeline just doesn't have the elements for that.
> > > > OTOH, maybe that ordering does not even make sense to have in hardware?
> > > > So maybe not all combinations are actually needed.    
> > > 
> > > If userspace wanted such a pipeline (I'm not sure why it would), then it
> > > could just combine the two LUTs in one.  
> > 
> > Maybe? You can also combine the 1D LUTs into the 3D LUT in the
> > middle, too, but the result is not generally the same as using them
> > separately when the 3D LUT size is limited.  
> 
> Yes, my 3D LUT has 17 points, I've heard about LUTs up to 33 points
> (with bilinear interpolation), that's quite little if used as a 1D LUT.
> There's also the fact that interpolation will result in cross-talk
> between the three colour channels.

Hi Laurent,

that is not what I meant, but that's true too.

What I meant was this:
https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/582#note_951271

pre_curve and post_curve are the 1D LUTs on each side of a size-limited
3D LUT. pre_curve controls the 3D LUT tap positioning to concentrate on
the areas where the approximated function needs more taps.


> > Why would a generic userspace library API be a more feasible effort?  
> 
> I'll make a parallel with the camera world again, because I think
> display hardware is taking a similar path, just later.
> 
> We used to have capture devices that offered a fairly high-level
> interface at the hardware or firmware level, with a set of features that
> were more or less standard. Devices could be exposed with an abstract
> model, as done in KMS today, with a set of standardized controls (V4L2
> uses the word control when KMS uses property). We had way less automated
> testing, and didn't enough attention to consistency of the userspace API
> (we're talking about code from the beginning of the century, if not
> earlier, the world was different), so different drivers were not
> consistent in how the API was implemented (for instance the API didn't
> specify gain units, so drivers would use different units), but more or
> less, you could write a standard V4L2 application that would work with
> most devices, with a few quirks here and there.
> 
> Then the world changed. Cameras become more complex, with the raw sensor
> and the ISP exposed to Linux in the OMAP3 that Nokia used in the N900
> and N9 phones. For a while we tried to stick with V4L2, and implement
> ISP control within the kernel, with the help of a userspace daemon
> behind the scenes to which the kernel driver called back. The daemon was
> required, as control of the camera required complicated computation that
> really benefited from floating point math. It was horrible, and the
> daemon was closed-source, but from an application point of view, it was
> still high-level V4L2 as before. This hit a big wall though, to expose a
> high-level API on top of a low-level hardware interface, we had to
> bridge the gap in the kernel. This meant hardcoding use cases on the
> kernel side, which clearly wouldn't scale, as not everybody had the same
> use cases (an industrial vision application and a smartphone camera
> application really don't see the world the same way).  We were facing a
> dead end.
> 
> We then decided to expose a lower-level API from kernel to userspace,
> giving userspace the ability to control everything. That's how the Media
> Controller API was born (its purpose is to expose the internal topology
> of the device as a graph of connected entities, that's about it), as
> well as the V4L2 subdev userspace API (in KMS term, that would mean more
> or less exposing DRM bridges to userspace, with the ability to control
> them individually). This simplified the kernel implementation (no more
> hardcoded use cases, no more daemon behind the scenes), but at the same
> time resulted in an API that wasn't a good fit for applications to
> consume directly. One step closer to the hardware, one step further from
> the user. You can think of this as the API exposed by GPU kernel
> drivers, it's not enough, a userspace component to translate to a
> higher-level API is needed.
> 
> This was followed by 10 years of limbo as Nokia pulled the plug from
> Linux (and then smartphone) development. Fast forward, at the end of
> 2018, we created libcamera as the mesa of the camera world to solve this
> issue. It offers a standard camera stack with a high-level API, where
> device-specific code can be added (and kept to the minimum, as a
> centralized framework means we can share code that is common between
> devices).
> 
> I see lots of parallel with display pipelines getting more complex. It
> wouldn't be fair to expose a very low-level API and ask every single
> compositor to know about all the devices out there, but at the same
> time, we're reaching a level of hardware complexity that I believe makes
> it a dead end to try to abstract and standardize everything at the
> kernel level. This doesn't mean we should allow all vendors to throw in
> any kind of crappy properties or other API extensions, we can still
> enforce some ground rules, such as proper documentation, reasonable
> effort for standardization, and reference open-source userspace
> implementation in a project we would consider to be the equalivant of
> mesa for KMS.

Thank you very much for explaining this! It makes perfect sense to me.
It also sounds like a ¤&!%¤# amount of work, but really does sound like
the best plan in the long term.

GPUs have OpenGL and Vulkan, but I suppose with cameras and display,
there is nothing standard to aim for, for either hardware designers nor
software designers? Chicken-and-egg.

> > > BTW, if we want to expose a finer-grained topology of processing blocks,
> > > I'd recommend looking at the Media Controller API, it was designed just
> > > for that purpose.  
> > 
> > Sure. However, right now we have KMS properties instead. Or maybe they
> > should just go all unused?
> > 
> > Maybe the rule should be to not add any more KMS properties and tell
> > people to design something completely new based on Media Controller API
> > instead?  
> 
> No, I think that would be too harsh, my point was that if we want to
> expose how the building blocks are layed out, the MC API is meant to
> expose a graph of connected entities, so it's a good match. Controlling
> the device should still go through KMS properties I believe.

Ok.

...

> > To be able to plug device-specific components in compositors, we need a
> > driver-agnostic API through which compositors can use those device
> > specific components. Still the same problem, now just in userspace than
> > at the UAPI level.  
> 
> Agreed, it pushes the problem to a different place :-) I however think
> it may be easier to solve it in userspace, as it's a better place to
> hardcode use cases. A general-purpose compositor may not want to make
> use of all hardware features, while a specific-purpose compositor may
> want to exercise the hardware features differently. This however opens
> the door to introducing badly-designed and un(der-)documented
> properties, which is not something I want, so we would still need to set
> ground rules, such as documentation and reference implementation in a
> given userspace compositor/stack.

Right, sounds fine, but I think it also shifts the KMS UAPI paradigm a
little, away from generic and towards driver-specific. If the DRM
maintainers agree to that, I'm fine with it. It's just different from
what I have understood so far, which is why I'm so keen on getting UAPI
documented precisely.

> On that note, I think we need to do a bit more hand-holding. I'm sitting
> on a patch series that is several years old to add 3D LUT support to my
> driver, and while I believe I'm quite experienced with KMS development,
> adding support for this in a reference compositor in userspace is a
> really big task that I don't know how to tackle. I also know very little
> about colour management, and have nobody in the team who has that
> knowledge and could help me. I'm thus facing a bit of a dead-end, not
> because I don't want to spend the time required for this, but because
> the bar is so high that I just can't do it. I would expect many
> developers to be facing the same challenges. If the community doesn't
> make an effort in fostering collaboration between developers (most
> kernel developers won't even know who to ask for help if they have to
> step out of the kernel to userspace) and providing good documentation,
> we'll just scare developers away. 

That's why I'm screaming here, to make contact. :-D

And Simor Ser is another, and I have seen him even offer to implement
userspace bits for new KMS properties in development.

But a 3D LUT in display is a really tiny piece in the landscape of color
management. Color management on Wayland has been talked about, oh I
don't remember, for a decade? Weston code has been experimented on for
the past few years at least. But only last week, the very first piece
of actual color management exclusive code was merged in Weston upstream.

Getting a display server project into a phase where it would actually
start experimenting on KMS features can take a decade - not just work
but finding the funding. Weston is not quite there yet either, not even
with all the currently open merge requests.

Then, if your 3D LUT is after blending, I'm not sure I would have use
for it in Weston! If it was on a KMS plane before blending, then yes, I
would have use for it. All this depends on how the blending color space
is chosen, and the idea we have right now does not have use for a 3D
LUT after blending - except maybe for fullscreen single-buffer
scenegraph, e.g. video and games.

> It took me weeks to figure out how to
> run Weston on this device, as it has no GPU that I can use, and falling
> back to a software compositor required finding out that lots of
> documentation is outdated and referenced command line options that were
> not accurate anymore (or I could mention painfully finding out that an
> issue I was blocked with for days was solved by plugging in a USB mouse
> to the board, as Weston didn't want to start without an input device).
> Another developer from my team tried before me and gave up, thinking it
> was impossible. That's the reality that many kernel developers are
> facing, the same way I suppose than me saying saying "you just need to
> recompile the kernel with this option enabled" is not very helpful for
> lots of userspace developers.

Why have I not heard of your problems with Weston?!

We have Gitlab issues, mailing list and IRC. You're warmly welcome to
ask when you hit a wall. :-)

I even read #dri-devel very often, so if you mentioned Weston there, it
would have had a good chance to reach me.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3] Documentation: gpu: Mention the requirements for new properties

[Pekka Paalanen]

[-- Attachment #1: Type: text/plain, Size: 12248 bytes --]

On Fri, 18 Jun 2021 15:19:15 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote:

> Hi Pekka,
> 
> On Fri, Jun 18, 2021 at 02:32:00PM +0300, Pekka Paalanen wrote:
> > On Fri, 18 Jun 2021 12:58:49 +0300 Laurent Pinchart wrote:  
> > > On Fri, Jun 18, 2021 at 11:55:38AM +0300, Pekka Paalanen wrote:  
> > > > On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:    
> > > > > On Thu, Jun 17, 2021 at 02:33:11PM +0300, Pekka Paalanen wrote:    
> > > > > > On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:      
> > > > > > > On Thu, Jun 17, 2021 at 10:27:01AM +0300, Pekka Paalanen wrote:      
> > > > > > > > On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:        
> > > > > > > > > On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:        

...

> > > > > One very typical difference between devices is the order of the
> > > > > processing blocks. By modelling the KMS pipeline as degamma -> ccm ->
> > > > > gamma, we can accommodate hardware that have any combination of
> > > > > [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at    
> > > > 
> > > > But you cannot represent pipelines like
> > > > 1D LUT -> 1D LUT -> CCM
> > > > because the abstract pipeline just doesn't have the elements for that.
> > > > OTOH, maybe that ordering does not even make sense to have in hardware?
> > > > So maybe not all combinations are actually needed.    
> > > 
> > > If userspace wanted such a pipeline (I'm not sure why it would), then it
> > > could just combine the two LUTs in one.  
> > 
> > Maybe? You can also combine the 1D LUTs into the 3D LUT in the
> > middle, too, but the result is not generally the same as using them
> > separately when the 3D LUT size is limited.  
> 
> Yes, my 3D LUT has 17 points, I've heard about LUTs up to 33 points
> (with bilinear interpolation), that's quite little if used as a 1D LUT.
> There's also the fact that interpolation will result in cross-talk
> between the three colour channels.

Hi Laurent,

that is not what I meant, but that's true too.

What I meant was this:
https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/582#note_951271

pre_curve and post_curve are the 1D LUTs on each side of a size-limited
3D LUT. pre_curve controls the 3D LUT tap positioning to concentrate on
the areas where the approximated function needs more taps.


> > Why would a generic userspace library API be a more feasible effort?  
> 
> I'll make a parallel with the camera world again, because I think
> display hardware is taking a similar path, just later.
> 
> We used to have capture devices that offered a fairly high-level
> interface at the hardware or firmware level, with a set of features that
> were more or less standard. Devices could be exposed with an abstract
> model, as done in KMS today, with a set of standardized controls (V4L2
> uses the word control when KMS uses property). We had way less automated
> testing, and didn't enough attention to consistency of the userspace API
> (we're talking about code from the beginning of the century, if not
> earlier, the world was different), so different drivers were not
> consistent in how the API was implemented (for instance the API didn't
> specify gain units, so drivers would use different units), but more or
> less, you could write a standard V4L2 application that would work with
> most devices, with a few quirks here and there.
> 
> Then the world changed. Cameras become more complex, with the raw sensor
> and the ISP exposed to Linux in the OMAP3 that Nokia used in the N900
> and N9 phones. For a while we tried to stick with V4L2, and implement
> ISP control within the kernel, with the help of a userspace daemon
> behind the scenes to which the kernel driver called back. The daemon was
> required, as control of the camera required complicated computation that
> really benefited from floating point math. It was horrible, and the
> daemon was closed-source, but from an application point of view, it was
> still high-level V4L2 as before. This hit a big wall though, to expose a
> high-level API on top of a low-level hardware interface, we had to
> bridge the gap in the kernel. This meant hardcoding use cases on the
> kernel side, which clearly wouldn't scale, as not everybody had the same
> use cases (an industrial vision application and a smartphone camera
> application really don't see the world the same way).  We were facing a
> dead end.
> 
> We then decided to expose a lower-level API from kernel to userspace,
> giving userspace the ability to control everything. That's how the Media
> Controller API was born (its purpose is to expose the internal topology
> of the device as a graph of connected entities, that's about it), as
> well as the V4L2 subdev userspace API (in KMS term, that would mean more
> or less exposing DRM bridges to userspace, with the ability to control
> them individually). This simplified the kernel implementation (no more
> hardcoded use cases, no more daemon behind the scenes), but at the same
> time resulted in an API that wasn't a good fit for applications to
> consume directly. One step closer to the hardware, one step further from
> the user. You can think of this as the API exposed by GPU kernel
> drivers, it's not enough, a userspace component to translate to a
> higher-level API is needed.
> 
> This was followed by 10 years of limbo as Nokia pulled the plug from
> Linux (and then smartphone) development. Fast forward, at the end of
> 2018, we created libcamera as the mesa of the camera world to solve this
> issue. It offers a standard camera stack with a high-level API, where
> device-specific code can be added (and kept to the minimum, as a
> centralized framework means we can share code that is common between
> devices).
> 
> I see lots of parallel with display pipelines getting more complex. It
> wouldn't be fair to expose a very low-level API and ask every single
> compositor to know about all the devices out there, but at the same
> time, we're reaching a level of hardware complexity that I believe makes
> it a dead end to try to abstract and standardize everything at the
> kernel level. This doesn't mean we should allow all vendors to throw in
> any kind of crappy properties or other API extensions, we can still
> enforce some ground rules, such as proper documentation, reasonable
> effort for standardization, and reference open-source userspace
> implementation in a project we would consider to be the equalivant of
> mesa for KMS.

Thank you very much for explaining this! It makes perfect sense to me.
It also sounds like a ¤&!%¤# amount of work, but really does sound like
the best plan in the long term.

GPUs have OpenGL and Vulkan, but I suppose with cameras and display,
there is nothing standard to aim for, for either hardware designers nor
software designers? Chicken-and-egg.

> > > BTW, if we want to expose a finer-grained topology of processing blocks,
> > > I'd recommend looking at the Media Controller API, it was designed just
> > > for that purpose.  
> > 
> > Sure. However, right now we have KMS properties instead. Or maybe they
> > should just go all unused?
> > 
> > Maybe the rule should be to not add any more KMS properties and tell
> > people to design something completely new based on Media Controller API
> > instead?  
> 
> No, I think that would be too harsh, my point was that if we want to
> expose how the building blocks are layed out, the MC API is meant to
> expose a graph of connected entities, so it's a good match. Controlling
> the device should still go through KMS properties I believe.

Ok.

...

> > To be able to plug device-specific components in compositors, we need a
> > driver-agnostic API through which compositors can use those device
> > specific components. Still the same problem, now just in userspace than
> > at the UAPI level.  
> 
> Agreed, it pushes the problem to a different place :-) I however think
> it may be easier to solve it in userspace, as it's a better place to
> hardcode use cases. A general-purpose compositor may not want to make
> use of all hardware features, while a specific-purpose compositor may
> want to exercise the hardware features differently. This however opens
> the door to introducing badly-designed and un(der-)documented
> properties, which is not something I want, so we would still need to set
> ground rules, such as documentation and reference implementation in a
> given userspace compositor/stack.

Right, sounds fine, but I think it also shifts the KMS UAPI paradigm a
little, away from generic and towards driver-specific. If the DRM
maintainers agree to that, I'm fine with it. It's just different from
what I have understood so far, which is why I'm so keen on getting UAPI
documented precisely.

> On that note, I think we need to do a bit more hand-holding. I'm sitting
> on a patch series that is several years old to add 3D LUT support to my
> driver, and while I believe I'm quite experienced with KMS development,
> adding support for this in a reference compositor in userspace is a
> really big task that I don't know how to tackle. I also know very little
> about colour management, and have nobody in the team who has that
> knowledge and could help me. I'm thus facing a bit of a dead-end, not
> because I don't want to spend the time required for this, but because
> the bar is so high that I just can't do it. I would expect many
> developers to be facing the same challenges. If the community doesn't
> make an effort in fostering collaboration between developers (most
> kernel developers won't even know who to ask for help if they have to
> step out of the kernel to userspace) and providing good documentation,
> we'll just scare developers away. 

That's why I'm screaming here, to make contact. :-D

And Simor Ser is another, and I have seen him even offer to implement
userspace bits for new KMS properties in development.

But a 3D LUT in display is a really tiny piece in the landscape of color
management. Color management on Wayland has been talked about, oh I
don't remember, for a decade? Weston code has been experimented on for
the past few years at least. But only last week, the very first piece
of actual color management exclusive code was merged in Weston upstream.

Getting a display server project into a phase where it would actually
start experimenting on KMS features can take a decade - not just work
but finding the funding. Weston is not quite there yet either, not even
with all the currently open merge requests.

Then, if your 3D LUT is after blending, I'm not sure I would have use
for it in Weston! If it was on a KMS plane before blending, then yes, I
would have use for it. All this depends on how the blending color space
is chosen, and the idea we have right now does not have use for a 3D
LUT after blending - except maybe for fullscreen single-buffer
scenegraph, e.g. video and games.

> It took me weeks to figure out how to
> run Weston on this device, as it has no GPU that I can use, and falling
> back to a software compositor required finding out that lots of
> documentation is outdated and referenced command line options that were
> not accurate anymore (or I could mention painfully finding out that an
> issue I was blocked with for days was solved by plugging in a USB mouse
> to the board, as Weston didn't want to start without an input device).
> Another developer from my team tried before me and gave up, thinking it
> was impossible. That's the reality that many kernel developers are
> facing, the same way I suppose than me saying saying "you just need to
> recompile the kernel with this option enabled" is not very helpful for
> lots of userspace developers.

Why have I not heard of your problems with Weston?!

We have Gitlab issues, mailing list and IRC. You're warmly welcome to
ask when you hit a wall. :-)

I even read #dri-devel very often, so if you mentioned Weston there, it
would have had a good chance to reach me.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]