liboutput: thoughts about shared library on top of DRM/KMS

liboutput: thoughts about shared library on top of DRM/KMS

["Keith Packard"]

[-- Attachment #1.1: Type: text/plain, Size: 1155 bytes --]


During XDC this year, we heard a few presentations and had a lot of
hallway talk about sharing code for driving DRM/KMS for display.

I think the general consensus is that there is enough shared
functionality between all of the various DRM/KMS clients that we can
start thinking about building a library that deals with the complexity
of DRM/KMS and can help take full advantage of the hardware.

I've started writing up some ideas, mostly just goals, but also some
random thoughts about how this might work. This is all in the
'liboutput' project on freedesktop.org:

        https://gitlab.freedesktop.org/xorg/liboutput

I picked a spot in the 'xorg' namespace, not to confine this to X
systems, but to reflect the goal that this be a library shared across
all x.org-related projects, including Wayland compositors, EGL and
Vulkan direct applications and the X server.

All there is at this point is a README.md file; I'd like to generate
some consensus on a direction for the project before we start writing
any actual design documents.

Please submit MRs to that repo, or respond to this email as you prefer.

-- 
-keith

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Robert Foss]

Hey Keith,

Thanks for setting this up, and picking a descriptive project name :)

How is liboutput going to relate to the libliftoff[1] project?


[1] https://github.com/emersion/libliftoff


Rob.

On 05.10.19 18:09, Keith Packard wrote:
> 
> During XDC this year, we heard a few presentations and had a lot of
> hallway talk about sharing code for driving DRM/KMS for display.
> 
> I think the general consensus is that there is enough shared
> functionality between all of the various DRM/KMS clients that we can
> start thinking about building a library that deals with the complexity
> of DRM/KMS and can help take full advantage of the hardware.
> 
> I've started writing up some ideas, mostly just goals, but also some
> random thoughts about how this might work. This is all in the
> 'liboutput' project on freedesktop.org:
> 
>          https://gitlab.freedesktop.org/xorg/liboutput
> 
> I picked a spot in the 'xorg' namespace, not to confine this to X
> systems, but to reflect the goal that this be a library shared across
> all x.org-related projects, including Wayland compositors, EGL and
> Vulkan direct applications and the X server.
> 
> All there is at this point is a README.md file; I'd like to generate
> some consensus on a direction for the project before we start writing
> any actual design documents.
> 
> Please submit MRs to that repo, or respond to this email as you prefer.
> 
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Keith Packard]

[-- Attachment #1.1: Type: text/plain, Size: 1096 bytes --]

Robert Foss <robert.foss@collabora.com> writes:

> Hey Keith,
>
> Thanks for setting this up, and picking a descriptive project name :)
>
> How is liboutput going to relate to the libliftoff[1] project?

We heard about libliftoff at XDC. It sounds like it does the plane
selection stuff, including searching for configurations that support
'most' of what the user asked for while passing back a list of 'uh,
these didn't fit'. That sounds like a good foundation for liboutput.

I think we want a bit more, including things like render-based
operations for compositing fallback, format conversion, transforms,
PRIME support etc. I think we might also want to be able to create
pseudo outputs (to memory only) for things like virtualization and
testing.

Mostly, I heard lots of ideas at XDC about more we might do to share
code between DRM/KMS users, especially compositors.

I'd love to keep the conversation going and see if we can generate a
solid set of shared goals, then put together some kind of architecture
and bits of code to see what might work.

-- 
-keith

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Daniel Stone]

Hi Keith,

On Sat, 5 Oct 2019 at 17:16, Keith Packard <keithp@keithp.com> wrote:
> During XDC this year, we heard a few presentations and had a lot of
> hallway talk about sharing code for driving DRM/KMS for display.

Definitely. That would be great.

> I think the general consensus is that there is enough shared
> functionality between all of the various DRM/KMS clients that we can
> start thinking about building a library that deals with the complexity
> of DRM/KMS and can help take full advantage of the hardware.

Yeeeessss. Ish.

There are definitely a few annoying problems which we should have
common resolution for. I'm thinking of:
  - libliftoff's mandate of solving the views -> planes conundrum;
it's a ton of work to do properly, is extremely easy to get wrong, and
will ultimately require platform-specific tweaks as well
  - similarly, working backwards from 'I want to light these
connectors up' to determine the plane -> CRTC -> connector routing,
which also requires a mixture of brute force and platform-specific
tweaks - currently this is mostly just to handle asymmetric CRTC
capabilities (e.g. max clock), but is being complicated by the
requirement for merged CRTCs
  - EDID/CEA/etc parsing, which is not only tedious and sucks, but
requires a quirks database
  - sharing a mode-selection description language and semantics might
also be nice
  - parsing DRM properties - particularly enums - is painful enough
that Weston wrote its own internal wrapper around it
  - TTY handling :(

Having those as helper libraries would solve a _lot_ of issues Weston
has today. But pushing much more than that - thinking of rendering and
transforms in particular - is going to be actively harmful to us.
Given that we want to work on not just KMS, but nested-Wayland,
nested-X11, remote protocols like RDP/Waltham, and even mixed
local/remote setups where some outputs are KMS and others are at the
end of an RTSP stream, I'm going to need to implement most of this
stuff anyway.

Delegating everything to a big does-everything-KMS library means that
I'll just have to do it twice, and hope there isn't too much impedance
mismatch between the results. I also worry about both colour
management and timing: the users are going to have radically different
wants and requirements for those, and a one-size-fits-all API seems
like it would be tough to deal with.

I think there would be a load of value in starting with simple helpers
which can be used independently of any larger scheme, tackling that
list above. Most of them have good prior art: libliftoff is working
from Weston's plane-assignment code, Mutter has pretty good CRTC <->
connector routing IIRC, EDID parsing is best in the X server, DRM
properties are probably best done in either Weston or kmscube
(depending on the enum vs. string efficiency tradeoff), and TTY
handling is probably most robust in the X server.

If I was starting a compositor from scratch then I'd probably think
pretty hard about a does-everything-for-you library, but with
libweston already existing, it's a much harder sell tbh.

Cheers,
Daniel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Daniel Stone]

Hi,

On Mon, 7 Oct 2019 at 18:35, Daniel Stone <daniel@fooishbar.org> wrote:
> There are definitely a few annoying problems which we should have
> common resolution for. I'm thinking of:
>   - [...]

Oh, and add backlight handling to that list.

Cheers,
Daniel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Keith Packard]

[-- Attachment #1.1: Type: text/plain, Size: 691 bytes --]

Daniel Stone <daniel@fooishbar.org> writes:

> I think there would be a load of value in starting with simple helpers
> which can be used independently of any larger scheme, tackling that
> list above.

Yeah, a helper library that didn't enforce at tonne of policy and just
let the user glue things together on their own is probably going to be
more generally usable by existing and new systems.

I definitely like the idea of stealing the best parts of all existing
systems and trying to make them work together.

How many libraries we end up with isn't nearly as important to me as
making sure they work well together; common data types, similar style,
etc.

-- 
-keith

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Daniel Stone]

On Mon, 7 Oct 2019 at 19:16, Keith Packard <keithp@keithp.com> wrote:
> Daniel Stone <daniel@fooishbar.org> writes:
> > I think there would be a load of value in starting with simple helpers
> > which can be used independently of any larger scheme, tackling that
> > list above.
>
> Yeah, a helper library that didn't enforce at tonne of policy and just
> let the user glue things together on their own is probably going to be
> more generally usable by existing and new systems.
>
> I definitely like the idea of stealing the best parts of all existing
> systems and trying to make them work together.
>
> How many libraries we end up with isn't nearly as important to me as
> making sure they work well together; common data types, similar style,
> etc.

Yeah, totally AOL.

Cheers,
Daniel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Pekka Paalanen]

[-- Attachment #1.1: Type: text/plain, Size: 1789 bytes --]

On Mon, 07 Oct 2019 14:16:32 -0400
"Keith Packard" <keithp@keithp.com> wrote:

> Daniel Stone <daniel@fooishbar.org> writes:
> 
> > I think there would be a load of value in starting with simple helpers
> > which can be used independently of any larger scheme, tackling that
> > list above.  
> 
> Yeah, a helper library that didn't enforce at tonne of policy and just
> let the user glue things together on their own is probably going to be
> more generally usable by existing and new systems.
> 
> I definitely like the idea of stealing the best parts of all existing
> systems and trying to make them work together.
> 
> How many libraries we end up with isn't nearly as important to me as
> making sure they work well together; common data types, similar style,
> etc.
> 

Hi,

a bunch of helpers is the more attractive idea to me too. We just have
to pay attention to API/ABI instability vs. usage: we should be able to
both redesign the API at will to go forward, but people will not be
happy using a library that breaks ABI all the time.

Libweston worked around that issue by making all major release versions
parallel-installable. That means literally all files or directories
have the major number in their base name. Another option would be to
have a static library only. I'm not sure which one would be more
attractive. If going with static-library-only, then we probably one day
want to convert into a dynamic library, and then the question of
whether the ABI is stable enough will arise again.

Using Meson should make the static-lib-only approach very easy,
downstream projects could include liboutput as a git sub-module or
such. That should eliminate the version dependency issues that builds
might otherwise have.


Thanks,
pq

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Neil Armstrong]

[-- Attachment #1.1.1: Type: text/plain, Size: 1660 bytes --]

Hi,

On 05/10/2019 18:09, Keith Packard wrote:
> 
> During XDC this year, we heard a few presentations and had a lot of
> hallway talk about sharing code for driving DRM/KMS for display.
> 
> I think the general consensus is that there is enough shared
> functionality between all of the various DRM/KMS clients that we can
> start thinking about building a library that deals with the complexity
> of DRM/KMS and can help take full advantage of the hardware.

Seeing the description, it seems to be a libdrm with steroids,
why libdrm doesn't handle all this already ?

Is there a plan to maybe use it as a foundation for projects like
wlroots or drm_hwcomposer for example ?

Neil

> 
> I've started writing up some ideas, mostly just goals, but also some
> random thoughts about how this might work. This is all in the
> 'liboutput' project on freedesktop.org:
> 
>         https://gitlab.freedesktop.org/xorg/liboutput
> 
> I picked a spot in the 'xorg' namespace, not to confine this to X
> systems, but to reflect the goal that this be a library shared across
> all x.org-related projects, including Wayland compositors, EGL and
> Vulkan direct applications and the X server.
> 
> All there is at this point is a README.md file; I'd like to generate
> some consensus on a direction for the project before we start writing
> any actual design documents.
> 
> Please submit MRs to that repo, or respond to this email as you prefer.
> 
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> 



[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Daniel Stone]

Hi,

On Mon, 7 Oct 2019 at 19:16, Keith Packard <keithp@keithp.com> wrote:
> Daniel Stone <daniel@fooishbar.org> writes:
> > I think there would be a load of value in starting with simple helpers
> > which can be used independently of any larger scheme, tackling that
> > list above.
>
> Yeah, a helper library that didn't enforce at tonne of policy and just
> let the user glue things together on their own is probably going to be
> more generally usable by existing and new systems.

To elaborate a little bit, one of the reasons I'm loath to hide
complexity like transforms, colour management, and timing away in an
encapsulated lower layer, is because I have to expose all those
details anyway. Ultimately to make those work properly, we'll require
awareness not just in the compositor itself, but pushed through to
clients.

Wayland already has facility for informing clients about output
transforms so they can render pre-rotated and avoid the
compositor-side transform; in order to make HDR and other colour
management (e.g. just simple calibration) properly we need to have
full plumbing back through to clients; doing timing properly,
particularly for multiple simultaneous clients, also requires a fair
bit of mechanics and back-and-forth.

There's a lot that we could usefully share between all the users, and
having a shared library to help with that would be great. But the
thought of tucking it all away in an opaque layer which (*waves
hands*) just does it, gives me cold EGLStreams sweats.

Maybe a good place to start is if we all listed the bits of code which
we'd be delighted to jettison?

Cheers,
Daniel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Heinrich Fink]

On Mon, 7 Oct 2019 at 20:16, Keith Packard <keithp@keithp.com> wrote:
>
> Daniel Stone <daniel@fooishbar.org> writes:
>
> > I think there would be a load of value in starting with simple helpers
> > which can be used independently of any larger scheme, tackling that
> > list above.
>
> Yeah, a helper library that didn't enforce at tonne of policy and just
> let the user glue things together on their own is probably going to be
> more generally usable by existing and new systems.

I really like that idea. For product development where the compositor
itself might not be open-source, it would also really help companies
behind these products to better work with the open-source community,
i.e. being able to contribute to a set of shared helper libraries.

- Heinrich
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Alex Deucher]

On Tue, Oct 8, 2019 at 5:32 AM Daniel Stone <daniel@fooishbar.org> wrote:
>
> Hi,
>
> On Mon, 7 Oct 2019 at 19:16, Keith Packard <keithp@keithp.com> wrote:
> > Daniel Stone <daniel@fooishbar.org> writes:
> > > I think there would be a load of value in starting with simple helpers
> > > which can be used independently of any larger scheme, tackling that
> > > list above.
> >
> > Yeah, a helper library that didn't enforce at tonne of policy and just
> > let the user glue things together on their own is probably going to be
> > more generally usable by existing and new systems.
>
> To elaborate a little bit, one of the reasons I'm loath to hide
> complexity like transforms, colour management, and timing away in an
> encapsulated lower layer, is because I have to expose all those
> details anyway. Ultimately to make those work properly, we'll require
> awareness not just in the compositor itself, but pushed through to
> clients.
>
> Wayland already has facility for informing clients about output
> transforms so they can render pre-rotated and avoid the
> compositor-side transform; in order to make HDR and other colour
> management (e.g. just simple calibration) properly we need to have
> full plumbing back through to clients; doing timing properly,
> particularly for multiple simultaneous clients, also requires a fair
> bit of mechanics and back-and-forth.
>
> There's a lot that we could usefully share between all the users, and
> having a shared library to help with that would be great. But the
> thought of tucking it all away in an opaque layer which (*waves
> hands*) just does it, gives me cold EGLStreams sweats.
>
> Maybe a good place to start is if we all listed the bits of code which
> we'd be delighted to jettison?

On the flipside, it would be nice to have one set of code to do
modesets from userspace.  Certainly simplifies QA since in theory we
should be seeing the same sequences from all apps using the helpers
rather than every app rolling their own and getting it subtly
different enough that strange things happen with different
compositors.

While we are on the topic, it would be nice to have a central place
for robustness/context lost handling for compositors so we can
properly handle GPU resets.  Not sure that is really possible though.

Alex
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Daniel Vetter]

On Tue, Oct 8, 2019 at 4:36 PM Alex Deucher <alexdeucher@gmail.com> wrote:
>
> On Tue, Oct 8, 2019 at 5:32 AM Daniel Stone <daniel@fooishbar.org> wrote:
> >
> > Hi,
> >
> > On Mon, 7 Oct 2019 at 19:16, Keith Packard <keithp@keithp.com> wrote:
> > > Daniel Stone <daniel@fooishbar.org> writes:
> > > > I think there would be a load of value in starting with simple helpers
> > > > which can be used independently of any larger scheme, tackling that
> > > > list above.
> > >
> > > Yeah, a helper library that didn't enforce at tonne of policy and just
> > > let the user glue things together on their own is probably going to be
> > > more generally usable by existing and new systems.
> >
> > To elaborate a little bit, one of the reasons I'm loath to hide
> > complexity like transforms, colour management, and timing away in an
> > encapsulated lower layer, is because I have to expose all those
> > details anyway. Ultimately to make those work properly, we'll require
> > awareness not just in the compositor itself, but pushed through to
> > clients.
> >
> > Wayland already has facility for informing clients about output
> > transforms so they can render pre-rotated and avoid the
> > compositor-side transform; in order to make HDR and other colour
> > management (e.g. just simple calibration) properly we need to have
> > full plumbing back through to clients; doing timing properly,
> > particularly for multiple simultaneous clients, also requires a fair
> > bit of mechanics and back-and-forth.
> >
> > There's a lot that we could usefully share between all the users, and
> > having a shared library to help with that would be great. But the
> > thought of tucking it all away in an opaque layer which (*waves
> > hands*) just does it, gives me cold EGLStreams sweats.
> >
> > Maybe a good place to start is if we all listed the bits of code which
> > we'd be delighted to jettison?
>
> On the flipside, it would be nice to have one set of code to do
> modesets from userspace.  Certainly simplifies QA since in theory we
> should be seeing the same sequences from all apps using the helpers
> rather than every app rolling their own and getting it subtly
> different enough that strange things happen with different
> compositors.

For atomic drivers, and atomic userspace this really doesn't matter
anymore. Because userspace just assembles an overall update, and the
kernel applies that update no matter in which order userspace listed
the property/value/object triples.

It does somewhat matter for legacy kms (but not from a driver
validation pov, anything legacy can do can also be done as a legit
request through atomic), but I really can't care about that one much.
Imo better to just cut everyone over to atomic. The only thing that
might make sense is the fake atomic support, where userspace tries to
not be too dumb about a transition on drivers without atomic (i.e.
shut down crtcs first, then light up the new ones, to avoid in-between
state that oversubscribes available hw resources). But that's really a
niche thing.

> While we are on the topic, it would be nice to have a central place
> for robustness/context lost handling for compositors so we can
> properly handle GPU resets.  Not sure that is really possible though.

I think this blows up the scope too much and will break the project.
In the past there's been attempts at a libcompositor (and you need
that much to recover from context loss), and that never got anywhere
at all. I think the modular helpers approach has a much bigger chance
at success.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: liboutput: thoughts about shared library on top of DRM/KMS

[Keith Packard]

[-- Attachment #1.1: Type: text/plain, Size: 616 bytes --]

Neil Armstrong <narmstrong@baylibre.com> writes:

> Seeing the description, it seems to be a libdrm with steroids,
> why libdrm doesn't handle all this already ?

That'd be a lot of steroids; we're talking about creating helper
functions all the way up to rendering images into scanout buffers
(presumably using Vulkan?) for format conversion or flattening.

> Is there a plan to maybe use it as a foundation for projects like
> wlroots or drm_hwcomposer for example ?

Yes, the goal is to start to share code across a wide range of DRM
users, instead of having everyone roll their own.

-- 
-keith

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel