All of lore.kernel.org
 help / color / mirror / Atom feed
* Handling DRM master transitions cooperatively
@ 2021-09-03 19:08 Dennis Filder
  2021-09-07 10:07 ` Pekka Paalanen
  0 siblings, 1 reply; 23+ messages in thread
From: Dennis Filder @ 2021-09-03 19:08 UTC (permalink / raw)
  To: dri-devel; +Cc: Hans de Goede

Hans de Goede asked me to take a topic from a private discussion here.
I must also preface that I'm not a graphics person and my knowledge of
DRI/DRM is cursory at best.

I initiated the conversation with de Goede after learning that the X
server now supports being started with an open DRM file descriptor
(this was added for Keith Packard's xlease project).  I wondered if
that could be used to smoothen the Plymouth->X transition somehow and
asked de Goede if there were any such plans.  He denied, but mentioned
that a new ioctl is in the works to prevent the kernel from wiping the
contents of a frame buffer after a device is closed, and that this
would help to keep transitions smooth.

I am a bit disappointed with this being considered a desirable way of
handling that transfer of control over a shared DRM device as it shows
a lack of ambition.  Sure, it's probably easy to implement, but it
will also greatly limit how such transitions can be presented to the
user.  In practice it would mean plymouthd closing the DRM device and
exiting so that systemd can start the display manager which then
starts an X server to present the login screen.  If for that several
shared libraries have to first be loaded and relocated while the
system is under heavy load then there will be a noticeable delay
manifesting as a frozen screen.  After that the best you can hope for
is blending the still-frame over into the login screen (or whatever
comes then).  The VT-API-based switching mechanism currently en vogue
suffers from similar limitations.

If the approach to transferring control were to be changed to a scheme
that involves both donor and recipient process connecting to each
other on a unix socket and actively coordinating the transfer
(i.e. the calls to drmSetMaster and drmDropMaster) then this would
open the door to a host of possibilities.  Not only could the
transition be kept infinitesimally short since both processes are
already up, but it could also involve e.g. the recipient continuing an
animation the donor had going reusing state that is transferred as a
memfd.  This way there wouldn't be any noticeable freezes on the
display making for a far more polished, and thus impressive
experience.  It would be a feat a program alone cannot achieve on its
own.  Another option made possible would be implementing a watchdog.
If the recipient transfers e.g. file descriptors for a pipe and a
pidfd of itself, then the donor could monitor those for a
heartbeat/process termination and take back control over the device if
something goes awry (deadlock/crash) and initiate a recovery
mechanism.  With the other approach implementing such features is
simply not possible.

Making processes talk to each other and work together like this would
also be a far more accurate software representation of what is
actually going on: different subsystems passing control over a shared
device around to work towards the common goal of a good user
experience.

A bit of context: The idea underlying this came from my experience
with accessibility technology under Linux where uncoordinated fighting
over the audio device among all kinds of processes led to countless
ways in which things would break with no hope of ever fixing anything.
It instilled in me the conviction that user-facing programs are broken
if they are not written to talk to each other to coordinate access to
shared resources for the goal of rendering a good user experience, but
instead leave it to the distro maintainer/user to set things up into a
static, brittle working order.  Seeing a much-needed cultural shift
begin somewhere would be nice.  The Plymouth->X transition would lend
itself well as a starting point since many building blocks are already
there.

Regards

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-03 19:08 Handling DRM master transitions cooperatively Dennis Filder
@ 2021-09-07 10:07 ` Pekka Paalanen
  2021-09-07 10:19   ` Simon Ser
  2021-09-07 12:42   ` Hans de Goede
  0 siblings, 2 replies; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-07 10:07 UTC (permalink / raw)
  To: Dennis Filder; +Cc: dri-devel, Hans de Goede

[-- Attachment #1: Type: text/plain, Size: 10039 bytes --]

On Fri, 3 Sep 2021 21:08:21 +0200
Dennis Filder <d.filder@web.de> wrote:

> Hans de Goede asked me to take a topic from a private discussion here.
> I must also preface that I'm not a graphics person and my knowledge of
> DRI/DRM is cursory at best.
> 
> I initiated the conversation with de Goede after learning that the X
> server now supports being started with an open DRM file descriptor
> (this was added for Keith Packard's xlease project).  I wondered if
> that could be used to smoothen the Plymouth->X transition somehow and
> asked de Goede if there were any such plans.  He denied, but mentioned
> that a new ioctl is in the works to prevent the kernel from wiping the
> contents of a frame buffer after a device is closed, and that this
> would help to keep transitions smooth.

Hi,

I believe the kernel is not wiping anything on device close. If
something in the KMS state is wiped, it originates in userspace:

- Plymouth doing something (e.g. RmFB on an in-use FB will turn the
  output off, you need to be careful to "leak" your FB if you want a
  smooth hand-over)

- Xorg doing something (e.g. resetting instead of inheriting KMS state)

- Something missed in the hand-off sequence which allows fbcon to
  momentarily take over between Plymouth and Xorg. This would need to
  be fixed between Plymouth and Xorg.

- Maybe systemd-logind does something odd to the KMS device? It has
  pretty wild code there. Or maybe it causes fbcon to take over.

What is the new ioctl you referred to?

Being able to be started with an open DRM file descriptor is not
necessary for a smooth hand-over as far as I know. There are tons of
other details that are, though.

> 
> I am a bit disappointed with this being considered a desirable way of
> handling that transfer of control over a shared DRM device as it shows
> a lack of ambition.  Sure, it's probably easy to implement, but it

Or more likely bigger fires and lack of time, like with everything.

> will also greatly limit how such transitions can be presented to the
> user.  In practice it would mean plymouthd closing the DRM device and
> exiting so that systemd can start the display manager which then
> starts an X server to present the login screen.  If for that several
> shared libraries have to first be loaded and relocated while the
> system is under heavy load then there will be a noticeable delay
> manifesting as a frozen screen.  After that the best you can hope for
> is blending the still-frame over into the login screen (or whatever
> comes then).  The VT-API-based switching mechanism currently en vogue
> suffers from similar limitations.

All that is already solvable purely in userspace in my opinion, today.
It's just a big project over several independent userspace software
projects.

> If the approach to transferring control were to be changed to a scheme
> that involves both donor and recipient process connecting to each
> other on a unix socket and actively coordinating the transfer
> (i.e. the calls to drmSetMaster and drmDropMaster) then this would
> open the door to a host of possibilities.  Not only could the
> transition be kept infinitesimally short since both processes are
> already up, but it could also involve e.g. the recipient continuing an
> animation the donor had going reusing state that is transferred as a
> memfd.  This way there wouldn't be any noticeable freezes on the
> display making for a far more polished, and thus impressive
> experience.  It would be a feat a program alone cannot achieve on its
> own.  Another option made possible would be implementing a watchdog.
> If the recipient transfers e.g. file descriptors for a pipe and a
> pidfd of itself, then the donor could monitor those for a
> heartbeat/process termination and take back control over the device if
> something goes awry (deadlock/crash) and initiate a recovery
> mechanism.  With the other approach implementing such features is
> simply not possible.

Nothing in the kernel stops userspace developers from doing exactly
that. Seems like you would be working on Plymouth and a display server
of your choice. Don't forget to count in systemd-logind as well, since
that is a popular component for managing sessions and is involved with
e.g. drmSetMaster.

It's a good goal.

It's also more or less necessary for a smooth hand-over of a KMS device
between any two processes also for other reasons I've discussed in the
past with DRM developers. This is the topic about any KMS client (a
program using KMS) needing to reach a guaranteed "clean" KMS state to
display correctly.

The kernel DRM subsystem never resets KMS state in any way, apart from
driver initialisation.

This means that when a new KMS client takes over, the KMS state could
be anything, whatever the previous KMS client left in. This is a
problem, because the KMS client may not know how to reset all the KMS
properties to clean, sane defaults. Currently there is also no reset
ioctl in the kernel either, and no userspace space solution for storing
a sane default state. The problem arises from KMS properties: each KMS
client may not know how to program all the KMS properties the kernel
supports on the device. For example, if one KMS client leaves the
output in HDR mode, and the next KMS client does not understand the HDR
property, then quite likely the latter KMS client will display an awful
image without knowing it.

There is also the convention of a KMS client not restoring the
inherited KMS state on exit or switch-out, because that could cause
unnecessary flicker on screen and delays. This amplifies the above
problem.

The only time the KMS state is at "sane" defaults is right after driver
initialisation. Presumably. So only the first KMS client after a reboot
can expect a sane KMS state.

Mind, also fbcon is a KMS client, it's just a kernel-internal one.
Letting fbcon take over momentarily can reset some KMS properties to
nice defaults, but not all, because when new KMS properties are added,
no-one usually remembers to patch fbcon/fbdev/whatever to reset it when
fbcon takes over. Switching temporarily to fbcon also causes flicker
and delays.

The above problems could perhaps be solved if there was a generic KMS
hand-over protocol in userspace. The two KMS clients could agree on
which KMS properties should be reset and by whom. Who understands and
programs which KMS properties. And perhaps, some system component in
early'ish boot could save the driver-initial KMS state which is
presumably the good default for any use case, on the properties any
particular KMS client does not know or bother to program by itself.

So yes, userspace protocol for KMS hand-off would be very welcome. But
who would have the time to develop it, when so far we can just limp
forward with the current undocumented conventions.

Such protocol, if widely used, might also make it unnecessary for KMS
clients to save and restore KMS properties they do not understand when
they switch out and later back in. When a KMS client has released
DRM master on the device, some other KMS client could have "messed up"
the KMS state, so restoring to what you used before is necessary. I
don't think anyone actually implements this save/restore yet for
unknown KMS properties, and it would be much easier to implement than
the hand-off protocol. Maybe switching is either not done, or it is
always done to/from a display manager process which sanitises the KMS
state or enforces that KMS clients do not leave random state behind. Or
maybe most KMS clients are just really good at agreeing on which KMS
properties everyone will use. If some stale KMS property causes a
problem on some KMS client (display server), it is pretty easy to just
add support for programming that property in the KMS client. Problem
solved and no hand-off protocol needed. Then the next KMS client hits
and does the same.

> Making processes talk to each other and work together like this would
> also be a far more accurate software representation of what is
> actually going on: different subsystems passing control over a shared
> device around to work towards the common goal of a good user
> experience.
> 
> A bit of context: The idea underlying this came from my experience
> with accessibility technology under Linux where uncoordinated fighting
> over the audio device among all kinds of processes led to countless
> ways in which things would break with no hope of ever fixing anything.
> It instilled in me the conviction that user-facing programs are broken
> if they are not written to talk to each other to coordinate access to
> shared resources for the goal of rendering a good user experience, but
> instead leave it to the distro maintainer/user to set things up into a
> static, brittle working order.  Seeing a much-needed cultural shift
> begin somewhere would be nice.  The Plymouth->X transition would lend
> itself well as a starting point since many building blocks are already
> there.

I might recommend picking a Wayland display server instead of Xorg
first. The thing is, Xorg is only a middle-man and it is some X11
client that decides what and how will be displayed. Therefore with X11
architecture, it's not just two processes that need to communicate the
hand-off, it's three: Plymouth, Xorg, and the X11 client that will
actually draw stuff.

In Wayland architecture, you only need to communicate between Plymouth
and the Wayland compositor you picked. How that Wayland compositor
draws anything is an internal detail to it, so you can solve that
in-project.

You could also think that how Xorg gets the content is an internal
detail to the X11 desktop, but that might lead to needing some new X11
protocol extension to be able to control Xorg's actions on the KMS
device sufficiently. It may also be hard to get any new feature code
into Xorg and released.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-07 10:07 ` Pekka Paalanen
@ 2021-09-07 10:19   ` Simon Ser
  2021-09-07 12:52     ` Pekka Paalanen
  2021-09-07 12:42   ` Hans de Goede
  1 sibling, 1 reply; 23+ messages in thread
From: Simon Ser @ 2021-09-07 10:19 UTC (permalink / raw)
  To: Pekka Paalanen; +Cc: Dennis Filder, dri-devel, Hans de Goede

FWIW, I've just hit a case where a compositor leaves a "rotation" KMS
prop set behind, then Xorg tries to startup and fails because it doesn't
reset this prop. So none of this is theoretical.

I still think a "reset all KMS props to an arbitrary default value" flag
in drmModeAtomicCommit is the best way forward. I'm not sure a user-space
protocol would help too much.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-07 10:07 ` Pekka Paalanen
  2021-09-07 10:19   ` Simon Ser
@ 2021-09-07 12:42   ` Hans de Goede
  2021-09-08  7:36     ` Pekka Paalanen
  1 sibling, 1 reply; 23+ messages in thread
From: Hans de Goede @ 2021-09-07 12:42 UTC (permalink / raw)
  To: Pekka Paalanen, Dennis Filder; +Cc: dri-devel

Hi,

On 9/7/21 12:07 PM, Pekka Paalanen wrote:
> On Fri, 3 Sep 2021 21:08:21 +0200
> Dennis Filder <d.filder@web.de> wrote:
> 
>> Hans de Goede asked me to take a topic from a private discussion here.
>> I must also preface that I'm not a graphics person and my knowledge of
>> DRI/DRM is cursory at best.
>>
>> I initiated the conversation with de Goede after learning that the X
>> server now supports being started with an open DRM file descriptor
>> (this was added for Keith Packard's xlease project).  I wondered if
>> that could be used to smoothen the Plymouth->X transition somehow and
>> asked de Goede if there were any such plans.  He denied, but mentioned
>> that a new ioctl is in the works to prevent the kernel from wiping the
>> contents of a frame buffer after a device is closed, and that this
>> would help to keep transitions smooth.
> 
> Hi,
> 
> I believe the kernel is not wiping anything on device close. If
> something in the KMS state is wiped, it originates in userspace:
> 
> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
>   output off, you need to be careful to "leak" your FB if you want a
>   smooth hand-over)

The "kernel is not wiping anything on device close" is not true,
when closing /dev/dri/card# any remaining FBs from the app closing
it will be dealt with as if they were RmFB-ed, causing the screen
to show what I call "the fallback fb", at least with the i915 driver.

> - Xorg doing something (e.g. resetting instead of inheriting KMS state)
> 
> - Something missed in the hand-off sequence which allows fbcon to
>   momentarily take over between Plymouth and Xorg. This would need to
>   be fixed between Plymouth and Xorg.
> 
> - Maybe systemd-logind does something odd to the KMS device? It has
>   pretty wild code there. Or maybe it causes fbcon to take over.
> 
> What is the new ioctl you referred to?

It is an ioctl to mark a FB to not have it auto-removed on device-close,
instead leaving it in place until some some kernel/userspace client
actively installs another FB. This was proposed by Rob Clark quite
a while ago, but it never got anywhere because of lack of userspace
actually interested in using it.

I've been thinking about reviving Rob's patch, since at least for
plymouth this would be pretty useful to have.

Regards,

Hans


^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-07 10:19   ` Simon Ser
@ 2021-09-07 12:52     ` Pekka Paalanen
  2021-09-07 15:52       ` Sebastian Wick
  2021-09-08  9:51       ` Simon Ser
  0 siblings, 2 replies; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-07 12:52 UTC (permalink / raw)
  To: Simon Ser; +Cc: Dennis Filder, dri-devel, Hans de Goede

[-- Attachment #1: Type: text/plain, Size: 1381 bytes --]

On Tue, 07 Sep 2021 10:19:03 +0000
Simon Ser <contact@emersion.fr> wrote:

> FWIW, I've just hit a case where a compositor leaves a "rotation" KMS
> prop set behind, then Xorg tries to startup and fails because it doesn't
> reset this prop. So none of this is theoretical.
> 
> I still think a "reset all KMS props to an arbitrary default value" flag
> in drmModeAtomicCommit is the best way forward. I'm not sure a user-space
> protocol would help too much.

Hi Simon,

for the "reset KMS state" problem, sure. Thanks for confirming the
problem, too.

The hand-off problem does need userspace protocol though, so that the
two parties can negotiate what part of KMS state can be inherited by
the receiver and who will do the animation from the first to the second
state in case you want to avoid abrupt changes. It would also be useful
for a cross-fade as a perhaps more flexible way than the current "leak
an FB, let the next KMS client scrape it via ioctls and copy it so it
can be textured from".

Userspace protocol is also useful for starting the next KMS client
first and handing off only later once it's actually running. I'm not
sure if that is already possible with the session switching stuff, but
I have a feeling it might be fragile or miss pieces like the next KMS
client signalling ready before actually switching to it.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-07 12:52     ` Pekka Paalanen
@ 2021-09-07 15:52       ` Sebastian Wick
  2021-09-08 16:21         ` Dennis Filder
  2021-09-08  9:51       ` Simon Ser
  1 sibling, 1 reply; 23+ messages in thread
From: Sebastian Wick @ 2021-09-07 15:52 UTC (permalink / raw)
  To: dri-devel

> On Tue, 07 Sep 2021 10:19:03 +0000
> Simon Ser <contact at emersion.fr> wrote:
> 
> > FWIW, I've just hit a case where a compositor leaves a "rotation" KMS
> > prop set behind, then Xorg tries to startup and fails because it doesn't
> > reset this prop. So none of this is theoretical.
> >
> > I still think a "reset all KMS props to an arbitrary default value" flag
> > in drmModeAtomicCommit is the best way forward. I'm not sure a user-space
> > protocol would help too much.
> 
> Hi Simon,
> 
> for the "reset KMS state" problem, sure. Thanks for confirming the
> problem, too.
> 
> The hand-off problem does need userspace protocol though, so that the
> two parties can negotiate what part of KMS state can be inherited by
> the receiver and who will do the animation from the first to the second
> state in case you want to avoid abrupt changes. It would also be useful
> for a cross-fade as a perhaps more flexible way than the current "leak
> an FB, let the next KMS client scrape it via ioctls and copy it so it
> can be textured from".

The state reset already is an implicit protocol. Another IPC mechanism
however could extend it to work the other way around: instead of
inheriting all the state and trying to transition from that to the
second client's desired state the second client would send its own
desired state back to the first (instead of applying it immediately)
which would then try to transition from its own state to the second
state (and if it can't you fall back to the implicit inherited state
protocol). However, this is only an improvement if the first client
knows how to do the transition and the second does not. All in all I
doubt that you can convince most people to add this kind of complexity
just for slightly higher chances of a good transition.

The reset state protocol on the other hand solves real problems and
gives you a good transition as long as the second client knows about the
same properties as the previous one which usually is the case for the
typical bootsplash->login manager->compositor chain.

Maybe I'm completely missing how such a protocol would work though.

> Userspace protocol is also useful for starting the next KMS client
> first and handing off only later once it's actually running. I'm not
> sure if that is already possible with the session switching stuff, but
> I have a feeling it might be fragile or miss pieces like the next KMS
> client signalling ready before actually switching to it.
> 
> 
> Thanks,
> pq

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-07 12:42   ` Hans de Goede
@ 2021-09-08  7:36     ` Pekka Paalanen
  2021-09-08  7:49       ` Hans de Goede
  2021-09-08 16:27       ` Daniel Vetter
  0 siblings, 2 replies; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-08  7:36 UTC (permalink / raw)
  To: Hans de Goede, Daniel Vetter; +Cc: Dennis Filder, dri-devel

[-- Attachment #1: Type: text/plain, Size: 3538 bytes --]

On Tue, 7 Sep 2021 14:42:56 +0200
Hans de Goede <hdegoede@redhat.com> wrote:

> Hi,
> 
> On 9/7/21 12:07 PM, Pekka Paalanen wrote:
> > On Fri, 3 Sep 2021 21:08:21 +0200
> > Dennis Filder <d.filder@web.de> wrote:
> >   
> >> Hans de Goede asked me to take a topic from a private discussion here.
> >> I must also preface that I'm not a graphics person and my knowledge of
> >> DRI/DRM is cursory at best.
> >>
> >> I initiated the conversation with de Goede after learning that the X
> >> server now supports being started with an open DRM file descriptor
> >> (this was added for Keith Packard's xlease project).  I wondered if
> >> that could be used to smoothen the Plymouth->X transition somehow and
> >> asked de Goede if there were any such plans.  He denied, but mentioned
> >> that a new ioctl is in the works to prevent the kernel from wiping the
> >> contents of a frame buffer after a device is closed, and that this
> >> would help to keep transitions smooth.  
> > 
> > Hi,
> > 
> > I believe the kernel is not wiping anything on device close. If
> > something in the KMS state is wiped, it originates in userspace:
> > 
> > - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
> >   output off, you need to be careful to "leak" your FB if you want a
> >   smooth hand-over)  
> 
> The "kernel is not wiping anything on device close" is not true,
> when closing /dev/dri/card# any remaining FBs from the app closing
> it will be dealt with as if they were RmFB-ed, causing the screen
> to show what I call "the fallback fb", at least with the i915 driver.

No, that's not what should happen AFAIK.

True, all FBs that are not referenced by active CRTCs or planes will
get freed, since their refcount drops to zero, but those CRTCs and
planes that are active will remain active and therefore keep their
reference to the respective FBs and so the FBs remain until replaced or
turned off explicitly (by e.g. fbcon if you switch to that rather than
another userspace KMS client). I believe that is the whole reason why
e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
would not have anything to scrape.

danvet, what is the DRM core intention?

Or am I confused because display servers do not tend to close the DRM
device fd on switch-out but Plymouth does (too early)?

If so, why can't Plymouth keep the device open longer and quit only
when the hand-off is complete? Not quitting too early would be a
prerequisite for any explicit hand-off protocol as well.


Thanks,
pq

> > - Xorg doing something (e.g. resetting instead of inheriting KMS state)
> > 
> > - Something missed in the hand-off sequence which allows fbcon to
> >   momentarily take over between Plymouth and Xorg. This would need to
> >   be fixed between Plymouth and Xorg.
> > 
> > - Maybe systemd-logind does something odd to the KMS device? It has
> >   pretty wild code there. Or maybe it causes fbcon to take over.
> > 
> > What is the new ioctl you referred to?  
> 
> It is an ioctl to mark a FB to not have it auto-removed on device-close,
> instead leaving it in place until some some kernel/userspace client
> actively installs another FB. This was proposed by Rob Clark quite
> a while ago, but it never got anywhere because of lack of userspace
> actually interested in using it.
> 
> I've been thinking about reviving Rob's patch, since at least for
> plymouth this would be pretty useful to have.
> 
> Regards,
> 
> Hans
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-08  7:36     ` Pekka Paalanen
@ 2021-09-08  7:49       ` Hans de Goede
  2021-09-08 16:27       ` Daniel Vetter
  1 sibling, 0 replies; 23+ messages in thread
From: Hans de Goede @ 2021-09-08  7:49 UTC (permalink / raw)
  To: Pekka Paalanen, Daniel Vetter; +Cc: Dennis Filder, dri-devel

Hi,

On 9/8/21 9:36 AM, Pekka Paalanen wrote:
> On Tue, 7 Sep 2021 14:42:56 +0200
> Hans de Goede <hdegoede@redhat.com> wrote:
> 
>> Hi,
>>
>> On 9/7/21 12:07 PM, Pekka Paalanen wrote:
>>> On Fri, 3 Sep 2021 21:08:21 +0200
>>> Dennis Filder <d.filder@web.de> wrote:
>>>   
>>>> Hans de Goede asked me to take a topic from a private discussion here.
>>>> I must also preface that I'm not a graphics person and my knowledge of
>>>> DRI/DRM is cursory at best.
>>>>
>>>> I initiated the conversation with de Goede after learning that the X
>>>> server now supports being started with an open DRM file descriptor
>>>> (this was added for Keith Packard's xlease project).  I wondered if
>>>> that could be used to smoothen the Plymouth->X transition somehow and
>>>> asked de Goede if there were any such plans.  He denied, but mentioned
>>>> that a new ioctl is in the works to prevent the kernel from wiping the
>>>> contents of a frame buffer after a device is closed, and that this
>>>> would help to keep transitions smooth.  
>>>
>>> Hi,
>>>
>>> I believe the kernel is not wiping anything on device close. If
>>> something in the KMS state is wiped, it originates in userspace:
>>>
>>> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
>>>   output off, you need to be careful to "leak" your FB if you want a
>>>   smooth hand-over)  
>>
>> The "kernel is not wiping anything on device close" is not true,
>> when closing /dev/dri/card# any remaining FBs from the app closing
>> it will be dealt with as if they were RmFB-ed, causing the screen
>> to show what I call "the fallback fb", at least with the i915 driver.
> 
> No, that's not what should happen AFAIK.

I'm pretty sure that that is what is happening though.

But hopefully someone else can either confirm or deny this :)

> True, all FBs that are not referenced by active CRTCs or planes will
> get freed, since their refcount drops to zero, but those CRTCs and
> planes that are active will remain active and therefore keep their
> reference to the respective FBs and so the FBs remain until replaced or
> turned off explicitly (by e.g. fbcon if you switch to that rather than
> another userspace KMS client). I believe that is the whole reason why
> e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
> would not have anything to scrape.
> 
> danvet, what is the DRM core intention?
> 
> Or am I confused because display servers do not tend to close the DRM
> device fd on switch-out but Plymouth does (too early)?
> 
> If so, why can't Plymouth keep the device open longer and quit only
> when the hand-off is complete? Not quitting too early would be a
> prerequisite for any explicit hand-off protocol as well.

plymouth is actually keeping the device open longer for exactly this
reason, the following happens:

1. plmouth starts
2. gdm starts and tells plymouth to "deactivate" which will stop it from
making drm ioctls and drop its drm master rights, while keeping the fb around
3. gdm waits for the greeter process to tell it that it has successfully
taken over the screen
4. gdm tells plymouth to quit

And something similar is happening on gdm greeter -> gnome user session
handover.

But we need the new ioctl at least on shutdown / reboot to avoid the
"fallback fb" (typically the EFI/BIOS setup fb which i915 inherited at boot)
showing for a brief moment when plymouth quits at shutdown / reboot and
there is nothing to hand-over the fb to in that case.

And the new ioctl would also make the above handover a lot simpler.

And we currently also have a flicker when going from user-session to
gdm on logout or from gdm/user-session to plymout on shutdown/reboot.

Basically we have quite a few transitions and currently only the
boot + login path is smooth and the rest needs more work, which either
requires a standardized handover method (instead of the current
hardcoded plymouth -> gdm stuff), or just allowing the FB to sit
around until the next drm-client installs its FB, which would be
much more KISS, so that has my preference.

And this KISS method will also work with transitions to a new
console-owner process which is not aware of any handover protocols,
as long as the old process uses the ioctl the transition will be
smooth. So e.g. gdm -> i3 on Xorg session will be smooth (1)

Regards,

Hans


1) I think this actually already is smooth because in this case gdm
just sleeps for 5 seconds before killing the greeter I believe, but
with the ioctl we could remove this hack



^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-07 12:52     ` Pekka Paalanen
  2021-09-07 15:52       ` Sebastian Wick
@ 2021-09-08  9:51       ` Simon Ser
  2021-09-08 11:13         ` Pekka Paalanen
  2021-09-08 16:14         ` Dennis Filder
  1 sibling, 2 replies; 23+ messages in thread
From: Simon Ser @ 2021-09-08  9:51 UTC (permalink / raw)
  To: Pekka Paalanen; +Cc: Dennis Filder, dri-devel, Hans de Goede

> On Tue, 07 Sep 2021 10:19:03 +0000
> Simon Ser <contact@emersion.fr> wrote:
>
> > FWIW, I've just hit a case where a compositor leaves a "rotation" KMS
> > prop set behind, then Xorg tries to startup and fails because it doesn't
> > reset this prop. So none of this is theoretical.
> >
> > I still think a "reset all KMS props to an arbitrary default value" flag
> > in drmModeAtomicCommit is the best way forward. I'm not sure a user-space
> > protocol would help too much.
>
> Hi Simon,
>
> for the "reset KMS state" problem, sure. Thanks for confirming the
> problem, too.
>
> The hand-off problem does need userspace protocol though, so that the
> two parties can negotiate what part of KMS state can be inherited by
> the receiver and who will do the animation from the first to the second
> state in case you want to avoid abrupt changes. It would also be useful
> for a cross-fade as a perhaps more flexible way than the current "leak
> an FB, let the next KMS client scrape it via ioctls and copy it so it
> can be textured from".

The KMS state can be limited to single FB on primary plane covering the whole
CRTC, no scaling, no other property set than FB_ID/CRTC_*/SRC_*.

Is it useful to make the previous client perform the animation? I don't really
understand the use-case here.

> Userspace protocol is also useful for starting the next KMS client
> first and handing off only later once it's actually running. I'm not
> sure if that is already possible with the session switching stuff, but
> I have a feeling it might be fragile or miss pieces like the next KMS
> client signalling ready before actually switching to it.

Hm, right. I'm not 100% clear if it's possible for the next client to set
everything up while the VT is not active.

It would help to make logind/seatd give a non-master DRM FD when VT-switched
away. Not sure they do it atm.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-08  9:51       ` Simon Ser
@ 2021-09-08 11:13         ` Pekka Paalanen
  2021-09-08 16:14         ` Dennis Filder
  1 sibling, 0 replies; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-08 11:13 UTC (permalink / raw)
  To: Simon Ser; +Cc: Dennis Filder, dri-devel, Hans de Goede

[-- Attachment #1: Type: text/plain, Size: 3220 bytes --]

On Wed, 08 Sep 2021 09:51:54 +0000
Simon Ser <contact@emersion.fr> wrote:

> > On Tue, 07 Sep 2021 10:19:03 +0000
> > Simon Ser <contact@emersion.fr> wrote:
> >  
> > > FWIW, I've just hit a case where a compositor leaves a "rotation" KMS
> > > prop set behind, then Xorg tries to startup and fails because it doesn't
> > > reset this prop. So none of this is theoretical.
> > >
> > > I still think a "reset all KMS props to an arbitrary default value" flag
> > > in drmModeAtomicCommit is the best way forward. I'm not sure a user-space
> > > protocol would help too much.  
> >
> > Hi Simon,
> >
> > for the "reset KMS state" problem, sure. Thanks for confirming the
> > problem, too.
> >
> > The hand-off problem does need userspace protocol though, so that the
> > two parties can negotiate what part of KMS state can be inherited by
> > the receiver and who will do the animation from the first to the second
> > state in case you want to avoid abrupt changes. It would also be useful
> > for a cross-fade as a perhaps more flexible way than the current "leak
> > an FB, let the next KMS client scrape it via ioctls and copy it so it
> > can be textured from".  
> 
> The KMS state can be limited to single FB on primary plane covering the whole
> CRTC, no scaling, no other property set than FB_ID/CRTC_*/SRC_*.
> 
> Is it useful to make the previous client perform the animation? I don't really
> understand the use-case here.

I guess the use cases are more or less imaginary for now.

Imagine one HDR-capable display server handing off to another
HDR-capable display server. If the releasing display server does not
know the receiving display server understands HDR, the releasing
display server might run an animation to turn HDR off - fade to black,
for instance, so that the impact from changing from HDR to SDR is
minimized. Then the receiving display server sees KMS is in SDR mode,
and maybe sets up a black image and then switches back to HDR.

If you're happy with fade-to-black on switch, then no problem. However,
the only way to not fade-to-black or even come cross-fade is
some negotiation to see that both sides understand HDR.

If the previous FB was rendered for HDR display, you will need to know
a lot from it if you want to do a cross-fade that doesn't glitch.

Also, while I don't see why changing between SDR and HDR would require
a modeset in KMS, I suppose it might take a moment for the monitor to
adapt. It might cause glitches similar to changing video modes.

> > Userspace protocol is also useful for starting the next KMS client
> > first and handing off only later once it's actually running. I'm not
> > sure if that is already possible with the session switching stuff, but
> > I have a feeling it might be fragile or miss pieces like the next KMS
> > client signalling ready before actually switching to it.  
> 
> Hm, right. I'm not 100% clear if it's possible for the next client to set
> everything up while the VT is not active.
> 
> It would help to make logind/seatd give a non-master DRM FD when VT-switched
> away. Not sure they do it atm.

Oh yeah, that may be an obvious gap I missed.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-08  9:51       ` Simon Ser
  2021-09-08 11:13         ` Pekka Paalanen
@ 2021-09-08 16:14         ` Dennis Filder
  1 sibling, 0 replies; 23+ messages in thread
From: Dennis Filder @ 2021-09-08 16:14 UTC (permalink / raw)
  To: Simon Ser, Pekka Paalanen; +Cc: dri-devel, Hans de Goede, Sebastian Wick

On Wed, Sep 08, 2021 at 09:51:54AM +0000, Simon Ser wrote:
> > On Tue, 07 Sep 2021 10:19:03 +0000
> > Simon Ser <contact@emersion.fr> wrote:
> >
> > > FWIW, I've just hit a case where a compositor leaves a "rotation" KMS
> > > prop set behind, then Xorg tries to startup and fails because it doesn't
> > > reset this prop. So none of this is theoretical.
> > >
> > > I still think a "reset all KMS props to an arbitrary default value" flag
> > > in drmModeAtomicCommit is the best way forward. I'm not sure a user-space
> > > protocol would help too much.
> >
> > Hi Simon,
> >
> > for the "reset KMS state" problem, sure. Thanks for confirming the
> > problem, too.
> >
> > The hand-off problem does need userspace protocol though, so that the
> > two parties can negotiate what part of KMS state can be inherited by
> > the receiver and who will do the animation from the first to the second
> > state in case you want to avoid abrupt changes. It would also be useful
> > for a cross-fade as a perhaps more flexible way than the current "leak
> > an FB, let the next KMS client scrape it via ioctls and copy it so it
> > can be textured from".
>
> The KMS state can be limited to single FB on primary plane covering the whole
> CRTC, no scaling, no other property set than FB_ID/CRTC_*/SRC_*.
>
> Is it useful to make the previous client perform the animation? I don't really
> understand the use-case here.

The use case for the animation is e.g. the transition from Plymouth to
the display server.  Currently it is done as a still frame transition,
maybe with a blend-over effect.  But with the current design it is not
possible to blend Plymouth's animation over into another animation in
the display server because the second client lacks the knowledge how
to keep it going for a little bit.

Another use case is switching between sessions which currently also is
only possible as a still frame transition.  However, if you wanted to
present the session switching by doing e.g. a shaking screen animation
and blending the old display content over into the new content then
the first client would have to render the first half of the animation,
and the second client would have to render the second half during
which it would then blend away the content of the first screen while
blending in its own content and also slowing the shaking to a stop.
For that to work the second client would need all the information
necessary to render that animation, and also a way to perform the
frame-perfect change-over.

Granted, that is a very complicated, eye-candy-oriented use case, but
it would serve to show-case the potential of the design.

Regards.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-07 15:52       ` Sebastian Wick
@ 2021-09-08 16:21         ` Dennis Filder
  2021-09-09  8:20           ` Pekka Paalanen
  0 siblings, 1 reply; 23+ messages in thread
From: Dennis Filder @ 2021-09-08 16:21 UTC (permalink / raw)
  To: Sebastian Wick, dri-devel; +Cc: Hans de Goede, Pekka Paalanen, Simon Ser

On Tue, Sep 07, 2021 at 05:52:41PM +0200, Sebastian Wick wrote:
> > On Tue, 07 Sep 2021 10:19:03 +0000
> > Simon Ser <contact at emersion.fr> wrote:
> >
> > > FWIW, I've just hit a case where a compositor leaves a "rotation" KMS
> > > prop set behind, then Xorg tries to startup and fails because it doesn't
> > > reset this prop. So none of this is theoretical.
> > >
> > > I still think a "reset all KMS props to an arbitrary default value" flag
> > > in drmModeAtomicCommit is the best way forward. I'm not sure a user-space
> > > protocol would help too much.
> >
> > Hi Simon,
> >
> > for the "reset KMS state" problem, sure. Thanks for confirming the
> > problem, too.
> >
> > The hand-off problem does need userspace protocol though, so that the
> > two parties can negotiate what part of KMS state can be inherited by
> > the receiver and who will do the animation from the first to the second
> > state in case you want to avoid abrupt changes. It would also be useful
> > for a cross-fade as a perhaps more flexible way than the current "leak
> > an FB, let the next KMS client scrape it via ioctls and copy it so it
> > can be textured from".
>
> The state reset already is an implicit protocol. Another IPC mechanism
> however could extend it to work the other way around: instead of
> inheriting all the state and trying to transition from that to the
> second client's desired state the second client would send its own
> desired state back to the first (instead of applying it immediately)
> which would then try to transition from its own state to the second
> state (and if it can't you fall back to the implicit inherited state
> protocol). However, this is only an improvement if the first client
> knows how to do the transition and the second does not. All in all I
> doubt that you can convince most people to add this kind of complexity
> just for slightly higher chances of a good transition.
>
> The reset state protocol on the other hand solves real problems and
> gives you a good transition as long as the second client knows about the
> same properties as the previous one which usually is the case for the
> typical bootsplash->login manager->compositor chain.
>
> Maybe I'm completely missing how such a protocol would work though.

The idea was that since you would have to have some IPC mechanism in
user space anyway to quickly effect a flicker-free transition from
Plymouth to the display manager (since, as de Goede reiterates in the
other message, both processes must have the device already open and
call drmSetMaster/drmDropMaster coordinatedly) you might just as well
look for ways how it could be designed for the benefit of everyone.
Using "implicit protocols" for things like this is usually the go-to
way, not because it's good design, but because it is easy to
implement.  But these "implicit protocols" have a tendency to greatly
limit what can be done and to not be easily adaptable once the use
cases become more complicated or refined, and thus they force
contortions on everyone eventually.

How such a protocol could look?  I don't know.  Maybe some DBus
interface for a broker/multiplexer for shared devices that would keep
track of the current DRM master and tell any process interested in
obtaining it what process to talk to.  It could then contact it either
via DBus or over a separate socket, communicate its capabilities,
negotiate the modalities for the transition and acquire the necessary
resources in the form of file descriptors passed over DBus/the socket.
Then both processes could set themselves up for the transition and
effect it, which could involve e.g. unlocking a locked mutex/semaphore
in shared memory.  Alternatively, the donor could refuse the handover,
e.g. if a screen locker is configured to prohibit release of the
device.  Complexitywise the sky would be the limit, of course, but it
needn't be this complicated from the beginning.  An initial version of
such a protocol could be held just as simple as the status quo.

As for the point raised by Paalanen that implementing something like
this would require a lot of effort I must state that, while certainly
true, many of the things I mentioned here are already implemented
somehow somewhere.  Plymouth has a control socket and protocol with
which the state of the splash screen can be controlled from the
outside to make the transition to gdm smoother.  The xlease project
apparently was designed with the intent that DRM devices should be
leased (and subleased) out to processes, and cross-process
coordination would be governed this way.  The kmscon project also had
to come up with something to govern device access since it could no
longer piggy-back on the VT-API.  systemd-logind also draws up a
framework for governance over a shared device and how to tie them to
sessions/seats (with such peculiarities that you cannot auto-spawn a
getty on tty1 since that is "reserved" for Wayland).  Then there is
the VT console, and probably lots of other little things I don't even
know about.

All this effort is already being expended, and IMO it proves the need
for some way to gracefully coordinate access to shared devices that
offers more than what can be done with current "implicit protocols".
The community should consider acknowledging this need and trying to
answer the question what such a subsystem should and should not look
like.  Once the nature of the problem is understood better
implementational questions will become easier.

Regards.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-08  7:36     ` Pekka Paalanen
  2021-09-08  7:49       ` Hans de Goede
@ 2021-09-08 16:27       ` Daniel Vetter
  2021-09-09  7:37         ` Pekka Paalanen
  1 sibling, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2021-09-08 16:27 UTC (permalink / raw)
  To: Pekka Paalanen; +Cc: Hans de Goede, Dennis Filder, dri-devel

On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:
>
> On Tue, 7 Sep 2021 14:42:56 +0200
> Hans de Goede <hdegoede@redhat.com> wrote:
>
> > Hi,
> >
> > On 9/7/21 12:07 PM, Pekka Paalanen wrote:
> > > On Fri, 3 Sep 2021 21:08:21 +0200
> > > Dennis Filder <d.filder@web.de> wrote:
> > >
> > >> Hans de Goede asked me to take a topic from a private discussion here.
> > >> I must also preface that I'm not a graphics person and my knowledge of
> > >> DRI/DRM is cursory at best.
> > >>
> > >> I initiated the conversation with de Goede after learning that the X
> > >> server now supports being started with an open DRM file descriptor
> > >> (this was added for Keith Packard's xlease project).  I wondered if
> > >> that could be used to smoothen the Plymouth->X transition somehow and
> > >> asked de Goede if there were any such plans.  He denied, but mentioned
> > >> that a new ioctl is in the works to prevent the kernel from wiping the
> > >> contents of a frame buffer after a device is closed, and that this
> > >> would help to keep transitions smooth.
> > >
> > > Hi,
> > >
> > > I believe the kernel is not wiping anything on device close. If
> > > something in the KMS state is wiped, it originates in userspace:
> > >
> > > - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
> > >   output off, you need to be careful to "leak" your FB if you want a
> > >   smooth hand-over)
> >
> > The "kernel is not wiping anything on device close" is not true,
> > when closing /dev/dri/card# any remaining FBs from the app closing
> > it will be dealt with as if they were RmFB-ed, causing the screen
> > to show what I call "the fallback fb", at least with the i915 driver.
>
> No, that's not what should happen AFAIK.
>
> True, all FBs that are not referenced by active CRTCs or planes will
> get freed, since their refcount drops to zero, but those CRTCs and
> planes that are active will remain active and therefore keep their
> reference to the respective FBs and so the FBs remain until replaced or
> turned off explicitly (by e.g. fbcon if you switch to that rather than
> another userspace KMS client). I believe that is the whole reason why
> e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
> would not have anything to scrape.
>
> danvet, what is the DRM core intention?

Historical accidents mostly. There's two things that foil easy
handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
- Forced fbdev restore on final close of all drm fd. This is only
prevented if there's a drm master left around (systemd-logind can keep
that instead of forcing the compositor to survive until the other has
taken over, which it needs to do anyway to prevent the drm master
handover from going sideways). This can be fixed by simply disabling
fbdev completely, which you really want to do anyway. Again it's uabi,
people will complain if we break this I think.

> Or am I confused because display servers do not tend to close the DRM
> device fd on switch-out but Plymouth does (too early)?

Yeah, that stops both forced restore/disable from kicking in.

> If so, why can't Plymouth keep the device open longer and quit only
> when the hand-off is complete? Not quitting too early would be a
> prerequisite for any explicit hand-off protocol as well.

With closefb semantics and fbdev disabled plymouth could quit early,
and things still work.
-Daniel

>
> Thanks,
> pq
>
> > > - Xorg doing something (e.g. resetting instead of inheriting KMS state)
> > >
> > > - Something missed in the hand-off sequence which allows fbcon to
> > >   momentarily take over between Plymouth and Xorg. This would need to
> > >   be fixed between Plymouth and Xorg.
> > >
> > > - Maybe systemd-logind does something odd to the KMS device? It has
> > >   pretty wild code there. Or maybe it causes fbcon to take over.
> > >
> > > What is the new ioctl you referred to?
> >
> > It is an ioctl to mark a FB to not have it auto-removed on device-close,
> > instead leaving it in place until some some kernel/userspace client
> > actively installs another FB. This was proposed by Rob Clark quite
> > a while ago, but it never got anywhere because of lack of userspace
> > actually interested in using it.
> >
> > I've been thinking about reviving Rob's patch, since at least for
> > plymouth this would be pretty useful to have.
> >
> > Regards,
> >
> > Hans
> >
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-08 16:27       ` Daniel Vetter
@ 2021-09-09  7:37         ` Pekka Paalanen
  2021-09-14 13:45           ` Daniel Vetter
  0 siblings, 1 reply; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-09  7:37 UTC (permalink / raw)
  To: Daniel Vetter; +Cc: Hans de Goede, Dennis Filder, dri-devel

[-- Attachment #1: Type: text/plain, Size: 4397 bytes --]

On Wed, 8 Sep 2021 18:27:09 +0200
Daniel Vetter <daniel@ffwll.ch> wrote:

> On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:
> >
> > On Tue, 7 Sep 2021 14:42:56 +0200
> > Hans de Goede <hdegoede@redhat.com> wrote:
> >  
> > > Hi,
> > >
> > > On 9/7/21 12:07 PM, Pekka Paalanen wrote:  
> > > > On Fri, 3 Sep 2021 21:08:21 +0200
> > > > Dennis Filder <d.filder@web.de> wrote:
> > > >  
> > > >> Hans de Goede asked me to take a topic from a private discussion here.
> > > >> I must also preface that I'm not a graphics person and my knowledge of
> > > >> DRI/DRM is cursory at best.
> > > >>
> > > >> I initiated the conversation with de Goede after learning that the X
> > > >> server now supports being started with an open DRM file descriptor
> > > >> (this was added for Keith Packard's xlease project).  I wondered if
> > > >> that could be used to smoothen the Plymouth->X transition somehow and
> > > >> asked de Goede if there were any such plans.  He denied, but mentioned
> > > >> that a new ioctl is in the works to prevent the kernel from wiping the
> > > >> contents of a frame buffer after a device is closed, and that this
> > > >> would help to keep transitions smooth.  
> > > >
> > > > Hi,
> > > >
> > > > I believe the kernel is not wiping anything on device close. If
> > > > something in the KMS state is wiped, it originates in userspace:
> > > >
> > > > - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
> > > >   output off, you need to be careful to "leak" your FB if you want a
> > > >   smooth hand-over)  
> > >
> > > The "kernel is not wiping anything on device close" is not true,
> > > when closing /dev/dri/card# any remaining FBs from the app closing
> > > it will be dealt with as if they were RmFB-ed, causing the screen
> > > to show what I call "the fallback fb", at least with the i915 driver.  
> >
> > No, that's not what should happen AFAIK.
> >
> > True, all FBs that are not referenced by active CRTCs or planes will
> > get freed, since their refcount drops to zero, but those CRTCs and
> > planes that are active will remain active and therefore keep their
> > reference to the respective FBs and so the FBs remain until replaced or
> > turned off explicitly (by e.g. fbcon if you switch to that rather than
> > another userspace KMS client). I believe that is the whole reason why
> > e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
> > would not have anything to scrape.
> >
> > danvet, what is the DRM core intention?  
> 
> Historical accidents mostly. There's two things that foil easy
> handover to the next compositor:
> - RMFB instead of CLOSEFB semantics, especially when closing the
> drmfd. This is uapi, so anything we change needs to be opt-in

What does this mean and refer to?

Are you trying to say, that closing the DRM device fd (freeing the file
description) causes an implicit RmFB on all the FBs tied to that DRM
device file description?

I never realised that before.

> - Forced fbdev restore on final close of all drm fd. This is only
> prevented if there's a drm master left around (systemd-logind can keep
> that instead of forcing the compositor to survive until the other has
> taken over, which it needs to do anyway to prevent the drm master
> handover from going sideways). This can be fixed by simply disabling
> fbdev completely, which you really want to do anyway. Again it's uabi,
> people will complain if we break this I think.

Do you mean that it is not enough to leave the tty in KD_GRAPHICS mode
to stop fbcon/fbdev from taking over?

Is it really fbdev on its own rather than fbcon (poking at fbdev) that
will change the KMS state?

That is, it's not enough to disable fbcon?

> > Or am I confused because display servers do not tend to close the DRM
> > device fd on switch-out but Plymouth does (too early)?  
> 
> Yeah, that stops both forced restore/disable from kicking in.

Which "that"?

> > If so, why can't Plymouth keep the device open longer and quit only
> > when the hand-off is complete? Not quitting too early would be a
> > prerequisite for any explicit hand-off protocol as well.  
> 
> With closefb semantics and fbdev disabled plymouth could quit early,
> and things still work.

What is "closefb semantics"?


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-08 16:21         ` Dennis Filder
@ 2021-09-09  8:20           ` Pekka Paalanen
  0 siblings, 0 replies; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-09  8:20 UTC (permalink / raw)
  To: Dennis Filder; +Cc: Sebastian Wick, dri-devel, Hans de Goede, Simon Ser

[-- Attachment #1: Type: text/plain, Size: 5610 bytes --]

On Wed, 8 Sep 2021 18:21:12 +0200
Dennis Filder <d.filder@web.de> wrote:

> The idea was that since you would have to have some IPC mechanism in
> user space anyway to quickly effect a flicker-free transition from
> Plymouth to the display manager (since, as de Goede reiterates in the
> other message, both processes must have the device already open and
> call drmSetMaster/drmDropMaster coordinatedly) you might just as well
> look for ways how it could be designed for the benefit of everyone.
> Using "implicit protocols" for things like this is usually the go-to
> way, not because it's good design, but because it is easy to
> implement.  But these "implicit protocols" have a tendency to greatly
> limit what can be done and to not be easily adaptable once the use
> cases become more complicated or refined, and thus they force
> contortions on everyone eventually.
> 
> How such a protocol could look?  I don't know.  Maybe some DBus
> interface for a broker/multiplexer for shared devices that would keep
> track of the current DRM master and tell any process interested in
> obtaining it what process to talk to.

Hi,

such broker daemon exists already. It is called systemd-logind, and
maybe some of the logind replacements as well. Currently it handles
session switching, the D-Bus API replacing the legacy VT switching API
and drmSetMaster/drmDropMaster. It's also independent of the VT
subsystem, so it can be used on all physical seats.

The existing D-Bus API there is probably not flexible enough for the
best hand-over experience though, as Simon said.

Since session switching is what the logind API does, hand-over protocols
would fit there IMO.

>  It could then contact it either
> via DBus or over a separate socket, communicate its capabilities,
> negotiate the modalities for the transition and acquire the necessary
> resources in the form of file descriptors passed over DBus/the socket.
> Then both processes could set themselves up for the transition and
> effect it, which could involve e.g. unlocking a locked mutex/semaphore
> in shared memory.  Alternatively, the donor could refuse the handover,
> e.g. if a screen locker is configured to prohibit release of the
> device.  Complexitywise the sky would be the limit, of course, but it
> needn't be this complicated from the beginning.  An initial version of
> such a protocol could be held just as simple as the status quo.
> 
> As for the point raised by Paalanen that implementing something like
> this would require a lot of effort I must state that, while certainly
> true, many of the things I mentioned here are already implemented
> somehow somewhere.  Plymouth has a control socket and protocol with
> which the state of the splash screen can be controlled from the
> outside to make the transition to gdm smoother.  The xlease project
> apparently was designed with the intent that DRM devices should be
> leased (and subleased) out to processes, and cross-process
> coordination would be governed this way.

DRM leasing was not designed for permanent hand-over. Instead, it was
designed for temporarily leasing some KMS resources to another process,
where the lessor can forcibly revoke them at any time.

That said, I'm not sure what happens to the lessee if the lessor quits.
I would expect the lease to get revoked, since how else would the next
display server instance gain control if the lessor e.g. crashed.

I'd keep the DRM leasing out of the permanent whole-device hand-off
design, it seems like a detour to me.

>  The kmscon project also had
> to come up with something to govern device access since it could no
> longer piggy-back on the VT-API.  systemd-logind also draws up a
> framework for governance over a shared device and how to tie them to
> sessions/seats (with such peculiarities that you cannot auto-spawn a
> getty on tty1 since that is "reserved" for Wayland).  Then there is
> the VT console, and probably lots of other little things I don't even
> know about.
> 
> All this effort is already being expended, and IMO it proves the need
> for some way to gracefully coordinate access to shared devices that
> offers more than what can be done with current "implicit protocols".
> The community should consider acknowledging this need and trying to
> answer the question what such a subsystem should and should not look
> like.  Once the nature of the problem is understood better
> implementational questions will become easier.

Your initial email came out a bit ranty rather than a technical
question. I think it would help if you had a tangible goal, some
specific behaviour you want to implement, and write that down in detail.
Then it would be easier to discuss how to achieve that while not making
the solution excessively single-purpose or hacky. Imaginary use cases
are both fairly abstract so hard to discuss, and uncertain if they would
ever actually be done - putting the effort needed at risk of being
wasted, reducing interest.

I acknowledge a lot of needs all over, but there are only so many hours
in a day. My wish for a "KMS state reset" feature is in the same
position. First I am working towards userspace features (color
management and HDR) that I expect to highlight the need for the
feature, and then see if the wish gets more traction - unless one day I
would be able to work on it myself. It is also perfectly possible that
KMS state reset feature will never happen, since it is too easy to fix
KMS clients to just program more KMS properties.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-09  7:37         ` Pekka Paalanen
@ 2021-09-14 13:45           ` Daniel Vetter
  2021-09-22  8:56             ` Pekka Paalanen
  0 siblings, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2021-09-14 13:45 UTC (permalink / raw)
  To: Pekka Paalanen; +Cc: Daniel Vetter, Hans de Goede, Dennis Filder, dri-devel

On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
> On Wed, 8 Sep 2021 18:27:09 +0200
> Daniel Vetter <daniel@ffwll.ch> wrote:
> 
> > On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:
> > >
> > > On Tue, 7 Sep 2021 14:42:56 +0200
> > > Hans de Goede <hdegoede@redhat.com> wrote:
> > >  
> > > > Hi,
> > > >
> > > > On 9/7/21 12:07 PM, Pekka Paalanen wrote:  
> > > > > On Fri, 3 Sep 2021 21:08:21 +0200
> > > > > Dennis Filder <d.filder@web.de> wrote:
> > > > >  
> > > > >> Hans de Goede asked me to take a topic from a private discussion here.
> > > > >> I must also preface that I'm not a graphics person and my knowledge of
> > > > >> DRI/DRM is cursory at best.
> > > > >>
> > > > >> I initiated the conversation with de Goede after learning that the X
> > > > >> server now supports being started with an open DRM file descriptor
> > > > >> (this was added for Keith Packard's xlease project).  I wondered if
> > > > >> that could be used to smoothen the Plymouth->X transition somehow and
> > > > >> asked de Goede if there were any such plans.  He denied, but mentioned
> > > > >> that a new ioctl is in the works to prevent the kernel from wiping the
> > > > >> contents of a frame buffer after a device is closed, and that this
> > > > >> would help to keep transitions smooth.  
> > > > >
> > > > > Hi,
> > > > >
> > > > > I believe the kernel is not wiping anything on device close. If
> > > > > something in the KMS state is wiped, it originates in userspace:
> > > > >
> > > > > - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
> > > > >   output off, you need to be careful to "leak" your FB if you want a
> > > > >   smooth hand-over)  
> > > >
> > > > The "kernel is not wiping anything on device close" is not true,
> > > > when closing /dev/dri/card# any remaining FBs from the app closing
> > > > it will be dealt with as if they were RmFB-ed, causing the screen
> > > > to show what I call "the fallback fb", at least with the i915 driver.  
> > >
> > > No, that's not what should happen AFAIK.
> > >
> > > True, all FBs that are not referenced by active CRTCs or planes will
> > > get freed, since their refcount drops to zero, but those CRTCs and
> > > planes that are active will remain active and therefore keep their
> > > reference to the respective FBs and so the FBs remain until replaced or
> > > turned off explicitly (by e.g. fbcon if you switch to that rather than
> > > another userspace KMS client). I believe that is the whole reason why
> > > e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
> > > would not have anything to scrape.
> > >
> > > danvet, what is the DRM core intention?  
> > 
> > Historical accidents mostly. There's two things that foil easy
> > handover to the next compositor:
> > - RMFB instead of CLOSEFB semantics, especially when closing the
> > drmfd. This is uapi, so anything we change needs to be opt-in
> 
> What does this mean and refer to?
> 
> Are you trying to say, that closing the DRM device fd (freeing the file
> description) causes an implicit RmFB on all the FBs tied to that DRM
> device file description?
> 
> I never realised that before.

Yes, final close does iterate over fb and do an RMFB. Which is why we've
had this discussion whether closefb semantics should be an ADDFB2 flag at
creation time instead.

> > - Forced fbdev restore on final close of all drm fd. This is only
> > prevented if there's a drm master left around (systemd-logind can keep
> > that instead of forcing the compositor to survive until the other has
> > taken over, which it needs to do anyway to prevent the drm master
> > handover from going sideways). This can be fixed by simply disabling
> > fbdev completely, which you really want to do anyway. Again it's uabi,
> > people will complain if we break this I think.
> 
> Do you mean that it is not enough to leave the tty in KD_GRAPHICS mode
> to stop fbcon/fbdev from taking over?

Nope. You need an open drm master.

This is because we do actually support /dev/fb clients rendering in
KD_GRAPHICS mode for backwards compat with the fbdev subsystem.

> Is it really fbdev on its own rather than fbcon (poking at fbdev) that
> will change the KMS state?
> 
> That is, it's not enough to disable fbcon?

fbcon doesn't disable fbdev, and the only way to block fbdev is to have a
drm master around.

I guess we could try and make this smarter by creating some kind of weak
master status for fbdev, but only when either fbcon or fbdev is opened.
Maybe this would help?

fbdev already keep track of this open count, so wouldn't be too onerous to
wire that up into drm_client.

The problem there is then that not yet all drivers use the drm_client
stuff for fbdev emulation, so you'd need to either convert more, or hack
up a few more things to make this consistent.

> > > Or am I confused because display servers do not tend to close the DRM
> > > device fd on switch-out but Plymouth does (too early)?  
> > 
> > Yeah, that stops both forced restore/disable from kicking in.
> 
> Which "that"?

that = open drm master. Open drm master alwasy wins agains fbdev/fbcon,
and with latest patches it's guaranteed to be race free.

> > > If so, why can't Plymouth keep the device open longer and quit only
> > > when the hand-off is complete? Not quitting too early would be a
> > > prerequisite for any explicit hand-off protocol as well.  
> > 
> > With closefb semantics and fbdev disabled plymouth could quit early,
> > and things still work.
> 
> What is "closefb semantics"?

closefb semantics = no forced plane/crtc disable, active plane keeps a
drm_fb reference

rmfb semantics = forced plane/crtc disable, the drm_fb is guaranteed to be
forcefully removed from the system

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-14 13:45           ` Daniel Vetter
@ 2021-09-22  8:56             ` Pekka Paalanen
  2021-09-22  9:21               ` Hans de Goede
  0 siblings, 1 reply; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-22  8:56 UTC (permalink / raw)
  To: Daniel Vetter; +Cc: Hans de Goede, Dennis Filder, dri-devel

[-- Attachment #1: Type: text/plain, Size: 6462 bytes --]

On Tue, 14 Sep 2021 15:45:21 +0200
Daniel Vetter <daniel@ffwll.ch> wrote:

> On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
> > On Wed, 8 Sep 2021 18:27:09 +0200
> > Daniel Vetter <daniel@ffwll.ch> wrote:
> >   
> > > On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:  
> > > >
> > > > On Tue, 7 Sep 2021 14:42:56 +0200
> > > > Hans de Goede <hdegoede@redhat.com> wrote:
> > > >    
> > > > > Hi,
> > > > >
> > > > > On 9/7/21 12:07 PM, Pekka Paalanen wrote:    
> > > > > > On Fri, 3 Sep 2021 21:08:21 +0200
> > > > > > Dennis Filder <d.filder@web.de> wrote:
> > > > > >    
> > > > > >> Hans de Goede asked me to take a topic from a private discussion here.
> > > > > >> I must also preface that I'm not a graphics person and my knowledge of
> > > > > >> DRI/DRM is cursory at best.
> > > > > >>
> > > > > >> I initiated the conversation with de Goede after learning that the X
> > > > > >> server now supports being started with an open DRM file descriptor
> > > > > >> (this was added for Keith Packard's xlease project).  I wondered if
> > > > > >> that could be used to smoothen the Plymouth->X transition somehow and
> > > > > >> asked de Goede if there were any such plans.  He denied, but mentioned
> > > > > >> that a new ioctl is in the works to prevent the kernel from wiping the
> > > > > >> contents of a frame buffer after a device is closed, and that this
> > > > > >> would help to keep transitions smooth.    
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I believe the kernel is not wiping anything on device close. If
> > > > > > something in the KMS state is wiped, it originates in userspace:
> > > > > >
> > > > > > - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
> > > > > >   output off, you need to be careful to "leak" your FB if you want a
> > > > > >   smooth hand-over)    
> > > > >
> > > > > The "kernel is not wiping anything on device close" is not true,
> > > > > when closing /dev/dri/card# any remaining FBs from the app closing
> > > > > it will be dealt with as if they were RmFB-ed, causing the screen
> > > > > to show what I call "the fallback fb", at least with the i915 driver.    
> > > >
> > > > No, that's not what should happen AFAIK.
> > > >
> > > > True, all FBs that are not referenced by active CRTCs or planes will
> > > > get freed, since their refcount drops to zero, but those CRTCs and
> > > > planes that are active will remain active and therefore keep their
> > > > reference to the respective FBs and so the FBs remain until replaced or
> > > > turned off explicitly (by e.g. fbcon if you switch to that rather than
> > > > another userspace KMS client). I believe that is the whole reason why
> > > > e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
> > > > would not have anything to scrape.
> > > >
> > > > danvet, what is the DRM core intention?    
> > > 
> > > Historical accidents mostly. There's two things that foil easy
> > > handover to the next compositor:
> > > - RMFB instead of CLOSEFB semantics, especially when closing the
> > > drmfd. This is uapi, so anything we change needs to be opt-in  
> > 
> > What does this mean and refer to?
> > 
> > Are you trying to say, that closing the DRM device fd (freeing the file
> > description) causes an implicit RmFB on all the FBs tied to that DRM
> > device file description?
> > 
> > I never realised that before.  
> 
> Yes, final close does iterate over fb and do an RMFB. Which is why we've
> had this discussion whether closefb semantics should be an ADDFB2 flag at
> creation time instead.

Hi Daniel,

such flag would make sense to me.

> > > - Forced fbdev restore on final close of all drm fd. This is only
> > > prevented if there's a drm master left around (systemd-logind can keep
> > > that instead of forcing the compositor to survive until the other has
> > > taken over, which it needs to do anyway to prevent the drm master
> > > handover from going sideways). This can be fixed by simply disabling
> > > fbdev completely, which you really want to do anyway. Again it's uabi,
> > > people will complain if we break this I think.  
> > 
> > Do you mean that it is not enough to leave the tty in KD_GRAPHICS mode
> > to stop fbcon/fbdev from taking over?  
> 
> Nope. You need an open drm master.
> 
> This is because we do actually support /dev/fb clients rendering in
> KD_GRAPHICS mode for backwards compat with the fbdev subsystem.
> 
> > Is it really fbdev on its own rather than fbcon (poking at fbdev) that
> > will change the KMS state?
> > 
> > That is, it's not enough to disable fbcon?  
> 
> fbcon doesn't disable fbdev, and the only way to block fbdev is to have a
> drm master around.

Thanks for pointing this out.

> I guess we could try and make this smarter by creating some kind of weak
> master status for fbdev, but only when either fbcon or fbdev is opened.
> Maybe this would help?

It would make sense to me if we wanted a bandaid, but do we? I don't
know.

> fbdev already keep track of this open count, so wouldn't be too onerous to
> wire that up into drm_client.
> 
> The problem there is then that not yet all drivers use the drm_client
> stuff for fbdev emulation, so you'd need to either convert more, or hack
> up a few more things to make this consistent.
> 
> > > > Or am I confused because display servers do not tend to close the DRM
> > > > device fd on switch-out but Plymouth does (too early)?    
> > > 
> > > Yeah, that stops both forced restore/disable from kicking in.  
> > 
> > Which "that"?  
> 
> that = open drm master. Open drm master alwasy wins agains fbdev/fbcon,
> and with latest patches it's guaranteed to be race free.
> 
> > > > If so, why can't Plymouth keep the device open longer and quit only
> > > > when the hand-off is complete? Not quitting too early would be a
> > > > prerequisite for any explicit hand-off protocol as well.    
> > > 
> > > With closefb semantics and fbdev disabled plymouth could quit early,
> > > and things still work.  
> > 
> > What is "closefb semantics"?  
> 
> closefb semantics = no forced plane/crtc disable, active plane keeps a
> drm_fb reference
> 
> rmfb semantics = forced plane/crtc disable, the drm_fb is guaranteed to be
> forcefully removed from the system

Very good.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-22  8:56             ` Pekka Paalanen
@ 2021-09-22  9:21               ` Hans de Goede
  2021-09-23  8:23                 ` Pekka Paalanen
  2021-10-01 16:33                 ` Simon Ser
  0 siblings, 2 replies; 23+ messages in thread
From: Hans de Goede @ 2021-09-22  9:21 UTC (permalink / raw)
  To: Pekka Paalanen, Daniel Vetter; +Cc: Dennis Filder, dri-devel

Hi,

On 9/22/21 10:56 AM, Pekka Paalanen wrote:
> On Tue, 14 Sep 2021 15:45:21 +0200
> Daniel Vetter <daniel@ffwll.ch> wrote:
> 
>> On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
>>> On Wed, 8 Sep 2021 18:27:09 +0200
>>> Daniel Vetter <daniel@ffwll.ch> wrote:
>>>   
>>>> On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:  
>>>>>
>>>>> On Tue, 7 Sep 2021 14:42:56 +0200
>>>>> Hans de Goede <hdegoede@redhat.com> wrote:
>>>>>    
>>>>>> Hi,
>>>>>>
>>>>>> On 9/7/21 12:07 PM, Pekka Paalanen wrote:    
>>>>>>> On Fri, 3 Sep 2021 21:08:21 +0200
>>>>>>> Dennis Filder <d.filder@web.de> wrote:
>>>>>>>    
>>>>>>>> Hans de Goede asked me to take a topic from a private discussion here.
>>>>>>>> I must also preface that I'm not a graphics person and my knowledge of
>>>>>>>> DRI/DRM is cursory at best.
>>>>>>>>
>>>>>>>> I initiated the conversation with de Goede after learning that the X
>>>>>>>> server now supports being started with an open DRM file descriptor
>>>>>>>> (this was added for Keith Packard's xlease project).  I wondered if
>>>>>>>> that could be used to smoothen the Plymouth->X transition somehow and
>>>>>>>> asked de Goede if there were any such plans.  He denied, but mentioned
>>>>>>>> that a new ioctl is in the works to prevent the kernel from wiping the
>>>>>>>> contents of a frame buffer after a device is closed, and that this
>>>>>>>> would help to keep transitions smooth.    
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I believe the kernel is not wiping anything on device close. If
>>>>>>> something in the KMS state is wiped, it originates in userspace:
>>>>>>>
>>>>>>> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
>>>>>>>   output off, you need to be careful to "leak" your FB if you want a
>>>>>>>   smooth hand-over)    
>>>>>>
>>>>>> The "kernel is not wiping anything on device close" is not true,
>>>>>> when closing /dev/dri/card# any remaining FBs from the app closing
>>>>>> it will be dealt with as if they were RmFB-ed, causing the screen
>>>>>> to show what I call "the fallback fb", at least with the i915 driver.    
>>>>>
>>>>> No, that's not what should happen AFAIK.
>>>>>
>>>>> True, all FBs that are not referenced by active CRTCs or planes will
>>>>> get freed, since their refcount drops to zero, but those CRTCs and
>>>>> planes that are active will remain active and therefore keep their
>>>>> reference to the respective FBs and so the FBs remain until replaced or
>>>>> turned off explicitly (by e.g. fbcon if you switch to that rather than
>>>>> another userspace KMS client). I believe that is the whole reason why
>>>>> e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
>>>>> would not have anything to scrape.
>>>>>
>>>>> danvet, what is the DRM core intention?    
>>>>
>>>> Historical accidents mostly. There's two things that foil easy
>>>> handover to the next compositor:
>>>> - RMFB instead of CLOSEFB semantics, especially when closing the
>>>> drmfd. This is uapi, so anything we change needs to be opt-in  
>>>
>>> What does this mean and refer to?
>>>
>>> Are you trying to say, that closing the DRM device fd (freeing the file
>>> description) causes an implicit RmFB on all the FBs tied to that DRM
>>> device file description?
>>>
>>> I never realised that before.  
>>
>> Yes, final close does iterate over fb and do an RMFB. Which is why we've
>> had this discussion whether closefb semantics should be an ADDFB2 flag at
>> creation time instead.
> 
> Hi Daniel,
> 
> such flag would make sense to me.

Hmm, I was thinking having a separate call to mark a FB to switch to
closefb semantics. But both plymouth (because of end of animation)
and GNOME (because a mostly empty gnome-shell needs to be rendered
to avoid leaking privacy sensitive info) will need to prepare a
special FB on exit anyways, so then an ADDFB2 flag would work fine.

I would be happy to work on the plymouth side of this, so that we
have at least one consumer of such a flag lined up for merging.

Regards,

Hans


^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-22  9:21               ` Hans de Goede
@ 2021-09-23  8:23                 ` Pekka Paalanen
  2021-09-23  8:52                   ` Hans de Goede
  2021-09-30  9:26                   ` Daniel Vetter
  2021-10-01 16:33                 ` Simon Ser
  1 sibling, 2 replies; 23+ messages in thread
From: Pekka Paalanen @ 2021-09-23  8:23 UTC (permalink / raw)
  To: Hans de Goede; +Cc: Daniel Vetter, Dennis Filder, dri-devel

[-- Attachment #1: Type: text/plain, Size: 5716 bytes --]

On Wed, 22 Sep 2021 11:21:16 +0200
Hans de Goede <hdegoede@redhat.com> wrote:

> Hi,
> 
> On 9/22/21 10:56 AM, Pekka Paalanen wrote:
> > On Tue, 14 Sep 2021 15:45:21 +0200
> > Daniel Vetter <daniel@ffwll.ch> wrote:
> >   
> >> On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:  
> >>> On Wed, 8 Sep 2021 18:27:09 +0200
> >>> Daniel Vetter <daniel@ffwll.ch> wrote:
> >>>     
> >>>> On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:    
> >>>>>
> >>>>> On Tue, 7 Sep 2021 14:42:56 +0200
> >>>>> Hans de Goede <hdegoede@redhat.com> wrote:
> >>>>>      
> >>>>>> Hi,
> >>>>>>
> >>>>>> On 9/7/21 12:07 PM, Pekka Paalanen wrote:      
> >>>>>>> On Fri, 3 Sep 2021 21:08:21 +0200
> >>>>>>> Dennis Filder <d.filder@web.de> wrote:
> >>>>>>>      
> >>>>>>>> Hans de Goede asked me to take a topic from a private discussion here.
> >>>>>>>> I must also preface that I'm not a graphics person and my knowledge of
> >>>>>>>> DRI/DRM is cursory at best.
> >>>>>>>>
> >>>>>>>> I initiated the conversation with de Goede after learning that the X
> >>>>>>>> server now supports being started with an open DRM file descriptor
> >>>>>>>> (this was added for Keith Packard's xlease project).  I wondered if
> >>>>>>>> that could be used to smoothen the Plymouth->X transition somehow and
> >>>>>>>> asked de Goede if there were any such plans.  He denied, but mentioned
> >>>>>>>> that a new ioctl is in the works to prevent the kernel from wiping the
> >>>>>>>> contents of a frame buffer after a device is closed, and that this
> >>>>>>>> would help to keep transitions smooth.      
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I believe the kernel is not wiping anything on device close. If
> >>>>>>> something in the KMS state is wiped, it originates in userspace:
> >>>>>>>
> >>>>>>> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
> >>>>>>>   output off, you need to be careful to "leak" your FB if you want a
> >>>>>>>   smooth hand-over)      
> >>>>>>
> >>>>>> The "kernel is not wiping anything on device close" is not true,
> >>>>>> when closing /dev/dri/card# any remaining FBs from the app closing
> >>>>>> it will be dealt with as if they were RmFB-ed, causing the screen
> >>>>>> to show what I call "the fallback fb", at least with the i915 driver.      
> >>>>>
> >>>>> No, that's not what should happen AFAIK.
> >>>>>
> >>>>> True, all FBs that are not referenced by active CRTCs or planes will
> >>>>> get freed, since their refcount drops to zero, but those CRTCs and
> >>>>> planes that are active will remain active and therefore keep their
> >>>>> reference to the respective FBs and so the FBs remain until replaced or
> >>>>> turned off explicitly (by e.g. fbcon if you switch to that rather than
> >>>>> another userspace KMS client). I believe that is the whole reason why
> >>>>> e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
> >>>>> would not have anything to scrape.
> >>>>>
> >>>>> danvet, what is the DRM core intention?      
> >>>>
> >>>> Historical accidents mostly. There's two things that foil easy
> >>>> handover to the next compositor:
> >>>> - RMFB instead of CLOSEFB semantics, especially when closing the
> >>>> drmfd. This is uapi, so anything we change needs to be opt-in    
> >>>
> >>> What does this mean and refer to?
> >>>
> >>> Are you trying to say, that closing the DRM device fd (freeing the file
> >>> description) causes an implicit RmFB on all the FBs tied to that DRM
> >>> device file description?
> >>>
> >>> I never realised that before.    
> >>
> >> Yes, final close does iterate over fb and do an RMFB. Which is why we've
> >> had this discussion whether closefb semantics should be an ADDFB2 flag at
> >> creation time instead.  
> > 
> > Hi Daniel,
> > 
> > such flag would make sense to me.  
> 
> Hmm, I was thinking having a separate call to mark a FB to switch to
> closefb semantics. But both plymouth (because of end of animation)
> and GNOME (because a mostly empty gnome-shell needs to be rendered
> to avoid leaking privacy sensitive info) will need to prepare a
> special FB on exit anyways, so then an ADDFB2 flag would work fine.
> 
> I would be happy to work on the plymouth side of this, so that we
> have at least one consumer of such a flag lined up for merging.

Right, but I'm thinking this from the other side: why would anyone
deliberately *want* RmFB semantics on device close?

I can't think of any, and hence I would be inclined to assume that
userspace would just switch to using closefb semantics for everything
all the time.

Legacy userspace is one thing, but userspace that is updated to set
closefb semantics will also be aware of what closefb means: it leaves
the FBs up and CRTCs and planes enabled, if you leave them like that.
So if they don't want that, they know they should not do that.

Asking in another way: why would the same program sometimes use RmFB
semantics and sometimes closefb semantics? Even more so, why would one
switch an FB from one to the other?

Hmm... to prevent leaking sensitive FBs on crash, perhaps? But you can
do that decision at AddFB2 time, right? Maybe not, as you can't really
force EGL to allocate a new buffer at will. Oh, but when EGL gives me a
buffer that I know is safe to leave up, I also know that it is not up
on any KMS plane (no front buffer rendering), so I can just RmFB and
AddFB2 again. That's a bit of a detour though.

At least a separate ioctl on an FB would be more flexible than a flag
at AddFB2.

Btw. what happens if I try to AddFB2 the same buffer twice?


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-23  8:23                 ` Pekka Paalanen
@ 2021-09-23  8:52                   ` Hans de Goede
  2021-09-30  9:26                   ` Daniel Vetter
  1 sibling, 0 replies; 23+ messages in thread
From: Hans de Goede @ 2021-09-23  8:52 UTC (permalink / raw)
  To: Pekka Paalanen; +Cc: Daniel Vetter, Dennis Filder, dri-devel

Hi,

On 9/23/21 10:23 AM, Pekka Paalanen wrote:
> On Wed, 22 Sep 2021 11:21:16 +0200
> Hans de Goede <hdegoede@redhat.com> wrote:
> 
>> Hi,
>>
>> On 9/22/21 10:56 AM, Pekka Paalanen wrote:
>>> On Tue, 14 Sep 2021 15:45:21 +0200
>>> Daniel Vetter <daniel@ffwll.ch> wrote:
>>>   
>>>> On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:  
>>>>> On Wed, 8 Sep 2021 18:27:09 +0200
>>>>> Daniel Vetter <daniel@ffwll.ch> wrote:
>>>>>     
>>>>>> On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:    
>>>>>>>
>>>>>>> On Tue, 7 Sep 2021 14:42:56 +0200
>>>>>>> Hans de Goede <hdegoede@redhat.com> wrote:
>>>>>>>      
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On 9/7/21 12:07 PM, Pekka Paalanen wrote:      
>>>>>>>>> On Fri, 3 Sep 2021 21:08:21 +0200
>>>>>>>>> Dennis Filder <d.filder@web.de> wrote:
>>>>>>>>>      
>>>>>>>>>> Hans de Goede asked me to take a topic from a private discussion here.
>>>>>>>>>> I must also preface that I'm not a graphics person and my knowledge of
>>>>>>>>>> DRI/DRM is cursory at best.
>>>>>>>>>>
>>>>>>>>>> I initiated the conversation with de Goede after learning that the X
>>>>>>>>>> server now supports being started with an open DRM file descriptor
>>>>>>>>>> (this was added for Keith Packard's xlease project).  I wondered if
>>>>>>>>>> that could be used to smoothen the Plymouth->X transition somehow and
>>>>>>>>>> asked de Goede if there were any such plans.  He denied, but mentioned
>>>>>>>>>> that a new ioctl is in the works to prevent the kernel from wiping the
>>>>>>>>>> contents of a frame buffer after a device is closed, and that this
>>>>>>>>>> would help to keep transitions smooth.      
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I believe the kernel is not wiping anything on device close. If
>>>>>>>>> something in the KMS state is wiped, it originates in userspace:
>>>>>>>>>
>>>>>>>>> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
>>>>>>>>>   output off, you need to be careful to "leak" your FB if you want a
>>>>>>>>>   smooth hand-over)      
>>>>>>>>
>>>>>>>> The "kernel is not wiping anything on device close" is not true,
>>>>>>>> when closing /dev/dri/card# any remaining FBs from the app closing
>>>>>>>> it will be dealt with as if they were RmFB-ed, causing the screen
>>>>>>>> to show what I call "the fallback fb", at least with the i915 driver.      
>>>>>>>
>>>>>>> No, that's not what should happen AFAIK.
>>>>>>>
>>>>>>> True, all FBs that are not referenced by active CRTCs or planes will
>>>>>>> get freed, since their refcount drops to zero, but those CRTCs and
>>>>>>> planes that are active will remain active and therefore keep their
>>>>>>> reference to the respective FBs and so the FBs remain until replaced or
>>>>>>> turned off explicitly (by e.g. fbcon if you switch to that rather than
>>>>>>> another userspace KMS client). I believe that is the whole reason why
>>>>>>> e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
>>>>>>> would not have anything to scrape.
>>>>>>>
>>>>>>> danvet, what is the DRM core intention?      
>>>>>>
>>>>>> Historical accidents mostly. There's two things that foil easy
>>>>>> handover to the next compositor:
>>>>>> - RMFB instead of CLOSEFB semantics, especially when closing the
>>>>>> drmfd. This is uapi, so anything we change needs to be opt-in    
>>>>>
>>>>> What does this mean and refer to?
>>>>>
>>>>> Are you trying to say, that closing the DRM device fd (freeing the file
>>>>> description) causes an implicit RmFB on all the FBs tied to that DRM
>>>>> device file description?
>>>>>
>>>>> I never realised that before.    
>>>>
>>>> Yes, final close does iterate over fb and do an RMFB. Which is why we've
>>>> had this discussion whether closefb semantics should be an ADDFB2 flag at
>>>> creation time instead.  
>>>
>>> Hi Daniel,
>>>
>>> such flag would make sense to me.  
>>
>> Hmm, I was thinking having a separate call to mark a FB to switch to
>> closefb semantics. But both plymouth (because of end of animation)
>> and GNOME (because a mostly empty gnome-shell needs to be rendered
>> to avoid leaking privacy sensitive info) will need to prepare a
>> special FB on exit anyways, so then an ADDFB2 flag would work fine.
>>
>> I would be happy to work on the plymouth side of this, so that we
>> have at least one consumer of such a flag lined up for merging.
> 
> Right, but I'm thinking this from the other side: why would anyone
> deliberately *want* RmFB semantics on device close?

Device-close also happens on application crashes. So basically
any time where the app (typically a display-server) may display
privacy sensitive contents which we don't want to leak to other
apps, we want RmFB semantics on close.

ATM e.g. GNOME just leaves everything on the screen in place
at logout / shutdown. So "apps" are definitely going to need to
have some work done to them to make sure that from a privacy
pov it is safe to leave the FB in place after they exit.

> I can't think of any, and hence I would be inclined to assume that
> userspace would just switch to using closefb semantics for everything
> all the time.
> 
> Legacy userspace is one thing, but userspace that is updated to set
> closefb semantics will also be aware of what closefb means: it leaves
> the FBs up and CRTCs and planes enabled, if you leave them like that.
> So if they don't want that, they know they should not do that.
> 
> Asking in another way: why would the same program sometimes use RmFB
> semantics and sometimes closefb semantics? Even more so, why would one
> switch an FB from one to the other?
> 
> Hmm... to prevent leaking sensitive FBs on crash, perhaps?

Right, I should have read your whole email before responding.

> But you can
> do that decision at AddFB2 time, right?

Yes I believe so.

> Maybe not, as you can't really
> force EGL to allocate a new buffer at will.

It does not need to be a new buffer as long as we are sure it has
safe contents in it, there will need to be a special render path
for rendering a "safe" desktop view on logout and once that
rendering has completed to a buffer, then that buffer can be
considered safe I guess (note I don't have a whole lot of
experience with EGL).

> Oh, but when EGL gives me a
> buffer that I know is safe to leave up, I also know that it is not up
> on any KMS plane (no front buffer rendering), so I can just RmFB and
> AddFB2 again. That's a bit of a detour though.
> 
> At least a separate ioctl on an FB would be more flexible than a flag
> at AddFB2.
> 
> Btw. what happens if I try to AddFB2 the same buffer twice?

These questions are best answered by someone who knows this stuff
better then me...

Regards,

Hans


^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-23  8:23                 ` Pekka Paalanen
  2021-09-23  8:52                   ` Hans de Goede
@ 2021-09-30  9:26                   ` Daniel Vetter
  1 sibling, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2021-09-30  9:26 UTC (permalink / raw)
  To: Pekka Paalanen; +Cc: Hans de Goede, Daniel Vetter, Dennis Filder, dri-devel

On Thu, Sep 23, 2021 at 11:23:00AM +0300, Pekka Paalanen wrote:
> On Wed, 22 Sep 2021 11:21:16 +0200
> Hans de Goede <hdegoede@redhat.com> wrote:
> 
> > Hi,
> > 
> > On 9/22/21 10:56 AM, Pekka Paalanen wrote:
> > > On Tue, 14 Sep 2021 15:45:21 +0200
> > > Daniel Vetter <daniel@ffwll.ch> wrote:
> > >   
> > >> On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:  
> > >>> On Wed, 8 Sep 2021 18:27:09 +0200
> > >>> Daniel Vetter <daniel@ffwll.ch> wrote:
> > >>>     
> > >>>> On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen <ppaalanen@gmail.com> wrote:    
> > >>>>>
> > >>>>> On Tue, 7 Sep 2021 14:42:56 +0200
> > >>>>> Hans de Goede <hdegoede@redhat.com> wrote:
> > >>>>>      
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> On 9/7/21 12:07 PM, Pekka Paalanen wrote:      
> > >>>>>>> On Fri, 3 Sep 2021 21:08:21 +0200
> > >>>>>>> Dennis Filder <d.filder@web.de> wrote:
> > >>>>>>>      
> > >>>>>>>> Hans de Goede asked me to take a topic from a private discussion here.
> > >>>>>>>> I must also preface that I'm not a graphics person and my knowledge of
> > >>>>>>>> DRI/DRM is cursory at best.
> > >>>>>>>>
> > >>>>>>>> I initiated the conversation with de Goede after learning that the X
> > >>>>>>>> server now supports being started with an open DRM file descriptor
> > >>>>>>>> (this was added for Keith Packard's xlease project).  I wondered if
> > >>>>>>>> that could be used to smoothen the Plymouth->X transition somehow and
> > >>>>>>>> asked de Goede if there were any such plans.  He denied, but mentioned
> > >>>>>>>> that a new ioctl is in the works to prevent the kernel from wiping the
> > >>>>>>>> contents of a frame buffer after a device is closed, and that this
> > >>>>>>>> would help to keep transitions smooth.      
> > >>>>>>>
> > >>>>>>> Hi,
> > >>>>>>>
> > >>>>>>> I believe the kernel is not wiping anything on device close. If
> > >>>>>>> something in the KMS state is wiped, it originates in userspace:
> > >>>>>>>
> > >>>>>>> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the
> > >>>>>>>   output off, you need to be careful to "leak" your FB if you want a
> > >>>>>>>   smooth hand-over)      
> > >>>>>>
> > >>>>>> The "kernel is not wiping anything on device close" is not true,
> > >>>>>> when closing /dev/dri/card# any remaining FBs from the app closing
> > >>>>>> it will be dealt with as if they were RmFB-ed, causing the screen
> > >>>>>> to show what I call "the fallback fb", at least with the i915 driver.      
> > >>>>>
> > >>>>> No, that's not what should happen AFAIK.
> > >>>>>
> > >>>>> True, all FBs that are not referenced by active CRTCs or planes will
> > >>>>> get freed, since their refcount drops to zero, but those CRTCs and
> > >>>>> planes that are active will remain active and therefore keep their
> > >>>>> reference to the respective FBs and so the FBs remain until replaced or
> > >>>>> turned off explicitly (by e.g. fbcon if you switch to that rather than
> > >>>>> another userspace KMS client). I believe that is the whole reason why
> > >>>>> e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client
> > >>>>> would not have anything to scrape.
> > >>>>>
> > >>>>> danvet, what is the DRM core intention?      
> > >>>>
> > >>>> Historical accidents mostly. There's two things that foil easy
> > >>>> handover to the next compositor:
> > >>>> - RMFB instead of CLOSEFB semantics, especially when closing the
> > >>>> drmfd. This is uapi, so anything we change needs to be opt-in    
> > >>>
> > >>> What does this mean and refer to?
> > >>>
> > >>> Are you trying to say, that closing the DRM device fd (freeing the file
> > >>> description) causes an implicit RmFB on all the FBs tied to that DRM
> > >>> device file description?
> > >>>
> > >>> I never realised that before.    
> > >>
> > >> Yes, final close does iterate over fb and do an RMFB. Which is why we've
> > >> had this discussion whether closefb semantics should be an ADDFB2 flag at
> > >> creation time instead.  
> > > 
> > > Hi Daniel,
> > > 
> > > such flag would make sense to me.  
> > 
> > Hmm, I was thinking having a separate call to mark a FB to switch to
> > closefb semantics. But both plymouth (because of end of animation)
> > and GNOME (because a mostly empty gnome-shell needs to be rendered
> > to avoid leaking privacy sensitive info) will need to prepare a
> > special FB on exit anyways, so then an ADDFB2 flag would work fine.
> > 
> > I would be happy to work on the plymouth side of this, so that we
> > have at least one consumer of such a flag lined up for merging.
> 
> Right, but I'm thinking this from the other side: why would anyone
> deliberately *want* RmFB semantics on device close?
> 
> I can't think of any, and hence I would be inclined to assume that
> userspace would just switch to using closefb semantics for everything
> all the time.
> 
> Legacy userspace is one thing, but userspace that is updated to set
> closefb semantics will also be aware of what closefb means: it leaves
> the FBs up and CRTCs and planes enabled, if you leave them like that.
> So if they don't want that, they know they should not do that.
> 
> Asking in another way: why would the same program sometimes use RmFB
> semantics and sometimes closefb semantics? Even more so, why would one
> switch an FB from one to the other?
> 
> Hmm... to prevent leaking sensitive FBs on crash, perhaps? But you can
> do that decision at AddFB2 time, right? Maybe not, as you can't really
> force EGL to allocate a new buffer at will. Oh, but when EGL gives me a
> buffer that I know is safe to leave up, I also know that it is not up
> on any KMS plane (no front buffer rendering), so I can just RmFB and
> AddFB2 again. That's a bit of a detour though.
> 
> At least a separate ioctl on an FB would be more flexible than a flag
> at AddFB2.
> 
> Btw. what happens if I try to AddFB2 the same buffer twice?

drm_fb are just refcounted metadata containers. So you could have the same
underlying gem bo wrapped in 2 addfb, one with rmfb and one with closefb
semantics. Depending which one you're using on which crtc, the crtc might
be shut off or not when you close the drmfd.

It's a bit silly, but no problem for the kernel, so I think this is all
fine.

Real use-case of multiple drm_fb for the same underlying object is stuff
like XRGB vs ARGB or different modifiers (like with compression enabled or
compression metadata not up to date).
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-09-22  9:21               ` Hans de Goede
  2021-09-23  8:23                 ` Pekka Paalanen
@ 2021-10-01 16:33                 ` Simon Ser
  2021-10-02 17:27                   ` Hans de Goede
  1 sibling, 1 reply; 23+ messages in thread
From: Simon Ser @ 2021-10-01 16:33 UTC (permalink / raw)
  To: Hans de Goede; +Cc: Pekka Paalanen, Daniel Vetter, Dennis Filder, dri-devel

On Wednesday, September 22nd, 2021 at 11:21, Hans de Goede <hdegoede@redhat.com> wrote:

> I would be happy to work on the plymouth side of this, so that we
> have at least one consumer of such a flag lined up for merging.

Do you have plans to work on the kernel side part of this? If so, feel free to
CC me for a review. If not, let me know if you'd like to switch roles, I could
work on a patch.

Simon

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: Handling DRM master transitions cooperatively
  2021-10-01 16:33                 ` Simon Ser
@ 2021-10-02 17:27                   ` Hans de Goede
  0 siblings, 0 replies; 23+ messages in thread
From: Hans de Goede @ 2021-10-02 17:27 UTC (permalink / raw)
  To: Simon Ser; +Cc: Pekka Paalanen, Daniel Vetter, Dennis Filder, dri-devel

Hi,

On 10/1/21 6:33 PM, Simon Ser wrote:
> On Wednesday, September 22nd, 2021 at 11:21, Hans de Goede <hdegoede@redhat.com> wrote:
> 
>> I would be happy to work on the plymouth side of this, so that we
>> have at least one consumer of such a flag lined up for merging.
> 
> Do you have plans to work on the kernel side part of this?

No I don't have any plans to work on the kernel side of this.

> If so, feel free to
> CC me for a review. If not, let me know if you'd like to switch roles, I could
> work on a patch.

I would be happy to review any kernel patches for this
(together with adding support to plymouth for this).

Regards,

Hans


^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2021-10-02 17:27 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-03 19:08 Handling DRM master transitions cooperatively Dennis Filder
2021-09-07 10:07 ` Pekka Paalanen
2021-09-07 10:19   ` Simon Ser
2021-09-07 12:52     ` Pekka Paalanen
2021-09-07 15:52       ` Sebastian Wick
2021-09-08 16:21         ` Dennis Filder
2021-09-09  8:20           ` Pekka Paalanen
2021-09-08  9:51       ` Simon Ser
2021-09-08 11:13         ` Pekka Paalanen
2021-09-08 16:14         ` Dennis Filder
2021-09-07 12:42   ` Hans de Goede
2021-09-08  7:36     ` Pekka Paalanen
2021-09-08  7:49       ` Hans de Goede
2021-09-08 16:27       ` Daniel Vetter
2021-09-09  7:37         ` Pekka Paalanen
2021-09-14 13:45           ` Daniel Vetter
2021-09-22  8:56             ` Pekka Paalanen
2021-09-22  9:21               ` Hans de Goede
2021-09-23  8:23                 ` Pekka Paalanen
2021-09-23  8:52                   ` Hans de Goede
2021-09-30  9:26                   ` Daniel Vetter
2021-10-01 16:33                 ` Simon Ser
2021-10-02 17:27                   ` Hans de Goede

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.