Display notch support

Display notch support

[Caleb Connolly]

With many more non-desktop form factor devices landing in the kernel, 
we're starting to run up against some limitations. Notably devices with 
display notches, cutouts and rounded corners.

Given that the DRI subsystem already deals with physical display 
properties like panel orientation which is fixed in device tree, it 
would make sense to also define other properties like parts of the 
display which are obscured there too. This can then be read by desktop 
environments and UI elements can be suitably adjusted.

Android already deals with non-square displays, however they also keep 
the device configuration in userspace: 
https://developer.android.com/guide/topics/display-cutout

A solution to make this configuration generic and exposed by the kernel 
would standardise this across Linux and potentially Android too which 
would be extremely beneficial to the mobile Linux community at large.

Does this seem like a feasible solution to solving the notch problem and 
avoiding a situation where every compositor have their own method of 
describing cutouts and rounded corners?

Regards,

Caleb



_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Display notch support

[Simon Ser]

> A solution to make this configuration generic and exposed by the kernel
> would standardise this across Linux

Having a KMS property for this makes sense to me.

Chatting with Jani on IRC, it doesn't seem like there's any EDID or
DisplayID block for this.

Note, Android exposes a data structure [1] with:

- Margin of the cut-out for each edge of the screen
- One rectangle per edge describing the cut-out region
- Size of the curved area for each edge of a waterfall display

I haven't found anything describing the rounded corners of the display.

[1]: https://developer.android.com/reference/android/view/DisplayCutout
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Display notch support

[Pekka Paalanen]

[-- Attachment #1.1: Type: text/plain, Size: 1953 bytes --]

On Wed, 28 Apr 2021 07:21:28 +0000
Simon Ser <contact@emersion.fr> wrote:

> > A solution to make this configuration generic and exposed by the kernel
> > would standardise this across Linux  
> 
> Having a KMS property for this makes sense to me.
> 
> Chatting with Jani on IRC, it doesn't seem like there's any EDID or
> DisplayID block for this.
> 
> Note, Android exposes a data structure [1] with:
> 
> - Margin of the cut-out for each edge of the screen
> - One rectangle per edge describing the cut-out region
> - Size of the curved area for each edge of a waterfall display
> 
> I haven't found anything describing the rounded corners of the display.
> 
> [1]: https://developer.android.com/reference/android/view/DisplayCutout

Hi,

I'm kind of worried whether you can design a description structure that
would be good for a long time. That list already looks quite
complicated. Add also watch-like devices with circular displays.

Would the kernel itself use this information at all?

If not, is there not a policy that DT is not a userspace configuration
store?

You mentioned the panel orientation property, but that is used by the
kernel for fbcon or something, is it not? Maybe as the default value
for the CRTC rotation property which actually turns the image?

Assuming that you succeed in describing these non-usable, funny
(waterfall edge), funny2 (e.g. behind a shade or filter so visible but
not normal), funny3 (e.g. phone button area with maybe tactile
markings), and normal areas, how would userspace handle this
information?

Funny2 and funny3 are hypothetical but maybe not too far-fetched.

Is there any provision for generic userspace to handle this generically?

This seems more like a job for the hypothetical liboutput, just like
recognising HMDs (yes, I know, kernel does that already, but there is a
point that kernel may not want to put fbcon on a HMD).


Thanks,
pq

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Display notch support

[Simon Ser]

On Wednesday, April 28th, 2021 at 9:44 AM, Pekka Paalanen <ppaalanen@gmail.com> wrote:

> I'm kind of worried whether you can design a description structure that
> would be good for a long time. That list already looks quite
> complicated. Add also watch-like devices with circular displays.
>
> Would the kernel itself use this information at all?

fbcon might want to letter-box its output to make sure it's not
obscured behind a cut-out area.

> If not, is there not a policy that DT is not a userspace configuration
> store?
>
> You mentioned the panel orientation property, but that is used by the
> kernel for fbcon or something, is it not? Maybe as the default value
> for the CRTC rotation property which actually turns the image?

I wonder if fbcon uses it at all. In general CRTC rotation is not
well-supported by HW drivers, at least for linear buffers. CRTC
rotation is just an optimization.

> Assuming that you succeed in describing these non-usable, funny
> (waterfall edge), funny2 (e.g. behind a shade or filter so visible but
> not normal), funny3 (e.g. phone button area with maybe tactile
> markings), and normal areas, how would userspace handle this
> information?
>
> Funny2 and funny3 are hypothetical but maybe not too far-fetched.
>
> Is there any provision for generic userspace to handle this generically?

I think the main use-case here is make sure there's nothing important
being cut out on screen. I agree we still don't know how the hw will
evolve and might design an API which is too restricted. But building
something that ends up too complicated and too generic wouldn't be
great either.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Display notch support

[Daniel Vetter]

On Wed, Apr 28, 2021 at 10:44:03AM +0300, Pekka Paalanen wrote:
> On Wed, 28 Apr 2021 07:21:28 +0000
> Simon Ser <contact@emersion.fr> wrote:
> 
> > > A solution to make this configuration generic and exposed by the kernel
> > > would standardise this across Linux  
> > 
> > Having a KMS property for this makes sense to me.
> > 
> > Chatting with Jani on IRC, it doesn't seem like there's any EDID or
> > DisplayID block for this.
> > 
> > Note, Android exposes a data structure [1] with:
> > 
> > - Margin of the cut-out for each edge of the screen
> > - One rectangle per edge describing the cut-out region
> > - Size of the curved area for each edge of a waterfall display
> > 
> > I haven't found anything describing the rounded corners of the display.
> > 
> > [1]: https://developer.android.com/reference/android/view/DisplayCutout
> 
> Hi,
> 
> I'm kind of worried whether you can design a description structure that
> would be good for a long time. That list already looks quite
> complicated. Add also watch-like devices with circular displays.
> 
> Would the kernel itself use this information at all?
> 
> If not, is there not a policy that DT is not a userspace configuration
> store?

If someone is sufficiently bored it would make sense to teach fbcon (but
not fbdev I guess for full sized boot splash) to avoid the edges/corners
for output.

But also fbcon/fbdev is I think finally dead on Android, so the
intersection of people who care about cut-outs and fbcon is likely 0.

Otherwise I can't think of anything.

> You mentioned the panel orientation property, but that is used by the
> kernel for fbcon or something, is it not? Maybe as the default value
> for the CRTC rotation property which actually turns the image?
> 
> Assuming that you succeed in describing these non-usable, funny
> (waterfall edge), funny2 (e.g. behind a shade or filter so visible but
> not normal), funny3 (e.g. phone button area with maybe tactile
> markings), and normal areas, how would userspace handle this
> information?
> 
> Funny2 and funny3 are hypothetical but maybe not too far-fetched.
> 
> Is there any provision for generic userspace to handle this generically?
> 
> This seems more like a job for the hypothetical liboutput, just like
> recognising HMDs (yes, I know, kernel does that already, but there is a
> point that kernel may not want to put fbcon on a HMD).

I think the desktop linux solution would be hwdb entries, except we've
never done this for anything display related. So yeah liboutput sounds
about right for this :-)

Btw on fbcon on HMD, I thought we're already taking care of that?
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Display notch support

[Jani Nikula]

On Wed, 28 Apr 2021, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Wed, Apr 28, 2021 at 10:44:03AM +0300, Pekka Paalanen wrote:
>> This seems more like a job for the hypothetical liboutput, just like
>> recognising HMDs (yes, I know, kernel does that already, but there is a
>> point that kernel may not want to put fbcon on a HMD).
>
> I think the desktop linux solution would be hwdb entries, except we've
> never done this for anything display related. So yeah liboutput sounds
> about right for this :-)
>
> Btw on fbcon on HMD, I thought we're already taking care of that?

This is a bit off-topic, but DisplayID 2.0 defines primary use cases for
head mounted VR and AR, so we wouldn't have to quirk them.

BR,
Jani.

-- 
Jani Nikula, Intel Open Source Graphics Center
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Display notch support

[Caleb Connolly]

On 28/04/2021 8:21 am, Simon Ser wrote:
>> A solution to make this configuration generic and exposed by the kernel
>> would standardise this across Linux
> Having a KMS property for this makes sense to me.
>
> Chatting with Jani on IRC, it doesn't seem like there's any EDID or
> DisplayID block for this.
>
> Note, Android exposes a data structure [1] with:
>
> - Margin of the cut-out for each edge of the screen
> - One rectangle per edge describing the cut-out region
> - Size of the curved area for each edge of a waterfall display
>
> I haven't found anything describing the rounded corners of the display.
>
> [1]: https://developer.android.com/reference/android/view/DisplayCutout

It looks like rounded corners only get a developer facing API in Android 
12 [1].

However from a vendor perspective it's possible to set padding for 
rounded corners [2], although this seems to mostly be a hack.

It seems like it would make sense to deal with cutouts and rounded 
corners separately, cutouts are always convex where rounded corners are 
always concave, they have different implications on how content should 
be adjusted.

[1]: 
https://developer.android.com/about/versions/12/features#rounded_corner_apis

[2]: 
https://github.com/LineageOS/android_device_oneplus_enchilada/commit/923c86a13b5ffb58683206a73a0813783e71e3fb





_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel