Hi All, As requested here is a copy of my LPC kernel summit track submission: Title: New userspace API for display-panel brightness control The current userspace API for brightness control offered by /sys/class/backlight devices has various problems: 1. There is no way to map the backlight device to a specific display-output / panel 2. On x86 there can be multiple firmware + direct-hw-access methods for controlling the backlight and the kernel may register multiple backlight-devices based on this which are all controlling the brightness for the same display-panel. To make things worse sometimes only one of the registered backlight devices actually works. 3. Controlling the brightness requires root-rights requiring desktop-environments to use suid-root helpers for this. 4. The scale of the brightness value is unclear, the API does not define if "perceived brightness" or electrical power is being controlled and in practice both are used without userspace knowing which is which. 5. The API does not define if a brightness value of 0 means off, or lowest brightness at which the screen is still readable (in a low lit room), again in practice both variants happen. This talk will present a proposal for a new userspace API which intends to address these problems in the form of a number of new properties for drm/kms properties on the drm_connector object for the display-panel. This talk will also focus on how to implement this proposal which brings several challenges with it: 1. The mess of having multiple interfaces to control a laptop's internal-panel will have to be sorted out because with the new API we can no longer just register multiple backlight devices and let userspace sort things out. 2. In various cases the drm/kms driver driving the panel does not control the brightness itself, but the brightness is controlled through some (ACPI) firmware interface such as the acpi_video or nvidia-wmi-ec-backlight interfaces. This introduces some challenging probe-ordering issues, the solution for which is not entirely clear yet, so this part of the talk will be actively seeking audience input on this topic. Comments: This is a duplicate submission with one which I submitted for the "LPC Refereed Track" before the "Kernel Summit 2022 CFP" posting. I recently send a RFC email about this to the relevant mailinglists: https://lore.kernel.org/dri-devel/0d188965-d809-81b5-74ce-7d30c49fee2d@redhat.com/ As well as another RFC laying out some initial backlight code refactoring steps. As there is a bunch of technical debt which needs to be addressed before work on a new uAPI can even begin: https://lore.kernel.org/dri-devel/98519ba0-7f18-201a-ea34-652f50343158@redhat.com/ I'm working on the refactoring now. I believe the refactoring is more likely to land in 5.21 then in 5.20. Let alone that the new uAPI on the kernel side + the mandatory userspace code consuming the uAPI will be ready before plumbers. IOW I expect this to still be very much in flux during Plumbers, so this won't be a presentation presenting only already finished work. Regards, Hans
On Thu, Jun 16, 2022 at 11:33 AM Hans de Goede <hdegoede@redhat.com> wrote: > The current userspace API for brightness control offered by > /sys/class/backlight devices has various problems: > > 1. There is no way to map the backlight device to a specific > display-output / panel For userspace, using just sysfs you mean? But that makes it sound like userspace needs to understand things like backlight-to-panel topology etc. If you add the presence of ambient light sensors to this mix things get even messier. I would rather make the analogy to the thermal subsystem: - Handles multiple thermal sensors - Handle linearization of measurements - Some accumulate and work to monitor a thermal zone - Handle multiple thermal zones - Also has cooling devices (such as CPU frequency and fans) - Policies are applied in the kernel to handle thermal sensors and cooling devices and control them in an orchestrated manner - Userspace can sit back and enjoy the show, but it works out-of-the box. No magic thermal daemon. Examples: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/thermal/thermal-zones.yaml Wouldn't backlight rather: - Handle multiple backlight devices. - Handle linearization of backlight intensity - Some accumulate and work as a composite backlight - Handle multiple composite backlights such as screens - Also handle ambient light sensors - Policies are applied in the kernel to handle backlight and ambient light sensors together - Userspace can sit back and enjoy the show but it works out-of-the box, no magic backlight daemon I understand userspace will want to force backlight to user preferences, older people need more backlight etc. But isn't it more compelling to handle that as a composite backlight device than to expose several of them to userspace? I imagine one big knob per screen 1-100 for userspace, a bool for on/off and a bool to select augmentation from ambient light sensors or not, the rest the kernel can figure out. My point is that this is not just a userspace API, it is a policy extension of the backlight subsystem. Maybe this is in line with what you're suggesting. I guess I just needed to mention ambient light sensors here. My personal annoyance is to see several diverging userspace implementations of policy for using ambient light sensors with backlight. It is already annoying, Android has something etc. I understand that this drives a truck through the old mantra to keep policy in userspace, but so does thermal already, so I'd just ask myself what makes most sense. Just my €0.01 Linus Walleij
<resend to both lists, because of confusion of which list to use> Hi All, As requested here is a copy of my LPC kernel summit track submission: Title: New userspace API for display-panel brightness control The current userspace API for brightness control offered by /sys/class/backlight devices has various problems: 1. There is no way to map the backlight device to a specific display-output / panel 2. On x86 there can be multiple firmware + direct-hw-access methods for controlling the backlight and the kernel may register multiple backlight-devices based on this which are all controlling the brightness for the same display-panel. To make things worse sometimes only one of the registered backlight devices actually works. 3. Controlling the brightness requires root-rights requiring desktop-environments to use suid-root helpers for this. 4. The scale of the brightness value is unclear, the API does not define if "perceived brightness" or electrical power is being controlled and in practice both are used without userspace knowing which is which. 5. The API does not define if a brightness value of 0 means off, or lowest brightness at which the screen is still readable (in a low lit room), again in practice both variants happen. This talk will present a proposal for a new userspace API which intends to address these problems in the form of a number of new properties for drm/kms properties on the drm_connector object for the display-panel. This talk will also focus on how to implement this proposal which brings several challenges with it: 1. The mess of having multiple interfaces to control a laptop's internal-panel will have to be sorted out because with the new API we can no longer just register multiple backlight devices and let userspace sort things out. 2. In various cases the drm/kms driver driving the panel does not control the brightness itself, but the brightness is controlled through some (ACPI) firmware interface such as the acpi_video or nvidia-wmi-ec-backlight interfaces. This introduces some challenging probe-ordering issues, the solution for which is not entirely clear yet, so this part of the talk will be actively seeking audience input on this topic. Comments: This is a duplicate submission with one which I submitted for the "LPC Refereed Track" before the "Kernel Summit 2022 CFP" posting. I recently send a RFC email about this to the relevant mailinglists: https://lore.kernel.org/dri-devel/0d188965-d809-81b5-74ce-7d30c49fee2d@redhat.com/ As well as another RFC laying out some initial backlight code refactoring steps. As there is a bunch of technical debt which needs to be addressed before work on a new uAPI can even begin: https://lore.kernel.org/dri-devel/98519ba0-7f18-201a-ea34-652f50343158@redhat.com/ I'm working on the refactoring now. I believe the refactoring is more likely to land in 5.21 then in 5.20. Let alone that the new uAPI on the kernel side + the mandatory userspace code consuming the uAPI will be ready before plumbers. IOW I expect this to still be very much in flux during Plumbers, so this won't be a presentation presenting only already finished work. Regards, Hans
Hi All, On 6/16/22 22:04, Linus Walleij wrote: > On Thu, Jun 16, 2022 at 11:33 AM Hans de Goede <hdegoede@redhat.com> wrote: > >> The current userspace API for brightness control offered by >> /sys/class/backlight devices has various problems: >> >> 1. There is no way to map the backlight device to a specific >> display-output / panel > > For userspace, using just sysfs you mean? Yes, > But that makes it sound like userspace needs to understand > things like backlight-to-panel topology etc. The linked RFC/proposal for the new API makes brightness a property on the drm connector object, making the backlight-to-panel topology mapping a kernel problem. One of the feature-requests behind this API is the ability to control the beightness of external monitors over DDC/DI this will mean (for laptops) multiple brightness controls (one per supported display) at which point userspace indeed needs to be aware of the brightness-control <-> panel mapping. Making this a property on the drm connector object makes this mapping very explicit. > If you add the presence of ambient light sensors to this mix > things get even messier. Mapping ambient light-sensors to dipslays is admittedly still an unsolved problem, even with my proposal. But I'm not aware of any external monitors with the capability to report ambient light-sensors back to the connected PC (or other display source). So for ambient light sensors for now userspace can keep assuming these belong to the device's internal panel, like it currently is also doing for the /sys/class/backlight devices. For external monitors with ambient light sensors, if anything I would expect the entire auto-brightness "stack" to be inside the external monitor, with the possibility to turn it on/off and set some parameters (like desired perceived brightness) through DDC/DI . In which case the auto setting + parameters can be made parameters on the drm-connector object just like brightness. And I guess we could make the ambient-light-level a drm-connector property too. This will require some kernel glue and is a bit outside of the scope of the current proposal. But it would make sense to do things this way and I think this would be a good/clean userspace API for the ambient light level reporting which automatically solves the display mapping problem. > I would rather make the analogy to the thermal subsystem: > > - Handles multiple thermal sensors > - Handle linearization of measurements > - Some accumulate and work to monitor a thermal zone > - Handle multiple thermal zones > - Also has cooling devices (such as CPU frequency and fans) > - Policies are applied in the kernel to handle thermal sensors > and cooling devices and control them in an orchestrated > manner > - Userspace can sit back and enjoy the show, but it works > out-of-the box. No magic thermal daemon. > > Examples: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/thermal/thermal-zones.yaml > > Wouldn't backlight rather: > > - Handle multiple backlight devices. > - Handle linearization of backlight intensity > - Some accumulate and work as a composite backlight > - Handle multiple composite backlights such as screens > - Also handle ambient light sensors > - Policies are applied in the kernel to handle backlight > and ambient light sensors together > - Userspace can sit back and enjoy the show but it works > out-of-the box, no magic backlight daemon > > I understand userspace will want to force backlight to user > preferences, older people need more backlight etc. > > But isn't it more compelling to handle that as a composite > backlight device than to expose several of them to > userspace? I imagine one big knob per screen > 1-100 for userspace, a bool for on/off and a bool to select > augmentation from ambient light sensors or not, the rest > the kernel can figure out. > > My point is that this is not just a userspace API, it is > a policy extension of the backlight subsystem. > > Maybe this is in line with what you're suggesting. > I guess I just needed to mention ambient light sensors > here. > > My personal annoyance is to see several diverging > userspace implementations of policy for using ambient > light sensors with backlight. It is already annoying, > Android has something etc. > > I understand that this drives a truck through the old mantra > to keep policy in userspace, but so does thermal already, > so I'd just ask myself what makes most sense. I'm actually a firm believer in keeping policy in userspace, especially for something like panel brightness control. There not only is an ambient-light sensor to take into account (with multiple possible algorithms to deal with + various knobs to tweak the algorithm) but also e.g. dimming the brightness when the machine is idle (no user input for X amount of time), when the battery goes below a certain treshold (battery saver mode) and when enabling the new-fangled builtin electronic privacy-screens. And more use-cases which impact the brightness control might popup in the future but just the above list is IMHO complicated enough to leave this to userspace. Thermal is different because letting things overheat is really really bad, so the kernel really must get involved here. But in general I do believe keeping policy in userspace is best (when possible). E.g. we also don't have any sound volume-control policy in the kernel and IMHO brightness control is more like sound volume-control then like thermalzones. Regards, Hans
On Mon, 20 Jun 2022, Hans de Goede <hdegoede@redhat.com> wrote: > <resend to both lists, because of confusion of which list to use> > > Hi All, > > As requested here is a copy of my LPC kernel summit track submission: > > Title: New userspace API for display-panel brightness control > > The current userspace API for brightness control offered by > /sys/class/backlight devices has various problems: > > 1. There is no way to map the backlight device to a specific > display-output / panel > > 2. On x86 there can be multiple firmware + direct-hw-access > methods for controlling the backlight and the kernel may > register multiple backlight-devices based on this which are > all controlling the brightness for the same display-panel. > To make things worse sometimes only one of the registered > backlight devices actually works. > > 3. Controlling the brightness requires root-rights requiring > desktop-environments to use suid-root helpers for this. > > 4. The scale of the brightness value is unclear, the API does > not define if "perceived brightness" or electrical power is > being controlled and in practice both are used without userspace > knowing which is which. > > 5. The API does not define if a brightness value of 0 means off, > or lowest brightness at which the screen is still readable > (in a low lit room), again in practice both variants happen. 6. It's not possible to change both the gamma and the brightness in the same KMS atomic commit. You'd want to be able to reduce brightness to conserve power, and counter the effects of that by changing gamma to reach a visually similar image. And you'd want to have the changes take effect at the same time instead of reducing brightness at some frame and change gamma at some other frame. This is pretty much impossible to do via the sysfs interface. BR, Jani. > This talk will present a proposal for a new userspace API > which intends to address these problems in the form of a > number of new properties for drm/kms properties on the > drm_connector object for the display-panel. > > This talk will also focus on how to implement this proposal > which brings several challenges with it: > > 1. The mess of having multiple interfaces to control a laptop's > internal-panel will have to be sorted out because with the new > API we can no longer just register multiple backlight devices > and let userspace sort things out. > > 2. In various cases the drm/kms driver driving the panel > does not control the brightness itself, but the brightness > is controlled through some (ACPI) firmware interface such > as the acpi_video or nvidia-wmi-ec-backlight interfaces. > > This introduces some challenging probe-ordering issues, > the solution for which is not entirely clear yet, so this > part of the talk will be actively seeking audience input > on this topic. > > > Comments: > This is a duplicate submission with one which I submitted for > the "LPC Refereed Track" before the "Kernel Summit 2022 CFP" posting. > > I recently send a RFC email about this to the relevant mailinglists: > https://lore.kernel.org/dri-devel/0d188965-d809-81b5-74ce-7d30c49fee2d@redhat.com/ > > As well as another RFC laying out some initial backlight code > refactoring steps. As there is a bunch of technical debt which > needs to be addressed before work on a new uAPI can even begin: > https://lore.kernel.org/dri-devel/98519ba0-7f18-201a-ea34-652f50343158@redhat.com/ > > I'm working on the refactoring now. I believe the refactoring > is more likely to land in 5.21 then in 5.20. Let alone that > the new uAPI on the kernel side + the mandatory userspace code > consuming the uAPI will be ready before plumbers. > > IOW I expect this to still be very much in flux during Plumbers, > so this won't be a presentation presenting only already finished > work. > > Regards, > > Hans > > -- Jani Nikula, Intel Open Source Graphics Center
Hi, On 6/28/22 15:53, Jani Nikula wrote: > On Mon, 20 Jun 2022, Hans de Goede <hdegoede@redhat.com> wrote: >> <resend to both lists, because of confusion of which list to use> >> >> Hi All, >> >> As requested here is a copy of my LPC kernel summit track submission: >> >> Title: New userspace API for display-panel brightness control >> >> The current userspace API for brightness control offered by >> /sys/class/backlight devices has various problems: >> >> 1. There is no way to map the backlight device to a specific >> display-output / panel >> >> 2. On x86 there can be multiple firmware + direct-hw-access >> methods for controlling the backlight and the kernel may >> register multiple backlight-devices based on this which are >> all controlling the brightness for the same display-panel. >> To make things worse sometimes only one of the registered >> backlight devices actually works. >> >> 3. Controlling the brightness requires root-rights requiring >> desktop-environments to use suid-root helpers for this. >> >> 4. The scale of the brightness value is unclear, the API does >> not define if "perceived brightness" or electrical power is >> being controlled and in practice both are used without userspace >> knowing which is which. >> >> 5. The API does not define if a brightness value of 0 means off, >> or lowest brightness at which the screen is still readable >> (in a low lit room), again in practice both variants happen. > > 6. It's not possible to change both the gamma and the brightness in the > same KMS atomic commit. You'd want to be able to reduce brightness to > conserve power, and counter the effects of that by changing gamma to > reach a visually similar image. And you'd want to have the changes take > effect at the same time instead of reducing brightness at some frame and > change gamma at some other frame. This is pretty much impossible to do > via the sysfs interface. Ack, that is a good point. Regards, Hans >> This talk will present a proposal for a new userspace API >> which intends to address these problems in the form of a >> number of new properties for drm/kms properties on the >> drm_connector object for the display-panel. >> >> This talk will also focus on how to implement this proposal >> which brings several challenges with it: >> >> 1. The mess of having multiple interfaces to control a laptop's >> internal-panel will have to be sorted out because with the new >> API we can no longer just register multiple backlight devices >> and let userspace sort things out. >> >> 2. In various cases the drm/kms driver driving the panel >> does not control the brightness itself, but the brightness >> is controlled through some (ACPI) firmware interface such >> as the acpi_video or nvidia-wmi-ec-backlight interfaces. >> >> This introduces some challenging probe-ordering issues, >> the solution for which is not entirely clear yet, so this >> part of the talk will be actively seeking audience input >> on this topic. >> >> >> Comments: >> This is a duplicate submission with one which I submitted for >> the "LPC Refereed Track" before the "Kernel Summit 2022 CFP" posting. >> >> I recently send a RFC email about this to the relevant mailinglists: >> https://lore.kernel.org/dri-devel/0d188965-d809-81b5-74ce-7d30c49fee2d@redhat.com/ >> >> As well as another RFC laying out some initial backlight code >> refactoring steps. As there is a bunch of technical debt which >> needs to be addressed before work on a new uAPI can even begin: >> https://lore.kernel.org/dri-devel/98519ba0-7f18-201a-ea34-652f50343158@redhat.com/ >> >> I'm working on the refactoring now. I believe the refactoring >> is more likely to land in 5.21 then in 5.20. Let alone that >> the new uAPI on the kernel side + the mandatory userspace code >> consuming the uAPI will be ready before plumbers. >> >> IOW I expect this to still be very much in flux during Plumbers, >> so this won't be a presentation presenting only already finished >> work. >> >> Regards, >> >> Hans >> >> >