Re: [PATCH 0/6] drm: Add mouse cursor hotspot support to atomic KMS

[Zack Rusin <zackr@vmware.com>]

> On Jun 5, 2022, at 3:30 AM, Simon Ser <contact@emersion.fr> wrote:
> 
> ⚠ External Email
> 
> On Friday, June 3rd, 2022 at 20:31, Zack Rusin <zackr@vmware.com> wrote:
> 
>>>>>>> My proposal was:
>>>>>>> 
>>>>>>> - Introduce DRM_CLIENT_CAP_CURSOR_PLANE_NO_POSITION (or a better name). Only
>>>>>>> user-space which supports the hotspot props will enable it.
>>>>>>> - By default, don't expose a cursor plane, because current user-space doesn't
>>>>>>> support it (Weston might put a window in the cursor plane, and nobody can
>>>>>>> report hotspot).
>>>>>>> - If the KMS client enables the cap, advertise the cursor
>>>>>>> plane, and make it so the plane doesn't have the CRTC_X/CRTC_Y properties
>>>>>>> since the driver will pick the position.
>>>>>>> 
>>>>>>> That way both old and new user-space are fixed.
>>>>>> 
>>>>>> I don’t think I see how that fixes anything. In particular I don’t see a way
>>>>>> of fixing the old user space at all. We require hotspot info, old user space
>>>>>> doesn’t set it because there’s no way of setting it on atomic kms.
>>>>> 
>>>>> Old atomic user-space is fixed by removing the cursor plane. Then old
>>>>> atomic user-space will fallback to drawing the cursor itself, e.g. via
>>>>> OpenGL.
>>>> 
>>>> But it’s not fixed, because the driver is still on a deny-list and
>>>> nothing implements this. You’re saying you could potentially “fix” by
>>>> implementing client side cursor handling in all kms clients? That’s a
>>>> hard sell if the user space can just put the virtualized driver on
>>>> deny-lists and fallback to use old kms which supports hotspots.
>>> 
>>> What deny-list are you referring to?
>>> 
>>> All compositors I know of implement a fallback when no cursor plane is
>>> usable.
>> 
>> I put the links in the first email in the
>> series:
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.gnome.org%2FGNOME%2Fmutter%2F-%2Fblob%2Fmain%2Fsrc%2Fbackends%2Fnative%2Fmeta-kms-impl-device-atomic.c%23L1188&amp;data=05%7C01%7Czackr%40vmware.com%7C2b0ab6c67e7d4bfb618a08da46c5394f%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637900110157712044%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=wggJaNScF0ziIG%2BfSdSUKBVZGoNjtm4Q8amS7SbJa%2FY%3D&amp;reserved=0
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Finvent.kde.org%2Fplasma%2Fkwin%2F-%2Fblob%2Fmaster%2Fsrc%2Fbackends%2Fdrm%2Fdrm_gpu.cpp%23L156&amp;data=05%7C01%7Czackr%40vmware.com%7C2b0ab6c67e7d4bfb618a08da46c5394f%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637900110157712044%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=%2BCTEJ9XtlI9kuKJJZVMNodtkZSkRIv8RN9jSRAM1pqM%3D&amp;reserved=0
> 
> I was not aware of these lists. Having to work-around drivers in user-space is
> really not great.
> 
> But regardless, that doesn't really change anything. With my proposal:
> 
> - Old user-space with hacky deny-lists (Mutter, KWin) still goes through the
>  legacy uAPI.
> - Old user-space without the hacky deny-lists (Weston, wlroots) uses software
>  cursors and is not broken anymore.
> - New user-space can report hotspot and is not broken.

Yea, that doesn’t work, but I think below I stopped where the issue is.

>> Also, let me point this out because it also seems to be a fundamental
>> misunderstanding, user space implementing software cursor doesn’t fix
>> anything. Just leaves everything broken in different ways. The reason
>> virtualized drivers went away from software cursors is because it makes it
>> impossible to make it work with a bunch of interesting and desirable
>> scenarios, e.g. unity mode where the guest might have a terminal and browser
>> open and then the virtual machine software creates windows out of those, so
>> you don’t have the entire desktop of the guest but instead native looking
>> windows with the apps. In that case guest has no way of knowing when the
>> cursor leaves the window, so to make seemless cursors work you’d need to
>> implement irc or backdoors that send that information from the host to the
>> guest, then have virtualized drivers create some sort of uevent, to send to
>> the userspace to let it know to hide the cursor because it actually left the
>> window. That’s a trivial scenario and there’s a lot more with mice that are
>> remoted themselves, guests with singular or maybe many apps exported,
>> possibly overlapping on the host but all within a desktop in the guest, etc.
> 
> Are you saying that the current broken behavior is better than software
> cursors?

They’re broken in very different ways. You’re asking me which bugs do I prefer. Ultimately the current lack of hotspots leaves mouse unusable but I don’t see any reason to trade that for another set of bugs instead of just fixing it (either via fallback to legacy or using the new hotspot api).

>>>>> New user-space supports setting the hotspot prop, and is aware that it can't
>>>>> set the cursor plane position, so the cursor plane can be exposed again when
>>>>> the cap is enabled.
>>>> 
>>>> But we still use cursor plane position. Hotspots are offsets from
>>>> cursor plane positions. Both are required.
>>> 
>>> No, VM drivers don't need and disregard the cursor position AFAIK. They
>>> replace it with the host cursor's position.
>> 
>> Iirc they all use it.
> 
> What do they use it for?
> 
> The whole point of this discussion is to be able to display the guest's cursor
> image on the host's cursor, so that latency is reduced?

Ah, I think I see now where the confusion is coming from. No, it’s definitely not. This has nothing to do with latency. By default position of a mouse cursor doesn’t tell us where the point that is actually used as an activation of a click is, e.g. a mouse cursor image with an arrow pointing to the top-left and a mouse cursor image pointing to the bottom right - both at the same position, as you can imagine it will become impossible to use the desktop if the click defaults to the top left, especially as the number of cursor images increases, you need information about which point within the cursor image activates the click, that’s the hotspot. You need to know the position of the image and you need to know which point within that image is responsible for actually activating the events.

So this has nothing to do with latency, this is about mouse cursor clicks/drags simply having wrong coordinates and mouse being effectively impossible to use on anything that doesn’t have the same workarounds as gnome-shell/kwin. So if you have user space code which hasn’t implemented the same workarounds gnome-shell/kwin that means that is has never been used with virtualized drivers because people tend to notice pretty quickly that when they click on a button/link a completely different button/link in a different part of the screen is activated...

z