Mouse pointer warping with gtk display

Mouse pointer warping with gtk display

[BALATON Zoltan]

Hello,

We've found a problem with mouse pointer jumping around then constantly 
warping making it unusable which only seems to happen with the gtk 
display. Similar problem was reported with MacOS guests but now we could 
also reproduce with MorphOS on pegasos2 and thus confirm it's not a guest 
issue or hardware emulation problem but seems to be related to QEMU's gtk 
display. This can be reproduced as following (boot.img is the file from 
the root dir of the iso which is available from 
https://www.morphos-team.net/morphos-3.15.iso ):

qemu-system-ppc -M pegasos2 -cdrom morphos-3.15.iso -display sdl \
-L ../pc-bios -kernel boot.img -device ati-vga,romfile=""

This does not have this problem and mouse works but running with -display 
gtk has the mouse pointer issue making it unusable. It can be seemingly 
avoided with using -device ati-vga,romfile="",guest_hwcursor=true so I 
think problem is somewhere in mouse handling in ui/gtk.c with host side 
mouse pointer. The gd_motion_event() in that file has a big comment about 
warping mouse back to the middle near the end of the function which seems 
to ne what's happening and causing this as this does not appear in sdl.c 
that does not show the same issue but I don't understand what that code is 
trying to do.

It's a known problem that host mouse pointer can get out of sync with the 
guest which I think is because mouse acceleration on the host is not 
correctly taken into account when reporting mouse moves to the guest and 
this may try to correct that issue but it seems to cause more harm than 
good in some cases. Any idea what might be the problem and how to solve 
this? I wonder why nobody else is seeing this with other guests. (I'm not 
direcly interested in fixing it as I'm using sdl display but this makes it 
hard to use some guests for users whose distros make gtk backend the 
default which may actually be most people so it might be a good idea to 
improve this for them so I thought at least reporting the problem is 
useful.)

Regards,
BALATON Zoltan

Re: Mouse pointer warping with gtk display

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 4328 bytes --]

Hi

On Thu, Jan 27, 2022 at 10:29 PM BALATON Zoltan <balaton@eik.bme.hu> wrote:

> Hello,
>
> We've found a problem with mouse pointer jumping around then constantly
> warping making it unusable which only seems to happen with the gtk
> display. Similar problem was reported with MacOS guests but now we could
> also reproduce with MorphOS on pegasos2 and thus confirm it's not a guest
> issue or hardware emulation problem but seems to be related to QEMU's gtk
> display. This can be reproduced as following (boot.img is the file from
> the root dir of the iso which is available from
> https://www.morphos-team.net/morphos-3.15.iso ):
>
> qemu-system-ppc -M pegasos2 -cdrom morphos-3.15.iso -display sdl \
> -L ../pc-bios -kernel boot.img -device ati-vga,romfile=""
>
> This does not have this problem and mouse works but running with -display
> gtk has the mouse pointer issue making it unusable. It can be seemingly
> avoided with using -device ati-vga,romfile="",guest_hwcursor=true so I
> think problem is somewhere in mouse handling in ui/gtk.c with host side
> mouse pointer. The gd_motion_event() in that file has a big comment about
> warping mouse back to the middle near the end of the function which seems
> to ne what's happening and causing this as this does not appear in sdl.c
> that does not show the same issue but I don't understand what that code is
> trying to do.
>

In general, a guest handles mouse input with relative motion events
(standard ps2).

A more VM-friendly guest can additionally handle mouse input with absolute
coordinates (vmmouse, usb tablet etc)

While giving absolute coordinates to a guest is quite a simple task from a
client (and offers some advantages) it is more tricky to deal with relative
mouse events...

At the toolkit level, iirc, you generally don't have low-level events &
relative motion (gtk doesn't let you grab the mouse exclusively to receive
all events). You can workaround this with some window managers (ex wayland
relative-pointer unstable extension).

The "old" strategy with gtk2/3 is to grab the mouse, which is then
constrained within the display widget iirc, and convert absolute events to
relative events by doing simple current_position - last_position. Since
your client mouse may reach the widget borders (and/or the client monitor
limits, can't remember which applies, I think it varies), you also need to
recenter the client mouse somewhere, and that's usually in the middle of
the display widget.

This is clunky at best, and gtk devs decided to no longer support that kind
of API with gtk4. The recommended approach is to use lower-level events,
like the one you get from the wayland extensions. This is a pain for
portability, but that's all we have for now.

spice-gtk implements the wayland stuff. Have you tried running -display
spice-app? It may have a more polished behavior than the one in qemu.






>
> It's a known problem that host mouse pointer can get out of sync with the
> guest which I think is because mouse acceleration on the host is not
> correctly taken into account when reporting mouse moves to the guest and
> this may try to correct that issue but it seems to cause more harm than
> good in some cases. Any idea what might be the problem and how to solve
> this? I wonder why nobody else is seeing this with other guests. (I'm not
> direcly interested in fixing it as I'm using sdl display but this makes it
> hard to use some guests for users whose distros make gtk backend the
> default which may actually be most people so it might be a good idea to
> improve this for them so I thought at least reporting the problem is
> useful.)
>

I suspect SDL offers an API for relative mouse events, which works better
than what we do in the gtk backend.

I hope in the future we can drop the gtk code from qemu, in favor of
dbus-display and rdw. Maintaining that kind of code is hard, and having the
same gtk widget base code for various remote desktop apps & protocols
should help maintenance and offer a consistent behavior, whether you use
QEMU/DBus, VNC, Spice, or RDP and whatever the future brings.

(I wrote the above mostly from memory, I agree it's convoluted and a bit
buggy, I doubt anyone likes it ;)

-- 
Marc-André Lureau

[-- Attachment #2: Type: text/html, Size: 5513 bytes --]

Re: Mouse pointer warping with gtk display

[Howard Spoelstra]

[-- Attachment #1: Type: text/plain, Size: 4885 bytes --]

On Thu, Jan 27, 2022 at 8:48 PM Marc-André Lureau <
marcandre.lureau@gmail.com> wrote:

> Hi
>
> On Thu, Jan 27, 2022 at 10:29 PM BALATON Zoltan <balaton@eik.bme.hu>
> wrote:
>
>> Hello,
>>
>> We've found a problem with mouse pointer jumping around then constantly
>> warping making it unusable which only seems to happen with the gtk
>> display. Similar problem was reported with MacOS guests but now we could
>> also reproduce with MorphOS on pegasos2 and thus confirm it's not a guest
>> issue or hardware emulation problem but seems to be related to QEMU's gtk
>> display. This can be reproduced as following (boot.img is the file from
>> the root dir of the iso which is available from
>> https://www.morphos-team.net/morphos-3.15.iso ):
>>
>> qemu-system-ppc -M pegasos2 -cdrom morphos-3.15.iso -display sdl \
>> -L ../pc-bios -kernel boot.img -device ati-vga,romfile=""
>>
>> This does not have this problem and mouse works but running with -display
>> gtk has the mouse pointer issue making it unusable. It can be seemingly
>> avoided with using -device ati-vga,romfile="",guest_hwcursor=true so I
>> think problem is somewhere in mouse handling in ui/gtk.c with host side
>> mouse pointer. The gd_motion_event() in that file has a big comment about
>> warping mouse back to the middle near the end of the function which seems
>> to ne what's happening and causing this as this does not appear in sdl.c
>> that does not show the same issue but I don't understand what that code
>> is
>> trying to do.
>>
>
> In general, a guest handles mouse input with relative motion events
> (standard ps2).
>
> A more VM-friendly guest can additionally handle mouse input with absolute
> coordinates (vmmouse, usb tablet etc)
>
> While giving absolute coordinates to a guest is quite a simple task from a
> client (and offers some advantages) it is more tricky to deal with relative
> mouse events...
>
> At the toolkit level, iirc, you generally don't have low-level events &
> relative motion (gtk doesn't let you grab the mouse exclusively to receive
> all events). You can workaround this with some window managers (ex wayland
> relative-pointer unstable extension).
>
> The "old" strategy with gtk2/3 is to grab the mouse, which is then
> constrained within the display widget iirc, and convert absolute events to
> relative events by doing simple current_position - last_position. Since
> your client mouse may reach the widget borders (and/or the client monitor
> limits, can't remember which applies, I think it varies), you also need to
> recenter the client mouse somewhere, and that's usually in the middle of
> the display widget.
>
> This is clunky at best, and gtk devs decided to no longer support that
> kind of API with gtk4. The recommended approach is to use lower-level
> events, like the one you get from the wayland extensions. This is a pain
> for portability, but that's all we have for now.
>
> spice-gtk implements the wayland stuff. Have you tried running -display
> spice-app? It may have a more polished behavior than the one in qemu.
>
>
>
>
>
>
>>
>> It's a known problem that host mouse pointer can get out of sync with the
>> guest which I think is because mouse acceleration on the host is not
>> correctly taken into account when reporting mouse moves to the guest and
>> this may try to correct that issue but it seems to cause more harm than
>> good in some cases. Any idea what might be the problem and how to solve
>> this? I wonder why nobody else is seeing this with other guests. (I'm not
>> direcly interested in fixing it as I'm using sdl display but this makes
>> it
>> hard to use some guests for users whose distros make gtk backend the
>> default which may actually be most people so it might be a good idea to
>> improve this for them so I thought at least reporting the problem is
>> useful.)
>>
>
> I suspect SDL offers an API for relative mouse events, which works better
> than what we do in the gtk backend.
>
> I hope in the future we can drop the gtk code from qemu, in favor of
> dbus-display and rdw. Maintaining that kind of code is hard, and having the
> same gtk widget base code for various remote desktop apps & protocols
> should help maintenance and offer a consistent behavior, whether you use
> QEMU/DBus, VNC, Spice, or RDP and whatever the future brings.
>
> (I wrote the above mostly from memory, I agree it's convoluted and a bit
> buggy, I doubt anyone likes it ;)
>
> --
> Marc-André Lureau
>

Hi Zoltan,

While I can reproduce the issue you report when running morphos with gtk, I
cannot currently reproduce with the Mac OS/OSX guests I tested on Linux
host. Both mac99 (adb mouse and kbd) and mac99,via=pmu (usb mouse and kbd)
keep the mouse inside the guest window with both sdl and gtk.

Best,
Howard

[-- Attachment #2: Type: text/html, Size: 6339 bytes --]

Re: Mouse pointer warping with gtk display

[Mark Cave-Ayland]

On 27/01/2022 20:28, Howard Spoelstra wrote:

> Hi Zoltan,
> 
> While I can reproduce the issue you report when running morphos with gtk, I cannot 
> currently reproduce with the Mac OS/OSX guests I tested on Linux host. Both mac99 
> (adb mouse and kbd) and mac99,via=pmu (usb mouse and kbd) keep the mouse inside the 
> guest window with both sdl and gtk.
> 
> Best,
> Howard

Does the ati-vga device have a hardware cursor? Maybe it could be related to that?


ATB,

Mark.

Re: Mouse pointer warping with gtk display

[BALATON Zoltan]

Hello,

On Thu, 27 Jan 2022, Howard Spoelstra wrote:
> While I can reproduce the issue you report when running morphos with gtk, I
> cannot currently reproduce with the Mac OS/OSX guests I tested on Linux
> host. Both mac99 (adb mouse and kbd) and mac99,via=pmu (usb mouse and kbd)
> keep the mouse inside the guest window with both sdl and gtk.

Interesting. I remember people seeing this with MacOS in the past but did 
some more testing now and indeed it does not seem to happen with MacOS and 
default VGA. But I could reproduce it with MorphOS on mac99.via=pmu and 
-device ati-vga but not with -vga none -device sm501 so maybe it's related 
to ati-vga then. (Booting MacOS with ati-vga to cross check is a bit 
complicated so I could not test that now and don't have a graphical Linux 
guest for mac99 either at the moment.)

I don't see how the display emulation can break mouse handling though. The 
ati-vga,guest_hwcursor=true option seems to help so it's something with 
host side mouse only. Maybe it's related to dpy_mouse_set() that we call 
when HW cursor regs are written but I have to check how this works in 
ati-vga because I've forgotten it by now. The strange thing is that the 
pointer seems to warp even when the mouse is not grabbed that I think 
should not happen but it just disappears when I click on the window and 
the mouse is grabbed. And it only happens with gtk, not with sdl. Some 
clues would still be welcome to get to the cause of this.

Regards,
BALATON Zoltan

Re: Mouse pointer warping with gtk display

[BALATON Zoltan]

On Thu, 27 Jan 2022, Mark Cave-Ayland wrote:
> On 27/01/2022 20:28, Howard Spoelstra wrote:
>
>> Hi Zoltan,
>> 
>> While I can reproduce the issue you report when running morphos with gtk, I 
>> cannot currently reproduce with the Mac OS/OSX guests I tested on Linux 
>> host. Both mac99 (adb mouse and kbd) and mac99,via=pmu (usb mouse and kbd) 
>> keep the mouse inside the guest window with both sdl and gtk.
>> 
>> Best,
>> Howard
>
> Does the ati-vga device have a hardware cursor? Maybe it could be related to 
> that?

Yes ir does and it seems to be related but still don't understand how. 
I've desctibed what I've found in another reply.

Regards,
BALATON Zoltan