[PATCH 0/2] ui/cocoa: updateUIInfo threading, autorelease pools

[PATCH 0/2] ui/cocoa: updateUIInfo threading, autorelease pools

[Peter Maydell]

This patchset was originally provoked by Akihiko Odaki noting
that we have some unnecessary creation and deletion of autorelease
pools in the Cocoa UI code. Patch 2 deletes them; but to get there
we need to do a bit of cleanup of the updateUIInfo support,
which wasn't considering threads.

Tested only very lightly.

Peter Maydell (2):
  ui/cocoa.m: Fix updateUIInfo threading issues
  ui/cocoa.m: Remove unnecessary NSAutoreleasePools

 ui/cocoa.m | 33 ++++++++++++++++++++++++---------
 1 file changed, 24 insertions(+), 9 deletions(-)

-- 
2.25.1

[PATCH 1/2] ui/cocoa.m: Fix updateUIInfo threading issues

[Peter Maydell]

The updateUIInfo method makes Cocoa API calls.  It also calls back
into QEMU functions like dpy_set_ui_info().  To do this safely, we
need to follow two rules:
 * Cocoa API calls are made on the Cocoa UI thread
 * When calling back into QEMU we must hold the iothread lock

Fix the places where we got this wrong, by taking the iothread lock
while executing updateUIInfo, and moving the call in cocoa_switch()
inside the dispatch_async block.

Some of the Cocoa UI methods which call updateUIInfo are invoked as
part of the initial application startup, while we're still doing the
little cross-thread dance described in the comment just above
call_qemu_main().  This meant they were calling back into the QEMU UI
layer before we'd actually finished initializing our display and
registered the DisplayChangeListener, which isn't really valid.  Once
updateUIInfo takes the iothread lock, we no longer get away with
this, because during this startup phase the iothread lock is held by
the QEMU main-loop thread which is waiting for us to finish our
display initialization.  So we must suppress updateUIInfo until
applicationDidFinishLaunching allows the QEMU main-loop thread to
continue, and instead defer telling the UI layer about our initial
window size until later.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 ui/cocoa.m | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/ui/cocoa.m b/ui/cocoa.m
index a8f1cdaf926..f8d3d8ad7a6 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -522,8 +522,9 @@ QemuCocoaView *cocoaView;
     }
 }
 
-- (void) updateUIInfo
+- (void) doUpdateUIInfo
 {
+    /* Must be called with the iothread lock, i.e. via updateUIInfo */
     NSSize frameSize;
     QemuUIInfo info;
 
@@ -554,6 +555,22 @@ QemuCocoaView *cocoaView;
     dpy_set_ui_info(dcl.con, &info, TRUE);
 }
 
+- (void) updateUIInfo
+{
+    if (!allow_events) {
+        /*
+         * Don't try to tell QEMU about UI information in the application
+         * startup phase -- we haven't yet registered dcl with the QEMU UI
+         * layer, and also trying to take the iothread lock would deadlock.
+         */
+        return;
+    }
+
+    with_iothread_lock(^{
+        [self doUpdateUIInfo];
+    });
+}
+
 - (void)viewDidMoveToWindow
 {
     [self updateUIInfo];
@@ -1985,8 +2002,6 @@ static void cocoa_switch(DisplayChangeListener *dcl,
 
     COCOA_DEBUG("qemu_cocoa: cocoa_switch\n");
 
-    [cocoaView updateUIInfo];
-
     // The DisplaySurface will be freed as soon as this callback returns.
     // We take a reference to the underlying pixman image here so it does
     // not disappear from under our feet; the switchSurface method will
@@ -1994,6 +2009,7 @@ static void cocoa_switch(DisplayChangeListener *dcl,
     pixman_image_ref(image);
 
     dispatch_async(dispatch_get_main_queue(), ^{
+        [cocoaView updateUIInfo];
         [cocoaView switchSurface:image];
     });
     [pool release];
@@ -2057,6 +2073,11 @@ static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
     qemu_event_init(&cbevent, false);
     cbowner = [[QemuCocoaPasteboardTypeOwner alloc] init];
     qemu_clipboard_peer_register(&cbpeer);
+
+    /* Now we're set up, tell the UI layer our initial window size */
+    dispatch_async(dispatch_get_main_queue(), ^{
+        [cocoaView updateUIInfo];
+    });
 }
 
 static QemuDisplay qemu_display_cocoa = {
-- 
2.25.1

Re: [PATCH 1/2] ui/cocoa.m: Fix updateUIInfo threading issues

[Akihiko Odaki]

On Wed, Feb 23, 2022 at 2:06 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> The updateUIInfo method makes Cocoa API calls.  It also calls back
> into QEMU functions like dpy_set_ui_info().  To do this safely, we
> need to follow two rules:
>  * Cocoa API calls are made on the Cocoa UI thread
>  * When calling back into QEMU we must hold the iothread lock
>
> Fix the places where we got this wrong, by taking the iothread lock
> while executing updateUIInfo, and moving the call in cocoa_switch()
> inside the dispatch_async block.
>
> Some of the Cocoa UI methods which call updateUIInfo are invoked as
> part of the initial application startup, while we're still doing the
> little cross-thread dance described in the comment just above
> call_qemu_main().  This meant they were calling back into the QEMU UI
> layer before we'd actually finished initializing our display and
> registered the DisplayChangeListener, which isn't really valid.  Once
> updateUIInfo takes the iothread lock, we no longer get away with
> this, because during this startup phase the iothread lock is held by
> the QEMU main-loop thread which is waiting for us to finish our
> display initialization.  So we must suppress updateUIInfo until
> applicationDidFinishLaunching allows the QEMU main-loop thread to
> continue, and instead defer telling the UI layer about our initial
> window size until later.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Thanks for fixing this bug.

> ---
>  ui/cocoa.m | 27 ++++++++++++++++++++++++---
>  1 file changed, 24 insertions(+), 3 deletions(-)
>
> diff --git a/ui/cocoa.m b/ui/cocoa.m
> index a8f1cdaf926..f8d3d8ad7a6 100644
> --- a/ui/cocoa.m
> +++ b/ui/cocoa.m
> @@ -522,8 +522,9 @@ QemuCocoaView *cocoaView;
>      }
>  }
>
> -- (void) updateUIInfo
> +- (void) doUpdateUIInfo

You may add the suffix "Locked" to align with handleEventLocked and
make the intention more explicit.

>  {
> +    /* Must be called with the iothread lock, i.e. via updateUIInfo */
>      NSSize frameSize;
>      QemuUIInfo info;
>
> @@ -554,6 +555,22 @@ QemuCocoaView *cocoaView;
>      dpy_set_ui_info(dcl.con, &info, TRUE);
>  }
>
> +- (void) updateUIInfo
> +{
> +    if (!allow_events) {
> +        /*
> +         * Don't try to tell QEMU about UI information in the application
> +         * startup phase -- we haven't yet registered dcl with the QEMU UI
> +         * layer, and also trying to take the iothread lock would deadlock.
> +         */
> +        return;
> +    }
> +
> +    with_iothread_lock(^{
> +        [self doUpdateUIInfo];
> +    });
> +}
> +
>  - (void)viewDidMoveToWindow
>  {
>      [self updateUIInfo];
> @@ -1985,8 +2002,6 @@ static void cocoa_switch(DisplayChangeListener *dcl,
>
>      COCOA_DEBUG("qemu_cocoa: cocoa_switch\n");
>
> -    [cocoaView updateUIInfo];
> -
>      // The DisplaySurface will be freed as soon as this callback returns.
>      // We take a reference to the underlying pixman image here so it does
>      // not disappear from under our feet; the switchSurface method will
> @@ -1994,6 +2009,7 @@ static void cocoa_switch(DisplayChangeListener *dcl,
>      pixman_image_ref(image);
>
>      dispatch_async(dispatch_get_main_queue(), ^{
> +        [cocoaView updateUIInfo];
>          [cocoaView switchSurface:image];
>      });
>      [pool release];
> @@ -2057,6 +2073,11 @@ static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
>      qemu_event_init(&cbevent, false);
>      cbowner = [[QemuCocoaPasteboardTypeOwner alloc] init];
>      qemu_clipboard_peer_register(&cbpeer);
> +
> +    /* Now we're set up, tell the UI layer our initial window size */
> +    dispatch_async(dispatch_get_main_queue(), ^{
> +        [cocoaView updateUIInfo];
> +    });
>  }

This is not necessary since register_displaychangelistener calls dpy_gfx_switch.

>
>  static QemuDisplay qemu_display_cocoa = {
> --
> 2.25.1
>

Re: [PATCH 1/2] ui/cocoa.m: Fix updateUIInfo threading issues

[Peter Maydell]

On Tue, 22 Feb 2022 at 17:13, Akihiko Odaki <akihiko.odaki@gmail.com> wrote:
>
> On Wed, Feb 23, 2022 at 2:06 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> > diff --git a/ui/cocoa.m b/ui/cocoa.m
> > index a8f1cdaf926..f8d3d8ad7a6 100644
> > --- a/ui/cocoa.m
> > +++ b/ui/cocoa.m
> > @@ -522,8 +522,9 @@ QemuCocoaView *cocoaView;
> >      }
> >  }
> >
> > -- (void) updateUIInfo
> > +- (void) doUpdateUIInfo
>
> You may add the suffix "Locked" to align with handleEventLocked and
> make the intention more explicit.

Ah, good idea.

> > +    /* Now we're set up, tell the UI layer our initial window size */
> > +    dispatch_async(dispatch_get_main_queue(), ^{
> > +        [cocoaView updateUIInfo];
> > +    });
> >  }
>
> This is not necessary since register_displaychangelistener calls dpy_gfx_switch.

Cool, I'll drop this bit.

-- PMM

[PATCH 2/2] ui/cocoa.m: Remove unnecessary NSAutoreleasePools

[Peter Maydell]

In commit 6e657e64cdc478 in 2013 we added some autorelease pools to
deal with complaints from macOS when we made calls into Cocoa from
threads that didn't have automatically created autorelease pools.
Later on, macOS got stricter about forbidding cross-thread Cocoa
calls, and in commit 5588840ff77800e839d8 we restructured the code to
avoid them.  This left the autorelease pool creation in several
functions without any purpose; delete it.

We still need the pool in cocoa_refresh() for the clipboard related
code which is called directly there.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 ui/cocoa.m | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/ui/cocoa.m b/ui/cocoa.m
index f8d3d8ad7a6..ea8084cda86 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1973,8 +1973,6 @@ int main (int argc, char **argv) {
 static void cocoa_update(DisplayChangeListener *dcl,
                          int x, int y, int w, int h)
 {
-    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
-
     COCOA_DEBUG("qemu_cocoa: cocoa_update\n");
 
     dispatch_async(dispatch_get_main_queue(), ^{
@@ -1990,14 +1988,11 @@ static void cocoa_update(DisplayChangeListener *dcl,
         }
         [cocoaView setNeedsDisplayInRect:rect];
     });
-
-    [pool release];
 }
 
 static void cocoa_switch(DisplayChangeListener *dcl,
                          DisplaySurface *surface)
 {
-    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
     pixman_image_t *image = surface->image;
 
     COCOA_DEBUG("qemu_cocoa: cocoa_switch\n");
@@ -2012,7 +2007,6 @@ static void cocoa_switch(DisplayChangeListener *dcl,
         [cocoaView updateUIInfo];
         [cocoaView switchSurface:image];
     });
-    [pool release];
 }
 
 static void cocoa_refresh(DisplayChangeListener *dcl)
-- 
2.25.1