[Qemu-devel] RFC: running the user interface in a thread ...

[Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi folks,

I'm starting to investigate if and how we can move the user interface
code into its own thread instead of running it in the iothread and
therefore avoid blocking the guest in case some UI actions take a little
longer.

opengl and toolkits tend to be bad at multithreading.  So my idea is to
have a single thread dedicated to all the UI + rendering stuff, possibly
let even the virglrenderer run in ui thread context.

I think we have to make that opt-in per user interface, so we can go
forward step by step.

The ui thread will need quite some stuff provided by the mainloop.  Wait
for all kinds of events (from vnc socket, x11 connection, ...).
Probably timers too.  Wait for events from other threads (guest screen
updates).

Suggestions how to tackle that?
Can I reuse the qemu mainloop code outside of the iothread?
Maybe it'll be better to go straight for a glib main loop?
Is it possible to wait for file handle events and posix condition
variables at the same time?

Other notes / hints / suggestions / ideas?

thanks,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

On 01/18/2016 02:54 AM, Gerd Hoffmann wrote:
>   Hi folks,
> 
> I'm starting to investigate if and how we can move the user interface
> code into its own thread instead of running it in the iothread and
> therefore avoid blocking the guest in case some UI actions take a little
> longer.

That, and so the UI can use localization without corrupting other
threads that MUST use the C locale (QMP parsing/output of floating point
is particularly important to get right with '.' instead of ',' for a
decimal separator).

> 
> Suggestions how to tackle that?

Sadly, I don't have good suggestions here, as I've never done much in
the way of UI coding.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi,

> That, and so the UI can use localization without corrupting other
> threads that MUST use the C locale (QMP parsing/output of floating point
> is particularly important to get right with '.' instead of ',' for a
> decimal separator).

A quick look at the setlocale manpage doesn't make clear whenever
threads can have different locales or not.

Related:  Possibly we should move UIs to another *process* instead?
At least the ones which use a UI toolkit (i.e. sdl, gtk, cocoa).

cheers,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
>   Hi,
> 
> > That, and so the UI can use localization without corrupting other
> > threads that MUST use the C locale (QMP parsing/output of floating point
> > is particularly important to get right with '.' instead of ',' for a
> > decimal separator).
> 
> A quick look at the setlocale manpage doesn't make clear whenever
> threads can have different locales or not.
> 
> Related:  Possibly we should move UIs to another *process* instead?
> At least the ones which use a UI toolkit (i.e. sdl, gtk, cocoa).

That does feel safer, if you can find an efficient way to do the graphics.
A lot less code running with the privs of the qemu process.

Dave

> 
> cheers,
>   Gerd
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi,

> That does feel safer, if you can find an efficient way to do the graphics.
> A lot less code running with the privs of the qemu process.

That is where it starts to become tricky ...

On modern linux we can simply use dma-bufs.

On older linux and generic posix (*bsd etc) we should be able to use
sysv shared memory.

As MacOS X is based on BSD I'd expect sysv shared memory is available
there too.  Possibly we can use something more efficient there, we have
macos-specific ui code anyway.

Windows?  Hmm, no idea how to shared memory there.

We also need a control channel, to manage the shm bufs, to send user
input (kbd/mouse), to implement some control things (machine menu in gtk
ui).  One option would be to simply use the monitor for that.  The other
one is to invent something limited to the stuff actually needed instead
of handing over a dangerous swiss army knife to the process running the
UI.  Hmm, maybe a new monitor mode with a limited set of commands would
do too.

cheers,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Kevin Wolf]

Am 19.01.2016 um 15:14 hat Gerd Hoffmann geschrieben:
>   Hi,
> 
> > That does feel safer, if you can find an efficient way to do the graphics.
> > A lot less code running with the privs of the qemu process.
> 
> That is where it starts to become tricky ...
> 
> On modern linux we can simply use dma-bufs.
> 
> On older linux and generic posix (*bsd etc) we should be able to use
> sysv shared memory.
> 
> As MacOS X is based on BSD I'd expect sysv shared memory is available
> there too.  Possibly we can use something more efficient there, we have
> macos-specific ui code anyway.
> 
> Windows?  Hmm, no idea how to shared memory there.
> 
> We also need a control channel, to manage the shm bufs, to send user
> input (kbd/mouse), to implement some control things (machine menu in gtk
> ui).  One option would be to simply use the monitor for that.  The other
> one is to invent something limited to the stuff actually needed instead
> of handing over a dangerous swiss army knife to the process running the
> UI.  Hmm, maybe a new monitor mode with a limited set of commands would
> do too.

I think you need to offer the full monitor command set anyway because
a UI that is on par with SDL or GTK must provide a monitor interface to
the user.

Kevin

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Markus Armbruster]

Gerd Hoffmann <kraxel@redhat.com> writes:

>   Hi,
>
>> That, and so the UI can use localization without corrupting other
>> threads that MUST use the C locale (QMP parsing/output of floating point
>> is particularly important to get right with '.' instead of ',' for a
>> decimal separator).
>
> A quick look at the setlocale manpage doesn't make clear whenever
> threads can have different locales or not.

Try uselocale(3).

> Related:  Possibly we should move UIs to another *process* instead?
> At least the ones which use a UI toolkit (i.e. sdl, gtk, cocoa).

If we can pull it off...

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Daniel P. Berrange]

On Tue, Jan 19, 2016 at 02:01:47PM +0100, Gerd Hoffmann wrote:
>   Hi,
> 
> > That, and so the UI can use localization without corrupting other
> > threads that MUST use the C locale (QMP parsing/output of floating point
> > is particularly important to get right with '.' instead of ',' for a
> > decimal separator).
> 
> A quick look at the setlocale manpage doesn't make clear whenever
> threads can have different locales or not.

glibc has per-thread locales, but this isn't guaranteed portable across
unix platforms. We could enable per-thread localization on Linux builds
but disable it on other platforms.

> Related:  Possibly we should move UIs to another *process* instead?
> At least the ones which use a UI toolkit (i.e. sdl, gtk, cocoa).

Is the worth the effort ? The majority of mgmt apps use spice/vnc which
is already out of process. Will the people do use SDL/GTK/Coca backends,
be doing so in a scenario where moving the UI to a separate process is
a benefit to them ?  I'm not saying we shouldn't separate the UI to
another process, just that we should consider whether there's other
things todo in QEMU UI layer that are a better payoff for the userbase.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]

On 01/19/2016 08:28 AM, Daniel P. Berrange wrote:
> On Tue, Jan 19, 2016 at 02:01:47PM +0100, Gerd Hoffmann wrote:
>>   Hi,
>>
>>> That, and so the UI can use localization without corrupting other
>>> threads that MUST use the C locale (QMP parsing/output of floating point
>>> is particularly important to get right with '.' instead of ',' for a
>>> decimal separator).
>>
>> A quick look at the setlocale manpage doesn't make clear whenever
>> threads can have different locales or not.
> 
> glibc has per-thread locales, but this isn't guaranteed portable across
> unix platforms.

POSIX requires it, but not all the world has caught up to POSIX.

> We could enable per-thread localization on Linux builds
> but disable it on other platforms.

Sure, that's a reasonable compromise.

> 
>> Related:  Possibly we should move UIs to another *process* instead?
>> At least the ones which use a UI toolkit (i.e. sdl, gtk, cocoa).
> 
> Is the worth the effort ? The majority of mgmt apps use spice/vnc which
> is already out of process. Will the people do use SDL/GTK/Coca backends,
> be doing so in a scenario where moving the UI to a separate process is
> a benefit to them ?  I'm not saying we shouldn't separate the UI to
> another process, just that we should consider whether there's other
> things todo in QEMU UI layer that are a better payoff for the userbase.

I haven't done enough UI programming to have an informed opinion here.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi,

> >> Related:  Possibly we should move UIs to another *process* instead?
> >> At least the ones which use a UI toolkit (i.e. sdl, gtk, cocoa).
> > 
> > Is the worth the effort ? The majority of mgmt apps use spice/vnc which
> > is already out of process.

When you are using libvirt you are not using sdl/gtk/cocoa, period.

It is quite inconvenient to setup.  gtk isn't supported in the first
place (possible to make work with <qemu:args> trickery though).  Google
finds nothing on the topic.  I highly doubt this is in widespread use.

> > Will the people do use SDL/GTK/Coca backends,
> > be doing so in a scenario where moving the UI to a separate process is
> > a benefit to them ?

Using gtk/sdl is very convenient when using qemu directly.  Running
"qemu -some-switches-to-test-stuff -cdrom /path/to/fedora-live.iso" and
have gtk ui come up is nice, so you don't have care about finding an
unused port and to startup a viewer process.

We hardly can take that away without having users scream.

We could try to go for better integration with viewers.  Launching
remote-viewer with correct parameters would be a simple start (i.e.
something along the lines of -spice port=auto,viewer=yes).  But that
wouldn't match neither gtk nor libvirt+virt-viewer feature-wise because
it wouldn't allow to control the machine (reset, shutdown, ...).  So if
we want deprecate and later drop toolkit (gtk/sdl/cocoa) UIs in favor of
something like this we need to do better.

We could build some qemu viewer app using the spice gtk widget instead
of trying to move our existing gtk code into a separate process.

spice would need a portable shared memory transport then, otherwise such
a setup wouldn't be able to match gtk/sdl in performance.

cheers,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Kevin Wolf]

Am 18.01.2016 um 10:54 hat Gerd Hoffmann geschrieben:
>   Hi folks,
> 
> I'm starting to investigate if and how we can move the user interface
> code into its own thread instead of running it in the iothread and
> therefore avoid blocking the guest in case some UI actions take a little
> longer.
> 
> opengl and toolkits tend to be bad at multithreading.  So my idea is to
> have a single thread dedicated to all the UI + rendering stuff, possibly
> let even the virglrenderer run in ui thread context.
> 
> I think we have to make that opt-in per user interface, so we can go
> forward step by step.
> 
> The ui thread will need quite some stuff provided by the mainloop.  Wait
> for all kinds of events (from vnc socket, x11 connection, ...).
> Probably timers too.  Wait for events from other threads (guest screen
> updates).
> 
> Suggestions how to tackle that?
> Can I reuse the qemu mainloop code outside of the iothread?

That should be possible. The block layer runs additional main loops in
dataplane threads. I think AioContext is the keyword here, so that you
process only events in your own UI context.

I'm copying Stefan who knows this stuff a bit better than me.

> Maybe it'll be better to go straight for a glib main loop?
> Is it possible to wait for file handle events and posix condition
> variables at the same time?
> 
> Other notes / hints / suggestions / ideas?

Kevin

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2174 bytes --]

On Tue, Jan 19, 2016 at 01:51:09PM +0100, Kevin Wolf wrote:
> Am 18.01.2016 um 10:54 hat Gerd Hoffmann geschrieben:
> >   Hi folks,
> > 
> > I'm starting to investigate if and how we can move the user interface
> > code into its own thread instead of running it in the iothread and
> > therefore avoid blocking the guest in case some UI actions take a little
> > longer.
> > 
> > opengl and toolkits tend to be bad at multithreading.  So my idea is to
> > have a single thread dedicated to all the UI + rendering stuff, possibly
> > let even the virglrenderer run in ui thread context.
> > 
> > I think we have to make that opt-in per user interface, so we can go
> > forward step by step.
> > 
> > The ui thread will need quite some stuff provided by the mainloop.  Wait
> > for all kinds of events (from vnc socket, x11 connection, ...).
> > Probably timers too.  Wait for events from other threads (guest screen
> > updates).
> > 
> > Suggestions how to tackle that?
> > Can I reuse the qemu mainloop code outside of the iothread?
> 
> That should be possible. The block layer runs additional main loops in
> dataplane threads. I think AioContext is the keyword here, so that you
> process only events in your own UI context.
> 
> I'm copying Stefan who knows this stuff a bit better than me.
> 
> > Maybe it'll be better to go straight for a glib main loop?

That sounds good in theory (but see below) since AioContext integrates
with the glib main loop because it is a GSource.  QEMU code should use
AioContext where we have high resolution timers and APIs for file
descriptor, EventNotifier, and BH.

But I think using multiple glib main loops has limitations/problems.
CCing Daniel Berrange because I vaguely remember this topic being
discussed.

> > Is it possible to wait for file handle events and posix condition
> > variables at the same time?

I don't think POSIX condition variables can be monitored from an event
loop.

Use BH or EventNotifier instead.  BH has optimizations to avoid syscalls
where possible while EventNotifier is just a plain eventfd/pipe
(requires a write() syscall for each notification).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi,

> That sounds good in theory (but see below) since AioContext integrates
> with the glib main loop because it is a GSource.  QEMU code should use
> AioContext where we have high resolution timers and APIs for file
> descriptor, EventNotifier, and BH.

Sounds good.

> Use BH or EventNotifier instead.  BH has optimizations to avoid syscalls
> where possible while EventNotifier is just a plain eventfd/pipe
> (requires a write() syscall for each notification).

How does BH signaling work?  I know I can qemu_bh_schedule from !main
thread to kick BH in main thread context.  The other way around works
using aio_bh_new + aio_bh_call I guess?

thanks,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Paolo Bonzini]

On 21/01/2016 11:39, Gerd Hoffmann wrote:
> How does BH signaling work?  I know I can qemu_bh_schedule from !main
> thread to kick BH in main thread context.  The other way around works
> using aio_bh_new + aio_bh_call I guess?

Yes.  The code will run in the iothread.  Note that I'm modifying
AioContext to *require* an IOThread if you use it outside the main thread.

Paolo

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Daniel P. Berrange]

On Thu, Jan 21, 2016 at 09:58:11AM +0000, Stefan Hajnoczi wrote:
> On Tue, Jan 19, 2016 at 01:51:09PM +0100, Kevin Wolf wrote:
> > Am 18.01.2016 um 10:54 hat Gerd Hoffmann geschrieben:
> > >   Hi folks,
> > > 
> > > I'm starting to investigate if and how we can move the user interface
> > > code into its own thread instead of running it in the iothread and
> > > therefore avoid blocking the guest in case some UI actions take a little
> > > longer.
> > > 
> > > opengl and toolkits tend to be bad at multithreading.  So my idea is to
> > > have a single thread dedicated to all the UI + rendering stuff, possibly
> > > let even the virglrenderer run in ui thread context.
> > > 
> > > I think we have to make that opt-in per user interface, so we can go
> > > forward step by step.
> > > 
> > > The ui thread will need quite some stuff provided by the mainloop.  Wait
> > > for all kinds of events (from vnc socket, x11 connection, ...).
> > > Probably timers too.  Wait for events from other threads (guest screen
> > > updates).
> > > 
> > > Suggestions how to tackle that?
> > > Can I reuse the qemu mainloop code outside of the iothread?
> > 
> > That should be possible. The block layer runs additional main loops in
> > dataplane threads. I think AioContext is the keyword here, so that you
> > process only events in your own UI context.
> > 
> > I'm copying Stefan who knows this stuff a bit better than me.
> > 
> > > Maybe it'll be better to go straight for a glib main loop?
> 
> That sounds good in theory (but see below) since AioContext integrates
> with the glib main loop because it is a GSource.  QEMU code should use
> AioContext where we have high resolution timers and APIs for file
> descriptor, EventNotifier, and BH.
> 
> But I think using multiple glib main loops has limitations/problems.
> CCing Daniel Berrange because I vaguely remember this topic being
> discussed.

WRT to GTK you need to be very careful to ensure all gtk_* and
gdk_* API calls are made from a single thread.  Running multiple
glib main loops isn't per-se a problem, as long as you respect
that threading requirement. FWIW, spice already has its own
background thread that runs its own main loop.

> > > Is it possible to wait for file handle events and posix condition
> > > variables at the same time?
> 
> I don't think POSIX condition variables can be monitored from an event
> loop.

Yep, you can't monitor condition vars from poll()

> Use BH or EventNotifier instead.  BH has optimizations to avoid syscalls
> where possible while EventNotifier is just a plain eventfd/pipe
> (requires a write() syscall for each notification).


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Daniel P. Berrange]

On Mon, Jan 18, 2016 at 10:54:40AM +0100, Gerd Hoffmann wrote:
>   Hi folks,
> 
> I'm starting to investigate if and how we can move the user interface
> code into its own thread instead of running it in the iothread and
> therefore avoid blocking the guest in case some UI actions take a little
> longer.
> 
> opengl and toolkits tend to be bad at multithreading.  So my idea is to
> have a single thread dedicated to all the UI + rendering stuff, possibly
> let even the virglrenderer run in ui thread context.
> 
> I think we have to make that opt-in per user interface, so we can go
> forward step by step.
> 
> The ui thread will need quite some stuff provided by the mainloop.  Wait
> for all kinds of events (from vnc socket, x11 connection, ...).
> Probably timers too.  Wait for events from other threads (guest screen
> updates).
> 
> Suggestions how to tackle that?
> Can I reuse the qemu mainloop code outside of the iothread?
> Maybe it'll be better to go straight for a glib main loop?
> Is it possible to wait for file handle events and posix condition
> variables at the same time?
> 
> Other notes / hints / suggestions / ideas?

IIUC, the spice backend already does alot of it work in a background
thread, to avoid stalling the main qemu event loop with potentially
slow tasks.

As a general note though, mixing GTK and threads is really not something
I'd encourage in general. I've battled it with my photo app entangle and
it has been nothing but pain. There are many behavioural differences in
GTK threading with different versions of GTK, so the app frequently broke
with new GTK releases. On Win32 you must *never* call gtk_* functions
from multiple threads, and even on Linux it is deprecated now. The best
pratice is for everything GTK related to be run from a single thread,
ideally the main thread. If any GTK event handler needs to do something
non-trivial it should use  GTask to fire off a background thread todo
the work without blocking the UI.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi,

> As a general note though, mixing GTK and threads is really not something
> I'd encourage in general.

Sure, thats why I think qemu should have *one* ui thread, and that one
will be the only one calling into gtk (and also the only one running
opengl in case that is enabled).

cheers,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Dave Airlie]

On 18 January 2016 at 19:54, Gerd Hoffmann <kraxel@redhat.com> wrote:
>   Hi folks,
>
> I'm starting to investigate if and how we can move the user interface
> code into its own thread instead of running it in the iothread and
> therefore avoid blocking the guest in case some UI actions take a little
> longer.
>
> opengl and toolkits tend to be bad at multithreading.  So my idea is to
> have a single thread dedicated to all the UI + rendering stuff, possibly
> let even the virglrenderer run in ui thread context.

I'd still like virgl to run it its own thread at some point like
dataplane but for virgl I suppose.

We see cases where backend GLSL compiles and end up taking quite a
while, and I'd prefer
to not block the UI or qemu on those.

I've hacked on this before, but only with SDL and it was pretty dirty,
and it gave a fairly decent
speed up.

My thoughts are to use dataplane like design to process the queue in a
separate thread,
and then have some sort of channel between the UI and virtio-gpu
thread to handle things like
screen resize etc.

http://cgit.freedesktop.org/~airlied/qemu/commit/?h=dave3d&id=fe22a0955255afef12b947c4a91efa57e7a7c429
is my previous horror patch.

Dave.

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Paolo Bonzini]

On 21/01/2016 09:44, Dave Airlie wrote:
> I've hacked on this before, but only with SDL and it was pretty dirty,
> and it gave a fairly decent
> speed up.
> 
> My thoughts are to use dataplane like design to process the queue in a
> separate thread,
> and then have some sort of channel between the UI and virtio-gpu
> thread to handle things like
> screen resize etc.
> 
> http://cgit.freedesktop.org/~airlied/qemu/commit/?h=dave3d&id=fe22a0955255afef12b947c4a91efa57e7a7c429
> is my previous horror patch.

Instead of having a full-blown thread, are there things (such as the
TGSI->GLSL conversion) that could be simply offloaded to a userspace
thread pool, either in QEMU or in virglrenderer?

If you can do this in QEMU, as an alternative to writing code with
callbacks we have coroutines based on a setjmp-longjmp and also a simple
thread pool (thread-pool.c).  If you want to allocate your own
thread-pool, you can create an additional AioContext (aio_context_new)
and add the AioContext to the glib main loop using aio_get_g_source.

Paolo

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

On Do, 2016-01-21 at 10:05 +0100, Paolo Bonzini wrote:
> 
> On 21/01/2016 09:44, Dave Airlie wrote:
> > I've hacked on this before, but only with SDL and it was pretty dirty,
> > and it gave a fairly decent
> > speed up.
> > 
> > My thoughts are to use dataplane like design to process the queue in a
> > separate thread,
> > and then have some sort of channel between the UI and virtio-gpu
> > thread to handle things like
> > screen resize etc.
> > 
> > http://cgit.freedesktop.org/~airlied/qemu/commit/?h=dave3d&id=fe22a0955255afef12b947c4a91efa57e7a7c429
> > is my previous horror patch.
> 
> Instead of having a full-blown thread, are there things (such as the
> TGSI->GLSL conversion) that could be simply offloaded to a userspace
> thread pool, either in QEMU or in virglrenderer?

I think virglrenderer would have to do that.  Unfortunaly opengl isn't
very good at multithreading, so I'm not sure a thread pool would work
well.  Compiling shaders could be a special case where threading
actually works because that isn't in the actual rendering workflow.  Not
fully sure though, Dave?

cheers,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Paolo Bonzini]

On 21/01/2016 10:52, Gerd Hoffmann wrote:
>> > Instead of having a full-blown thread, are there things (such as the
>> > TGSI->GLSL conversion) that could be simply offloaded to a userspace
>> > thread pool, either in QEMU or in virglrenderer?
> I think virglrenderer would have to do that.  Unfortunaly opengl isn't
> very good at multithreading, so I'm not sure a thread pool would work
> well.  Compiling shaders could be a special case where threading
> actually works because that isn't in the actual rendering workflow.  Not
> fully sure though, Dave?

Or even MESA could. Perhaps we (you :)) could add an extension for
asynchronous shader compilation.  But the TGSI->GLSL conversion is done
by virglrender, with no opengl calls (with a big IIRC in front).

Paolo

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Dave Airlie]

On 21 January 2016 at 19:05, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 21/01/2016 09:44, Dave Airlie wrote:
>> I've hacked on this before, but only with SDL and it was pretty dirty,
>> and it gave a fairly decent
>> speed up.
>>
>> My thoughts are to use dataplane like design to process the queue in a
>> separate thread,
>> and then have some sort of channel between the UI and virtio-gpu
>> thread to handle things like
>> screen resize etc.
>>
>> http://cgit.freedesktop.org/~airlied/qemu/commit/?h=dave3d&id=fe22a0955255afef12b947c4a91efa57e7a7c429
>> is my previous horror patch.
>
> Instead of having a full-blown thread, are there things (such as the
> TGSI->GLSL conversion) that could be simply offloaded to a userspace
> thread pool, either in QEMU or in virglrenderer?

Not really, the TGSI->GLSL overhead is quite minor, the problem is the
produced GLSL then gets passed to
OpenGL to compile, and depending on the quality of the original
program and my conversion code, the
GLSL compiler can get quite upset.

In theory Mesa could help here, but GL isn't thread friendly at all,
so it probably won't help in the virgl
case even if it did. Since most GL apps compile a shader and block on
using it straight away doing it
in a thread won't help unblock things.

So I think it would be best to have all the virgl vq processing happen
in it's own thread with some API
to the UI to do UI resizes (the most difficult) and dirty regions etc.

Dave.

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi,

> In theory Mesa could help here, but GL isn't thread friendly at all,
> so it probably won't help in the virgl
> case even if it did. Since most GL apps compile a shader and block on
> using it straight away doing it
> in a thread won't help unblock things.
> 
> So I think it would be best to have all the virgl vq processing happen
> in it's own thread with some API
> to the UI to do UI resizes (the most difficult) and dirty regions etc.

We can move only virgl into its own thread, but then we'll have two
threads (virgl and main which runs ui) which use opengl.  So I was
thinking maybe it is better to have a single thread which runs both
virgl and ui (and thats why I've started this thread ...).

cheers,
  Gerd

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Dave Airlie]

On 22 January 2016 at 16:59, Gerd Hoffmann <kraxel@redhat.com> wrote:
>   Hi,
>
>> In theory Mesa could help here, but GL isn't thread friendly at all,
>> so it probably won't help in the virgl
>> case even if it did. Since most GL apps compile a shader and block on
>> using it straight away doing it
>> in a thread won't help unblock things.
>>
>> So I think it would be best to have all the virgl vq processing happen
>> in it's own thread with some API
>> to the UI to do UI resizes (the most difficult) and dirty regions etc.
>
> We can move only virgl into its own thread, but then we'll have two
> threads (virgl and main which runs ui) which use opengl.  So I was
> thinking maybe it is better to have a single thread which runs both
> virgl and ui (and thats why I've started this thread ...).

In theory as long as we have separate contexts bound in each thread it
should be fine.

OpenGL is okay as long as one context is bound in one thread at a time.

Since the only sharing we have between contexts are textures, they
should be okay
between shared contexts.

Dave.

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Fam Zheng]

On Mon, 01/18 10:54, Gerd Hoffmann wrote:
>   Hi folks,
> 
> I'm starting to investigate if and how we can move the user interface
> code into its own thread instead of running it in the iothread and
> therefore avoid blocking the guest in case some UI actions take a little
> longer.
> 
> opengl and toolkits tend to be bad at multithreading.  So my idea is to
> have a single thread dedicated to all the UI + rendering stuff, possibly
> let even the virglrenderer run in ui thread context.
> 
> I think we have to make that opt-in per user interface, so we can go
> forward step by step.
> 
> The ui thread will need quite some stuff provided by the mainloop.  Wait
> for all kinds of events (from vnc socket, x11 connection, ...).
> Probably timers too.  Wait for events from other threads (guest screen
> updates).
> 
> Suggestions how to tackle that?
> Can I reuse the qemu mainloop code outside of the iothread?
> Maybe it'll be better to go straight for a glib main loop?
> Is it possible to wait for file handle events and posix condition
> variables at the same time?
> 
> Other notes / hints / suggestions / ideas?

What are taking long time? Can you give a few examples?

Fam

Re: [Qemu-devel] RFC: running the user interface in a thread ...

[Gerd Hoffmann]

  Hi,

> What are taking long time? Can you give a few examples?

Didn't profile things, but UI code is handling bulky graphics data,
which of course takes time, especially in case some software scaling is
involved.

For sdl-1 I remember complains about it slowing down guests alot.  IIRC
I checked a while back and figured it does some X11 stuff synchronously
(i.e. waits on X server reply) somewhere deep in the library.

With virglrenderer a new issue came up: compiling shaders can take time.

Given both opengl and ui toolkits tend to be pretty bad at
multithreading I'm not sure how well helper threads for heavier UI tasks
are going to work (I suspect that is the background of your question).

cheers,
  Gerd