[Qemu-devel] multihead & multiseat in qemu

[Qemu-devel] multihead & multiseat in qemu

[Gerd Hoffmann]

  Hi,

I'm thinking about how to handle multihead and multiseat in qemu best.

On multihead:  Mouse in virtual machines works best with absolute
coordinates, and the way this is done today is to assign a (virtual) usb
tablet to the guest.  With multihead this becomes a bit difficuilt.
Today we try to calculate the coordinates for the tablet that they cover
all displays, which has a number of drawbacks.  I think it would be
better to operate more like a touchscreen, i.e. have one touch input
device for each monitor.  For that to work the guest must know which
touch device belongs to which monitor.

On multiseat:  Very simliar problem here (thats why both issues in one
mail):  The guest needs to know which devices belong to which seat.

Qemu needs the grouping/assignment information too, to the the input
routing right (i.e. input from this window needs to go to that virtual
input device etc).  Doing the configuration twice (once for qemu, once
for the guest) and make sure they actually match would be annoying
though.  So I think we should configure qemu only, then pass that
information to the guest somehow.

Question is how to do that best?

I'd like to have some way to assign tags such as seat=foo or head=1 to
devices.  Preferably in some way which can easily picked up with udev
rules, so it is easily usable by system-logind and Xorg server.

We have virtio devices (virtio-gpu for example).  For these it is easy,
we can put something into the virtio protocol, and the guest driver can
create a sysfs file where udev/systemd can pick it up.

We have pci devices (cirrus for example).  One idea for them would be to
create a vendor-specific pci capabiliy for tags.  Probably needs some
small utility to read them, or kernel driver support to export them via
sysfs.

We have usb devices (kbd/mouse/tablet).  We could put something into the
string table, or have some vendor-specific descriptor.  Same problem
here, not easy accessible, needs a tool or kernel support.

Comments?  Better ideas?  Other suggestions?

cheers,
  Gerd

Re: [Qemu-devel] multihead & multiseat in qemu

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
>   Hi,
> 
> I'm thinking about how to handle multihead and multiseat in qemu best.
> 
> On multihead:  Mouse in virtual machines works best with absolute
> coordinates, and the way this is done today is to assign a (virtual) usb
> tablet to the guest.  With multihead this becomes a bit difficuilt.
> Today we try to calculate the coordinates for the tablet that they cover
> all displays, which has a number of drawbacks.  I think it would be
> better to operate more like a touchscreen, i.e. have one touch input
> device for each monitor.  For that to work the guest must know which
> touch device belongs to which monitor.
> 
> On multiseat:  Very simliar problem here (thats why both issues in one
> mail):  The guest needs to know which devices belong to which seat.
> 
> Qemu needs the grouping/assignment information too, to the the input
> routing right (i.e. input from this window needs to go to that virtual
> input device etc).  Doing the configuration twice (once for qemu, once
> for the guest) and make sure they actually match would be annoying
> though.  So I think we should configure qemu only, then pass that
> information to the guest somehow.
> 
> Question is how to do that best?
> 
> I'd like to have some way to assign tags such as seat=foo or head=1 to
> devices.  Preferably in some way which can easily picked up with udev
> rules, so it is easily usable by system-logind and Xorg server.

Can't you just do this in the serial number strings for both monitor EDID
and USB device, I'm assuming that's how current multiseat setups figure
stuff out.

I think with the touchscreen idea that means qemu never needs itself
to have a view of where it's displays are logically left/right/above/down
which is probably fortunate.

Dave

> We have virtio devices (virtio-gpu for example).  For these it is easy,
> we can put something into the virtio protocol, and the guest driver can
> create a sysfs file where udev/systemd can pick it up.
> 
> We have pci devices (cirrus for example).  One idea for them would be to
> create a vendor-specific pci capabiliy for tags.  Probably needs some
> small utility to read them, or kernel driver support to export them via
> sysfs.
> 
> We have usb devices (kbd/mouse/tablet).  We could put something into the
> string table, or have some vendor-specific descriptor.  Same problem
> here, not easy accessible, needs a tool or kernel support.
> 
> Comments?  Better ideas?  Other suggestions?
> 
> cheers,
>   Gerd
> 
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] multihead & multiseat in qemu

[Dave Airlie]

On Fri, Mar 21, 2014 at 11:27 PM, Gerd Hoffmann <kraxel@redhat.com> wrote:
>   Hi,
>
> I'm thinking about how to handle multihead and multiseat in qemu best.
>
> On multihead:  Mouse in virtual machines works best with absolute
> coordinates, and the way this is done today is to assign a (virtual) usb
> tablet to the guest.  With multihead this becomes a bit difficuilt.
> Today we try to calculate the coordinates for the tablet that they cover
> all displays, which has a number of drawbacks.  I think it would be
> better to operate more like a touchscreen, i.e. have one touch input
> device for each monitor.  For that to work the guest must know which
> touch device belongs to which monitor.

Yeah I think I'm in agreement with this, one touch device per monitor,

> On multiseat:  Very simliar problem here (thats why both issues in one
> mail):  The guest needs to know which devices belong to which seat.
>
> Qemu needs the grouping/assignment information too, to the the input
> routing right (i.e. input from this window needs to go to that virtual
> input device etc).  Doing the configuration twice (once for qemu, once
> for the guest) and make sure they actually match would be annoying
> though.  So I think we should configure qemu only, then pass that
> information to the guest somehow.
>
> Question is how to do that best?

The GNOME guys have been working on auto config of touch devices,
I need to find who is doing that work and what they are triggering on,
since X just provides the devices currently, I think we might need some agent
in the guest for this.

>
> I'd like to have some way to assign tags such as seat=foo or head=1 to
> devices.  Preferably in some way which can easily picked up with udev
> rules, so it is easily usable by system-logind and Xorg server.

The only current example of seat autoconfiguration is for the usb hubs
from displaylink I think, which have video/audio/ethernet/usb behind a
hub that udev detects as being part of a seat, I'd need to look up the
specifics,

> We have virtio devices (virtio-gpu for example).  For these it is easy,
> we can put something into the virtio protocol, and the guest driver can
> create a sysfs file where udev/systemd can pick it up.
>
> We have pci devices (cirrus for example).  One idea for them would be to
> create a vendor-specific pci capabiliy for tags.  Probably needs some
> small utility to read them, or kernel driver support to export them via
> sysfs.
>
> We have usb devices (kbd/mouse/tablet).  We could put something into the
> string table, or have some vendor-specific descriptor.  Same problem
> here, not easy accessible, needs a tool or kernel support.
>
> Comments?  Better ideas?  Other suggestions?

It does seems like tagging the devices somehow would be better than providing
a seat device, like we could in theory have a pci and usb controller
per seat, and
have devices move around between them, this would be like what we for
the real hw,
however per-device tags does look like it might be nicer in the long run.

Dave.

Re: [Qemu-devel] multihead & multiseat in qemu

[Gerd Hoffmann]

  Hi,

> > I'd like to have some way to assign tags such as seat=foo or head=1 to
> > devices.  Preferably in some way which can easily picked up with udev
> > rules, so it is easily usable by system-logind and Xorg server.
> 
> The only current example of seat autoconfiguration is for the usb hubs
> from displaylink I think, which have video/audio/ethernet/usb behind a
> hub that udev detects as being part of a seat, I'd need to look up the
> specifics,

udev can match upstream devices, so it's easy to tag all devices below
the usb hub as belonging to the seat.

> > We have virtio devices (virtio-gpu for example).  For these it is easy,
> > we can put something into the virtio protocol, and the guest driver can
> > create a sysfs file where udev/systemd can pick it up.
> >
> > We have pci devices (cirrus for example).  One idea for them would be to
> > create a vendor-specific pci capabiliy for tags.  Probably needs some
> > small utility to read them, or kernel driver support to export them via
> > sysfs.
> >
> > We have usb devices (kbd/mouse/tablet).  We could put something into the
> > string table, or have some vendor-specific descriptor.  Same problem
> > here, not easy accessible, needs a tool or kernel support.
> >
> > Comments?  Better ideas?  Other suggestions?
> 
> It does seems like tagging the devices somehow would be better than providing
> a seat device, like we could in theory have a pci and usb controller
> per seat, and
> have devices move around between them, this would be like what we for
> the real hw,
> however per-device tags does look like it might be nicer in the long run.

I've been thinking about this too.  Use a pci-bridge, then group the
devices belonging to the seat below the pci-bridge for easy udev
matching.  That doesn't help for the multihead case though.  And we have
to tag the devices for qemu's internal usage (host-side input routing)
anyway.  Thats why I think the tagging will work better.

cheers,
  Gerd

Re: [Qemu-devel] multihead & multiseat in qemu

[Gerd Hoffmann]

  Hi,

> > I'd like to have some way to assign tags such as seat=foo or head=1 to
> > devices.  Preferably in some way which can easily picked up with udev
> > rules, so it is easily usable by system-logind and Xorg server.
> 
> Can't you just do this in the serial number strings for both monitor EDID
> and USB device, I'm assuming that's how current multiseat setups figure
> stuff out.

We don't emulate monitor EDID ...

Also the serial number for HID devices is (ab-)used to signal remote
wakeup support (see commit 7b074a22dab4bdda9864b933f1bc811a3db42845).

> I think with the touchscreen idea that means qemu never needs itself
> to have a view of where it's displays are logically left/right/above/down
> which is probably fortunate.

Yes, that is one of the reasons.

cheers,
  Gerd

Re: [Qemu-devel] multihead & multiseat in qemu

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
>   Hi,
> 
> > > I'd like to have some way to assign tags such as seat=foo or head=1 to
> > > devices.  Preferably in some way which can easily picked up with udev
> > > rules, so it is easily usable by system-logind and Xorg server.
> > 
> > Can't you just do this in the serial number strings for both monitor EDID
> > and USB device, I'm assuming that's how current multiseat setups figure
> > stuff out.
> 
> We don't emulate monitor EDID ...
> 
> Also the serial number for HID devices is (ab-)used to signal remote
> wakeup support (see commit 7b074a22dab4bdda9864b933f1bc811a3db42845).

Oh dear, that's a shame to use a field like that for a single bit of info.

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] multihead & multiseat in qemu

[Gerd Hoffmann]

On Mo, 2014-03-24 at 09:36 +0000, Dr. David Alan Gilbert wrote:
> * Gerd Hoffmann (kraxel@redhat.com) wrote:
> >   Hi,
> > 
> > > > I'd like to have some way to assign tags such as seat=foo or head=1 to
> > > > devices.  Preferably in some way which can easily picked up with udev
> > > > rules, so it is easily usable by system-logind and Xorg server.
> > > 
> > > Can't you just do this in the serial number strings for both monitor EDID
> > > and USB device, I'm assuming that's how current multiseat setups figure
> > > stuff out.
> > 
> > We don't emulate monitor EDID ...
> > 
> > Also the serial number for HID devices is (ab-)used to signal remote
> > wakeup support (see commit 7b074a22dab4bdda9864b933f1bc811a3db42845).
> 
> Oh dear, that's a shame to use a field like that for a single bit of info.

Indeed.  Patch sent to systemd-devel to get that fixed.

Matching by serial number certainly makes sense.  EDID emulation /
pass-through has been discussed before, so we might get that some day.
Also for virtio-gpu (which is the most important case) we have the
option to simply pass on serial numbers without the edid, maybe in
virtgpu_display_info.

cheers,
  Gerd

Re: [Qemu-devel] multihead & multiseat in qemu

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
> On Mo, 2014-03-24 at 09:36 +0000, Dr. David Alan Gilbert wrote:
> > * Gerd Hoffmann (kraxel@redhat.com) wrote:
> > >   Hi,
> > > 
> > > > > I'd like to have some way to assign tags such as seat=foo or head=1 to
> > > > > devices.  Preferably in some way which can easily picked up with udev
> > > > > rules, so it is easily usable by system-logind and Xorg server.
> > > > 
> > > > Can't you just do this in the serial number strings for both monitor EDID
> > > > and USB device, I'm assuming that's how current multiseat setups figure
> > > > stuff out.
> > > 
> > > We don't emulate monitor EDID ...
> > > 
> > > Also the serial number for HID devices is (ab-)used to signal remote
> > > wakeup support (see commit 7b074a22dab4bdda9864b933f1bc811a3db42845).
> > 
> > Oh dear, that's a shame to use a field like that for a single bit of info.
> 
> Indeed.  Patch sent to systemd-devel to get that fixed.

Yes, what are the logistics of when QEMU could change without breaking things.

> Matching by serial number certainly makes sense.  EDID emulation /
> pass-through has been discussed before, so we might get that some day.
> Also for virtio-gpu (which is the most important case) we have the
> option to simply pass on serial numbers without the edid, maybe in
> virtgpu_display_info.

Yeh I guess for virtio/spice it's easy, and probably the only thing
to worry about.

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] multihead & multiseat in qemu

[Gerd Hoffmann]

> > 
> > Indeed.  Patch sent to systemd-devel to get that fixed.
> 
> Yes, what are the logistics of when QEMU could change without breaking things.

There aren't any explicit rules.  Also depends on the use case.  We
better should not change the default serial number for a number of
years, to not break old guests.  Making it configurable for the purpose
of tablet/monitor matching is alot less problematic I think as you need
pretty recent bits anyway to use virtio-gpu in the first place.

cheers,
  Gerd