Compatibility between -device sga and -machine graphics=off

Compatibility between -device sga and -machine graphics=off

[Daniel P. Berrangé]

Given the libvirt XML snippet

  <os>
    ...
    <bios useserial="yes"/>
    ...
  </os>

libvirt QEMU driver will always format

  -device sga

Libguestfs uses this syntax, so we need to make sure it still works
in future even if 'sga' is disabled or removed in a QEMU build in
favour of SeaBIOS' built-in support.


There are the following combinations that matter

 1. Graphical display only, no serial port, BIOS to graphical display

     <video>
       <model type='vga'/>
     </video>

   QEMU cli

     -device VGA


 2. Serial port only, no graphics, BIOS to nowhere

     <serial type='pty'/>

   QEMU cli

     -serial pty


 3. Serial port only, no graphics, BIOS to serial

     <os>
       <bios useserial='yes'/>
     </os>
     <serial type='pty'/>

   QEMU cli

     -serial pty -device sga
     -serial pty -M graphics=off


 4. Graphical display, serial port, BIOS only to graphical display

     <video>
       <model type='vga'/>
     </video>
     <serial type='pty'/>

   QEMU cli

     -device VGA -serial pty


 5. Graphical display, serial port, BIOS to graphical display + serial

     <os>
       <bios useserial='yes'/>
     </os>
     <video>
       <model type='vga'/>
     </video>
     <serial type='pty'/>

   QEMU cli

     -device VGA -serial pty -device sga
     -device VGA -serial pty -M graphics=off



The cases 3 and 5 appear to have the same effect on output with x86
for SGA and graphics=off, which is good.

If I do 'info mtree' though, I do see a different memory layout
when changing from SGA to graphics=off


@@ -18,7 +18,7 @@
     00000000000c0000-00000000000c3fff (prio 1, ram): alias pam-rom @pc.ram 00000000000c0000-00000000000c3fff
     00000000000c4000-00000000000c7fff (prio 1, ram): alias pam-rom @pc.ram 00000000000c4000-00000000000c7fff
     00000000000c8000-00000000000cbfff (prio 1, ram): alias pam-rom @pc.ram 00000000000c8000-00000000000cbfff
-    00000000000cb000-00000000000cdfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000cb000-00000000000cdfff
+    00000000000ca000-00000000000ccfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000ca000-00000000000ccfff
     00000000000cc000-00000000000cffff (prio 1, ram): alias pam-rom @pc.ram 00000000000cc000-00000000000cffff
     00000000000d0000-00000000000d3fff (prio 1, ram): alias pam-rom @pc.ram 00000000000d0000-00000000000d3fff
     00000000000d4000-00000000000d7fff (prio 1, ram): alias pam-rom @pc.ram 00000000000d4000-00000000000d7fff
@@ -109,7 +109,7 @@
     00000000000c0000-00000000000c3fff (prio 1, ram): alias pam-rom @pc.ram 00000000000c0000-00000000000c3fff
     00000000000c4000-00000000000c7fff (prio 1, ram): alias pam-rom @pc.ram 00000000000c4000-00000000000c7fff
     00000000000c8000-00000000000cbfff (prio 1, ram): alias pam-rom @pc.ram 00000000000c8000-00000000000cbfff
-    00000000000cb000-00000000000cdfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000cb000-00000000000cdfff
+    00000000000ca000-00000000000ccfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000ca000-00000000000ccfff
     00000000000cc000-00000000000cffff (prio 1, ram): alias pam-rom @pc.ram 00000000000cc000-00000000000cffff
     00000000000d0000-00000000000d3fff (prio 1, ram): alias pam-rom @pc.ram 00000000000d0000-00000000000d3fff
     00000000000d4000-00000000000d7fff (prio 1, ram): alias pam-rom @pc.ram 00000000000d4000-00000000000d7fff
@@ -185,7 +185,7 @@
     00000000000c0000-00000000000c3fff (prio 1, ram): alias pam-rom @pc.ram 00000000000c0000-00000000000c3fff
     00000000000c4000-00000000000c7fff (prio 1, ram): alias pam-rom @pc.ram 00000000000c4000-00000000000c7fff
     00000000000c8000-00000000000cbfff (prio 1, ram): alias pam-rom @pc.ram 00000000000c8000-00000000000cbfff
-    00000000000cb000-00000000000cdfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000cb000-00000000000cdfff
+    00000000000ca000-00000000000ccfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000ca000-00000000000ccfff
     00000000000cc000-00000000000cffff (prio 1, ram): alias pam-rom @pc.ram 00000000000cc000-00000000000cffff
     00000000000d0000-00000000000d3fff (prio 1, ram): alias pam-rom @pc.ram 00000000000d0000-00000000000d3fff
     00000000000d4000-00000000000d7fff (prio 1, ram): alias pam-rom @pc.ram 00000000000d4000-00000000000d7fff


Despite this difference, I was able migrate from a x86 guest using SGA
to a guest using graphics=off without errors being reported. So it
doesn't seem to change the migration data sections sent on the wire
at least.

With this in mind, is the different memory layout harmless from the
POV of migration, or is it none the less going to cause subtle
problems for the guest at some point in future ?


On non-x86 emulators I see graphics=off has semantic effects beyond
just controlling whether the firmware prints to the serial or not
though. IOW it is overloaded to do multiple jobs, while -device sga
only did one specific job. This makes graphics=off somewhat undesirable
to use.  We're possibly lucky in this specific case though, because
the 'sgabios.bin' ROM is x86 asm code, so was never valid to use in
the non-x86 case.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Compatibility between -device sga and -machine graphics=off

[Paolo Bonzini]

On 08/09/21 18:08, Daniel P. Berrangé wrote:
> Despite this difference, I was able migrate from a x86 guest using SGA
> to a guest using graphics=off without errors being reported. So it
> doesn't seem to change the migration data sections sent on the wire
> at least.

It would probably break with Windows XP/2003 guests on AMD processors, 
as those are the ones that use kvmvapic.  On other guests and 
processors, it's a no-op.

Paolo

Re: Compatibility between -device sga and -machine graphics=off

[Richard W.M. Jones]

Just commenting from the libguestfs POV:

We don't care about migration, and:

On Wed, Sep 08, 2021 at 05:08:08PM +0100, Daniel P. Berrangé wrote:
> On non-x86 emulators I see graphics=off has semantic effects beyond
> just controlling whether the firmware prints to the serial or not
> though. IOW it is overloaded to do multiple jobs, while -device sga
> only did one specific job. This makes graphics=off somewhat undesirable
> to use.  We're possibly lucky in this specific case though, because
> the 'sgabios.bin' ROM is x86 asm code, so was never valid to use in
> the non-x86 case.

we only enable <bios useserial=yes/> on i686 & x86-64.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: Compatibility between -device sga and -machine graphics=off

[Daniel P. Berrangé]

On Wed, Sep 08, 2021 at 06:28:01PM +0200, Paolo Bonzini wrote:
> On 08/09/21 18:08, Daniel P. Berrangé wrote:
> > Despite this difference, I was able migrate from a x86 guest using SGA
> > to a guest using graphics=off without errors being reported. So it
> > doesn't seem to change the migration data sections sent on the wire
> > at least.
> 
> It would probably break with Windows XP/2003 guests on AMD processors, as
> those are the ones that use kvmvapic.  On other guests and processors, it's
> a no-op.

After running an incoming migrate, the target QEMU's  'info mtree'
changed to reflect what the src QEMU originally reported.  So whatever
difference 'graphics=off' causes compared to -device sga initially,
appears to have been eliminated by running the migrate. Not sure if
that's true in the general case or not - I'm just testing with a
minimalist CLI and no real OS running

 qemu-system-x86_64 -nodefaults -display sdl -device VGA -serial stdio  -device sga

vs

 qemu-system-x86_64 -nodefaults -display sdl -device VGA -serial stdio  -machine graphics=off

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Compatibility between -device sga and -machine graphics=off

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

Oh, that's cool. It must be part of the kvmvapic migration data. Still,
there are very likely some rare cases that would break (on any machine),
e.g. if migrating while seabios is accessing the list of option roms.

Paolo

Il mer 8 set 2021, 18:36 Daniel P. Berrangé <berrange@redhat.com> ha
scritto:

> On Wed, Sep 08, 2021 at 06:28:01PM +0200, Paolo Bonzini wrote:
> > On 08/09/21 18:08, Daniel P. Berrangé wrote:
> > > Despite this difference, I was able migrate from a x86 guest using SGA
> > > to a guest using graphics=off without errors being reported. So it
> > > doesn't seem to change the migration data sections sent on the wire
> > > at least.
> >
> > It would probably break with Windows XP/2003 guests on AMD processors, as
> > those are the ones that use kvmvapic.  On other guests and processors,
> it's
> > a no-op.
>
> After running an incoming migrate, the target QEMU's  'info mtree'
> changed to reflect what the src QEMU originally reported.  So whatever
> difference 'graphics=off' causes compared to -device sga initially,
> appears to have been eliminated by running the migrate. Not sure if
> that's true in the general case or not - I'm just testing with a
> minimalist CLI and no real OS running
>
>  qemu-system-x86_64 -nodefaults -display sdl -device VGA -serial stdio
> -device sga
>
> vs
>
>  qemu-system-x86_64 -nodefaults -display sdl -device VGA -serial stdio
> -machine graphics=off
>
> Regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|
>
>

[-- Attachment #2: Type: text/html, Size: 2824 bytes --]

Re: Compatibility between -device sga and -machine graphics=off

[Gerd Hoffmann]

On Wed, Sep 08, 2021 at 05:08:08PM +0100, Daniel P. Berrangé wrote:
> Given the libvirt XML snippet
> 
>   <os>
>     ...
>     <bios useserial="yes"/>
>     ...
>   </os>
> 
> libvirt QEMU driver will always format
> 
>   -device sga
> 
> Libguestfs uses this syntax, so we need to make sure it still works
> in future even if 'sga' is disabled or removed in a QEMU build in
> favour of SeaBIOS' built-in support.

Just replacing '-device sga' with '-machine graphics=off' in case sga is
not available should work fine.

serial console support in seabios is available for quite a while
(merged in 2017, seabios 1.11 in rhel-7 has it), so switching over
unconditionally is possibly an option too.  Not sure what the libvirt
backward compatibility policy is though.

>  1. Graphical display only, no serial port, BIOS to graphical display
>  2. Serial port only, no graphics, BIOS to nowhere
>  3. Serial port only, no graphics, BIOS to serial
>  4. Graphical display, serial port, BIOS only to graphical display
>  5. Graphical display, serial port, BIOS to graphical display + serial

Should all work fine.

> If I do 'info mtree' though, I do see a different memory layout
> when changing from SGA to graphics=off

> -    00000000000cb000-00000000000cdfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000cb000-00000000000cdfff
> +    00000000000ca000-00000000000ccfff (prio 1000, ram): alias kvmvapic-rom @pc.ram 00000000000ca000-00000000000ccfff

probably sgabios.bin is loaded to ca000 when enabled.

> On non-x86 emulators I see graphics=off has semantic effects beyond
> just controlling whether the firmware prints to the serial or not
> though.

It's been a while, but yes, IIRC on ppc this is passed to the linux
kernel somehow (device tree?) so it also affects the default console
device used by linux.

But sgabios is x86-only anyway so that should not be a problem here.

HTH & take care,
  Gerd

Re: Compatibility between -device sga and -machine graphics=off

[Daniel P. Berrangé]

On Wed, Sep 08, 2021 at 10:50:53PM +0200, Gerd Hoffmann wrote:
> On Wed, Sep 08, 2021 at 05:08:08PM +0100, Daniel P. Berrangé wrote:
> > Given the libvirt XML snippet
> > 
> >   <os>
> >     ...
> >     <bios useserial="yes"/>
> >     ...
> >   </os>
> > 
> > libvirt QEMU driver will always format
> > 
> >   -device sga
> > 
> > Libguestfs uses this syntax, so we need to make sure it still works
> > in future even if 'sga' is disabled or removed in a QEMU build in
> > favour of SeaBIOS' built-in support.
> 
> Just replacing '-device sga' with '-machine graphics=off' in case sga is
> not available should work fine.
> 
> serial console support in seabios is available for quite a while
> (merged in 2017, seabios 1.11 in rhel-7 has it), so switching over
> unconditionally is possibly an option too.  Not sure what the libvirt
> backward compatibility policy is though.

libvirt currently targets QEMU 2.11.0 and newer.

Fortunately it appears that QEMU 2.11.0 included SeaBIOS 1.11.0,
so we can thus eliminate usage of 'sga' from libvirt entirely,
provided I can convince myself migration is safe, which looks
probable.

> > On non-x86 emulators I see graphics=off has semantic effects beyond
> > just controlling whether the firmware prints to the serial or not
> > though.
> 
> It's been a while, but yes, IIRC on ppc this is passed to the linux
> kernel somehow (device tree?) so it also affects the default console
> device used by linux.
> 
> But sgabios is x86-only anyway so that should not be a problem here.

libvirt has never validated the arch when enabling sga, but clearly
it can't ever have been usable on non-x86. So I'll probably restrict
the usage of graphics=off to only those arches where I can validate
it something useful for the firmware output. 


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|