All of lore.kernel.org
 help / color / mirror / Atom feed
* Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-21 13:51 ` Paolo Bonzini
  0 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-21 13:51 UTC (permalink / raw)
  To: qemu-devel, KVM list

Some of you may have heard about the "Clear Containers" initiative from
Intel, which couple KVM with various kernel tricks to create extremely
lightweight virtual machines.  The experimental Clear Containers setup
requires only 18-20 MB to launch a virtual machine, and needs about 60
ms to boot.

Now, as all of you probably know, "QEMU is great for running Windows or
legacy Linux guests, but that flexibility comes at a hefty price. Not
only does all of the emulation consume memory, it also requires some
form of low-level firmware in the guest as well. All of this adds quite
a bit to virtual-machine startup times (500 to 700 milliseconds is not
unusual)".

Right?  In fact, it's for this reason that Clear Containers uses kvmtool
instead of QEMU.

No, wrong!  In fact, reporting bad performance is pretty much the same
as throwing down the gauntlet.

Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
a slimmed-down QEMU configuration, boots a virtual machine in 40
milliseconds[2] on an Ivy Bridge Core i7 processor.

qboot is available at git://github.com/bonzini/qboot.git.  In all the
glory of its 8KB of code, it brings together various existing open
source components:

* a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS

* a couple hardware initialization routines written mostly from scratch
but with good help from SeaBIOS source code

* a minimal 32-bit libc based on kvm-unit-tests

* the Linux loader from QEMU itself

The repository has more information on how to achieve fast boot times,
and examples of using qboot.  Right now there is a limit of 8 MB for
vmlinuz+initrd+cmdline, which however should be enough for initrd-less
containers.

The first commit to qboot is more or less 24 hours old, so there is
definitely more work to do, in particular to extract ACPI tables from
QEMU and present them to the guest.  This is probably another day of
work or so, and it will enable multiprocessor guests with little or no
impact on the boot times.  SMBIOS information is also available from QEMU.

On the QEMU side, there is no support yet for persistent memory and the
NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
will automatically start using it.

Happy hacking!

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-21 13:51 ` Paolo Bonzini
  0 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-21 13:51 UTC (permalink / raw)
  To: qemu-devel, KVM list

Some of you may have heard about the "Clear Containers" initiative from
Intel, which couple KVM with various kernel tricks to create extremely
lightweight virtual machines.  The experimental Clear Containers setup
requires only 18-20 MB to launch a virtual machine, and needs about 60
ms to boot.

Now, as all of you probably know, "QEMU is great for running Windows or
legacy Linux guests, but that flexibility comes at a hefty price. Not
only does all of the emulation consume memory, it also requires some
form of low-level firmware in the guest as well. All of this adds quite
a bit to virtual-machine startup times (500 to 700 milliseconds is not
unusual)".

Right?  In fact, it's for this reason that Clear Containers uses kvmtool
instead of QEMU.

No, wrong!  In fact, reporting bad performance is pretty much the same
as throwing down the gauntlet.

Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
a slimmed-down QEMU configuration, boots a virtual machine in 40
milliseconds[2] on an Ivy Bridge Core i7 processor.

qboot is available at git://github.com/bonzini/qboot.git.  In all the
glory of its 8KB of code, it brings together various existing open
source components:

* a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS

* a couple hardware initialization routines written mostly from scratch
but with good help from SeaBIOS source code

* a minimal 32-bit libc based on kvm-unit-tests

* the Linux loader from QEMU itself

The repository has more information on how to achieve fast boot times,
and examples of using qboot.  Right now there is a limit of 8 MB for
vmlinuz+initrd+cmdline, which however should be enough for initrd-less
containers.

The first commit to qboot is more or less 24 hours old, so there is
definitely more work to do, in particular to extract ACPI tables from
QEMU and present them to the guest.  This is probably another day of
work or so, and it will enable multiprocessor guests with little or no
impact on the boot times.  SMBIOS information is also available from QEMU.

On the QEMU side, there is no support yet for persistent memory and the
NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
will automatically start using it.

Happy hacking!

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 13:51 ` [Qemu-devel] " Paolo Bonzini
  (?)
@ 2015-05-21 15:48 ` Avi Kivity
  2015-05-21 16:21   ` Paolo Bonzini
  -1 siblings, 1 reply; 50+ messages in thread
From: Avi Kivity @ 2015-05-21 15:48 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel, KVM list

On 05/21/2015 04:51 PM, Paolo Bonzini wrote:
> Some of you may have heard about the "Clear Containers" initiative from
> Intel, which couple KVM with various kernel tricks to create extremely
> lightweight virtual machines.  The experimental Clear Containers setup
> requires only 18-20 MB to launch a virtual machine, and needs about 60
> ms to boot.
>
> Now, as all of you probably know, "QEMU is great for running Windows or
> legacy Linux guests, but that flexibility comes at a hefty price. Not
> only does all of the emulation consume memory, it also requires some
> form of low-level firmware in the guest as well. All of this adds quite
> a bit to virtual-machine startup times (500 to 700 milliseconds is not
> unusual)".
>
> Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> instead of QEMU.
>
> No, wrong!  In fact, reporting bad performance is pretty much the same
> as throwing down the gauntlet.
>
> Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
> a slimmed-down QEMU configuration, boots a virtual machine in 40
> milliseconds[2] on an Ivy Bridge Core i7 processor.
>
> qboot is available at git://github.com/bonzini/qboot.git.  In all the
> glory of its 8KB of code, it brings together various existing open
> source components:
>
> * a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS
>
> * a couple hardware initialization routines written mostly from scratch
> but with good help from SeaBIOS source code
>
> * a minimal 32-bit libc based on kvm-unit-tests
>
> * the Linux loader from QEMU itself
>
> The repository has more information on how to achieve fast boot times,
> and examples of using qboot.  Right now there is a limit of 8 MB for
> vmlinuz+initrd+cmdline, which however should be enough for initrd-less
> containers.
>
> The first commit to qboot is more or less 24 hours old, so there is
> definitely more work to do, in particular to extract ACPI tables from
> QEMU and present them to the guest.  This is probably another day of
> work or so, and it will enable multiprocessor guests with little or no
> impact on the boot times.  SMBIOS information is also available from QEMU.
>
> On the QEMU side, there is no support yet for persistent memory and the
> NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
> will automatically start using it.
>
> Happy hacking!
>

Lovely!

Note you have memcpy.o instead of memcpy.c.

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 15:48 ` Avi Kivity
@ 2015-05-21 16:21   ` Paolo Bonzini
  2015-05-21 18:28     ` Avi Kivity
  0 siblings, 1 reply; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-21 16:21 UTC (permalink / raw)
  To: Avi Kivity, qemu-devel, KVM list



On 21/05/2015 17:48, Avi Kivity wrote:
> Lovely!
> 
> Note you have memcpy.o instead of memcpy.c.

Doh, and it's not used anyway.  Check the repository, and let me know if
OSv boots with it (it probably needs ACPI; Linux doesn't boot virtio
without ACPI).

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 13:51 ` [Qemu-devel] " Paolo Bonzini
@ 2015-05-21 17:04   ` Jan Kiszka
  -1 siblings, 0 replies; 50+ messages in thread
From: Jan Kiszka @ 2015-05-21 17:04 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel, KVM list

On 2015-05-21 15:51, Paolo Bonzini wrote:
> Some of you may have heard about the "Clear Containers" initiative from
> Intel, which couple KVM with various kernel tricks to create extremely
> lightweight virtual machines.  The experimental Clear Containers setup
> requires only 18-20 MB to launch a virtual machine, and needs about 60
> ms to boot.
> 
> Now, as all of you probably know, "QEMU is great for running Windows or
> legacy Linux guests, but that flexibility comes at a hefty price. Not
> only does all of the emulation consume memory, it also requires some
> form of low-level firmware in the guest as well. All of this adds quite
> a bit to virtual-machine startup times (500 to 700 milliseconds is not
> unusual)".
> 
> Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> instead of QEMU.
> 
> No, wrong!  In fact, reporting bad performance is pretty much the same
> as throwing down the gauntlet.
> 
> Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
> a slimmed-down QEMU configuration, boots a virtual machine in 40
> milliseconds[2] on an Ivy Bridge Core i7 processor.
> 
> qboot is available at git://github.com/bonzini/qboot.git.  In all the
> glory of its 8KB of code, it brings together various existing open
> source components:
> 
> * a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS
> 
> * a couple hardware initialization routines written mostly from scratch
> but with good help from SeaBIOS source code
> 
> * a minimal 32-bit libc based on kvm-unit-tests
> 
> * the Linux loader from QEMU itself
> 
> The repository has more information on how to achieve fast boot times,
> and examples of using qboot.  Right now there is a limit of 8 MB for
> vmlinuz+initrd+cmdline, which however should be enough for initrd-less
> containers.
> 
> The first commit to qboot is more or less 24 hours old, so there is
> definitely more work to do, in particular to extract ACPI tables from
> QEMU and present them to the guest.  This is probably another day of
> work or so, and it will enable multiprocessor guests with little or no
> impact on the boot times.  SMBIOS information is also available from QEMU.
> 
> On the QEMU side, there is no support yet for persistent memory and the
> NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
> will automatically start using it.
> 
> Happy hacking!

Incidentally, I did something similar these days to get Linux booting in
Jailhouse non-root cells, i.e without BIOS and almost no hardware except
memory, cpus and pci devices. Yes, requires a bit pv for Linux, but
really little. Not aiming for speed (yet), just for less hypervisor
work. Maybe there are some milliseconds to save when cutting off more
hardware in an analogous way...

PV pat^Whacks are here:
http://git.kiszka.org/?p=linux.git;a=shortlog;h=refs/heads/queues/jailhouse.
The boot loader is a combination of a python script [1] (result can be
saved and reused - replaces ACPI) and really few lines of code [2][3].

Jan

[1]
https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/tools/jailhouse-cell-linux
[2]
https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/inmates/lib/x86/header.S
[3]
https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/inmates/tools/x86/linux-loader.c

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-21 17:04   ` Jan Kiszka
  0 siblings, 0 replies; 50+ messages in thread
From: Jan Kiszka @ 2015-05-21 17:04 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel, KVM list

On 2015-05-21 15:51, Paolo Bonzini wrote:
> Some of you may have heard about the "Clear Containers" initiative from
> Intel, which couple KVM with various kernel tricks to create extremely
> lightweight virtual machines.  The experimental Clear Containers setup
> requires only 18-20 MB to launch a virtual machine, and needs about 60
> ms to boot.
> 
> Now, as all of you probably know, "QEMU is great for running Windows or
> legacy Linux guests, but that flexibility comes at a hefty price. Not
> only does all of the emulation consume memory, it also requires some
> form of low-level firmware in the guest as well. All of this adds quite
> a bit to virtual-machine startup times (500 to 700 milliseconds is not
> unusual)".
> 
> Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> instead of QEMU.
> 
> No, wrong!  In fact, reporting bad performance is pretty much the same
> as throwing down the gauntlet.
> 
> Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
> a slimmed-down QEMU configuration, boots a virtual machine in 40
> milliseconds[2] on an Ivy Bridge Core i7 processor.
> 
> qboot is available at git://github.com/bonzini/qboot.git.  In all the
> glory of its 8KB of code, it brings together various existing open
> source components:
> 
> * a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS
> 
> * a couple hardware initialization routines written mostly from scratch
> but with good help from SeaBIOS source code
> 
> * a minimal 32-bit libc based on kvm-unit-tests
> 
> * the Linux loader from QEMU itself
> 
> The repository has more information on how to achieve fast boot times,
> and examples of using qboot.  Right now there is a limit of 8 MB for
> vmlinuz+initrd+cmdline, which however should be enough for initrd-less
> containers.
> 
> The first commit to qboot is more or less 24 hours old, so there is
> definitely more work to do, in particular to extract ACPI tables from
> QEMU and present them to the guest.  This is probably another day of
> work or so, and it will enable multiprocessor guests with little or no
> impact on the boot times.  SMBIOS information is also available from QEMU.
> 
> On the QEMU side, there is no support yet for persistent memory and the
> NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
> will automatically start using it.
> 
> Happy hacking!

Incidentally, I did something similar these days to get Linux booting in
Jailhouse non-root cells, i.e without BIOS and almost no hardware except
memory, cpus and pci devices. Yes, requires a bit pv for Linux, but
really little. Not aiming for speed (yet), just for less hypervisor
work. Maybe there are some milliseconds to save when cutting off more
hardware in an analogous way...

PV pat^Whacks are here:
http://git.kiszka.org/?p=linux.git;a=shortlog;h=refs/heads/queues/jailhouse.
The boot loader is a combination of a python script [1] (result can be
saved and reused - replaces ACPI) and really few lines of code [2][3].

Jan

[1]
https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/tools/jailhouse-cell-linux
[2]
https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/inmates/lib/x86/header.S
[3]
https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/inmates/tools/x86/linux-loader.c

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 16:21   ` Paolo Bonzini
@ 2015-05-21 18:28     ` Avi Kivity
  0 siblings, 0 replies; 50+ messages in thread
From: Avi Kivity @ 2015-05-21 18:28 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel, KVM list

On 05/21/2015 07:21 PM, Paolo Bonzini wrote:
>
> On 21/05/2015 17:48, Avi Kivity wrote:
>> Lovely!
>>
>> Note you have memcpy.o instead of memcpy.c.
> Doh, and it's not used anyway.  Check the repository, and let me know if
> OSv boots with it (it probably needs ACPI; Linux doesn't boot virtio
> without ACPI).
>

Yes, it requires ACPI.  We don't implement the pre-ACPI bootstrap methods.

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 13:51 ` [Qemu-devel] " Paolo Bonzini
@ 2015-05-22  2:53   ` Yong Wang
  -1 siblings, 0 replies; 50+ messages in thread
From: Yong Wang @ 2015-05-22  2:53 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> On the QEMU side, there is no support yet for persistent memory and the
> NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
> will automatically start using it.
> 

We are working on adding NFIT support into virtual bios.


^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-22  2:53   ` Yong Wang
  0 siblings, 0 replies; 50+ messages in thread
From: Yong Wang @ 2015-05-22  2:53 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> On the QEMU side, there is no support yet for persistent memory and the
> NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
> will automatically start using it.
> 

We are working on adding NFIT support into virtual bios.

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 13:51 ` [Qemu-devel] " Paolo Bonzini
                   ` (3 preceding siblings ...)
  (?)
@ 2015-05-22 11:01 ` Daniel P. Berrange
  2015-05-22 11:04   ` Peter Maydell
  -1 siblings, 1 reply; 50+ messages in thread
From: Daniel P. Berrange @ 2015-05-22 11:01 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> Some of you may have heard about the "Clear Containers" initiative from
> Intel, which couple KVM with various kernel tricks to create extremely
> lightweight virtual machines.  The experimental Clear Containers setup
> requires only 18-20 MB to launch a virtual machine, and needs about 60
> ms to boot.
> 
> Now, as all of you probably know, "QEMU is great for running Windows or
> legacy Linux guests, but that flexibility comes at a hefty price. Not
> only does all of the emulation consume memory, it also requires some
> form of low-level firmware in the guest as well. All of this adds quite
> a bit to virtual-machine startup times (500 to 700 milliseconds is not
> unusual)".
> 
> Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> instead of QEMU.
> 
> No, wrong!  In fact, reporting bad performance is pretty much the same
> as throwing down the gauntlet.

On the QEMU side of things I wonder if there is scope for taking AArch64's
'virt' machine type concept and duplicating it on all architectures. It
would be nice to have a common minimal machine type on all architectures
that discards all legacy platform stuff and focuses on the minimum needed
to run modern virtual machine optimized guest OS. People would always know
that a machine type called 'virt' was the minimal virtualization platform,
while the others all target emulation of realworld (legacy) baremetal
platforms.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 11:01 ` Daniel P. Berrange
@ 2015-05-22 11:04   ` Peter Maydell
  2015-05-22 11:12     ` Daniel P. Berrange
  0 siblings, 1 reply; 50+ messages in thread
From: Peter Maydell @ 2015-05-22 11:04 UTC (permalink / raw)
  To: Daniel P. Berrange; +Cc: Paolo Bonzini, qemu-devel, KVM list

On 22 May 2015 at 12:01, Daniel P. Berrange <berrange@redhat.com> wrote:
> On the QEMU side of things I wonder if there is scope for taking AArch64's
> 'virt' machine type concept and duplicating it on all architectures.

Experience suggests that holding the line on "minimal" is really
quite tricky, though -- there's always one more thing that
somebody really wants to add...

-- PMM

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 11:04   ` Peter Maydell
@ 2015-05-22 11:12     ` Daniel P. Berrange
  2015-05-22 11:21       ` Peter Maydell
  2015-05-25 12:53       ` Paolo Bonzini
  0 siblings, 2 replies; 50+ messages in thread
From: Daniel P. Berrange @ 2015-05-22 11:12 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Paolo Bonzini, qemu-devel, KVM list

On Fri, May 22, 2015 at 12:04:54PM +0100, Peter Maydell wrote:
> On 22 May 2015 at 12:01, Daniel P. Berrange <berrange@redhat.com> wrote:
> > On the QEMU side of things I wonder if there is scope for taking AArch64's
> > 'virt' machine type concept and duplicating it on all architectures.
> 
> Experience suggests that holding the line on "minimal" is really
> quite tricky, though -- there's always one more thing that
> somebody really wants to add...

Yep, it is hard saying no - but I'd think as long as it was possible to add
the extra features using -device, it ought to be practical to keep a "virt"
machine types "-nodefaults -nodefconfig" base setup pretty minimal. In
particular I don't see why we need to have a SATA controller and ISA/LPC
bridge in every virt machine - root PCI bus only should be possible, as you
can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
floppy via PCI devices and/or by adding a USB bus in the cases where you
really need one.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 11:12     ` Daniel P. Berrange
@ 2015-05-22 11:21       ` Peter Maydell
  2015-05-22 11:33         ` Daniel P. Berrange
                           ` (2 more replies)
  2015-05-25 12:53       ` Paolo Bonzini
  1 sibling, 3 replies; 50+ messages in thread
From: Peter Maydell @ 2015-05-22 11:21 UTC (permalink / raw)
  To: Daniel P. Berrange; +Cc: Paolo Bonzini, qemu-devel, KVM list

On 22 May 2015 at 12:12, Daniel P. Berrange <berrange@redhat.com> wrote:
> Yep, it is hard saying no - but I'd think as long as it was possible to add
> the extra features using -device, it ought to be practical to keep a "virt"
> machine types "-nodefaults -nodefconfig" base setup pretty minimal.

Mmm, but -device only works for pluggable devices really. We don't
have a coherent mechanism for saying "put the PS/2 keyboard controller
into the system at its usual IO ports" on the command line.

-- PMM

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 11:21       ` Peter Maydell
@ 2015-05-22 11:33         ` Daniel P. Berrange
  2015-05-22 11:34           ` Gerd Hoffmann
  2015-05-22 11:42           ` Markus Armbruster
  2 siblings, 0 replies; 50+ messages in thread
From: Daniel P. Berrange @ 2015-05-22 11:33 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Paolo Bonzini, qemu-devel, KVM list

On Fri, May 22, 2015 at 12:21:27PM +0100, Peter Maydell wrote:
> On 22 May 2015 at 12:12, Daniel P. Berrange <berrange@redhat.com> wrote:
> > Yep, it is hard saying no - but I'd think as long as it was possible to add
> > the extra features using -device, it ought to be practical to keep a "virt"
> > machine types "-nodefaults -nodefconfig" base setup pretty minimal.
> 
> Mmm, but -device only works for pluggable devices really. We don't
> have a coherent mechanism for saying "put the PS/2 keyboard controller
> into the system at its usual IO ports" on the command line.

Oh, I didn't neccessarily mean that we'd need the ability to add a
ps/2 keyboard via -device. I meant that there just need to be able
to add /some/ kind of keyboard. eg we have a usb-kbd device that
could potentially fill that role. Likewise for mouse pointer. Serial
ports, etc.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 11:21       ` Peter Maydell
@ 2015-05-22 11:34           ` Gerd Hoffmann
  2015-05-22 11:34           ` Gerd Hoffmann
  2015-05-22 11:42           ` Markus Armbruster
  2 siblings, 0 replies; 50+ messages in thread
From: Gerd Hoffmann @ 2015-05-22 11:34 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Daniel P. Berrange, Paolo Bonzini, qemu-devel, KVM list

On Fr, 2015-05-22 at 12:21 +0100, Peter Maydell wrote:
> On 22 May 2015 at 12:12, Daniel P. Berrange <berrange@redhat.com> wrote:
> > Yep, it is hard saying no - but I'd think as long as it was possible to add
> > the extra features using -device, it ought to be practical to keep a "virt"
> > machine types "-nodefaults -nodefconfig" base setup pretty minimal.
> 
> Mmm, but -device only works for pluggable devices really. We don't
> have a coherent mechanism for saying "put the PS/2 keyboard controller
> into the system at its usual IO ports" on the command line.

Do we need that in the first place?
You can plugin a usb keyboard today.
You'll be able to plugin a virtio keyboard soon.

I think alot of hardware where this applies to is the legacy stuff we
want to get rid of for '-M virt' ...

cheers,
  Gerd



^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-22 11:34           ` Gerd Hoffmann
  0 siblings, 0 replies; 50+ messages in thread
From: Gerd Hoffmann @ 2015-05-22 11:34 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Paolo Bonzini, qemu-devel, KVM list

On Fr, 2015-05-22 at 12:21 +0100, Peter Maydell wrote:
> On 22 May 2015 at 12:12, Daniel P. Berrange <berrange@redhat.com> wrote:
> > Yep, it is hard saying no - but I'd think as long as it was possible to add
> > the extra features using -device, it ought to be practical to keep a "virt"
> > machine types "-nodefaults -nodefconfig" base setup pretty minimal.
> 
> Mmm, but -device only works for pluggable devices really. We don't
> have a coherent mechanism for saying "put the PS/2 keyboard controller
> into the system at its usual IO ports" on the command line.

Do we need that in the first place?
You can plugin a usb keyboard today.
You'll be able to plugin a virtio keyboard soon.

I think alot of hardware where this applies to is the legacy stuff we
want to get rid of for '-M virt' ...

cheers,
  Gerd

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 11:21       ` Peter Maydell
@ 2015-05-22 11:42           ` Markus Armbruster
  2015-05-22 11:34           ` Gerd Hoffmann
  2015-05-22 11:42           ` Markus Armbruster
  2 siblings, 0 replies; 50+ messages in thread
From: Markus Armbruster @ 2015-05-22 11:42 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Daniel P. Berrange, Paolo Bonzini, qemu-devel, KVM list

Peter Maydell <peter.maydell@linaro.org> writes:

> On 22 May 2015 at 12:12, Daniel P. Berrange <berrange@redhat.com> wrote:
>> Yep, it is hard saying no - but I'd think as long as it was possible to add
>> the extra features using -device, it ought to be practical to keep a "virt"
>> machine types "-nodefaults -nodefconfig" base setup pretty minimal.
>
> Mmm, but -device only works for pluggable devices really. We don't
> have a coherent mechanism for saying "put the PS/2 keyboard controller
> into the system at its usual IO ports" on the command line.

... yet.

The qdev long-term vision has always been to support starting with an
empty board, then add device models and their wiring.  Unfortunately,
progress towards that goal has been slow.

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-22 11:42           ` Markus Armbruster
  0 siblings, 0 replies; 50+ messages in thread
From: Markus Armbruster @ 2015-05-22 11:42 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Paolo Bonzini, qemu-devel, KVM list

Peter Maydell <peter.maydell@linaro.org> writes:

> On 22 May 2015 at 12:12, Daniel P. Berrange <berrange@redhat.com> wrote:
>> Yep, it is hard saying no - but I'd think as long as it was possible to add
>> the extra features using -device, it ought to be practical to keep a "virt"
>> machine types "-nodefaults -nodefconfig" base setup pretty minimal.
>
> Mmm, but -device only works for pluggable devices really. We don't
> have a coherent mechanism for saying "put the PS/2 keyboard controller
> into the system at its usual IO ports" on the command line.

... yet.

The qdev long-term vision has always been to support starting with an
empty board, then add device models and their wiring.  Unfortunately,
progress towards that goal has been slow.

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 13:51 ` [Qemu-devel] " Paolo Bonzini
                   ` (4 preceding siblings ...)
  (?)
@ 2015-05-22 14:06 ` Gerd Hoffmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Gerd Hoffmann @ 2015-05-22 14:06 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

  Hi,

> qboot is available at git://github.com/bonzini/qboot.git.

Firmware repo has packages now.

https://www.kraxel.org/repos/firmware.repo
https://www.kraxel.org/repos/jenkins/qboot/

enjoy,
  Gerd



^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-21 13:51 ` [Qemu-devel] " Paolo Bonzini
@ 2015-05-22 23:23   ` Kevin O'Connor
  -1 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-22 23:23 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> Some of you may have heard about the "Clear Containers" initiative from
> Intel, which couple KVM with various kernel tricks to create extremely
> lightweight virtual machines.  The experimental Clear Containers setup
> requires only 18-20 MB to launch a virtual machine, and needs about 60
> ms to boot.
> 
> Now, as all of you probably know, "QEMU is great for running Windows or
> legacy Linux guests, but that flexibility comes at a hefty price. Not
> only does all of the emulation consume memory, it also requires some
> form of low-level firmware in the guest as well. All of this adds quite
> a bit to virtual-machine startup times (500 to 700 milliseconds is not
> unusual)".
> 
> Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> instead of QEMU.
> 
> No, wrong!  In fact, reporting bad performance is pretty much the same
> as throwing down the gauntlet.
> 
> Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
> a slimmed-down QEMU configuration, boots a virtual machine in 40
> milliseconds[2] on an Ivy Bridge Core i7 processor.

Hi Paolo,

I'm curious if you've tried profiling SeaBIOS to see where it is
spending unnecessary time?  I wonder if a stripped down SeaBIOS could
obtain sufficient performance.

The page at http://seabios.org/Debugging#Timing_debug_messages
describes how to do basic profiling via timing of debug messages.

The default SeaBIOS build takes ~180ms on my (old AMD) system.  But,
by removing drivers and options via Kconfig I was able to bring it
down to ~25ms.  I suspect some additional Kconfig settings and a few
optimizations would make it possible to significantly reduce this
time.

Cheers,
-Kevin

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-22 23:23   ` Kevin O'Connor
  0 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-22 23:23 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> Some of you may have heard about the "Clear Containers" initiative from
> Intel, which couple KVM with various kernel tricks to create extremely
> lightweight virtual machines.  The experimental Clear Containers setup
> requires only 18-20 MB to launch a virtual machine, and needs about 60
> ms to boot.
> 
> Now, as all of you probably know, "QEMU is great for running Windows or
> legacy Linux guests, but that flexibility comes at a hefty price. Not
> only does all of the emulation consume memory, it also requires some
> form of low-level firmware in the guest as well. All of this adds quite
> a bit to virtual-machine startup times (500 to 700 milliseconds is not
> unusual)".
> 
> Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> instead of QEMU.
> 
> No, wrong!  In fact, reporting bad performance is pretty much the same
> as throwing down the gauntlet.
> 
> Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
> a slimmed-down QEMU configuration, boots a virtual machine in 40
> milliseconds[2] on an Ivy Bridge Core i7 processor.

Hi Paolo,

I'm curious if you've tried profiling SeaBIOS to see where it is
spending unnecessary time?  I wonder if a stripped down SeaBIOS could
obtain sufficient performance.

The page at http://seabios.org/Debugging#Timing_debug_messages
describes how to do basic profiling via timing of debug messages.

The default SeaBIOS build takes ~180ms on my (old AMD) system.  But,
by removing drivers and options via Kconfig I was able to bring it
down to ~25ms.  I suspect some additional Kconfig settings and a few
optimizations would make it possible to significantly reduce this
time.

Cheers,
-Kevin

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 23:23   ` [Qemu-devel] " Kevin O'Connor
@ 2015-05-23  3:55     ` Kevin O'Connor
  -1 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-23  3:55 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Fri, May 22, 2015 at 07:23:27PM -0400, Kevin O'Connor wrote:
> On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> > Some of you may have heard about the "Clear Containers" initiative from
> > Intel, which couple KVM with various kernel tricks to create extremely
> > lightweight virtual machines.  The experimental Clear Containers setup
> > requires only 18-20 MB to launch a virtual machine, and needs about 60
> > ms to boot.
> > 
> > Now, as all of you probably know, "QEMU is great for running Windows or
> > legacy Linux guests, but that flexibility comes at a hefty price. Not
> > only does all of the emulation consume memory, it also requires some
> > form of low-level firmware in the guest as well. All of this adds quite
> > a bit to virtual-machine startup times (500 to 700 milliseconds is not
> > unusual)".
> > 
> > Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> > instead of QEMU.
> > 
> > No, wrong!  In fact, reporting bad performance is pretty much the same
> > as throwing down the gauntlet.
> > 
> > Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
> > a slimmed-down QEMU configuration, boots a virtual machine in 40
> > milliseconds[2] on an Ivy Bridge Core i7 processor.
> 
> Hi Paolo,
> 
> I'm curious if you've tried profiling SeaBIOS to see where it is
> spending unnecessary time?  I wonder if a stripped down SeaBIOS could
> obtain sufficient performance.
> 
> The page at http://seabios.org/Debugging#Timing_debug_messages
> describes how to do basic profiling via timing of debug messages.
> 
> The default SeaBIOS build takes ~180ms on my (old AMD) system.  But,
> by removing drivers and options via Kconfig I was able to bring it
> down to ~25ms.  I suspect some additional Kconfig settings and a few
> optimizations would make it possible to significantly reduce this
> time.

Out of curiosity, I ran some additional timing tests.  With SeaBIOS
fully stripped down (via Kconfig), it takes ~20ms to get to the boot
phase on my old AMD system.  Of that 20ms, ~7ms is to enable shadow
ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci
init, and ~6ms is to make the shadow ram area read-only.  The time in
the remaining parts of the SeaBIOS code is so small that it's hard to
measure.

The above delays could be removed without much work:  It would be
trivial to add kconfig options to support disabling the pci init and
shadow read-only functionality.  The cpu timestamp calibrations could
be reworked so that the existence of the ACPI timer was checked first
so as to avoid that work.  The enabling of shadow ram could also be
bypassed if QEMU was changed to start with a ram based shadow instead
of a rom shadow (since QEMU doesn't fully implement the chipset shadow
registers, it's really pointless to start in rom shadow mode anyway).

-Kevin

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-23  3:55     ` Kevin O'Connor
  0 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-23  3:55 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Fri, May 22, 2015 at 07:23:27PM -0400, Kevin O'Connor wrote:
> On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> > Some of you may have heard about the "Clear Containers" initiative from
> > Intel, which couple KVM with various kernel tricks to create extremely
> > lightweight virtual machines.  The experimental Clear Containers setup
> > requires only 18-20 MB to launch a virtual machine, and needs about 60
> > ms to boot.
> > 
> > Now, as all of you probably know, "QEMU is great for running Windows or
> > legacy Linux guests, but that flexibility comes at a hefty price. Not
> > only does all of the emulation consume memory, it also requires some
> > form of low-level firmware in the guest as well. All of this adds quite
> > a bit to virtual-machine startup times (500 to 700 milliseconds is not
> > unusual)".
> > 
> > Right?  In fact, it's for this reason that Clear Containers uses kvmtool
> > instead of QEMU.
> > 
> > No, wrong!  In fact, reporting bad performance is pretty much the same
> > as throwing down the gauntlet.
> > 
> > Enter qboot, a minimal x86 firmware that runs on QEMU and, together with
> > a slimmed-down QEMU configuration, boots a virtual machine in 40
> > milliseconds[2] on an Ivy Bridge Core i7 processor.
> 
> Hi Paolo,
> 
> I'm curious if you've tried profiling SeaBIOS to see where it is
> spending unnecessary time?  I wonder if a stripped down SeaBIOS could
> obtain sufficient performance.
> 
> The page at http://seabios.org/Debugging#Timing_debug_messages
> describes how to do basic profiling via timing of debug messages.
> 
> The default SeaBIOS build takes ~180ms on my (old AMD) system.  But,
> by removing drivers and options via Kconfig I was able to bring it
> down to ~25ms.  I suspect some additional Kconfig settings and a few
> optimizations would make it possible to significantly reduce this
> time.

Out of curiosity, I ran some additional timing tests.  With SeaBIOS
fully stripped down (via Kconfig), it takes ~20ms to get to the boot
phase on my old AMD system.  Of that 20ms, ~7ms is to enable shadow
ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci
init, and ~6ms is to make the shadow ram area read-only.  The time in
the remaining parts of the SeaBIOS code is so small that it's hard to
measure.

The above delays could be removed without much work:  It would be
trivial to add kconfig options to support disabling the pci init and
shadow read-only functionality.  The cpu timestamp calibrations could
be reworked so that the existence of the ACPI timer was checked first
so as to avoid that work.  The enabling of shadow ram could also be
bypassed if QEMU was changed to start with a ram based shadow instead
of a rom shadow (since QEMU doesn't fully implement the chipset shadow
registers, it's really pointless to start in rom shadow mode anyway).

-Kevin

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-23  3:55     ` [Qemu-devel] " Kevin O'Connor
@ 2015-05-25  6:21       ` Vasiliy Tolstov
  -1 siblings, 0 replies; 50+ messages in thread
From: Vasiliy Tolstov @ 2015-05-25  6:21 UTC (permalink / raw)
  To: Kevin O'Connor; +Cc: Paolo Bonzini, qemu-devel, KVM list

2015-05-23 6:55 GMT+03:00 Kevin O'Connor <kevin@koconnor.net>:
> Out of curiosity, I ran some additional timing tests.  With SeaBIOS
> fully stripped down (via Kconfig), it takes ~20ms to get to the boot
> phase on my old AMD system.  Of that 20ms, ~7ms is to enable shadow
> ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci
> init, and ~6ms is to make the shadow ram area read-only.  The time in
> the remaining parts of the SeaBIOS code is so small that it's hard to
> measure.


Can you share config for seabios? As i understand i can safety to
remove keybord, ps2, usb, ata/ahci and leave only virtio (in case of
using qemu to boo linux/freebsd/windows systems) ?

-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-25  6:21       ` Vasiliy Tolstov
  0 siblings, 0 replies; 50+ messages in thread
From: Vasiliy Tolstov @ 2015-05-25  6:21 UTC (permalink / raw)
  To: Kevin O'Connor; +Cc: Paolo Bonzini, qemu-devel, KVM list

2015-05-23 6:55 GMT+03:00 Kevin O'Connor <kevin@koconnor.net>:
> Out of curiosity, I ran some additional timing tests.  With SeaBIOS
> fully stripped down (via Kconfig), it takes ~20ms to get to the boot
> phase on my old AMD system.  Of that 20ms, ~7ms is to enable shadow
> ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci
> init, and ~6ms is to make the shadow ram area read-only.  The time in
> the remaining parts of the SeaBIOS code is so small that it's hard to
> measure.


Can you share config for seabios? As i understand i can safety to
remove keybord, ps2, usb, ata/ahci and leave only virtio (in case of
using qemu to boo linux/freebsd/windows systems) ?

-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 23:23   ` [Qemu-devel] " Kevin O'Connor
@ 2015-05-25 12:52     ` Paolo Bonzini
  -1 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-25 12:52 UTC (permalink / raw)
  To: Kevin O'Connor; +Cc: qemu-devel, KVM list



On 23/05/2015 01:23, Kevin O'Connor wrote:
> Hi Paolo,
> 
> I'm curious if you've tried profiling SeaBIOS to see where it is
> spending unnecessary time?

No, I really wanted to get the absolute minimum time needed to get to
the kernel.  I announced it publicly because I think it's also instructive.

> I wonder if a stripped down SeaBIOS could
> obtain sufficient performance.

It most likely could.  However, some of the features (e.g. booting
vmlinuz from parallel flash or, in the future, from persistent memory)
probably do not fit SeaBIOS very well.  One would have to redo them
otherwise, for example using a DMA interface in fw_cfg.

Paolo

> The page at http://seabios.org/Debugging#Timing_debug_messages
> describes how to do basic profiling via timing of debug messages.
> 
> The default SeaBIOS build takes ~180ms on my (old AMD) system.  But,
> by removing drivers and options via Kconfig I was able to bring it
> down to ~25ms.  I suspect some additional Kconfig settings and a few
> optimizations would make it possible to significantly reduce this
> time.
> 
> Cheers,
> -Kevin
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-25 12:52     ` Paolo Bonzini
  0 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-25 12:52 UTC (permalink / raw)
  To: Kevin O'Connor; +Cc: qemu-devel, KVM list



On 23/05/2015 01:23, Kevin O'Connor wrote:
> Hi Paolo,
> 
> I'm curious if you've tried profiling SeaBIOS to see where it is
> spending unnecessary time?

No, I really wanted to get the absolute minimum time needed to get to
the kernel.  I announced it publicly because I think it's also instructive.

> I wonder if a stripped down SeaBIOS could
> obtain sufficient performance.

It most likely could.  However, some of the features (e.g. booting
vmlinuz from parallel flash or, in the future, from persistent memory)
probably do not fit SeaBIOS very well.  One would have to redo them
otherwise, for example using a DMA interface in fw_cfg.

Paolo

> The page at http://seabios.org/Debugging#Timing_debug_messages
> describes how to do basic profiling via timing of debug messages.
> 
> The default SeaBIOS build takes ~180ms on my (old AMD) system.  But,
> by removing drivers and options via Kconfig I was able to bring it
> down to ~25ms.  I suspect some additional Kconfig settings and a few
> optimizations would make it possible to significantly reduce this
> time.
> 
> Cheers,
> -Kevin
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22 11:12     ` Daniel P. Berrange
  2015-05-22 11:21       ` Peter Maydell
@ 2015-05-25 12:53       ` Paolo Bonzini
  2015-05-26 21:25         ` Christopher Covington
  1 sibling, 1 reply; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-25 12:53 UTC (permalink / raw)
  To: Daniel P. Berrange, Peter Maydell; +Cc: qemu-devel, KVM list



On 22/05/2015 13:12, Daniel P. Berrange wrote:
> In
> particular I don't see why we need to have a SATA controller and ISA/LPC
> bridge in every virt machine - root PCI bus only should be possible, as you
> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
> floppy via PCI devices and/or by adding a USB bus in the cases where you
> really need one.

I think removing the ISA/LPC bridge is hard.  It includes the real-time
clock and fw_cfg, for example.

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-25  6:21       ` [Qemu-devel] " Vasiliy Tolstov
@ 2015-05-25 15:05         ` Kevin O'Connor
  -1 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-25 15:05 UTC (permalink / raw)
  To: Vasiliy Tolstov; +Cc: Paolo Bonzini, qemu-devel, KVM list

[-- Attachment #1: Type: text/plain, Size: 2779 bytes --]

On Mon, May 25, 2015 at 09:21:49AM +0300, Vasiliy Tolstov wrote:
> 2015-05-23 6:55 GMT+03:00 Kevin O'Connor <kevin@koconnor.net>:
> > Out of curiosity, I ran some additional timing tests.  With SeaBIOS
> > fully stripped down (via Kconfig), it takes ~20ms to get to the boot
> > phase on my old AMD system.  Of that 20ms, ~7ms is to enable shadow
> > ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci
> > init, and ~6ms is to make the shadow ram area read-only.  The time in
> > the remaining parts of the SeaBIOS code is so small that it's hard to
> > measure.
> 
> Can you share config for seabios? As i understand i can safety to
> remove keybord, ps2, usb, ata/ahci and leave only virtio (in case of
> using qemu to boo linux/freebsd/windows systems) ?

To get to 20ms, I basically disabled everything.  (Actually, after
sending my last email I found I cound get to 16ms by setting
CONFIG_WRITABLE_UPPERMEMORY=y - it avoids much of the shadow ram
read-only cost.)  It's not possible to actually boot anything with
this config - I was instead timing to the point where one could add a
multiboot or vmlinux boot capability to SeaBIOS.

To get to these times I disabled debug messages themselves (they take
a few ms of the boot time).  So, to time events without debug messages
I added a few outb() calls to seabios at critical points.  See patch
and config below.

-Kevin


--- a/src/post.c
+++ b/src/post.c
@@ -167,11 +167,15 @@ platform_hardware_setup(void)
     // Init base pc hardware.
     pic_setup();
     mathcp_setup();
+    outb('3', 0x402); outb('\n', 0x402);
     timer_setup();
+    outb('4', 0x402); outb('\n', 0x402);
     clock_setup();
 
     // Platform specific setup
+    outb('5', 0x402); outb('\n', 0x402);
     qemu_platform_setup();
+    outb('6', 0x402); outb('\n', 0x402);
     coreboot_platform_setup();
 }
 
@@ -200,6 +204,7 @@ startBoot(void)
     // Clear low-memory allocations (required by PMM spec).
     memset((void*)BUILD_STACK_ADDR, 0, BUILD_EBDA_MINIMUM - BUILD_STACK_ADDR);
 
+    outb('8', 0x402); outb('\n', 0x402);
     dprintf(3, "Jump to int19\n");
     struct bregs br;
     memset(&br, 0, sizeof(br));
@@ -247,6 +252,7 @@ maininit(void)
     prepareboot();
 
     // Write protect bios memory.
+    outb('7', 0x402); outb('\n', 0x402);
     make_bios_readonly();
 
     // Invoke int 19 to start boot process.
@@ -331,12 +337,14 @@ handle_post(void)
 
     serial_debug_preinit();
     debug_banner();
+    outb('1', 0x402); outb('\n', 0x402);
 
     // Check if we are running under Xen.
     xen_preinit();
 
     // Allow writes to modify bios area (0xf0000)
     make_bios_writable();
+    outb('2', 0x402); outb('\n', 0x402);
 
     // Now that memory is read/writable - start post process.
     dopost();



[-- Attachment #2: .config --]
[-- Type: text/plain, Size: 1735 bytes --]

#
# Automatically generated file; DO NOT EDIT.
# SeaBIOS Configuration
#

#
# General Features
#
# CONFIG_COREBOOT is not set
CONFIG_QEMU=y
# CONFIG_CSM is not set
CONFIG_QEMU_HARDWARE=y
# CONFIG_XEN is not set
# CONFIG_THREADS is not set
# CONFIG_RELOCATE_INIT is not set
# CONFIG_BOOTMENU is not set
# CONFIG_BOOTORDER is not set
CONFIG_ENTRY_EXTRASTACK=y
CONFIG_MALLOC_UPPERMEMORY=y
CONFIG_ROM_SIZE=0

#
# Hardware support
#
# CONFIG_ATA is not set
# CONFIG_AHCI is not set
# CONFIG_SDCARD is not set
# CONFIG_VIRTIO_BLK is not set
# CONFIG_VIRTIO_SCSI is not set
# CONFIG_PVSCSI is not set
# CONFIG_ESP_SCSI is not set
# CONFIG_LSI_SCSI is not set
# CONFIG_MEGASAS is not set
# CONFIG_FLOPPY is not set
# CONFIG_PS2PORT is not set
# CONFIG_USB is not set
# CONFIG_SERIAL is not set
# CONFIG_LPT is not set
# CONFIG_USE_SMM is not set
CONFIG_MTRR_INIT=y
CONFIG_PMTIMER=y

#
# BIOS interfaces
#
CONFIG_DRIVES=y
CONFIG_CDROM_BOOT=y
CONFIG_CDROM_EMU=y
CONFIG_PCIBIOS=y
CONFIG_APMBIOS=y
CONFIG_PNPBIOS=y
# CONFIG_OPTIONROMS is not set
CONFIG_BOOT=y
CONFIG_KEYBOARD=y
CONFIG_KBD_CALL_INT15_4F=y
CONFIG_MOUSE=y
CONFIG_S3_RESUME=y
# CONFIG_VGAHOOKS is not set
# CONFIG_DISABLE_A20 is not set
CONFIG_WRITABLE_UPPERMEMORY=y
# CONFIG_TCGBIOS is not set

#
# BIOS Tables
#
# CONFIG_PIRTABLE is not set
# CONFIG_MPTABLE is not set
# CONFIG_SMBIOS is not set
# CONFIG_ACPI is not set
# CONFIG_FW_ROMFILE_LOAD is not set

#
# VGA ROM
#
CONFIG_NO_VGABIOS=y
# CONFIG_VGA_STANDARD_VGA is not set
# CONFIG_VGA_CIRRUS is not set
# CONFIG_VGA_BOCHS is not set
# CONFIG_VGA_GEODEGX2 is not set
# CONFIG_VGA_GEODELX is not set
# CONFIG_BUILD_VGABIOS is not set
CONFIG_VGA_FIXUP_ASM=y
CONFIG_VGA_EXTRA_STACK_SIZE=512

#
# Debugging
#
CONFIG_DEBUG_LEVEL=0

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-25 15:05         ` Kevin O'Connor
  0 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-25 15:05 UTC (permalink / raw)
  To: Vasiliy Tolstov; +Cc: Paolo Bonzini, qemu-devel, KVM list

[-- Attachment #1: Type: text/plain, Size: 2779 bytes --]

On Mon, May 25, 2015 at 09:21:49AM +0300, Vasiliy Tolstov wrote:
> 2015-05-23 6:55 GMT+03:00 Kevin O'Connor <kevin@koconnor.net>:
> > Out of curiosity, I ran some additional timing tests.  With SeaBIOS
> > fully stripped down (via Kconfig), it takes ~20ms to get to the boot
> > phase on my old AMD system.  Of that 20ms, ~7ms is to enable shadow
> > ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci
> > init, and ~6ms is to make the shadow ram area read-only.  The time in
> > the remaining parts of the SeaBIOS code is so small that it's hard to
> > measure.
> 
> Can you share config for seabios? As i understand i can safety to
> remove keybord, ps2, usb, ata/ahci and leave only virtio (in case of
> using qemu to boo linux/freebsd/windows systems) ?

To get to 20ms, I basically disabled everything.  (Actually, after
sending my last email I found I cound get to 16ms by setting
CONFIG_WRITABLE_UPPERMEMORY=y - it avoids much of the shadow ram
read-only cost.)  It's not possible to actually boot anything with
this config - I was instead timing to the point where one could add a
multiboot or vmlinux boot capability to SeaBIOS.

To get to these times I disabled debug messages themselves (they take
a few ms of the boot time).  So, to time events without debug messages
I added a few outb() calls to seabios at critical points.  See patch
and config below.

-Kevin


--- a/src/post.c
+++ b/src/post.c
@@ -167,11 +167,15 @@ platform_hardware_setup(void)
     // Init base pc hardware.
     pic_setup();
     mathcp_setup();
+    outb('3', 0x402); outb('\n', 0x402);
     timer_setup();
+    outb('4', 0x402); outb('\n', 0x402);
     clock_setup();
 
     // Platform specific setup
+    outb('5', 0x402); outb('\n', 0x402);
     qemu_platform_setup();
+    outb('6', 0x402); outb('\n', 0x402);
     coreboot_platform_setup();
 }
 
@@ -200,6 +204,7 @@ startBoot(void)
     // Clear low-memory allocations (required by PMM spec).
     memset((void*)BUILD_STACK_ADDR, 0, BUILD_EBDA_MINIMUM - BUILD_STACK_ADDR);
 
+    outb('8', 0x402); outb('\n', 0x402);
     dprintf(3, "Jump to int19\n");
     struct bregs br;
     memset(&br, 0, sizeof(br));
@@ -247,6 +252,7 @@ maininit(void)
     prepareboot();
 
     // Write protect bios memory.
+    outb('7', 0x402); outb('\n', 0x402);
     make_bios_readonly();
 
     // Invoke int 19 to start boot process.
@@ -331,12 +337,14 @@ handle_post(void)
 
     serial_debug_preinit();
     debug_banner();
+    outb('1', 0x402); outb('\n', 0x402);
 
     // Check if we are running under Xen.
     xen_preinit();
 
     // Allow writes to modify bios area (0xf0000)
     make_bios_writable();
+    outb('2', 0x402); outb('\n', 0x402);
 
     // Now that memory is read/writable - start post process.
     dopost();



[-- Attachment #2: .config --]
[-- Type: text/plain, Size: 1735 bytes --]

#
# Automatically generated file; DO NOT EDIT.
# SeaBIOS Configuration
#

#
# General Features
#
# CONFIG_COREBOOT is not set
CONFIG_QEMU=y
# CONFIG_CSM is not set
CONFIG_QEMU_HARDWARE=y
# CONFIG_XEN is not set
# CONFIG_THREADS is not set
# CONFIG_RELOCATE_INIT is not set
# CONFIG_BOOTMENU is not set
# CONFIG_BOOTORDER is not set
CONFIG_ENTRY_EXTRASTACK=y
CONFIG_MALLOC_UPPERMEMORY=y
CONFIG_ROM_SIZE=0

#
# Hardware support
#
# CONFIG_ATA is not set
# CONFIG_AHCI is not set
# CONFIG_SDCARD is not set
# CONFIG_VIRTIO_BLK is not set
# CONFIG_VIRTIO_SCSI is not set
# CONFIG_PVSCSI is not set
# CONFIG_ESP_SCSI is not set
# CONFIG_LSI_SCSI is not set
# CONFIG_MEGASAS is not set
# CONFIG_FLOPPY is not set
# CONFIG_PS2PORT is not set
# CONFIG_USB is not set
# CONFIG_SERIAL is not set
# CONFIG_LPT is not set
# CONFIG_USE_SMM is not set
CONFIG_MTRR_INIT=y
CONFIG_PMTIMER=y

#
# BIOS interfaces
#
CONFIG_DRIVES=y
CONFIG_CDROM_BOOT=y
CONFIG_CDROM_EMU=y
CONFIG_PCIBIOS=y
CONFIG_APMBIOS=y
CONFIG_PNPBIOS=y
# CONFIG_OPTIONROMS is not set
CONFIG_BOOT=y
CONFIG_KEYBOARD=y
CONFIG_KBD_CALL_INT15_4F=y
CONFIG_MOUSE=y
CONFIG_S3_RESUME=y
# CONFIG_VGAHOOKS is not set
# CONFIG_DISABLE_A20 is not set
CONFIG_WRITABLE_UPPERMEMORY=y
# CONFIG_TCGBIOS is not set

#
# BIOS Tables
#
# CONFIG_PIRTABLE is not set
# CONFIG_MPTABLE is not set
# CONFIG_SMBIOS is not set
# CONFIG_ACPI is not set
# CONFIG_FW_ROMFILE_LOAD is not set

#
# VGA ROM
#
CONFIG_NO_VGABIOS=y
# CONFIG_VGA_STANDARD_VGA is not set
# CONFIG_VGA_CIRRUS is not set
# CONFIG_VGA_BOCHS is not set
# CONFIG_VGA_GEODEGX2 is not set
# CONFIG_VGA_GEODELX is not set
# CONFIG_BUILD_VGABIOS is not set
CONFIG_VGA_FIXUP_ASM=y
CONFIG_VGA_EXTRA_STACK_SIZE=512

#
# Debugging
#
CONFIG_DEBUG_LEVEL=0

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-25 12:52     ` [Qemu-devel] " Paolo Bonzini
@ 2015-05-25 15:11       ` Kevin O'Connor
  -1 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-25 15:11 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Mon, May 25, 2015 at 02:52:51PM +0200, Paolo Bonzini wrote:
> On 23/05/2015 01:23, Kevin O'Connor wrote:
> > I'm curious if you've tried profiling SeaBIOS to see where it is
> > spending unnecessary time?
> 
> No, I really wanted to get the absolute minimum time needed to get to
> the kernel.  I announced it publicly because I think it's also instructive.
> 
> > I wonder if a stripped down SeaBIOS could
> > obtain sufficient performance.
> 
> It most likely could.  However, some of the features (e.g. booting
> vmlinuz from parallel flash or, in the future, from persistent memory)
> probably do not fit SeaBIOS very well.  One would have to redo them
> otherwise, for example using a DMA interface in fw_cfg.

I think adding vmlinuz and/or multiboot support to SeaBIOS would be
generally useful (and not difficult).

If a kernel was placed in memory (or memory-mapped flash) then I think
it would be reasonable to have SeaBIOS obtain its address from (an
unoptimized) fw_cfg and deploy it.

-Kevin

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-25 15:11       ` Kevin O'Connor
  0 siblings, 0 replies; 50+ messages in thread
From: Kevin O'Connor @ 2015-05-25 15:11 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, KVM list

On Mon, May 25, 2015 at 02:52:51PM +0200, Paolo Bonzini wrote:
> On 23/05/2015 01:23, Kevin O'Connor wrote:
> > I'm curious if you've tried profiling SeaBIOS to see where it is
> > spending unnecessary time?
> 
> No, I really wanted to get the absolute minimum time needed to get to
> the kernel.  I announced it publicly because I think it's also instructive.
> 
> > I wonder if a stripped down SeaBIOS could
> > obtain sufficient performance.
> 
> It most likely could.  However, some of the features (e.g. booting
> vmlinuz from parallel flash or, in the future, from persistent memory)
> probably do not fit SeaBIOS very well.  One would have to redo them
> otherwise, for example using a DMA interface in fw_cfg.

I think adding vmlinuz and/or multiboot support to SeaBIOS would be
generally useful (and not difficult).

If a kernel was placed in memory (or memory-mapped flash) then I think
it would be reasonable to have SeaBIOS obtain its address from (an
unoptimized) fw_cfg and deploy it.

-Kevin

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-22  2:53   ` [Qemu-devel] " Yong Wang
@ 2015-05-26  8:47     ` Stefan Hajnoczi
  -1 siblings, 0 replies; 50+ messages in thread
From: Stefan Hajnoczi @ 2015-05-26  8:47 UTC (permalink / raw)
  To: Yong Wang; +Cc: Paolo Bonzini, qemu-devel, KVM list

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

On Fri, May 22, 2015 at 10:53:54AM +0800, Yong Wang wrote:
> On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> > On the QEMU side, there is no support yet for persistent memory and the
> > NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
> > will automatically start using it.
> > 
> 
> We are working on adding NFIT support into virtual bios.

Great.  I asked about this on the #pmem (irc.oftc.net) IRC channel last week.

Which virtual bios are you targeting?

Stefan

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-26  8:47     ` Stefan Hajnoczi
  0 siblings, 0 replies; 50+ messages in thread
From: Stefan Hajnoczi @ 2015-05-26  8:47 UTC (permalink / raw)
  To: Yong Wang; +Cc: Paolo Bonzini, qemu-devel, KVM list

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

On Fri, May 22, 2015 at 10:53:54AM +0800, Yong Wang wrote:
> On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
> > On the QEMU side, there is no support yet for persistent memory and the
> > NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
> > will automatically start using it.
> > 
> 
> We are working on adding NFIT support into virtual bios.

Great.  I asked about this on the #pmem (irc.oftc.net) IRC channel last week.

Which virtual bios are you targeting?

Stefan

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-25 12:53       ` Paolo Bonzini
@ 2015-05-26 21:25         ` Christopher Covington
  2015-05-27  9:30           ` Paolo Bonzini
  0 siblings, 1 reply; 50+ messages in thread
From: Christopher Covington @ 2015-05-26 21:25 UTC (permalink / raw)
  To: Paolo Bonzini, Daniel P. Berrange, Peter Maydell; +Cc: qemu-devel, KVM list

On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
> 
> On 22/05/2015 13:12, Daniel P. Berrange wrote:
>> In
>> particular I don't see why we need to have a SATA controller and ISA/LPC
>> bridge in every virt machine - root PCI bus only should be possible, as you
>> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
>> floppy via PCI devices and/or by adding a USB bus in the cases where you
>> really need one.
> 
> I think removing the ISA/LPC bridge is hard.  It includes the real-time
> clock and fw_cfg, for example.

Could VirtIO specified replacements make sense for these peripherals?

Chris

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-26 21:25         ` Christopher Covington
@ 2015-05-27  9:30           ` Paolo Bonzini
  2015-05-27  9:36               ` [Qemu-devel] " Avi Kivity
                               ` (3 more replies)
  0 siblings, 4 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-27  9:30 UTC (permalink / raw)
  To: Christopher Covington, Daniel P. Berrange, Peter Maydell
  Cc: qemu-devel, KVM list



On 26/05/2015 23:25, Christopher Covington wrote:
> On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
>>
>> On 22/05/2015 13:12, Daniel P. Berrange wrote:
>>> In
>>> particular I don't see why we need to have a SATA controller and ISA/LPC
>>> bridge in every virt machine - root PCI bus only should be possible, as you
>>> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
>>> floppy via PCI devices and/or by adding a USB bus in the cases where you
>>> really need one.
>>
>> I think removing the ISA/LPC bridge is hard.  It includes the real-time
>> clock and fw_cfg, for example.
> 
> Could VirtIO specified replacements make sense for these peripherals?

Not really.  virtio is too heavyweight and you'd be reinventing the
wheel unnecessarily.

For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
uses fw_cfg.  Another commonly used ISA device is the UART, for which
again -M virt uses a pl031.

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-27  9:30           ` Paolo Bonzini
@ 2015-05-27  9:36               ` Avi Kivity
  2015-05-27 11:54               ` Peter Maydell
                                 ` (2 subsequent siblings)
  3 siblings, 0 replies; 50+ messages in thread
From: Avi Kivity @ 2015-05-27  9:36 UTC (permalink / raw)
  To: Paolo Bonzini, Christopher Covington, Daniel P. Berrange, Peter Maydell
  Cc: qemu-devel, KVM list



On 05/27/2015 12:30 PM, Paolo Bonzini wrote:
>
> On 26/05/2015 23:25, Christopher Covington wrote:
>> On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
>>> On 22/05/2015 13:12, Daniel P. Berrange wrote:
>>>> In
>>>> particular I don't see why we need to have a SATA controller and ISA/LPC
>>>> bridge in every virt machine - root PCI bus only should be possible, as you
>>>> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
>>>> floppy via PCI devices and/or by adding a USB bus in the cases where you
>>>> really need one.
>>> I think removing the ISA/LPC bridge is hard.  It includes the real-time
>>> clock and fw_cfg, for example.
>> Could VirtIO specified replacements make sense for these peripherals?
> Not really.  virtio is too heavyweight and you'd be reinventing the
> wheel unnecessarily.
>
> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
> uses fw_cfg.  Another commonly used ISA device is the UART, for which
> again -M virt uses a pl031.
>

The RTC can be replaced by kvmclock, the keyboard by virtio-console.  
Maybe we can provide an msr- or pci- based interface to fw_cfg.

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-27  9:36               ` Avi Kivity
  0 siblings, 0 replies; 50+ messages in thread
From: Avi Kivity @ 2015-05-27  9:36 UTC (permalink / raw)
  To: Paolo Bonzini, Christopher Covington, Daniel P. Berrange, Peter Maydell
  Cc: qemu-devel, KVM list



On 05/27/2015 12:30 PM, Paolo Bonzini wrote:
>
> On 26/05/2015 23:25, Christopher Covington wrote:
>> On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
>>> On 22/05/2015 13:12, Daniel P. Berrange wrote:
>>>> In
>>>> particular I don't see why we need to have a SATA controller and ISA/LPC
>>>> bridge in every virt machine - root PCI bus only should be possible, as you
>>>> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
>>>> floppy via PCI devices and/or by adding a USB bus in the cases where you
>>>> really need one.
>>> I think removing the ISA/LPC bridge is hard.  It includes the real-time
>>> clock and fw_cfg, for example.
>> Could VirtIO specified replacements make sense for these peripherals?
> Not really.  virtio is too heavyweight and you'd be reinventing the
> wheel unnecessarily.
>
> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
> uses fw_cfg.  Another commonly used ISA device is the UART, for which
> again -M virt uses a pl031.
>

The RTC can be replaced by kvmclock, the keyboard by virtio-console.  
Maybe we can provide an msr- or pci- based interface to fw_cfg.

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-27  9:36               ` [Qemu-devel] " Avi Kivity
@ 2015-05-27 11:00                 ` Paolo Bonzini
  -1 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-27 11:00 UTC (permalink / raw)
  To: Avi Kivity, Christopher Covington, Daniel P. Berrange, Peter Maydell
  Cc: qemu-devel, KVM list



On 27/05/2015 11:36, Avi Kivity wrote:
> 
> 
> On 05/27/2015 12:30 PM, Paolo Bonzini wrote:
>>
>> On 26/05/2015 23:25, Christopher Covington wrote:
>>> On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
>>>> On 22/05/2015 13:12, Daniel P. Berrange wrote:
>>>>> In
>>>>> particular I don't see why we need to have a SATA controller and
>>>>> ISA/LPC
>>>>> bridge in every virt machine - root PCI bus only should be
>>>>> possible, as you
>>>>> can provide disks via virtio-blk or virtio-scsi and serial,
>>>>> parallel, mouse,
>>>>> floppy via PCI devices and/or by adding a USB bus in the cases
>>>>> where you
>>>>> really need one.
>>>> I think removing the ISA/LPC bridge is hard.  It includes the real-time
>>>> clock and fw_cfg, for example.
>>> Could VirtIO specified replacements make sense for these peripherals?
>> Not really.  virtio is too heavyweight and you'd be reinventing the
>> wheel unnecessarily.
>>
>> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
>> uses fw_cfg.  Another commonly used ISA device is the UART, for which
>> again -M virt uses a pl031.
>>
> 
> The RTC can be replaced by kvmclock, the keyboard by virtio-console. 
> Maybe we can provide an msr- or pci- based interface to fw_cfg.

The RTC is used for more than clock unfortunately.  S3 support uses it
for example, both to tell the firmware that it's an S3 resume and for
resuming when the alarm fires.

All in all, getting rid of ISA seems like chasing windmills.  If you
want to do that, fine, but then do not do that on a minimal firmware
like qboot or SeaBIOS.  For example, UEFI provides run-time services
that let you access some low-level devices like this, and that's part of
why the ARM virtual machine image specification mandates UEFI support.
But even then, the ARM virtual machine image specification lets you
choose between pl031 and a Xen pv console, and doesn't specify
virtio-console...

Paolo

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-27 11:00                 ` Paolo Bonzini
  0 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-27 11:00 UTC (permalink / raw)
  To: Avi Kivity, Christopher Covington, Daniel P. Berrange, Peter Maydell
  Cc: qemu-devel, KVM list



On 27/05/2015 11:36, Avi Kivity wrote:
> 
> 
> On 05/27/2015 12:30 PM, Paolo Bonzini wrote:
>>
>> On 26/05/2015 23:25, Christopher Covington wrote:
>>> On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
>>>> On 22/05/2015 13:12, Daniel P. Berrange wrote:
>>>>> In
>>>>> particular I don't see why we need to have a SATA controller and
>>>>> ISA/LPC
>>>>> bridge in every virt machine - root PCI bus only should be
>>>>> possible, as you
>>>>> can provide disks via virtio-blk or virtio-scsi and serial,
>>>>> parallel, mouse,
>>>>> floppy via PCI devices and/or by adding a USB bus in the cases
>>>>> where you
>>>>> really need one.
>>>> I think removing the ISA/LPC bridge is hard.  It includes the real-time
>>>> clock and fw_cfg, for example.
>>> Could VirtIO specified replacements make sense for these peripherals?
>> Not really.  virtio is too heavyweight and you'd be reinventing the
>> wheel unnecessarily.
>>
>> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
>> uses fw_cfg.  Another commonly used ISA device is the UART, for which
>> again -M virt uses a pl031.
>>
> 
> The RTC can be replaced by kvmclock, the keyboard by virtio-console. 
> Maybe we can provide an msr- or pci- based interface to fw_cfg.

The RTC is used for more than clock unfortunately.  S3 support uses it
for example, both to tell the firmware that it's an S3 resume and for
resuming when the alarm fires.

All in all, getting rid of ISA seems like chasing windmills.  If you
want to do that, fine, but then do not do that on a minimal firmware
like qboot or SeaBIOS.  For example, UEFI provides run-time services
that let you access some low-level devices like this, and that's part of
why the ARM virtual machine image specification mandates UEFI support.
But even then, the ARM virtual machine image specification lets you
choose between pl031 and a Xen pv console, and doesn't specify
virtio-console...

Paolo

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-27  9:30           ` Paolo Bonzini
@ 2015-05-27 11:54               ` Peter Maydell
  2015-05-27 11:54               ` Peter Maydell
                                 ` (2 subsequent siblings)
  3 siblings, 0 replies; 50+ messages in thread
From: Peter Maydell @ 2015-05-27 11:54 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Christopher Covington, Daniel P. Berrange, qemu-devel, KVM list

On 27 May 2015 at 10:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
> uses fw_cfg.  Another commonly used ISA device is the UART, for which
> again -M virt uses a pl031.

Partly we do that because there were a number of reports that trying
to use virtio for the console didn't work reliably... Using the
stock UART that is widely supported in UEFI/uboot/kernel was a
conservative design choice.

The next thing that's likely to appear in "virt" is a PL061
GPIO device, which you need for CPU hotplug and external-shutdown-request
notifications.

-- PMM

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-27 11:54               ` Peter Maydell
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Maydell @ 2015-05-27 11:54 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Christopher Covington, KVM list, qemu-devel

On 27 May 2015 at 10:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
> uses fw_cfg.  Another commonly used ISA device is the UART, for which
> again -M virt uses a pl031.

Partly we do that because there were a number of reports that trying
to use virtio for the console didn't work reliably... Using the
stock UART that is widely supported in UEFI/uboot/kernel was a
conservative design choice.

The next thing that's likely to appear in "virt" is a PL061
GPIO device, which you need for CPU hotplug and external-shutdown-request
notifications.

-- PMM

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-27 11:54               ` Peter Maydell
@ 2015-05-27 12:05                 ` Paolo Bonzini
  -1 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-27 12:05 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Christopher Covington, Daniel P. Berrange, qemu-devel, KVM list



On 27/05/2015 13:54, Peter Maydell wrote:
> On 27 May 2015 at 10:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> > For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
>> > uses fw_cfg.  Another commonly used ISA device is the UART, for which
>> > again -M virt uses a pl031.
> Partly we do that because there were a number of reports that trying
> to use virtio for the console didn't work reliably... Using the
> stock UART that is widely supported in UEFI/uboot/kernel was a
> conservative design choice.
> 
> The next thing that's likely to appear in "virt" is a PL061
> GPIO device, which you need for CPU hotplug and external-shutdown-request
> notifications.

Indeed, and the x86 Q35 chipset puts the ACPI registers... in the
ISA/LPC bridge. :)

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-27 12:05                 ` Paolo Bonzini
  0 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-27 12:05 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Christopher Covington, KVM list, qemu-devel



On 27/05/2015 13:54, Peter Maydell wrote:
> On 27 May 2015 at 10:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> > For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
>> > uses fw_cfg.  Another commonly used ISA device is the UART, for which
>> > again -M virt uses a pl031.
> Partly we do that because there were a number of reports that trying
> to use virtio for the console didn't work reliably... Using the
> stock UART that is widely supported in UEFI/uboot/kernel was a
> conservative design choice.
> 
> The next thing that's likely to appear in "virt" is a PL061
> GPIO device, which you need for CPU hotplug and external-shutdown-request
> notifications.

Indeed, and the x86 Q35 chipset puts the ACPI registers... in the
ISA/LPC bridge. :)

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-27  9:30           ` Paolo Bonzini
  2015-05-27  9:36               ` [Qemu-devel] " Avi Kivity
  2015-05-27 11:54               ` Peter Maydell
@ 2015-05-27 12:50             ` Christopher Covington
  2015-05-27 12:59               ` Paolo Bonzini
  2015-05-27 16:24               ` Dr. David Alan Gilbert
  3 siblings, 1 reply; 50+ messages in thread
From: Christopher Covington @ 2015-05-27 12:50 UTC (permalink / raw)
  To: Paolo Bonzini, Daniel P. Berrange, Peter Maydell; +Cc: qemu-devel, KVM list

On 05/27/2015 05:30 AM, Paolo Bonzini wrote:
> 
> 
> On 26/05/2015 23:25, Christopher Covington wrote:
>> On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
>>>
>>> On 22/05/2015 13:12, Daniel P. Berrange wrote:
>>>> In
>>>> particular I don't see why we need to have a SATA controller and ISA/LPC
>>>> bridge in every virt machine - root PCI bus only should be possible, as you
>>>> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
>>>> floppy via PCI devices and/or by adding a USB bus in the cases where you
>>>> really need one.
>>>
>>> I think removing the ISA/LPC bridge is hard.  It includes the real-time
>>> clock and fw_cfg, for example.
>>
>> Could VirtIO specified replacements make sense for these peripherals?
> 
> Not really.  virtio is too heavyweight

I'd be curious to read where in your estimation this weight lies. Is it
one-time initialization or recurring? Is it specific to the PCI transport or
does MMIO suffer from it as well?

> and you'd be reinventing the wheel unnecessarily.

In my mind the utility of peripherals that are instruction set architecture
agnostic and can work over several transports is in reducing the amount of
(emulator/hypervisor, firmware, and OS) code used, and therefore in need of
maintenance, for common system emulation and virtualization use cases.

> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
> uses fw_cfg.  Another commonly used ISA device is the UART, for which
> again -M virt uses a pl031.

(UART is PL011; RTC is PL031)

Thanks,
Chris

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-27 12:50             ` Christopher Covington
@ 2015-05-27 12:59               ` Paolo Bonzini
  0 siblings, 0 replies; 50+ messages in thread
From: Paolo Bonzini @ 2015-05-27 12:59 UTC (permalink / raw)
  To: Christopher Covington, Daniel P. Berrange, Peter Maydell
  Cc: qemu-devel, KVM list



On 27/05/2015 14:50, Christopher Covington wrote:
>> Not really.  virtio is too heavyweight
> 
> I'd be curious to read where in your estimation this weight lies. Is it
> one-time initialization or recurring? Is it specific to the PCI transport or
> does MMIO suffer from it as well?

It's heavyweight in the sense that virtio requires you to design a
communication mechanism based on ring buffers.  It's much harder than a
few ad-hoc registers.

And I know everyone is upset about the attack surface of QEMU these
days, but effort would be much better spent adding QEMU-specific
customizations to a static analysis tool (that e.g. would derive bound
checks for the MemoryRegion read/write ops and be able to prove that
said ops cannot access arrays out of their bounds).

>> and you'd be reinventing the wheel unnecessarily.
> 
> In my mind the utility of peripherals that are instruction set architecture
> agnostic and can work over several transports is in reducing the amount of
> (emulator/hypervisor, firmware, and OS) code used, and therefore in need of
> maintenance, for common system emulation and virtualization use cases.

A fully processor-agnostic hardware architecture is a non-goal.  You'll
always have stuff like interrupt controllers that is extremely tied to
the processor.

If you want to abstract hardware, use the firmware, Luke!  Things such
as UEFI and ACPI are there for exactly this reason.  We will be able to
reuse a lot of x86 hotplug code on ARM using ACPI.  And if you don't
want to use ACPI, you can always write native OS drivers for the same
hotplug hardware.

Paolo

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-27  9:30           ` Paolo Bonzini
@ 2015-05-27 16:24               ` Dr. David Alan Gilbert
  2015-05-27 11:54               ` Peter Maydell
                                 ` (2 subsequent siblings)
  3 siblings, 0 replies; 50+ messages in thread
From: Dr. David Alan Gilbert @ 2015-05-27 16:24 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Christopher Covington, Daniel P. Berrange, Peter Maydell,
	qemu-devel, KVM list

* Paolo Bonzini (pbonzini@redhat.com) wrote:
> 
> 
> On 26/05/2015 23:25, Christopher Covington wrote:
> > On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
> >>
> >> On 22/05/2015 13:12, Daniel P. Berrange wrote:
> >>> In
> >>> particular I don't see why we need to have a SATA controller and ISA/LPC
> >>> bridge in every virt machine - root PCI bus only should be possible, as you
> >>> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
> >>> floppy via PCI devices and/or by adding a USB bus in the cases where you
> >>> really need one.
> >>
> >> I think removing the ISA/LPC bridge is hard.  It includes the real-time
> >> clock and fw_cfg, for example.
> > 
> > Could VirtIO specified replacements make sense for these peripherals?
> 
> Not really.  virtio is too heavyweight and you'd be reinventing the
> wheel unnecessarily.

I see reasons to replace some but not all these components;  and there's no point
in replacing the ISA/LPC bridge since it's got nothing at all in it.

> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
> uses fw_cfg.  Another commonly used ISA device is the UART, for which
> again -M virt uses a pl031.

I don't see much point in replacing the simple PC uart with
anything virtio; I can imagine that you might want to go down
to something really trivial with non of the bells and whistles;
but a UART is pretty simple.

The PC RTC though, it's a bit of a disaster that's had 30 years
of random cruft added into it to hold random things that should never
have been there.

Dave

> 
> Paolo
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-05-27 16:24               ` Dr. David Alan Gilbert
  0 siblings, 0 replies; 50+ messages in thread
From: Dr. David Alan Gilbert @ 2015-05-27 16:24 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Peter Maydell, Christopher Covington, KVM list, qemu-devel

* Paolo Bonzini (pbonzini@redhat.com) wrote:
> 
> 
> On 26/05/2015 23:25, Christopher Covington wrote:
> > On 05/25/2015 08:53 AM, Paolo Bonzini wrote:
> >>
> >> On 22/05/2015 13:12, Daniel P. Berrange wrote:
> >>> In
> >>> particular I don't see why we need to have a SATA controller and ISA/LPC
> >>> bridge in every virt machine - root PCI bus only should be possible, as you
> >>> can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse,
> >>> floppy via PCI devices and/or by adding a USB bus in the cases where you
> >>> really need one.
> >>
> >> I think removing the ISA/LPC bridge is hard.  It includes the real-time
> >> clock and fw_cfg, for example.
> > 
> > Could VirtIO specified replacements make sense for these peripherals?
> 
> Not really.  virtio is too heavyweight and you'd be reinventing the
> wheel unnecessarily.

I see reasons to replace some but not all these components;  and there's no point
in replacing the ISA/LPC bridge since it's got nothing at all in it.

> For example, ARM's "-M virt" uses a pl011 block for the RTC, and also
> uses fw_cfg.  Another commonly used ISA device is the UART, for which
> again -M virt uses a pl031.

I don't see much point in replacing the simple PC uart with
anything virtio; I can imagine that you might want to go down
to something really trivial with non of the bells and whistles;
but a UART is pretty simple.

The PC RTC though, it's a bit of a disaster that's had 30 years
of random cruft added into it to hold random things that should never
have been there.

Dave

> 
> Paolo
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: Announcing qboot, a minimal x86 firmware for QEMU
  2015-05-26  8:47     ` [Qemu-devel] " Stefan Hajnoczi
@ 2015-06-05 10:42       ` Stefan Hajnoczi
  -1 siblings, 0 replies; 50+ messages in thread
From: Stefan Hajnoczi @ 2015-06-05 10:42 UTC (permalink / raw)
  To: Yong Wang; +Cc: Paolo Bonzini, qemu-devel, KVM list

On Tue, May 26, 2015 at 9:47 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Fri, May 22, 2015 at 10:53:54AM +0800, Yong Wang wrote:
>> On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
>> > On the QEMU side, there is no support yet for persistent memory and the
>> > NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
>> > will automatically start using it.
>> >
>>
>> We are working on adding NFIT support into virtual bios.
>
> Great.  I asked about this on the #pmem (irc.oftc.net) IRC channel last week.
>
> Which virtual bios are you targeting?

Ping?

Interest in persistent memory is picking up and I'd like to avoid
duplicating work.  Which pieces do you have patches for?

1. QEMU -device pmem,file=/path/to/dax/file,id=pmem1 and fw_cfg/ACPI
info that gets passed to the guest
2. SeaBIOS NFIT ACPI table
3. ACPI NVDIMM DSM (probably not much needed, most features would be disabled)

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
@ 2015-06-05 10:42       ` Stefan Hajnoczi
  0 siblings, 0 replies; 50+ messages in thread
From: Stefan Hajnoczi @ 2015-06-05 10:42 UTC (permalink / raw)
  To: Yong Wang; +Cc: Paolo Bonzini, qemu-devel, KVM list

On Tue, May 26, 2015 at 9:47 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Fri, May 22, 2015 at 10:53:54AM +0800, Yong Wang wrote:
>> On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote:
>> > On the QEMU side, there is no support yet for persistent memory and the
>> > NFIT tables from ACPI 6.0.  Once that (and ACPI support) is added, qboot
>> > will automatically start using it.
>> >
>>
>> We are working on adding NFIT support into virtual bios.
>
> Great.  I asked about this on the #pmem (irc.oftc.net) IRC channel last week.
>
> Which virtual bios are you targeting?

Ping?

Interest in persistent memory is picking up and I'd like to avoid
duplicating work.  Which pieces do you have patches for?

1. QEMU -device pmem,file=/path/to/dax/file,id=pmem1 and fw_cfg/ACPI
info that gets passed to the guest
2. SeaBIOS NFIT ACPI table
3. ACPI NVDIMM DSM (probably not much needed, most features would be disabled)

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 50+ messages in thread

end of thread, other threads:[~2015-06-05 10:42 UTC | newest]

Thread overview: 50+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-05-21 13:51 Announcing qboot, a minimal x86 firmware for QEMU Paolo Bonzini
2015-05-21 13:51 ` [Qemu-devel] " Paolo Bonzini
2015-05-21 15:48 ` Avi Kivity
2015-05-21 16:21   ` Paolo Bonzini
2015-05-21 18:28     ` Avi Kivity
2015-05-21 17:04 ` Jan Kiszka
2015-05-21 17:04   ` [Qemu-devel] " Jan Kiszka
2015-05-22  2:53 ` Yong Wang
2015-05-22  2:53   ` [Qemu-devel] " Yong Wang
2015-05-26  8:47   ` Stefan Hajnoczi
2015-05-26  8:47     ` [Qemu-devel] " Stefan Hajnoczi
2015-06-05 10:42     ` Stefan Hajnoczi
2015-06-05 10:42       ` [Qemu-devel] " Stefan Hajnoczi
2015-05-22 11:01 ` Daniel P. Berrange
2015-05-22 11:04   ` Peter Maydell
2015-05-22 11:12     ` Daniel P. Berrange
2015-05-22 11:21       ` Peter Maydell
2015-05-22 11:33         ` Daniel P. Berrange
2015-05-22 11:34         ` Gerd Hoffmann
2015-05-22 11:34           ` Gerd Hoffmann
2015-05-22 11:42         ` Markus Armbruster
2015-05-22 11:42           ` Markus Armbruster
2015-05-25 12:53       ` Paolo Bonzini
2015-05-26 21:25         ` Christopher Covington
2015-05-27  9:30           ` Paolo Bonzini
2015-05-27  9:36             ` Avi Kivity
2015-05-27  9:36               ` [Qemu-devel] " Avi Kivity
2015-05-27 11:00               ` Paolo Bonzini
2015-05-27 11:00                 ` [Qemu-devel] " Paolo Bonzini
2015-05-27 11:54             ` Peter Maydell
2015-05-27 11:54               ` Peter Maydell
2015-05-27 12:05               ` Paolo Bonzini
2015-05-27 12:05                 ` Paolo Bonzini
2015-05-27 12:50             ` Christopher Covington
2015-05-27 12:59               ` Paolo Bonzini
2015-05-27 16:24             ` Dr. David Alan Gilbert
2015-05-27 16:24               ` Dr. David Alan Gilbert
2015-05-22 14:06 ` Gerd Hoffmann
2015-05-22 23:23 ` Kevin O'Connor
2015-05-22 23:23   ` [Qemu-devel] " Kevin O'Connor
2015-05-23  3:55   ` Kevin O'Connor
2015-05-23  3:55     ` [Qemu-devel] " Kevin O'Connor
2015-05-25  6:21     ` Vasiliy Tolstov
2015-05-25  6:21       ` [Qemu-devel] " Vasiliy Tolstov
2015-05-25 15:05       ` Kevin O'Connor
2015-05-25 15:05         ` [Qemu-devel] " Kevin O'Connor
2015-05-25 12:52   ` Paolo Bonzini
2015-05-25 12:52     ` [Qemu-devel] " Paolo Bonzini
2015-05-25 15:11     ` Kevin O'Connor
2015-05-25 15:11       ` [Qemu-devel] " Kevin O'Connor

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.