From: Sergio Lopez <slp@redhat.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: ehabkost@redhat.com, maran.wilson@oracle.com, mst@redhat.com,
Stefan Hajnoczi <stefanha@redhat.com>,
qemu-devel@nongnu.org, Gerd Hoffmann <kraxel@redhat.com>,
Stefano Garzarella <sgarzare@redhat.com>,
rth@twiddle.net
Subject: Re: [Qemu-devel] [PATCH v2 4/4] hw/i386: Introduce the microvm machine type
Date: Thu, 18 Jul 2019 16:34:42 +0200 [thread overview]
Message-ID: <87ef2ntnr1.fsf@redhat.com> (raw)
In-Reply-To: <8736joxzv4.fsf@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 6834 bytes --]
Sergio Lopez <slp@redhat.com> writes:
> Paolo Bonzini <pbonzini@redhat.com> writes:
>
>> On 02/07/19 12:52, Sergio Lopez wrote:
>>> As I said, I'm also in favor of microvm supporting booting from
>>> firmware in the future, as long we keep the simple mode too.
>>
>> The simple mode adds code to QEMU's x86 target that only exists to
>> support microvm. It should be motivated by a clear win in boot times.
>
> OK. When I'm back from my PTO, I'll work on adding the firmware
> support to microvm. I'll run and share some numbers to see whether the
> simple mode makes sense or we can just rely on qboot for lower boot
> times plus SeaBIOS for compatibility.
I've just added support for starting the machine from SeaBIOS (Stefan
Hajnoczi pointed in another thread that it can be as fast as qboot, and
given that the latter doesn't support mptables, I just tested this
one). I tried to keep it as minimalistic as possible, but it still
required an RTC (mc146818), which dragged in an ISA BUS, and this one a
KVM PIT.
I ran some numbers using Stefano Garzarella's qemu-boot-time scripts
[1] on a server with 2xIntel Xeon Silver 4114 2.20GHz, using the
upstream QEMU (474f3938d79ab36b9231c9ad3b5a9314c2aeacde) built with
minimal features [2]. The VM boots a minimal kernel [3] without initrd,
using a kata container image as root via virtio-blk (though this isn't
really relevant, as we're just taking measurements until the kernel is
about to exec init).
---------------------
| QEMU with SeaBIOS |
---------------------
Command line:
./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M microvm,legacy -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet virtio_mmio.device=512@0xd0000600:15 virtio_mmio.device=512@0xd0000400:14" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial-device -device virtconsole,chardev=virtiocon0 -drive id=test,file=/usr/share/kata-containers/kata-containers.img,format=raw,if=none -device virtio-blk-device,drive=test -monitor stdio
Average boot times after 10 consecutive runs:
qemu_init_end: 65.958714
linux_start_kernel: 77.735803 (+11.777089)
linux_start_user: 127.360739 (+49.624936)
Exposed I/O ports and MMIOs:
address-space: memory
0000000000000000-ffffffffffffffff (prio 0, i/o): system
0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @microvm.ram 0000000000000000-000000001fffffff
00000000000e0000-00000000000fffff (prio 1, i/o): alias isa-bios @pc.bios 0000000000000000-000000000001ffff
00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio
00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio
00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio
00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio
00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi
00000000fffe0000-00000000ffffffff (prio 0, ram): pc.bios
address-space: I/O
0000000000000000-000000000000ffff (prio 0, i/o): io
0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic
0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit
0000000000000070-0000000000000071 (prio 0, i/o): rtc
0000000000000070-0000000000000070 (prio 0, i/o): rtc-index
000000000000007e-000000000000007f (prio 0, i/o): kvmvapic
00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic
00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr
00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr
0000000000000510-0000000000000511 (prio 0, i/o): fwcfg
0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma
-------------------
| QEMU direct PVH |
-------------------
Command line:
./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M microvm -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial-device -device virtconsole,chardev=virtiocon0 -drive id=test,file=/usr/share/kata-containers/kata-containers.img,format=raw,if=none -device virtio-blk-device,drive=test -monitor stdio
Average boot times after 10 consecutive runs:
qemu_init_end: 64.043264
linux_start_kernel: 65.481782 (+1.438518)
linux_start_user: 114.938353 (+49.456571)
Exposed I/O ports and MMIOs:
address-space: memory
0000000000000000-ffffffffffffffff (prio 0, i/o): system
0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @microvm.ram 0000000000000000-000000001fffffff
00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio
00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio
00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio
00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio
00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic
00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi
address-space: I/O
0000000000000000-000000000000ffff (prio 0, i/o): io
000000000000007e-000000000000007f (prio 0, i/o): kvmvapic
--------------
| Comparison |
--------------
Average boot time:
* Relying on SeaBIOS, when compared with direct PVH boot, as a total
average overhead of ~12ms. The cost of initializing QEMU increases in
~2ms (probably due to need to instantiate more devices), while the
other ~10ms is the SeaBIOS overhead.
Exposed I/O ports and MMIOs:
* The following 8 I/O ports are only present in the version relying on SeaBIOS:
0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic
0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit
0000000000000070-0000000000000071 (prio 0, i/o): rtc
0000000000000070-0000000000000070 (prio 0, i/o): rtc-index
00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic
00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr
00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr
0000000000000510-0000000000000511 (prio 0, i/o): fwcfg
0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma
* The following MMIO region is only present in the direct boot version:
00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic
---------------
| Conclusions |
---------------
Objectively, the version that boots directly the kernel using PVH is 10%
faster and has a slightly larger exposed surface. Whether this is enough
to justify its existence is quite subjective.
In my opinion, not only I think it makes sense to have it, but I also
think there's little reason to have the firmware reliant version, given
the nature and purpose of microvm.
Sergio.
[1] https://github.com/stefano-garzarella/qemu-boot-time
[2] https://paste.fedoraproject.org/paste/YZ9Ok-dJtQrc0xxctFm-nw
[3] https://paste.fedoraproject.org/paste/sck0jfioAJdMq51HH6wkmA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
next prev parent reply other threads:[~2019-07-18 14:35 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-01 14:47 [Qemu-devel] [PATCH v2 0/4] Introduce the microvm machine type Sergio Lopez
2019-07-01 14:47 ` [Qemu-devel] [PATCH v2 1/4] hw/virtio: Factorize virtio-mmio headers Sergio Lopez
2019-07-01 14:47 ` [Qemu-devel] [PATCH v2 2/4] hw/i386: Add an Intel MPTable generator Sergio Lopez
2019-07-02 8:02 ` Gerd Hoffmann
2019-07-02 8:37 ` Sergio Lopez
2019-07-02 9:33 ` Gerd Hoffmann
2019-07-01 14:47 ` [Qemu-devel] [PATCH v2 3/4] hw/i386: Factorize PVH related functions Sergio Lopez
2019-07-01 14:47 ` [Qemu-devel] [PATCH v2 4/4] hw/i386: Introduce the microvm machine type Sergio Lopez
2019-07-02 8:17 ` Gerd Hoffmann
2019-07-02 8:42 ` Sergio Lopez
2019-07-02 10:16 ` Gerd Hoffmann
2019-07-02 10:52 ` Sergio Lopez
2019-07-02 11:50 ` Gerd Hoffmann
2019-07-02 14:06 ` Paolo Bonzini
2019-07-02 14:41 ` Sergio Lopez
2019-07-18 14:34 ` Sergio Lopez [this message]
2019-07-18 15:48 ` Paolo Bonzini
2019-07-19 10:30 ` Sergio Lopez
2019-07-19 11:49 ` Paolo Bonzini
2019-07-02 8:19 ` Stefano Garzarella
2019-07-02 8:47 ` Sergio Lopez
2019-07-02 10:37 ` Paolo Bonzini
2019-07-02 11:16 ` Sergio Lopez
2019-07-01 18:32 ` [Qemu-devel] [PATCH v2 0/4] " no-reply
2019-07-01 19:06 ` no-reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ef2ntnr1.fsf@redhat.com \
--to=slp@redhat.com \
--cc=ehabkost@redhat.com \
--cc=kraxel@redhat.com \
--cc=maran.wilson@oracle.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=rth@twiddle.net \
--cc=sgarzare@redhat.com \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).