Stefan Hajnoczi writes: > On Tue, Jul 02, 2019 at 02:11:02PM +0200, Sergio Lopez wrote: >> Microvm is a machine type inspired by both NEMU and Firecracker, and >> constructed after the machine model implemented by the latter. >> >> It's main purpose is providing users a KVM-only machine type with fast >> boot times, minimal attack surface (measured as the number of IO ports >> and MMIO regions exposed to the Guest) and small footprint (specially >> when combined with the ongoing QEMU modularization effort). >> >> Normally, other than the device support provided by KVM itself, >> microvm only supports virtio-mmio devices. Microvm also includes a >> legacy mode, which adds an ISA bus with a 16550A serial port, useful >> for being able to see the early boot kernel messages. >> >> Microvm only supports booting PVH-enabled Linux ELF images. Booting >> other PVH-enabled kernels may be possible, but due to the lack of ACPI >> and firmware, we're relying on the command line for specifying the >> location of the virtio-mmio transports. If there's an interest on >> using this machine type with other kernels, we'll try to find some >> kind of middle ground solution. >> >> This is the list of the exposed IO ports and MMIO regions when running >> in non-legacy mode: >> >> address-space: memory >> 00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio >> 00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio >> 00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio >> 00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio >> 00000000d0000800-00000000d00009ff (prio 0, i/o): virtio-mmio >> 00000000d0000a00-00000000d0000bff (prio 0, i/o): virtio-mmio >> 00000000d0000c00-00000000d0000dff (prio 0, i/o): virtio-mmio >> 00000000d0000e00-00000000d0000fff (prio 0, i/o): virtio-mmio >> 00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi >> >> address-space: I/O >> 0000000000000000-000000000000ffff (prio 0, i/o): io >> 0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic >> 0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit >> 000000000000007e-000000000000007f (prio 0, i/o): kvmvapic >> 00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic >> 00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr >> 00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr >> >> A QEMU instance with the microvm machine type can be invoked this way: >> >> - Normal mode: >> >> qemu-system-x86_64 -M microvm -m 512m -smp 2 \ >> -kernel vmlinux -append "console=hvc0 root=/dev/vda" \ >> -nodefaults -no-user-config \ >> -chardev pty,id=virtiocon0,server \ >> -device virtio-serial-device \ >> -device virtconsole,chardev=virtiocon0 \ >> -drive id=test,file=test.img,format=raw,if=none \ >> -device virtio-blk-device,drive=test \ >> -netdev tap,id=tap0,script=no,downscript=no \ >> -device virtio-net-device,netdev=tap0 >> >> - Legacy mode: >> >> qemu-system-x86_64 -M microvm,legacy -m 512m -smp 2 \ >> -kernel vmlinux -append "console=ttyS0 root=/dev/vda" \ >> -nodefaults -no-user-config \ >> -drive id=test,file=test.img,format=raw,if=none \ >> -device virtio-blk-device,drive=test \ >> -netdev tap,id=tap0,script=no,downscript=no \ >> -device virtio-net-device,netdev=tap0 \ >> -serial stdio > > Please post metrics that compare this against a minimal Q35. > > With qboot it was later found that SeaBIOS can achieve comparable boot > times, so it wasn't worth maintaining qboot. > > Data is needed to show that microvm is really a significant improvement > over a minimal Q35. I've just ran some numbers using Stefano Garzarella's qemu-boot-time scripts [1] on a server with 2xIntel Xeon Silver 4114 2.20GHz, using the upstream QEMU (474f3938d79ab36b9231c9ad3b5a9314c2aeacde) built with minimal features [2]. The VM boots a minimal kernel [3] without initrd, using a kata container image as root via virtio-blk (though this isn't really relevant, as we're just taking measurements until the kernel is about to exec init). To try to make the comparison as fair as possible, I've used a minimal q35 machine with as few devices as possible. Disabling HPET and PIT at the same time caused the kernel to get stuck on boot, so I ran two iterations, one without HPET and the other without PIT: ----------------- | Q35 with HPET | ----------------- Command line: ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M q35,smbus=off,nvdimm=off,pit=off,vmport=off,sata=off,usb=off,graphics=off -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial -device virtconsole,chardev=virtiocon0 -drive id=test,file=/root/src/images/hello-rootfs.ext4,format=raw,if=none -device virtio-blk,drive=test Average boot times after 10 consecutive runs: qemu_init_end: 77.637936 linux_start_kernel: 117.082526 (+39.44459) linux_start_user: 364.629972 (+247.547446) Memory tree: address-space: memory 0000000000000000-ffffffffffffffff (prio 0, i/o): system 0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @pc.ram 0000000000000000-000000001fffffff 0000000000000000-ffffffffffffffff (prio -1, i/o): pci 00000000000c0000-00000000000dffff (prio 1, rom): pc.rom 00000000000e0000-00000000000fffff (prio 1, i/o): alias isa-bios @pc.bios 0000000000020000-000000000003ffff 00000000febf4000-00000000febf7fff (prio 1, i/o): virtio-pci 00000000febf4000-00000000febf4fff (prio 0, i/o): virtio-pci-common 00000000febf5000-00000000febf5fff (prio 0, i/o): virtio-pci-isr 00000000febf6000-00000000febf6fff (prio 0, i/o): virtio-pci-device 00000000febf7000-00000000febf7fff (prio 0, i/o): virtio-pci-notify 00000000febf8000-00000000febfbfff (prio 1, i/o): virtio-pci 00000000febf8000-00000000febf8fff (prio 0, i/o): virtio-pci-common 00000000febf9000-00000000febf9fff (prio 0, i/o): virtio-pci-isr 00000000febfa000-00000000febfafff (prio 0, i/o): virtio-pci-device 00000000febfb000-00000000febfbfff (prio 0, i/o): virtio-pci-notify 00000000febfe000-00000000febfefff (prio 1, i/o): virtio-serial-pci-msix 00000000febfe000-00000000febfe01f (prio 0, i/o): msix-table 00000000febfe800-00000000febfe807 (prio 0, i/o): msix-pba 00000000febff000-00000000febfffff (prio 1, i/o): virtio-blk-pci-msix 00000000febff000-00000000febff01f (prio 0, i/o): msix-table 00000000febff800-00000000febff807 (prio 0, i/o): msix-pba 00000000fffc0000-00000000ffffffff (prio 0, rom): pc.bios 00000000000a0000-00000000000bffff (prio 1, i/o): alias smram-region @pci 00000000000a0000-00000000000bffff 00000000000c0000-00000000000c2fff (prio 1000, i/o): alias kvmvapic-rom @pc.ram 00000000000c0000-00000000000c2fff 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c0000-00000000000c3fff [disabled] 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c0000-00000000000c3fff [disabled] 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c0000-00000000000c3fff 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pci 00000000000c0000-00000000000c3fff [disabled] 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c4000-00000000000c7fff [disabled] 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c4000-00000000000c7fff [disabled] 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c4000-00000000000c7fff 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pci 00000000000c4000-00000000000c7fff [disabled] 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c8000-00000000000cbfff [disabled] 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c8000-00000000000cbfff [disabled] 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c8000-00000000000cbfff 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pci 00000000000c8000-00000000000cbfff [disabled] 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000cc000-00000000000cffff [disabled] 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000cc000-00000000000cffff [disabled] 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000cc000-00000000000cffff 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pci 00000000000cc000-00000000000cffff [disabled] 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d0000-00000000000d3fff [disabled] 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d0000-00000000000d3fff [disabled] 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d0000-00000000000d3fff 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pci 00000000000d0000-00000000000d3fff [disabled] 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d4000-00000000000d7fff [disabled] 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d4000-00000000000d7fff [disabled] 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d4000-00000000000d7fff 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pci 00000000000d4000-00000000000d7fff [disabled] 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d8000-00000000000dbfff [disabled] 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d8000-00000000000dbfff [disabled] 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d8000-00000000000dbfff 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pci 00000000000d8000-00000000000dbfff [disabled] 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000dc000-00000000000dffff [disabled] 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000dc000-00000000000dffff [disabled] 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000dc000-00000000000dffff 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pci 00000000000dc000-00000000000dffff [disabled] 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e0000-00000000000e3fff [disabled] 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e0000-00000000000e3fff [disabled] 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e0000-00000000000e3fff 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pci 00000000000e0000-00000000000e3fff [disabled] 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e4000-00000000000e7fff [disabled] 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e4000-00000000000e7fff [disabled] 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e4000-00000000000e7fff 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pci 00000000000e4000-00000000000e7fff [disabled] 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e8000-00000000000ebfff 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e8000-00000000000ebfff [disabled] 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e8000-00000000000ebfff [disabled] 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pci 00000000000e8000-00000000000ebfff [disabled] 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-ram @pc.ram 00000000000ec000-00000000000effff 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pc.ram 00000000000ec000-00000000000effff [disabled] 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-rom @pc.ram 00000000000ec000-00000000000effff [disabled] 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pci 00000000000ec000-00000000000effff [disabled] 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000f0000-00000000000fffff [disabled] 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000f0000-00000000000fffff [disabled] 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000f0000-00000000000fffff 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pci 00000000000f0000-00000000000fffff [disabled] 0000000020000000-0000000020000000 (prio 1, i/o): tseg-blackhole [disabled] 00000000b0000000-00000000bfffffff (prio 0, i/o): pcie-mmcfg-mmio 00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic 00000000fed00000-00000000fed003ff (prio 0, i/o): hpet 00000000fed1c000-00000000fed1ffff (prio 1, i/o): lpc-rcrb-mmio 00000000feda0000-00000000fedbffff (prio 1, i/o): alias smram-open-high @pc.ram 00000000000a0000-00000000000bffff [disabled] 00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi address-space: I/O 0000000000000000-000000000000ffff (prio 0, i/o): io 0000000000000000-0000000000000007 (prio 0, i/o): dma-chan 0000000000000008-000000000000000f (prio 0, i/o): dma-cont 0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic 0000000000000060-0000000000000060 (prio 0, i/o): i8042-data 0000000000000064-0000000000000064 (prio 0, i/o): i8042-cmd 0000000000000070-0000000000000071 (prio 0, i/o): rtc 0000000000000070-0000000000000070 (prio 0, i/o): rtc-index 000000000000007e-000000000000007f (prio 0, i/o): kvmvapic 0000000000000080-0000000000000080 (prio 0, i/o): ioport80 0000000000000081-0000000000000083 (prio 0, i/o): dma-page 0000000000000087-0000000000000087 (prio 0, i/o): dma-page 0000000000000089-000000000000008b (prio 0, i/o): dma-page 000000000000008f-000000000000008f (prio 0, i/o): dma-page 0000000000000092-0000000000000092 (prio 0, i/o): port92 00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic 00000000000000b2-00000000000000b3 (prio 0, i/o): apm-io 00000000000000c0-00000000000000cf (prio 0, i/o): dma-chan 00000000000000d0-00000000000000df (prio 0, i/o): dma-cont 00000000000000f0-00000000000000f0 (prio 0, i/o): ioportF0 00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr 00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr 0000000000000510-0000000000000511 (prio 0, i/o): fwcfg 0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma 0000000000000600-000000000000067f (prio 0, i/o): ich9-pm 0000000000000600-0000000000000603 (prio 0, i/o): acpi-evt 0000000000000604-0000000000000605 (prio 0, i/o): acpi-cnt 0000000000000608-000000000000060b (prio 0, i/o): acpi-tmr 0000000000000620-000000000000062f (prio 0, i/o): acpi-gpe0 0000000000000630-0000000000000637 (prio 0, i/o): acpi-smi 0000000000000660-000000000000067f (prio 0, i/o): sm-tco 0000000000000cd8-0000000000000ce3 (prio 0, i/o): acpi-mem-hotplug 0000000000000cf8-0000000000000cfb (prio 0, i/o): pci-conf-idx 0000000000000cf9-0000000000000cf9 (prio 1, i/o): lpc-reset-control 0000000000000cfc-0000000000000cff (prio 0, i/o): pci-conf-data 000000000000c000-000000000000c07f (prio 1, i/o): virtio-pci 000000000000c080-000000000000c0bf (prio 1, i/o): virtio-pci ---------------- | Q35 with PIT | ---------------- Command line: ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M q35,smbus=off,nvdimm=off,pit=on,vmport=off,sata=off,usb=off,graphics=off -no-hpet -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial -device virtconsole,chardev=virtiocon0 -drive id=test,file=/root/src/images/hello-rootfs.ext4,format=raw,if=none -device virtio-blk,drive=test Average boot times after 10 consecutive runs: qemu_init_end: 77.467852 linux_start_kernel: 116.688472 (+39.22062) linux_start_user: 363.033365 (+246.344893) Memory tree: address-space: memory 0000000000000000-ffffffffffffffff (prio 0, i/o): system 0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @pc.ram 0000000000000000-000000001fffffff 0000000000000000-ffffffffffffffff (prio -1, i/o): pci 00000000000c0000-00000000000dffff (prio 1, rom): pc.rom 00000000000e0000-00000000000fffff (prio 1, i/o): alias isa-bios @pc.bios 0000000000020000-000000000003ffff 00000000febf4000-00000000febf7fff (prio 1, i/o): virtio-pci 00000000febf4000-00000000febf4fff (prio 0, i/o): virtio-pci-common 00000000febf5000-00000000febf5fff (prio 0, i/o): virtio-pci-isr 00000000febf6000-00000000febf6fff (prio 0, i/o): virtio-pci-device 00000000febf7000-00000000febf7fff (prio 0, i/o): virtio-pci-notify 00000000febf8000-00000000febfbfff (prio 1, i/o): virtio-pci 00000000febf8000-00000000febf8fff (prio 0, i/o): virtio-pci-common 00000000febf9000-00000000febf9fff (prio 0, i/o): virtio-pci-isr 00000000febfa000-00000000febfafff (prio 0, i/o): virtio-pci-device 00000000febfb000-00000000febfbfff (prio 0, i/o): virtio-pci-notify 00000000febfe000-00000000febfefff (prio 1, i/o): virtio-serial-pci-msix 00000000febfe000-00000000febfe01f (prio 0, i/o): msix-table 00000000febfe800-00000000febfe807 (prio 0, i/o): msix-pba 00000000febff000-00000000febfffff (prio 1, i/o): virtio-blk-pci-msix 00000000febff000-00000000febff01f (prio 0, i/o): msix-table 00000000febff800-00000000febff807 (prio 0, i/o): msix-pba 00000000fffc0000-00000000ffffffff (prio 0, rom): pc.bios 00000000000a0000-00000000000bffff (prio 1, i/o): alias smram-region @pci 00000000000a0000-00000000000bffff 00000000000c0000-00000000000c2fff (prio 1000, i/o): alias kvmvapic-rom @pc.ram 00000000000c0000-00000000000c2fff 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c0000-00000000000c3fff [disabled] 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c0000-00000000000c3fff [disabled] 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c0000-00000000000c3fff 00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pci 00000000000c0000-00000000000c3fff [disabled] 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c4000-00000000000c7fff [disabled] 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c4000-00000000000c7fff [disabled] 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c4000-00000000000c7fff 00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pci 00000000000c4000-00000000000c7fff [disabled] 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c8000-00000000000cbfff [disabled] 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c8000-00000000000cbfff [disabled] 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c8000-00000000000cbfff 00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pci 00000000000c8000-00000000000cbfff [disabled] 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000cc000-00000000000cffff [disabled] 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000cc000-00000000000cffff [disabled] 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000cc000-00000000000cffff 00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pci 00000000000cc000-00000000000cffff [disabled] 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d0000-00000000000d3fff [disabled] 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d0000-00000000000d3fff [disabled] 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d0000-00000000000d3fff 00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pci 00000000000d0000-00000000000d3fff [disabled] 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d4000-00000000000d7fff [disabled] 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d4000-00000000000d7fff [disabled] 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d4000-00000000000d7fff 00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pci 00000000000d4000-00000000000d7fff [disabled] 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d8000-00000000000dbfff [disabled] 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d8000-00000000000dbfff [disabled] 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d8000-00000000000dbfff 00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pci 00000000000d8000-00000000000dbfff [disabled] 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000dc000-00000000000dffff [disabled] 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000dc000-00000000000dffff [disabled] 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000dc000-00000000000dffff 00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pci 00000000000dc000-00000000000dffff [disabled] 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e0000-00000000000e3fff [disabled] 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e0000-00000000000e3fff [disabled] 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e0000-00000000000e3fff 00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pci 00000000000e0000-00000000000e3fff [disabled] 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e4000-00000000000e7fff [disabled] 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e4000-00000000000e7fff [disabled] 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e4000-00000000000e7fff 00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pci 00000000000e4000-00000000000e7fff [disabled] 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e8000-00000000000ebfff 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e8000-00000000000ebfff [disabled] 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e8000-00000000000ebfff [disabled] 00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pci 00000000000e8000-00000000000ebfff [disabled] 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-ram @pc.ram 00000000000ec000-00000000000effff 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pc.ram 00000000000ec000-00000000000effff [disabled] 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-rom @pc.ram 00000000000ec000-00000000000effff [disabled] 00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pci 00000000000ec000-00000000000effff [disabled] 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000f0000-00000000000fffff [disabled] 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000f0000-00000000000fffff [disabled] 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000f0000-00000000000fffff 00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pci 00000000000f0000-00000000000fffff [disabled] 0000000020000000-0000000020000000 (prio 1, i/o): tseg-blackhole [disabled] 00000000b0000000-00000000bfffffff (prio 0, i/o): pcie-mmcfg-mmio 00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic 00000000fed1c000-00000000fed1ffff (prio 1, i/o): lpc-rcrb-mmio 00000000feda0000-00000000fedbffff (prio 1, i/o): alias smram-open-high @pc.ram 00000000000a0000-00000000000bffff [disabled] 00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi address-space: I/O 0000000000000000-000000000000ffff (prio 0, i/o): io 0000000000000000-0000000000000007 (prio 0, i/o): dma-chan 0000000000000008-000000000000000f (prio 0, i/o): dma-cont 0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic 0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit 0000000000000060-0000000000000060 (prio 0, i/o): i8042-data 0000000000000061-0000000000000061 (prio 0, i/o): pcspk 0000000000000064-0000000000000064 (prio 0, i/o): i8042-cmd 0000000000000070-0000000000000071 (prio 0, i/o): rtc 0000000000000070-0000000000000070 (prio 0, i/o): rtc-index 000000000000007e-000000000000007f (prio 0, i/o): kvmvapic 0000000000000080-0000000000000080 (prio 0, i/o): ioport80 0000000000000081-0000000000000083 (prio 0, i/o): dma-page 0000000000000087-0000000000000087 (prio 0, i/o): dma-page 0000000000000089-000000000000008b (prio 0, i/o): dma-page 000000000000008f-000000000000008f (prio 0, i/o): dma-page 0000000000000092-0000000000000092 (prio 0, i/o): port92 00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic 00000000000000b2-00000000000000b3 (prio 0, i/o): apm-io 00000000000000c0-00000000000000cf (prio 0, i/o): dma-chan 00000000000000d0-00000000000000df (prio 0, i/o): dma-cont 00000000000000f0-00000000000000f0 (prio 0, i/o): ioportF0 00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr 00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr 0000000000000510-0000000000000511 (prio 0, i/o): fwcfg 0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma 0000000000000600-000000000000067f (prio 0, i/o): ich9-pm 0000000000000600-0000000000000603 (prio 0, i/o): acpi-evt 0000000000000604-0000000000000605 (prio 0, i/o): acpi-cnt 0000000000000608-000000000000060b (prio 0, i/o): acpi-tmr 0000000000000620-000000000000062f (prio 0, i/o): acpi-gpe0 0000000000000630-0000000000000637 (prio 0, i/o): acpi-smi 0000000000000660-000000000000067f (prio 0, i/o): sm-tco 0000000000000cd8-0000000000000ce3 (prio 0, i/o): acpi-mem-hotplug 0000000000000cf8-0000000000000cfb (prio 0, i/o): pci-conf-idx 0000000000000cf9-0000000000000cf9 (prio 1, i/o): lpc-reset-control 0000000000000cfc-0000000000000cff (prio 0, i/o): pci-conf-data 000000000000c000-000000000000c07f (prio 1, i/o): virtio-pci 000000000000c080-000000000000c0bf (prio 1, i/o): virtio-pci ----------- | microvm | ----------- Command line: ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M microvm -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial-device -device virtconsole,chardev=virtiocon0 -drive id=test,file=/root/src/images/hello-rootfs.ext4,format=raw,if=none -device virtio-blk-device,drive=test Average boot times after 10 consecutive runs: qemu_init_end: 64.043264 linux_start_kernel: 65.481782 (+1.438518) linux_start_user: 114.938353 (+49.456571) Memory tree: address-space: memory 0000000000000000-ffffffffffffffff (prio 0, i/o): system 0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @microvm.ram 0000000000000000-000000001fffffff 00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio 00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio 00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio 00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio 00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic 00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi address-space: I/O 0000000000000000-000000000000ffff (prio 0, i/o): io 000000000000007e-000000000000007f (prio 0, i/o): kvmvapic -------------- | Conclusion | -------------- The average boot time of microvm is a third of Q35's (115ms vs. 363ms), and is smaller on all sections (QEMU initialization, firmware overhead and kernel start-to-user). Microvm's memory tree is also visibly simpler, significantly reducing the exposed surface to the guest. While we can certainly work on making Q35 smaller, I definitely think it's better (and way safer!) having a specialized machine type for a specific use case, than a minimal Q35 whose behavior significantly diverges from a conventional Q35. Sergio. [1] https://github.com/stefano-garzarella/qemu-boot-time [2] https://paste.fedoraproject.org/paste/YZ9Ok-dJtQrc0xxctFm-nw [3] https://paste.fedoraproject.org/paste/sck0jfioAJdMq51HH6wkmA