Paolo Bonzini writes: > On 18/07/19 16:34, Sergio Lopez wrote: >> I've just added support for starting the machine from SeaBIOS (Stefan >> Hajnoczi pointed in another thread that it can be as fast as qboot, and >> given that the latter doesn't support mptables, I just tested this >> one). I tried to keep it as minimalistic as possible, but it still >> required an RTC (mc146818), which dragged in an ISA BUS, and this one a >> KVM PIT. >> >> I ran some numbers using Stefano Garzarella's qemu-boot-time scripts >> [1] on a server with 2xIntel Xeon Silver 4114 2.20GHz, using the >> upstream QEMU (474f3938d79ab36b9231c9ad3b5a9314c2aeacde) built with >> minimal features [2]. The VM boots a minimal kernel [3] without initrd, >> using a kata container image as root via virtio-blk (though this isn't >> really relevant, as we're just taking measurements until the kernel is >> about to exec init). >> >> --------------------- >> | QEMU with SeaBIOS | >> --------------------- >> >> qemu_init_end: 65.958714 >> linux_start_kernel: 77.735803 (+11.777089) >> linux_start_user: 127.360739 (+49.624936) >> >> ------------------- >> | QEMU direct PVH | >> ------------------- >> >> qemu_init_end: 64.043264 >> linux_start_kernel: 65.481782 (+1.438518) >> linux_start_user: 114.938353 (+49.456571) >> >> -------------- >> | Comparison | >> -------------- >> >> Average boot time: >> >> * Relying on SeaBIOS, when compared with direct PVH boot, as a total >> average overhead of ~12ms. The cost of initializing QEMU increases in >> ~2ms (probably due to need to instantiate more devices), while the >> other ~10ms is the SeaBIOS overhead. >> >> --------------- >> | Conclusions | >> --------------- >> >> Objectively, the version that boots directly the kernel using PVH is 10% >> faster and has a slightly larger exposed surface. Whether this is enough >> to justify its existence is quite subjective. >> >> In my opinion, not only I think it makes sense to have it, but I also >> think there's little reason to have the firmware reliant version, given >> the nature and purpose of microvm. > > The advantage of firmware is support for vmlinuz and multiboot in > addition to PVH, and removing code from QEMU. So I think it's still > worth doing a comparison with qboot, and trying to understand where > SeaBIOS is spending its time (qboot should not need additional devices > other than fw_cfg, and since SeaBIOS has never been optimized for > microvm I expect it's possible to shave quite a few of those 12 ms). I tweaked qboot a bit to make it work with microvm (basically, if it can't identify a PCI vendor, it assumes it's running in a microvm, and avoids PCI and ACPI initialization, writing an minimal mptable instead) and I've just got some numbers: Average boot time: qemu_init_end: 64.502199 linux_start_kernel: 66.644311 (+2.142112) linux_start_user: 116.279127 (+49.634816) To avoid having to scroll up, these are the numbers with the direct PVH boot version: Average boot time: qemu_init_end: 64.043264 linux_start_kernel: 65.481782 (+1.438518) linux_start_user: 114.938353 (+49.456571) So starting from qboot is just a bit over 1ms slower, which I think it's quite reasonable. And, as you said, the only additional device is fw_cfg: address-space: I/O 0000000000000000-000000000000ffff (prio 0, i/o): io 000000000000007e-000000000000007f (prio 0, i/o): kvmvapic 0000000000000510-0000000000000511 (prio 0, i/o): fwcfg 0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma If people feel more comfortable keeping QEMU on x86_64 starting on Real Mode and having the firmware writing mptables, I'm fine with taking this approach. That said, to avoid confusing users, I think it'd be better to drop the legacy mode for SeaBIOS, and support just qboot. That would mean having to add a binary from the latter to "pc-bios" with some descriptive name ("qboot-microvm.bin"?) and making microvm look for that one by default. Does this sound like a reasonable plan? > On the other hand I agree that microvm is showing great promise compared > to PCI-based machine types. > > Paolo