All of lore.kernel.org
 help / color / mirror / Atom feed
From: Stefan Hajnoczi <stefanha@gmail.com>
To: Sergio Lopez <slp@redhat.com>
Cc: ehabkost@redhat.com, maran.wilson@oracle.com, mst@redhat.com,
	qemu-devel@nongnu.org, kraxel@redhat.com, pbonzini@redhat.com,
	sgarzare@redhat.com, rth@twiddle.net
Subject: Re: [Qemu-devel] [PATCH v3 0/4] Introduce the microvm machine type
Date: Fri, 19 Jul 2019 11:29:15 +0100	[thread overview]
Message-ID: <20190719102915.GG18585@stefanha-x1.localdomain> (raw)
In-Reply-To: <87d0i7tlkl.fsf@redhat.com>

[-- Attachment #1: Type: text/plain, Size: 31402 bytes --]

On Thu, Jul 18, 2019 at 05:21:46PM +0200, Sergio Lopez wrote:
> 
> Stefan Hajnoczi <stefanha@gmail.com> writes:
> 
> > On Tue, Jul 02, 2019 at 02:11:02PM +0200, Sergio Lopez wrote:
> >> Microvm is a machine type inspired by both NEMU and Firecracker, and
> >> constructed after the machine model implemented by the latter.
> >> 
> >> It's main purpose is providing users a KVM-only machine type with fast
> >> boot times, minimal attack surface (measured as the number of IO ports
> >> and MMIO regions exposed to the Guest) and small footprint (specially
> >> when combined with the ongoing QEMU modularization effort).
> >> 
> >> Normally, other than the device support provided by KVM itself,
> >> microvm only supports virtio-mmio devices. Microvm also includes a
> >> legacy mode, which adds an ISA bus with a 16550A serial port, useful
> >> for being able to see the early boot kernel messages.
> >> 
> >> Microvm only supports booting PVH-enabled Linux ELF images. Booting
> >> other PVH-enabled kernels may be possible, but due to the lack of ACPI
> >> and firmware, we're relying on the command line for specifying the
> >> location of the virtio-mmio transports. If there's an interest on
> >> using this machine type with other kernels, we'll try to find some
> >> kind of middle ground solution.
> >> 
> >> This is the list of the exposed IO ports and MMIO regions when running
> >> in non-legacy mode:
> >> 
> >> address-space: memory
> >>     00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio
> >>     00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio
> >>     00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio
> >>     00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio
> >>     00000000d0000800-00000000d00009ff (prio 0, i/o): virtio-mmio
> >>     00000000d0000a00-00000000d0000bff (prio 0, i/o): virtio-mmio
> >>     00000000d0000c00-00000000d0000dff (prio 0, i/o): virtio-mmio
> >>     00000000d0000e00-00000000d0000fff (prio 0, i/o): virtio-mmio
> >>     00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi
> >> 
> >> address-space: I/O
> >>   0000000000000000-000000000000ffff (prio 0, i/o): io
> >>     0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic
> >>     0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit
> >>     000000000000007e-000000000000007f (prio 0, i/o): kvmvapic
> >>     00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic
> >>     00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr
> >>     00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr
> >> 
> >> A QEMU instance with the microvm machine type can be invoked this way:
> >> 
> >>  - Normal mode:
> >> 
> >> qemu-system-x86_64 -M microvm -m 512m -smp 2 \
> >>  -kernel vmlinux -append "console=hvc0 root=/dev/vda" \
> >>  -nodefaults -no-user-config \
> >>  -chardev pty,id=virtiocon0,server \
> >>  -device virtio-serial-device \
> >>  -device virtconsole,chardev=virtiocon0 \
> >>  -drive id=test,file=test.img,format=raw,if=none \
> >>  -device virtio-blk-device,drive=test \
> >>  -netdev tap,id=tap0,script=no,downscript=no \
> >>  -device virtio-net-device,netdev=tap0
> >> 
> >>  - Legacy mode:
> >> 
> >> qemu-system-x86_64 -M microvm,legacy -m 512m -smp 2 \
> >>  -kernel vmlinux -append "console=ttyS0 root=/dev/vda" \
> >>  -nodefaults -no-user-config \
> >>  -drive id=test,file=test.img,format=raw,if=none \
> >>  -device virtio-blk-device,drive=test \
> >>  -netdev tap,id=tap0,script=no,downscript=no \
> >>  -device virtio-net-device,netdev=tap0 \
> >>  -serial stdio
> >
> > Please post metrics that compare this against a minimal Q35.
> >
> > With qboot it was later found that SeaBIOS can achieve comparable boot
> > times, so it wasn't worth maintaining qboot.
> >
> > Data is needed to show that microvm is really a significant improvement
> > over a minimal Q35.
> 
> I've just ran some numbers using Stefano Garzarella's qemu-boot-time
> scripts [1] on a server with 2xIntel Xeon Silver 4114 2.20GHz, using the
> upstream QEMU (474f3938d79ab36b9231c9ad3b5a9314c2aeacde) built with
> minimal features [2]. The VM boots a minimal kernel [3] without initrd,
> using a kata container image as root via virtio-blk (though this isn't
> really relevant, as we're just taking measurements until the kernel is
> about to exec init).
> 
> To try to make the comparison as fair as possible, I've used a minimal
> q35 machine with as few devices as possible. Disabling HPET and PIT at
> the same time caused the kernel to get stuck on boot, so I ran two
> iterations, one without HPET and the other without PIT:
> 
> 
> -----------------
>  | Q35 with HPET |
>  -----------------
> 
> Command line:
> 
> ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M q35,smbus=off,nvdimm=off,pit=off,vmport=off,sata=off,usb=off,graphics=off -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial -device virtconsole,chardev=virtiocon0 -drive id=test,file=/root/src/images/hello-rootfs.ext4,format=raw,if=none -device virtio-blk,drive=test
> 
> Average boot times after 10 consecutive runs:
> 
>  qemu_init_end: 77.637936
>  linux_start_kernel: 117.082526 (+39.44459)
>  linux_start_user: 364.629972 (+247.547446)
> 
> Memory tree:
> 
>  address-space: memory
>   0000000000000000-ffffffffffffffff (prio 0, i/o): system
>     0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @pc.ram 0000000000000000-000000001fffffff
>     0000000000000000-ffffffffffffffff (prio -1, i/o): pci
>       00000000000c0000-00000000000dffff (prio 1, rom): pc.rom
>       00000000000e0000-00000000000fffff (prio 1, i/o): alias isa-bios @pc.bios 0000000000020000-000000000003ffff
>       00000000febf4000-00000000febf7fff (prio 1, i/o): virtio-pci
>         00000000febf4000-00000000febf4fff (prio 0, i/o): virtio-pci-common
>         00000000febf5000-00000000febf5fff (prio 0, i/o): virtio-pci-isr
>         00000000febf6000-00000000febf6fff (prio 0, i/o): virtio-pci-device
>         00000000febf7000-00000000febf7fff (prio 0, i/o): virtio-pci-notify
>       00000000febf8000-00000000febfbfff (prio 1, i/o): virtio-pci
>         00000000febf8000-00000000febf8fff (prio 0, i/o): virtio-pci-common
>         00000000febf9000-00000000febf9fff (prio 0, i/o): virtio-pci-isr
>         00000000febfa000-00000000febfafff (prio 0, i/o): virtio-pci-device
>         00000000febfb000-00000000febfbfff (prio 0, i/o): virtio-pci-notify
>       00000000febfe000-00000000febfefff (prio 1, i/o): virtio-serial-pci-msix
>         00000000febfe000-00000000febfe01f (prio 0, i/o): msix-table
>         00000000febfe800-00000000febfe807 (prio 0, i/o): msix-pba
>       00000000febff000-00000000febfffff (prio 1, i/o): virtio-blk-pci-msix
>         00000000febff000-00000000febff01f (prio 0, i/o): msix-table
>         00000000febff800-00000000febff807 (prio 0, i/o): msix-pba
>       00000000fffc0000-00000000ffffffff (prio 0, rom): pc.bios
>     00000000000a0000-00000000000bffff (prio 1, i/o): alias smram-region @pci 00000000000a0000-00000000000bffff
>     00000000000c0000-00000000000c2fff (prio 1000, i/o): alias kvmvapic-rom @pc.ram 00000000000c0000-00000000000c2fff
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c0000-00000000000c3fff [disabled]
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c0000-00000000000c3fff [disabled]
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c0000-00000000000c3fff
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pci 00000000000c0000-00000000000c3fff [disabled]
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c4000-00000000000c7fff [disabled]
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c4000-00000000000c7fff [disabled]
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c4000-00000000000c7fff
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pci 00000000000c4000-00000000000c7fff [disabled]
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c8000-00000000000cbfff [disabled]
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c8000-00000000000cbfff [disabled]
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c8000-00000000000cbfff
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pci 00000000000c8000-00000000000cbfff [disabled]
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000cc000-00000000000cffff [disabled]
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000cc000-00000000000cffff [disabled]
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000cc000-00000000000cffff
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pci 00000000000cc000-00000000000cffff [disabled]
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d0000-00000000000d3fff [disabled]
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d0000-00000000000d3fff [disabled]
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d0000-00000000000d3fff
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pci 00000000000d0000-00000000000d3fff [disabled]
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d4000-00000000000d7fff [disabled]
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d4000-00000000000d7fff [disabled]
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d4000-00000000000d7fff
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pci 00000000000d4000-00000000000d7fff [disabled]
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d8000-00000000000dbfff [disabled]
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d8000-00000000000dbfff [disabled]
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d8000-00000000000dbfff
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pci 00000000000d8000-00000000000dbfff [disabled]
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000dc000-00000000000dffff [disabled]
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000dc000-00000000000dffff [disabled]
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000dc000-00000000000dffff
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pci 00000000000dc000-00000000000dffff [disabled]
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e0000-00000000000e3fff [disabled]
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e0000-00000000000e3fff [disabled]
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e0000-00000000000e3fff
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pci 00000000000e0000-00000000000e3fff [disabled]
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e4000-00000000000e7fff [disabled]
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e4000-00000000000e7fff [disabled]
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e4000-00000000000e7fff
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pci 00000000000e4000-00000000000e7fff [disabled]
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e8000-00000000000ebfff
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e8000-00000000000ebfff [disabled]
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e8000-00000000000ebfff [disabled]
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pci 00000000000e8000-00000000000ebfff [disabled]
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-ram @pc.ram 00000000000ec000-00000000000effff
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pc.ram 00000000000ec000-00000000000effff [disabled]
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-rom @pc.ram 00000000000ec000-00000000000effff [disabled]
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pci 00000000000ec000-00000000000effff [disabled]
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000f0000-00000000000fffff [disabled]
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000f0000-00000000000fffff [disabled]
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000f0000-00000000000fffff
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pci 00000000000f0000-00000000000fffff [disabled]
>     0000000020000000-0000000020000000 (prio 1, i/o): tseg-blackhole [disabled]
>     00000000b0000000-00000000bfffffff (prio 0, i/o): pcie-mmcfg-mmio
>     00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic
>     00000000fed00000-00000000fed003ff (prio 0, i/o): hpet
>     00000000fed1c000-00000000fed1ffff (prio 1, i/o): lpc-rcrb-mmio
>     00000000feda0000-00000000fedbffff (prio 1, i/o): alias smram-open-high @pc.ram 00000000000a0000-00000000000bffff [disabled]
>     00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi
> 
>  address-space: I/O
>   0000000000000000-000000000000ffff (prio 0, i/o): io
>     0000000000000000-0000000000000007 (prio 0, i/o): dma-chan
>     0000000000000008-000000000000000f (prio 0, i/o): dma-cont
>     0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic
>     0000000000000060-0000000000000060 (prio 0, i/o): i8042-data
>     0000000000000064-0000000000000064 (prio 0, i/o): i8042-cmd
>     0000000000000070-0000000000000071 (prio 0, i/o): rtc
>       0000000000000070-0000000000000070 (prio 0, i/o): rtc-index
>     000000000000007e-000000000000007f (prio 0, i/o): kvmvapic
>     0000000000000080-0000000000000080 (prio 0, i/o): ioport80
>     0000000000000081-0000000000000083 (prio 0, i/o): dma-page
>     0000000000000087-0000000000000087 (prio 0, i/o): dma-page
>     0000000000000089-000000000000008b (prio 0, i/o): dma-page
>     000000000000008f-000000000000008f (prio 0, i/o): dma-page
>     0000000000000092-0000000000000092 (prio 0, i/o): port92
>     00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic
>     00000000000000b2-00000000000000b3 (prio 0, i/o): apm-io
>     00000000000000c0-00000000000000cf (prio 0, i/o): dma-chan
>     00000000000000d0-00000000000000df (prio 0, i/o): dma-cont
>     00000000000000f0-00000000000000f0 (prio 0, i/o): ioportF0
>     00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr
>     00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr
>     0000000000000510-0000000000000511 (prio 0, i/o): fwcfg
>     0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma
>     0000000000000600-000000000000067f (prio 0, i/o): ich9-pm
>       0000000000000600-0000000000000603 (prio 0, i/o): acpi-evt
>       0000000000000604-0000000000000605 (prio 0, i/o): acpi-cnt
>       0000000000000608-000000000000060b (prio 0, i/o): acpi-tmr
>       0000000000000620-000000000000062f (prio 0, i/o): acpi-gpe0
>       0000000000000630-0000000000000637 (prio 0, i/o): acpi-smi
>       0000000000000660-000000000000067f (prio 0, i/o): sm-tco
>     0000000000000cd8-0000000000000ce3 (prio 0, i/o): acpi-mem-hotplug
>     0000000000000cf8-0000000000000cfb (prio 0, i/o): pci-conf-idx
>     0000000000000cf9-0000000000000cf9 (prio 1, i/o): lpc-reset-control
>     0000000000000cfc-0000000000000cff (prio 0, i/o): pci-conf-data
>     000000000000c000-000000000000c07f (prio 1, i/o): virtio-pci
>     000000000000c080-000000000000c0bf (prio 1, i/o): virtio-pci
> 
> 
>  ----------------
>  | Q35 with PIT |
>  ----------------
> 
> Command line:
> 
> ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M q35,smbus=off,nvdimm=off,pit=on,vmport=off,sata=off,usb=off,graphics=off -no-hpet -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial -device virtconsole,chardev=virtiocon0 -drive id=test,file=/root/src/images/hello-rootfs.ext4,format=raw,if=none -device virtio-blk,drive=test
> 
> Average boot times after 10 consecutive runs:
> 
>  qemu_init_end: 77.467852
>  linux_start_kernel: 116.688472 (+39.22062)
>  linux_start_user: 363.033365 (+246.344893)
> 
> Memory tree:
> 
> address-space: memory
>   0000000000000000-ffffffffffffffff (prio 0, i/o): system
>     0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @pc.ram 0000000000000000-000000001fffffff
>     0000000000000000-ffffffffffffffff (prio -1, i/o): pci
>       00000000000c0000-00000000000dffff (prio 1, rom): pc.rom
>       00000000000e0000-00000000000fffff (prio 1, i/o): alias isa-bios @pc.bios 0000000000020000-000000000003ffff
>       00000000febf4000-00000000febf7fff (prio 1, i/o): virtio-pci
>         00000000febf4000-00000000febf4fff (prio 0, i/o): virtio-pci-common
>         00000000febf5000-00000000febf5fff (prio 0, i/o): virtio-pci-isr
>         00000000febf6000-00000000febf6fff (prio 0, i/o): virtio-pci-device
>         00000000febf7000-00000000febf7fff (prio 0, i/o): virtio-pci-notify
>       00000000febf8000-00000000febfbfff (prio 1, i/o): virtio-pci
>         00000000febf8000-00000000febf8fff (prio 0, i/o): virtio-pci-common
>         00000000febf9000-00000000febf9fff (prio 0, i/o): virtio-pci-isr
>         00000000febfa000-00000000febfafff (prio 0, i/o): virtio-pci-device
>         00000000febfb000-00000000febfbfff (prio 0, i/o): virtio-pci-notify
>       00000000febfe000-00000000febfefff (prio 1, i/o): virtio-serial-pci-msix
>         00000000febfe000-00000000febfe01f (prio 0, i/o): msix-table
>         00000000febfe800-00000000febfe807 (prio 0, i/o): msix-pba
>       00000000febff000-00000000febfffff (prio 1, i/o): virtio-blk-pci-msix
>         00000000febff000-00000000febff01f (prio 0, i/o): msix-table
>         00000000febff800-00000000febff807 (prio 0, i/o): msix-pba
>       00000000fffc0000-00000000ffffffff (prio 0, rom): pc.bios
>     00000000000a0000-00000000000bffff (prio 1, i/o): alias smram-region @pci 00000000000a0000-00000000000bffff
>     00000000000c0000-00000000000c2fff (prio 1000, i/o): alias kvmvapic-rom @pc.ram 00000000000c0000-00000000000c2fff
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c0000-00000000000c3fff [disabled]
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c0000-00000000000c3fff [disabled]
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c0000-00000000000c3fff
>     00000000000c0000-00000000000c3fff (prio 1, i/o): alias pam-pci @pci 00000000000c0000-00000000000c3fff [disabled]
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c4000-00000000000c7fff [disabled]
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c4000-00000000000c7fff [disabled]
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c4000-00000000000c7fff
>     00000000000c4000-00000000000c7fff (prio 1, i/o): alias pam-pci @pci 00000000000c4000-00000000000c7fff [disabled]
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000c8000-00000000000cbfff [disabled]
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000c8000-00000000000cbfff [disabled]
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000c8000-00000000000cbfff
>     00000000000c8000-00000000000cbfff (prio 1, i/o): alias pam-pci @pci 00000000000c8000-00000000000cbfff [disabled]
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000cc000-00000000000cffff [disabled]
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000cc000-00000000000cffff [disabled]
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000cc000-00000000000cffff
>     00000000000cc000-00000000000cffff (prio 1, i/o): alias pam-pci @pci 00000000000cc000-00000000000cffff [disabled]
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d0000-00000000000d3fff [disabled]
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d0000-00000000000d3fff [disabled]
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d0000-00000000000d3fff
>     00000000000d0000-00000000000d3fff (prio 1, i/o): alias pam-pci @pci 00000000000d0000-00000000000d3fff [disabled]
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d4000-00000000000d7fff [disabled]
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d4000-00000000000d7fff [disabled]
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d4000-00000000000d7fff
>     00000000000d4000-00000000000d7fff (prio 1, i/o): alias pam-pci @pci 00000000000d4000-00000000000d7fff [disabled]
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000d8000-00000000000dbfff [disabled]
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000d8000-00000000000dbfff [disabled]
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000d8000-00000000000dbfff
>     00000000000d8000-00000000000dbfff (prio 1, i/o): alias pam-pci @pci 00000000000d8000-00000000000dbfff [disabled]
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000dc000-00000000000dffff [disabled]
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000dc000-00000000000dffff [disabled]
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000dc000-00000000000dffff
>     00000000000dc000-00000000000dffff (prio 1, i/o): alias pam-pci @pci 00000000000dc000-00000000000dffff [disabled]
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e0000-00000000000e3fff [disabled]
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e0000-00000000000e3fff [disabled]
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e0000-00000000000e3fff
>     00000000000e0000-00000000000e3fff (prio 1, i/o): alias pam-pci @pci 00000000000e0000-00000000000e3fff [disabled]
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e4000-00000000000e7fff [disabled]
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e4000-00000000000e7fff [disabled]
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e4000-00000000000e7fff
>     00000000000e4000-00000000000e7fff (prio 1, i/o): alias pam-pci @pci 00000000000e4000-00000000000e7fff [disabled]
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-ram @pc.ram 00000000000e8000-00000000000ebfff
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pc.ram 00000000000e8000-00000000000ebfff [disabled]
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-rom @pc.ram 00000000000e8000-00000000000ebfff [disabled]
>     00000000000e8000-00000000000ebfff (prio 1, i/o): alias pam-pci @pci 00000000000e8000-00000000000ebfff [disabled]
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-ram @pc.ram 00000000000ec000-00000000000effff
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pc.ram 00000000000ec000-00000000000effff [disabled]
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-rom @pc.ram 00000000000ec000-00000000000effff [disabled]
>     00000000000ec000-00000000000effff (prio 1, i/o): alias pam-pci @pci 00000000000ec000-00000000000effff [disabled]
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-ram @pc.ram 00000000000f0000-00000000000fffff [disabled]
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pc.ram 00000000000f0000-00000000000fffff [disabled]
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-rom @pc.ram 00000000000f0000-00000000000fffff
>     00000000000f0000-00000000000fffff (prio 1, i/o): alias pam-pci @pci 00000000000f0000-00000000000fffff [disabled]
>     0000000020000000-0000000020000000 (prio 1, i/o): tseg-blackhole [disabled]
>     00000000b0000000-00000000bfffffff (prio 0, i/o): pcie-mmcfg-mmio
>     00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic
>     00000000fed1c000-00000000fed1ffff (prio 1, i/o): lpc-rcrb-mmio
>     00000000feda0000-00000000fedbffff (prio 1, i/o): alias smram-open-high @pc.ram 00000000000a0000-00000000000bffff [disabled]
>     00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi
> 
> address-space: I/O
>   0000000000000000-000000000000ffff (prio 0, i/o): io
>     0000000000000000-0000000000000007 (prio 0, i/o): dma-chan
>     0000000000000008-000000000000000f (prio 0, i/o): dma-cont
>     0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic
>     0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit
>     0000000000000060-0000000000000060 (prio 0, i/o): i8042-data
>     0000000000000061-0000000000000061 (prio 0, i/o): pcspk
>     0000000000000064-0000000000000064 (prio 0, i/o): i8042-cmd
>     0000000000000070-0000000000000071 (prio 0, i/o): rtc
>       0000000000000070-0000000000000070 (prio 0, i/o): rtc-index
>     000000000000007e-000000000000007f (prio 0, i/o): kvmvapic
>     0000000000000080-0000000000000080 (prio 0, i/o): ioport80
>     0000000000000081-0000000000000083 (prio 0, i/o): dma-page
>     0000000000000087-0000000000000087 (prio 0, i/o): dma-page
>     0000000000000089-000000000000008b (prio 0, i/o): dma-page
>     000000000000008f-000000000000008f (prio 0, i/o): dma-page
>     0000000000000092-0000000000000092 (prio 0, i/o): port92
>     00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic
>     00000000000000b2-00000000000000b3 (prio 0, i/o): apm-io
>     00000000000000c0-00000000000000cf (prio 0, i/o): dma-chan
>     00000000000000d0-00000000000000df (prio 0, i/o): dma-cont
>     00000000000000f0-00000000000000f0 (prio 0, i/o): ioportF0
>     00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr
>     00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr
>     0000000000000510-0000000000000511 (prio 0, i/o): fwcfg
>     0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma
>     0000000000000600-000000000000067f (prio 0, i/o): ich9-pm
>       0000000000000600-0000000000000603 (prio 0, i/o): acpi-evt
>       0000000000000604-0000000000000605 (prio 0, i/o): acpi-cnt
>       0000000000000608-000000000000060b (prio 0, i/o): acpi-tmr
>       0000000000000620-000000000000062f (prio 0, i/o): acpi-gpe0
>       0000000000000630-0000000000000637 (prio 0, i/o): acpi-smi
>       0000000000000660-000000000000067f (prio 0, i/o): sm-tco
>     0000000000000cd8-0000000000000ce3 (prio 0, i/o): acpi-mem-hotplug
>     0000000000000cf8-0000000000000cfb (prio 0, i/o): pci-conf-idx
>     0000000000000cf9-0000000000000cf9 (prio 1, i/o): lpc-reset-control
>     0000000000000cfc-0000000000000cff (prio 0, i/o): pci-conf-data
>     000000000000c000-000000000000c07f (prio 1, i/o): virtio-pci
>     000000000000c080-000000000000c0bf (prio 1, i/o): virtio-pci
> 
> 
>  -----------
>  | microvm |
>  -----------
> 
> Command line:
> 
> ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M microvm -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial-device -device virtconsole,chardev=virtiocon0 -drive id=test,file=/root/src/images/hello-rootfs.ext4,format=raw,if=none -device virtio-blk-device,drive=test
> 
> Average boot times after 10 consecutive runs:
> 
>  qemu_init_end: 64.043264
>  linux_start_kernel: 65.481782 (+1.438518)
>  linux_start_user: 114.938353 (+49.456571)
> 
> Memory tree:
> 
>  address-space: memory
>   0000000000000000-ffffffffffffffff (prio 0, i/o): system
>     0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @microvm.ram 0000000000000000-000000001fffffff
>     00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio
>     00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio
>     00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio
>     00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio
>     00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic
>     00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi
> 
>  address-space: I/O
>   0000000000000000-000000000000ffff (prio 0, i/o): io
>     000000000000007e-000000000000007f (prio 0, i/o): kvmvapic
> 
> 
>  --------------
>  | Conclusion |
>  --------------
> 
> The average boot time of microvm is a third of Q35's (115ms vs. 363ms),
> and is smaller on all sections (QEMU initialization, firmware overhead
> and kernel start-to-user).
> 
> Microvm's memory tree is also visibly simpler, significantly reducing
> the exposed surface to the guest.
> 
> While we can certainly work on making Q35 smaller, I definitely think
> it's better (and way safer!) having a specialized machine type for a
> specific use case, than a minimal Q35 whose behavior significantly
> diverges from a conventional Q35.

Interesting, so not a 10x difference!  This might be amenable to
optimization.

My concern with microvm is that it's so limited that few users will be
able to benefit from the reduced attack surface and faster startup time.
I think it's worth investigating slimming down Q35 further first.

In terms of startup time the first step would be profiling Q35 kernel
startup to find out what's taking so long (firmware initialization, PCI
probing, etc)?

> Sergio.
> 
> [1] https://github.com/stefano-garzarella/qemu-boot-time
> [2] https://paste.fedoraproject.org/paste/YZ9Ok-dJtQrc0xxctFm-nw
> [3] https://paste.fedoraproject.org/paste/sck0jfioAJdMq51HH6wkmA



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

  reply	other threads:[~2019-07-19 10:29 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-02 12:11 [Qemu-devel] [PATCH v3 0/4] Introduce the microvm machine type Sergio Lopez
2019-07-02 12:11 ` [Qemu-devel] [PATCH v3 1/4] hw/virtio: Factorize virtio-mmio headers Sergio Lopez
2019-07-25  9:46   ` Liam Merwick
2019-07-25  9:58     ` Michael S. Tsirkin
2019-07-25 10:03       ` Peter Maydell
2019-07-25 10:36       ` Paolo Bonzini
2019-07-02 12:11 ` [Qemu-devel] [PATCH v3 2/4] hw/i386: Add an Intel MPTable generator Sergio Lopez
2019-07-02 12:11 ` [Qemu-devel] [PATCH v3 3/4] hw/i386: Factorize PVH related functions Sergio Lopez
2019-07-23  8:39   ` Liam Merwick
2019-07-02 12:11 ` [Qemu-devel] [PATCH v3 4/4] hw/i386: Introduce the microvm machine type Sergio Lopez
2019-07-02 13:58   ` Gerd Hoffmann
2019-07-25 10:47   ` Paolo Bonzini
2019-07-02 15:01 ` [Qemu-devel] [PATCH v3 0/4] " no-reply
2019-07-02 15:23 ` Peter Maydell
2019-07-02 17:34   ` Sergio Lopez
2019-07-02 18:04     ` Peter Maydell
2019-07-02 22:04       ` Sergio Lopez
2019-07-25  9:59         ` Michael S. Tsirkin
2019-07-25 10:05           ` Peter Maydell
2019-07-25 10:10             ` Michael S. Tsirkin
2019-07-25 14:52               ` Sergio Lopez
2019-07-25 10:42             ` Sergio Lopez
2019-07-25 11:23               ` Paolo Bonzini
2019-07-25 12:01                 ` Stefan Hajnoczi
2019-07-25 12:10                   ` Michael S. Tsirkin
2019-07-25 13:26                     ` Stefan Hajnoczi
2019-07-25 13:43                       ` Paolo Bonzini
2019-07-25 13:54                         ` Michael S. Tsirkin
2019-07-25 14:13                           ` Paolo Bonzini
2019-07-25 14:42                             ` Michael S. Tsirkin
2019-07-25 14:04                         ` Peter Maydell
2019-07-25 14:26                           ` Paolo Bonzini
2019-07-25 14:35                             ` Michael S. Tsirkin
2019-07-25 14:42                         ` Sergio Lopez
2019-07-25 14:58                           ` Michael S. Tsirkin
2019-07-25 15:01                             ` Michael S. Tsirkin
2019-07-25 15:39                               ` Paolo Bonzini
2019-07-25 17:38                                 ` Michael S. Tsirkin
2019-07-26 12:46                                   ` Igor Mammedov
2019-07-25 15:49                               ` Sergio Lopez
2019-07-25 13:48                       ` Michael S. Tsirkin
2019-07-02 15:30 ` no-reply
2019-07-03  9:58 ` Stefan Hajnoczi
2019-07-18 15:21   ` Sergio Lopez
2019-07-19 10:29     ` Stefan Hajnoczi [this message]
2019-07-19 13:48       ` Sergio Lopez
2019-07-19 15:09         ` Stefan Hajnoczi
2019-07-19 15:42           ` Montes, Julio
2019-07-23  8:43             ` Sergio Lopez
2019-07-23  9:47               ` Stefan Hajnoczi
2019-07-23 10:01                 ` Paolo Bonzini
2019-07-24 11:14                   ` Paolo Bonzini
2019-07-25  9:35                     ` Sergio Lopez
2019-07-25 10:03                     ` Michael S. Tsirkin
2019-07-25 10:55                       ` Paolo Bonzini
2019-07-25 14:46                     ` Michael S. Tsirkin
2019-07-25 15:35                       ` Paolo Bonzini
2019-07-25 17:33                         ` Michael S. Tsirkin
2019-07-25 20:30                         ` Michael S. Tsirkin
2019-07-26  7:57                           ` Paolo Bonzini
2019-07-26 11:10                             ` Michael S. Tsirkin
2019-07-23 11:30                 ` Stefano Garzarella
2019-07-24 15:23                   ` Stefano Garzarella
2019-08-29  9:02 ` Jing Liu
2019-08-29 15:46   ` Sergio Lopez
2019-08-30  4:53     ` Jing Liu
2019-08-30 14:27       ` Sergio Lopez
2019-09-02  5:43         ` Jing Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190719102915.GG18585@stefanha-x1.localdomain \
    --to=stefanha@gmail.com \
    --cc=ehabkost@redhat.com \
    --cc=kraxel@redhat.com \
    --cc=maran.wilson@oracle.com \
    --cc=mst@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=rth@twiddle.net \
    --cc=sgarzare@redhat.com \
    --cc=slp@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.