Re: [PATCH v2 0/3] KVM: x86: KVM_MEM_PCI_HOLE memory

From: "Michael S. Tsirkin" <mst@redhat.com>
To: Sean Christopherson <sean.j.christopherson@intel.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>,
	Peter Xu <peterx@redhat.com>,
	kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>,
	Julia Suvorova <jsuvorov@redhat.com>,
	Andy Lutomirski <luto@kernel.org>,
	Andrew Jones <drjones@redhat.com>,
	linux-kernel@vger.kernel.org, Gerd Hoffmann <kraxel@redhat.com>
Subject: Re: [PATCH v2 0/3] KVM: x86: KVM_MEM_PCI_HOLE memory
Date: Fri, 18 Sep 2020 08:34:37 -0400	[thread overview]
Message-ID: <20200918083134-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <20200911170031.GD4344@sjchrist-ice>

On Fri, Sep 11, 2020 at 10:00:31AM -0700, Sean Christopherson wrote:
> On Mon, Sep 07, 2020 at 07:32:23AM -0400, Michael S. Tsirkin wrote:
> > On Mon, Sep 07, 2020 at 10:37:39AM +0200, Vitaly Kuznetsov wrote:
> > > Sean Christopherson <sean.j.christopherson@intel.com> writes:
> > > 
> > > > On Fri, Sep 04, 2020 at 09:29:05AM +0200, Gerd Hoffmann wrote:
> > > >>   Hi,
> > > >> 
> > > >> > Unless I'm mistaken, microvm doesn't even support PCI, does it?
> > > >> 
> > > >> Correct, no pci support right now.
> > > >> 
> > > >> We could probably wire up ecam (arm/virt style) for pcie support, once
> > > >> the acpi support for mictovm finally landed (we need acpi for that
> > > >> because otherwise the kernel wouldn't find the pcie bus).
> > > >> 
> > > >> Question is whenever there is a good reason to do so.  Why would someone
> > > >> prefer microvm with pcie support over q35?
> > > >> 
> > > >> > If all of the above is true, this can be handled by adding "pci=lastbus=0"
> > > >> > as a guest kernel param to override its scanning of buses.  And couldn't
> > > >> > that be done by QEMU's microvm_fix_kernel_cmdline() to make it transparent
> > > >> > to the end user?
> > > >> 
> > > >> microvm_fix_kernel_cmdline() is a hack, not a solution.
> > > >> 
> > > >> Beside that I doubt this has much of an effect on microvm because
> > > >> it doesn't support pcie in the first place.
> > > >
> > > > I am so confused.  Vitaly, can you clarify exactly what QEMU VM type this
> > > > series is intended to help?  If this is for microvm, then why is the guest
> > > > doing PCI scanning in the first place?  If it's for q35, why is the
> > > > justification for microvm-like workloads?
> > > 
> > > I'm not exactly sure about the plans for particular machine types, the
> > > intention was to use this for pcie in QEMU in general so whatever
> > > machine type uses pcie will benefit. 
> > > 
> > > Now, it seems that we have a more sophisticated landscape. The
> > > optimization will only make sense to speed up boot so all 'traditional'
> > > VM types with 'traditional' firmware are out of
> > > question. 'Container-like' VMs seem to avoid PCI for now, I'm not sure
> > > if it's because they're in early stages of their development, because
> > > they can get away without PCI or, actually, because of slowness at boot
> > > (which we're trying to tackle with this feature). I'd definitely like to
> > > hear more what people think about this.
> > 
> > I suspect microvms will need pci eventually. I would much rather KVM
> > had an exit-less discovery mechanism in place by then because
> > learning from history if it doesn't they will do some kind of
> > hack on the kernel command line, and everyone will be stuck
> > supporting that for years ...
> 
> Is it not an option for the VMM to "accurately" enumerate the number of buses?
> E.g. if the VMM has devices on only bus 0, then enumerate that there is one
> bus so that the guest doesn't try and probe devices that can't possibly exist.
> Or is that completely non-sensical and/or violate PCIe spec?

There is some tension here, in that one way to make guest boot faster
is to defer hotplug of devices until after it booted.

-- 
MST