Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

From: Avi Kivity <avi@redhat.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"ddutile@redhat.com" <ddutile@redhat.com>,
	"mst@redhat.com" <mst@redhat.com>,
	"chrisw@redhat.com" <chrisw@redhat.com>
Subject: Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts
Date: Tue, 25 Jan 2011 19:11:36 +0200	[thread overview]
Message-ID: <4D3F0448.7030702@redhat.com> (raw)
In-Reply-To: <1295967460.3230.67.camel@x201>

On 01/25/2011 04:57 PM, Alex Williamson wrote:
> On Tue, 2011-01-25 at 12:23 +0200, Avi Kivity wrote:
> >  On 01/25/2011 07:37 AM, Alex Williamson wrote:
> >  >  On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote:
> >  >  >   I'll look at how we might be
> >  >  >   able to allocate slots on demand.  Thanks,
> >  >
> >  >  Here's a first cut just to see if this looks agreeable.  This allows the
> >  >  slot array to grow on demand.  This works with current userspace, as
> >  >  well as userspace trivially modified to double KVMState.slots and
> >  >  hotplugging enough pci-assign devices to exceed the previous limit (w/&
> >  >  w/o ept).  Hopefully I got the rcu bits correct.  Does this look like
> >  >  the right path?
> >
> >  This can be trivially exhausted to pin all RAM.
>
> What's a reasonable upper limit?  A PCI device can have at most 6 MMIO
> BARs, each taking one slot.

A BAR can take no slots, or several slots.  For example a BAR might have 
a framebuffer, followed by an off-screen framebuffer, followed by an 
mmio register area in one BAR.  You'd want the framebuffer to be dirty 
logged while the offscreen framebuffer is not tracked (so one slot for 
each) while the mmio range cannot be used as a slot.

That only holds for emulated devices.

>   It might also support MSI-X in a way that
> splits a BAR, so an absolute max of 7 slots per PCI device.  Assuming
> only 1 PCI bus for the moment and also assuming assigned devices are
> typically single function, 32 devices * 7 slots/device = 224 slots, so
> maybe a 256 limit?  Maybe we even bump it up to a limit of 64 devices
> with a slot limit of 512.  It would be easier in device assignment code
> to keep track of a number of devices limit than trying to guess whether
> slots will be available when we need them.

Sounds reasonable.  But we'd need a faster search for 512 slots.

-- 
error compiling committee.c: too many arguments to function