problems starting kvm-qemu with 5+ virtual functions

problems starting kvm-qemu with 5+ virtual functions

[Chris Friesen]

Hi,

We're running into a problem where we can't start up a single instance 
of kvm-qemu with 5 or more virtual functions (for the ethernet card) 
being passed to the guest.  It's an Intel I350 NIC if it matters.

I noticed a discussion in a thread titled "[RFC PATCH 0/2] Expose 
available KVM free memory slot count to help avoid aborts" that seemed 
to point to the hardcoded definition of KVM_MEMORY_SLOTS as being a 
possible problem, and proposed some patches to allow dynamic sizing of 
the number of memory slots.  It looks like that patch never went through 
to mainline though.

Is this likely the source of my problem?  Why didn't it ever get 
submitted to mainline.

What's the proper method of supporting 8 VFs per VM, with possibly 
several VMs?

Thanks,
Chris

Re: problems starting kvm-qemu with 5+ virtual functions

[Alex Williamson]

On Wed, 2012-07-11 at 10:52 -0600, Chris Friesen wrote:
> Hi,
> 
> We're running into a problem where we can't start up a single instance 
> of kvm-qemu with 5 or more virtual functions (for the ethernet card) 
> being passed to the guest.  It's an Intel I350 NIC if it matters.
> 
> I noticed a discussion in a thread titled "[RFC PATCH 0/2] Expose 
> available KVM free memory slot count to help avoid aborts" that seemed 
> to point to the hardcoded definition of KVM_MEMORY_SLOTS as being a 
> possible problem, and proposed some patches to allow dynamic sizing of 
> the number of memory slots.  It looks like that patch never went through 
> to mainline though.
> 
> Is this likely the source of my problem?

Yes, that's the problem.

> Why didn't it ever get submitted to mainline.

There were issues with extending the number of available memory slots.
By itself, those patches just prevent an abort, but don't really let you
add more devices.

> What's the proper method of supporting 8 VFs per VM, with possibly 
> several VMs?

The limiting factor to increasing memory slots was searching the array.
That's since been fixed by caching mmio page table entries.  AIUI, Avi
is now willing to accept a patch to bump memory slots to a much higher
value.  The number of available slots can be queried by qemu using
KVM_CAP_NR_MEMSLOTS, then something like the RFC you point at could be
used to allow device assignment to take advantage of that.  Thanks,

Alex

Re: problems starting kvm-qemu with 5+ virtual functions

[Chris Friesen]

On 07/11/2012 01:34 PM, Alex Williamson wrote:
>
> The limiting factor to increasing memory slots was searching the array.
> That's since been fixed by caching mmio page table entries.

Thanks for the confirmation of my suspicions.

Do you know roughly when this went in?  A commit ID would be great.

Chris

Re: problems starting kvm-qemu with 5+ virtual functions

[Alex Williamson]

On Wed, 2012-07-11 at 13:56 -0600, Chris Friesen wrote:
> On 07/11/2012 01:34 PM, Alex Williamson wrote:
> >
> > The limiting factor to increasing memory slots was searching the array.
> > That's since been fixed by caching mmio page table entries.
> 
> Thanks for the confirmation of my suspicions.
> 
> Do you know roughly when this went in?  A commit ID would be great.

Almost exactly a year ago:

ce88decffd17bf9f373cc233c961ad2054965667

Re: problems starting kvm-qemu with 5+ virtual functions

[Chris Friesen]

On 07/11/2012 02:06 PM, Alex Williamson wrote:
> On Wed, 2012-07-11 at 13:56 -0600, Chris Friesen wrote:
>> On 07/11/2012 01:34 PM, Alex Williamson wrote:
>>> The limiting factor to increasing memory slots was searching the array.
>>> That's since been fixed by caching mmio page table entries.
>> Thanks for the confirmation of my suspicions.
>>
>> Do you know roughly when this went in?  A commit ID would be great.
> Almost exactly a year ago:
>
> ce88decffd17bf9f373cc233c961ad2054965667
>

Thanks, that's useful.

Chris