Re: PV-vNUMA issue: topology is misinterpreted by the guest

From: Andrew Cooper <andrew.cooper3@citrix.com>
To: George Dunlap <George.Dunlap@eu.citrix.com>,
	Juergen Gross <jgross@suse.com>
Cc: Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	Wei Liu <wei.liu2@citrix.com>,
	Dario Faggioli <dario.faggioli@citrix.com>,
	David Vrabel <david.vrabel@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>
Subject: Re: PV-vNUMA issue: topology is misinterpreted by the guest
Date: Mon, 27 Jul 2015 11:54:27 +0100	[thread overview]
Message-ID: <55B60DE3.2050209@citrix.com> (raw)
In-Reply-To: <CAFLBxZbYR--Uq0bZxwSsX3HhUm5fa5U-Jtct6a=B8q8jS6WEfQ@mail.gmail.com>

On 27/07/15 11:43, George Dunlap wrote:
> On Mon, Jul 27, 2015 at 5:35 AM, Juergen Gross <jgross@suse.com> wrote:
>> On 07/24/2015 06:44 PM, Boris Ostrovsky wrote:
>>> On 07/24/2015 12:39 PM, Juergen Gross wrote:
>>>>
>>>>
>>>> I don't say mangling cpuids can't solve the scheduling problem. It
>>>> surely can. But it can't solve the scheduling problem without hiding
>>>> information like number of sockets or cores which might be required
>>>> for license purposes. If we don't care, fine.
>>>>
>>> (this is somewhat repeating the email I just sent)
>>>
>>> Why can's we construct socket/core info with CPUID (and *possibly* ACPI
>>> changes) that we present a reasonable (licensing-wise) picture?
>>>
>>> Can you suggest an example where it will not work and then maybe we can
>>> figure something out?
>>
>> Let's assume a software with license based on core count. You have a
>> system with a 2 8 core processors and hyperthreads enabled, summing up
>> to 32 logical processors. Your license is valid for up to 16 cores, so
>> running the software on bare metal on your system is fine.
>>
>> Now you are running the software inside a virtual machine with 24 vcpus
>> in a cpupool with 24 logical cpus limited to 12 cores (6 cores of each
>> processor). As we have to hide hyperthreading in order to not to have
>> to pin each vcpu to just a single logical processor, the topology
>> resulting from this picture will have to present 24 cores. The license
>> will not cover this hardware.
> But how does doing a PV topology help this situation?  Because we're
> telling one thing to the OS (via our PV interface) and another thing
> to applications (via direct CPUID access)?

I expressed exactly these concerns right back at the start of the vnuma
work.

The OS and its userspace can and will use cpuid.  Most examples will
only use cpuid.  The only thing worse that providing no NUMA information
at all is providing conflicting information between cpuid and vnuma.

IMO, HVM guests should get all their NUMA information from the same
sources as native hardware would provide.  PV guests are admittedly
harder as in generally we cannot hide the real topology information in
cpuid.

~Andrew