Re: [PATCH v3 1/4] x86/kvm: add boot parameter for adding vcpu-id bits

From: Juergen Gross <jgross@suse.com>
To: Sean Christopherson <seanjc@google.com>
Cc: kvm@vger.kernel.org, x86@kernel.org, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org, Jonathan Corbet <corbet@lwn.net>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: Re: [PATCH v3 1/4] x86/kvm: add boot parameter for adding vcpu-id bits
Date: Thu, 18 Nov 2021 08:45:19 +0100	[thread overview]
Message-ID: <731540b4-e8fc-0322-5aa0-e134bc55a397@suse.com> (raw)
In-Reply-To: <YZWUV2jvoOS9RSq8@google.com>

[-- Attachment #1.1.1: Type: text/plain, Size: 2440 bytes --]

On 18.11.21 00:46, Sean Christopherson wrote:
> On Wed, Nov 17, 2021, Juergen Gross wrote:
>> On 16.11.21 15:10, Juergen Gross wrote:
>>> Today the maximum vcpu-id of a kvm guest's vcpu on x86 systems is set
>>> via a #define in a header file.
>>>
>>> In order to support higher vcpu-ids without generally increasing the
>>> memory consumption of guests on the host (some guest structures contain
>>> arrays sized by KVM_MAX_VCPU_IDS) add a boot parameter for adding some
>>> bits to the vcpu-id. Additional bits are needed as the vcpu-id is
>>> constructed via bit-wise concatenation of socket-id, core-id, etc.
>>> As those ids maximum values are not always a power of 2, the vcpu-ids
>>> are sparse.
>>>
>>> The additional number of bits needed is basically the number of
>>> topology levels with a non-power-of-2 maximum value, excluding the top
>>> most level.
>>>
>>> The default value of the new parameter will be 2 in order to support
>>> today's possible topologies. The special value of -1 will use the
>>> number of bits needed for a guest with the current host's topology.
>>>
>>> Calculating the maximum vcpu-id dynamically requires to allocate the
>>> arrays using KVM_MAX_VCPU_IDS as the size dynamically.
>>>
>>> Signed-of-by: Juergen Gross <jgross@suse.com>
>>
>> Just thought about vcpu-ids a little bit more.
>>
>> It would be possible to replace the topology games completely by an
>> arbitrary rather high vcpu-id limit (65536?) and to allocate the memory
>> depending on the max vcpu-id just as needed.
>>
>> Right now the only vcpu-id dependent memory is for the ioapic consisting
>> of a vcpu-id indexed bitmap and a vcpu-id indexed byte array (vectors).
>>
>> We could start with a minimal size when setting up an ioapic and extend
>> the areas in case a new vcpu created would introduce a vcpu-id outside
>> the currently allocated memory. Both arrays are protected by the ioapic
>> specific lock (at least I couldn't spot any unprotected usage when
>> looking briefly into the code), so reallocating those arrays shouldn't
>> be hard. In case of ENOMEM the related vcpu creation would just fail.
>>
>> Thoughts?
> 
> Why not have userspace state the max vcpu_id it intends to creates on a per-VM
> basis?  Same end result, but doesn't require the complexity of reallocating the
> I/O APIC stuff.
> 

And if the userspace doesn't do it (like today)?

Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]