Re: [PATCH v3 4/4] x86/kvm: add boot parameter for setting max number of vcpus per guest

From: Juergen Gross <jgross@suse.com>
To: Sean Christopherson <seanjc@google.com>
Cc: kvm@vger.kernel.org, x86@kernel.org, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org, Jonathan Corbet <corbet@lwn.net>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: Re: [PATCH v3 4/4] x86/kvm: add boot parameter for setting max number of vcpus per guest
Date: Thu, 18 Nov 2021 16:15:34 +0100	[thread overview]
Message-ID: <d5c57c27-e237-ef84-96c7-f50619597023@suse.com> (raw)
In-Reply-To: <YZZrzSi1rdaP0ETF@google.com>

[-- Attachment #1.1.1: Type: text/plain, Size: 2520 bytes --]

On 18.11.21 16:05, Sean Christopherson wrote:
> On Thu, Nov 18, 2021, Juergen Gross wrote:
>> On 17.11.21 21:57, Sean Christopherson wrote:
>>> Rather than makes this a module param, I would prefer to start with the below
>>> patch (originally from TDX pre-enabling) and then wire up a way for userspace to
>>> _lower_ the max on a per-VM basis, e.g. add a capability.
>>
>> The main reason for this whole series is a request by a partner
>> to enable huge VMs on huge machines (huge meaning thousands of
>> vcpus on thousands of physical cpus).
>>
>> Making this large number a compile time setting would hurt all
>> the users who have more standard requirements by allocating the
>> needed resources even on small systems, so I've switched to a boot
>> parameter in order to enable those huge numbers only when required.
>>
>> With Marc's series to use an xarray for the vcpu pointers only the
>> bitmaps for sending IRQs to vcpus are left which need to be sized
>> according to the max vcpu limit. Your patch below seems to be fine, but
>> doesn't help for that case.
> 
> Ah, you want to let userspace define a MAX_VCPUS that goes well beyond the current
> limit without negatively impacting existing setups.  My idea of a per-VM capability

Correct.

> still works, it would simply require separating the default max from the absolute
> max, which this patch mostly does already, it just neglects to set an absolute max.
> 
> Which is a good segue into pointing out that if a module param is added, it needs
> to be sanity checked against a KVM-defined max.  The admin may be trusted to some
> extent, but there is zero reason to let userspace set max_vcspus to 4 billion.
> At that point, it really is just a param vs. capability question.

I agree. Capping it at e.g. 65536 would probably be a good idea.

> I like the idea of a capability because there are already two known use cases,
> arm64's GIC and x86's TDX, and it could also be used to reduce the kernel's footprint
> for use cases that run large numbers of smaller VMs.
> 
> The other alternative would be to turn KVM_MAX_VCPUS into a Kconfig knob.  I assume

I like combining the capping and a Kconfig knob. So let the distro (or
whoever is building the kernel) decide, which is the max allowed value
(e.g. above 65536 per default).

> the partner isn't running a vanilla distro build and could set it as they see fit.

And here you are wrong. They'd like to use standard SUSE Linux (SLE).

Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]