From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14047C4332F for ; Fri, 12 Nov 2021 14:02:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EFD4E60EE0 for ; Fri, 12 Nov 2021 14:02:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235070AbhKLOE7 (ORCPT ); Fri, 12 Nov 2021 09:04:59 -0500 Received: from mail.kernel.org ([198.145.29.99]:47218 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231617AbhKLOE4 (ORCPT ); Fri, 12 Nov 2021 09:04:56 -0500 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0148060F4F; Fri, 12 Nov 2021 14:02:06 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mlX7z-0051uE-PW; Fri, 12 Nov 2021 14:02:03 +0000 Date: Fri, 12 Nov 2021 14:02:03 +0000 Message-ID: <87k0hd8obo.wl-maz@kernel.org> From: Marc Zyngier To: Vitaly Kuznetsov Cc: kvm@vger.kernel.org, Paolo Bonzini , Sean Christopherson , Wanpeng Li , Jim Mattson , Eduardo Habkost , Andrew Jones , Huacai Chen , Aleksandar Markovic , Anup Patel , Paul Mackerras , Michael Ellerman , kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/5] KVM: arm64: Cap KVM_CAP_NR_VCPUS by KVM_CAP_MAX_VCPUS In-Reply-To: <875ysxg0s1.fsf@redhat.com> References: <20211111162746.100598-1-vkuznets@redhat.com> <20211111162746.100598-2-vkuznets@redhat.com> <875ysxg0s1.fsf@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: vkuznets@redhat.com, kvm@vger.kernel.org, pbonzini@redhat.com, seanjc@google.com, wanpengli@tencent.com, jmattson@google.com, ehabkost@redhat.com, drjones@redhat.com, chenhuacai@kernel.org, aleksandar.qemu.devel@gmail.com, anup.patel@wdc.com, paulus@ozlabs.org, mpe@ellerman.id.au, kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 12 Nov 2021 09:51:10 +0000, Vitaly Kuznetsov wrote: > > Marc Zyngier writes: > > > Hi Vitaly, > > > > On 2021-11-11 16:27, Vitaly Kuznetsov wrote: > >> It doesn't make sense to return the recommended maximum number of > >> vCPUs which exceeds the maximum possible number of vCPUs. > >> > >> Signed-off-by: Vitaly Kuznetsov > >> --- > >> arch/arm64/kvm/arm.c | 7 ++++++- > >> 1 file changed, 6 insertions(+), 1 deletion(-) > >> > >> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > >> index 7838e9fb693e..391dc7a921d5 100644 > >> --- a/arch/arm64/kvm/arm.c > >> +++ b/arch/arm64/kvm/arm.c > >> @@ -223,7 +223,12 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, > >> long ext) > >> r = 1; > >> break; > >> case KVM_CAP_NR_VCPUS: > >> - r = num_online_cpus(); > >> + if (kvm) > >> + r = min_t(unsigned int, num_online_cpus(), > >> + kvm->arch.max_vcpus); > >> + else > >> + r = min_t(unsigned int, num_online_cpus(), > >> + kvm_arm_default_max_vcpus()); > >> break; > >> case KVM_CAP_MAX_VCPUS: > >> case KVM_CAP_MAX_VCPU_ID: > > > > This looks odd. This means that depending on the phase userspace is > > in while initialising the VM, KVM_CAP_NR_VCPUS can return one thing > > or the other. > > > > For example, I create a VM on a 32 CPU system, NR_VCPUS says 32. > > I create a GICv2 interrupt controller, it now says 8. > > > > That's a change in behaviour that is visible by userspace > > Yes, I realize this is a userspace visible change. The reason I suggest > it is that logically, it seems very odd that the maximum recommended > number of vCPUs (KVM_CAP_NR_VCPUS) can be higher, than the maximum > supported number of vCPUs (KVM_CAP_MAX_VCPUS). I'm all for this change. > All userspaces which use > this information somehow should already contain some workaround for this > case. (maybe it's a rare one and nobody hit it yet or maybe there are no > userspaces using KVM_CAP_NR_VCPUS for anything besides complaining -- > like QEMU). > > I'd like KVM to be consistent across architectures and have the same > (similar) meaning for KVM_CAP_NR_VCPUS. Sure, but this is a pretty useless piece of information anyway. As Andrew pointed out, the information is available somewhere else, and all we need to do is to cap it to the number of supported vcpus, which is effectively a KVM limitation. Also, we are talking about representing the architecture to userspace. No amount of massaging is going to make an arm64 box look like an x86. > > which I'm keen on avoiding. I'd rather have the kvm and !kvm cases > > return the same thing. > > Forgive me my (ARM?) ignorance but what would it be then? If we go for > min(num_online_cpus(), kvm_arm_default_max_vcpus()) in both cases, cat > this can still go above KVM_CAP_MAX_VCPUS after vGIC is created? "min(num_online_cpus(), kvm_arm_default_max_vcpus())" is probably the right thing in all cases. Yes, KVM_CAP_NR_VCPUS will keep reporting more than the VM can actually support. But that's why we have KVM_CAP_MAX_VCPUS, which tells you now many vcpus you can create for a given configuration. This shows how useless KVM_CAP_NR_VCPUS is, and I wouldn't mind a documentation patch stating this. Thanks, M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E484FC433EF for ; Fri, 12 Nov 2021 14:03:39 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ABB0760EC0 for ; Fri, 12 Nov 2021 14:03:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org ABB0760EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=V47Z4weF0mJslyqJaxxPQrDaq0KYkvZ6q5RGq06YMtM=; b=hpi6amvtZ4fl7M Z4Ywuvvd64P9ormHjYC3iXmU5FUokwFvAXt5g00tO/ITvC2sNS8BriclRSE/u6WPA3K8C0J7DhWwm afU9UOSJ4+33H86WbGwyqH8opyFDZRu6dijoveHi9pRkYmpLGKP6CwL5I6ewiPW5mJPvahvLvxRib qmLEJqI3sK1h7buc4ccd+U5fDUhkqHOXeaJAMg36CTkTuqcV7lbvN2wZA3uB3tOvcXOPtr2QLv2Fw b0lTLzcIr5u66eCw2DdvgojQ5gUouqp+hsvCrjkwNuBn5ppcUPWVRMN3kmzHGVsNWOFX4VZKjeG9a V6lONmGN8SDTCOC8RkMg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mlX87-00Ah9M-L8; Fri, 12 Nov 2021 14:02:11 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mlX82-00Ah81-Et; Fri, 12 Nov 2021 14:02:07 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0148060F4F; Fri, 12 Nov 2021 14:02:06 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mlX7z-0051uE-PW; Fri, 12 Nov 2021 14:02:03 +0000 Date: Fri, 12 Nov 2021 14:02:03 +0000 Message-ID: <87k0hd8obo.wl-maz@kernel.org> From: Marc Zyngier To: Vitaly Kuznetsov Cc: kvm@vger.kernel.org, Paolo Bonzini , Sean Christopherson , Wanpeng Li , Jim Mattson , Eduardo Habkost , Andrew Jones , Huacai Chen , Aleksandar Markovic , Anup Patel , Paul Mackerras , Michael Ellerman , kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/5] KVM: arm64: Cap KVM_CAP_NR_VCPUS by KVM_CAP_MAX_VCPUS In-Reply-To: <875ysxg0s1.fsf@redhat.com> References: <20211111162746.100598-1-vkuznets@redhat.com> <20211111162746.100598-2-vkuznets@redhat.com> <875ysxg0s1.fsf@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: vkuznets@redhat.com, kvm@vger.kernel.org, pbonzini@redhat.com, seanjc@google.com, wanpengli@tencent.com, jmattson@google.com, ehabkost@redhat.com, drjones@redhat.com, chenhuacai@kernel.org, aleksandar.qemu.devel@gmail.com, anup.patel@wdc.com, paulus@ozlabs.org, mpe@ellerman.id.au, kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211112_060206_556710_2E168289 X-CRM114-Status: GOOD ( 39.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 12 Nov 2021 09:51:10 +0000, Vitaly Kuznetsov wrote: > > Marc Zyngier writes: > > > Hi Vitaly, > > > > On 2021-11-11 16:27, Vitaly Kuznetsov wrote: > >> It doesn't make sense to return the recommended maximum number of > >> vCPUs which exceeds the maximum possible number of vCPUs. > >> > >> Signed-off-by: Vitaly Kuznetsov > >> --- > >> arch/arm64/kvm/arm.c | 7 ++++++- > >> 1 file changed, 6 insertions(+), 1 deletion(-) > >> > >> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > >> index 7838e9fb693e..391dc7a921d5 100644 > >> --- a/arch/arm64/kvm/arm.c > >> +++ b/arch/arm64/kvm/arm.c > >> @@ -223,7 +223,12 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, > >> long ext) > >> r = 1; > >> break; > >> case KVM_CAP_NR_VCPUS: > >> - r = num_online_cpus(); > >> + if (kvm) > >> + r = min_t(unsigned int, num_online_cpus(), > >> + kvm->arch.max_vcpus); > >> + else > >> + r = min_t(unsigned int, num_online_cpus(), > >> + kvm_arm_default_max_vcpus()); > >> break; > >> case KVM_CAP_MAX_VCPUS: > >> case KVM_CAP_MAX_VCPU_ID: > > > > This looks odd. This means that depending on the phase userspace is > > in while initialising the VM, KVM_CAP_NR_VCPUS can return one thing > > or the other. > > > > For example, I create a VM on a 32 CPU system, NR_VCPUS says 32. > > I create a GICv2 interrupt controller, it now says 8. > > > > That's a change in behaviour that is visible by userspace > > Yes, I realize this is a userspace visible change. The reason I suggest > it is that logically, it seems very odd that the maximum recommended > number of vCPUs (KVM_CAP_NR_VCPUS) can be higher, than the maximum > supported number of vCPUs (KVM_CAP_MAX_VCPUS). I'm all for this change. > All userspaces which use > this information somehow should already contain some workaround for this > case. (maybe it's a rare one and nobody hit it yet or maybe there are no > userspaces using KVM_CAP_NR_VCPUS for anything besides complaining -- > like QEMU). > > I'd like KVM to be consistent across architectures and have the same > (similar) meaning for KVM_CAP_NR_VCPUS. Sure, but this is a pretty useless piece of information anyway. As Andrew pointed out, the information is available somewhere else, and all we need to do is to cap it to the number of supported vcpus, which is effectively a KVM limitation. Also, we are talking about representing the architecture to userspace. No amount of massaging is going to make an arm64 box look like an x86. > > which I'm keen on avoiding. I'd rather have the kvm and !kvm cases > > return the same thing. > > Forgive me my (ARM?) ignorance but what would it be then? If we go for > min(num_online_cpus(), kvm_arm_default_max_vcpus()) in both cases, cat > this can still go above KVM_CAP_MAX_VCPUS after vGIC is created? "min(num_online_cpus(), kvm_arm_default_max_vcpus())" is probably the right thing in all cases. Yes, KVM_CAP_NR_VCPUS will keep reporting more than the VM can actually support. But that's why we have KVM_CAP_MAX_VCPUS, which tells you now many vcpus you can create for a given configuration. This shows how useless KVM_CAP_NR_VCPUS is, and I wouldn't mind a documentation patch stating this. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel