From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93984C433FE for ; Fri, 12 Nov 2021 09:51:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7D5A060F51 for ; Fri, 12 Nov 2021 09:51:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234827AbhKLJyL (ORCPT ); Fri, 12 Nov 2021 04:54:11 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:56464 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234793AbhKLJyK (ORCPT ); Fri, 12 Nov 2021 04:54:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1636710679; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Q3tNE6PVDp1t883O718wrcDgXu0HBWeQVivnEf04yl8=; b=RkO3qVub/Trsy4BlS9rHaB1lpRaw7zBkJbbB8wb7OJVsi7FSN2IAu1qTQRVwDxKnCEYBEr Z4bZY6LxqGIHdWnRNwGp1oMIUrmnuUlaD2rYAoCiCHHowtkpfIWZojrYfVYIm/UD4GBCId FgB/XfwUoyJQkALDpUgsbnFKrmIVlIw= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-181-FtreiHwyM8yqtIqBQrQrhw-1; Fri, 12 Nov 2021 04:51:14 -0500 X-MC-Unique: FtreiHwyM8yqtIqBQrQrhw-1 Received: by mail-ed1-f69.google.com with SMTP id f4-20020a50e084000000b003db585bc274so7754091edl.17 for ; Fri, 12 Nov 2021 01:51:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=Q3tNE6PVDp1t883O718wrcDgXu0HBWeQVivnEf04yl8=; b=GRcOwpLZ4CWu557wwTOk6aC3izhiI5hrKwsCwHayUzQW+/8zIChGxczb50vgIojHVQ tadk8gbLI9Gd464GE/4V7bvBZ5W9PAJztgyddA5tKG1WDLBkKvfrq+TnvfgSK926JzQ9 iUMaaGWlbsTlJwyiZFHL64LTM7Qg1MHfmrQyKmPe9omCUHWgXsNoYWeHHfjmut0hWVhL I5DBt3DxbwSJeQIM/GPC2ADsCoOWVqP68uVKGays04VDuzz6dS8WpMbj5lWz5qJpfiT9 LDo3YaUvmy+cC8Qisj3tILt/VRPRh9UsbQTIvOW9Bo6VHa6n4znw8W6arZbN8ROfE2e9 4IqA== X-Gm-Message-State: AOAM532FxPA12XfyKObw0sbntL7HKXAf9DtbSp4JJfpHwFRmVNIg0op8 T6gQLe0fPy22F+Ng++TKkAIBHBJvma/G1da+ncwJuecM9EZ4icMk3Y0Lhv63PiZCbc6R6onC3UG ptwwvbY0XNnOKs2OOApX4n2s0iLCGSiqvSKItr4uq060m1NdnoOYgkSNL6OwBrlTQjvm3K3ZPRL /h X-Received: by 2002:a05:6402:27d3:: with SMTP id c19mr19433495ede.2.1636710672898; Fri, 12 Nov 2021 01:51:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJzLDjahuFseIZ7e+RowWVF5yMybji+/f1TpttlCU7zwauHRgLTGqMPciQsWHxQYl16h4+En2Q== X-Received: by 2002:a05:6402:27d3:: with SMTP id c19mr19433437ede.2.1636710672630; Fri, 12 Nov 2021 01:51:12 -0800 (PST) Received: from fedora (g-server-2.ign.cz. [91.219.240.2]) by smtp.gmail.com with ESMTPSA id q14sm2845328edj.42.2021.11.12.01.51.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Nov 2021 01:51:12 -0800 (PST) From: Vitaly Kuznetsov To: Marc Zyngier Cc: kvm@vger.kernel.org, Paolo Bonzini , Sean Christopherson , Wanpeng Li , Jim Mattson , Eduardo Habkost , Andrew Jones , Huacai Chen , Aleksandar Markovic , Anup Patel , Paul Mackerras , Michael Ellerman , kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/5] KVM: arm64: Cap KVM_CAP_NR_VCPUS by KVM_CAP_MAX_VCPUS In-Reply-To: References: <20211111162746.100598-1-vkuznets@redhat.com> <20211111162746.100598-2-vkuznets@redhat.com> Date: Fri, 12 Nov 2021 10:51:10 +0100 Message-ID: <875ysxg0s1.fsf@redhat.com> MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Marc Zyngier writes: > Hi Vitaly, > > On 2021-11-11 16:27, Vitaly Kuznetsov wrote: >> It doesn't make sense to return the recommended maximum number of >> vCPUs which exceeds the maximum possible number of vCPUs. >> >> Signed-off-by: Vitaly Kuznetsov >> --- >> arch/arm64/kvm/arm.c | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c >> index 7838e9fb693e..391dc7a921d5 100644 >> --- a/arch/arm64/kvm/arm.c >> +++ b/arch/arm64/kvm/arm.c >> @@ -223,7 +223,12 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, >> long ext) >> r = 1; >> break; >> case KVM_CAP_NR_VCPUS: >> - r = num_online_cpus(); >> + if (kvm) >> + r = min_t(unsigned int, num_online_cpus(), >> + kvm->arch.max_vcpus); >> + else >> + r = min_t(unsigned int, num_online_cpus(), >> + kvm_arm_default_max_vcpus()); >> break; >> case KVM_CAP_MAX_VCPUS: >> case KVM_CAP_MAX_VCPU_ID: > > This looks odd. This means that depending on the phase userspace is > in while initialising the VM, KVM_CAP_NR_VCPUS can return one thing > or the other. > > For example, I create a VM on a 32 CPU system, NR_VCPUS says 32. > I create a GICv2 interrupt controller, it now says 8. > > That's a change in behaviour that is visible by userspace Yes, I realize this is a userspace visible change. The reason I suggest it is that logically, it seems very odd that the maximum recommended number of vCPUs (KVM_CAP_NR_VCPUS) can be higher, than the maximum supported number of vCPUs (KVM_CAP_MAX_VCPUS). All userspaces which use this information somehow should already contain some workaround for this case. (maybe it's a rare one and nobody hit it yet or maybe there are no userspaces using KVM_CAP_NR_VCPUS for anything besides complaining -- like QEMU). I'd like KVM to be consistent across architectures and have the same (similar) meaning for KVM_CAP_NR_VCPUS. > which I'm keen on avoiding. I'd rather have the kvm and !kvm cases > return the same thing. Forgive me my (ARM?) ignorance but what would it be then? If we go for min(num_online_cpus(), kvm_arm_default_max_vcpus()) in both cases, cat this can still go above KVM_CAP_MAX_VCPUS after vGIC is created? Thanks for the feedback! -- Vitaly From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EB5EC433F5 for ; Fri, 12 Nov 2021 09:52:42 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F328761038 for ; Fri, 12 Nov 2021 09:52:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F328761038 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=h0NCajCBfTlvsIO41yfRpZ67h9I5+C71IyWwMqNDtjA=; b=FMtfqoZaYC7v/R N+poLtf1LOYbd7/mIBhO+/XNowin58DLsOLNqPSlfjNy4ycn2sgQARKyp0sRRogfIYTkv5Wln2mJQ p9Qw/YH2lLswM+XywBZfpnjr7oJR6P5426FRtipmiclte02u4fcdOzILSuVyO2+tuf7r01B2eZxtJ ihmHf8z587l4gyxoTJVbauRlfe0xJOGebxUle9xz8LZfC+nANqHLdE+9bmVV8QC3WmHHVkpJJQFBG /CKjKLUxiBbIgu4KqIlHpanlIl8k1BwzfHDOpjZVZ3IZzxmGo/lV9mJ/mfikMEX9yH9h992E51PH8 SUc/8p4AGfEWPobByo8w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mlTDT-009tr0-S2; Fri, 12 Nov 2021 09:51:28 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mlTDP-009tnx-8J for linux-arm-kernel@lists.infradead.org; Fri, 12 Nov 2021 09:51:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1636710679; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Q3tNE6PVDp1t883O718wrcDgXu0HBWeQVivnEf04yl8=; b=RkO3qVub/Trsy4BlS9rHaB1lpRaw7zBkJbbB8wb7OJVsi7FSN2IAu1qTQRVwDxKnCEYBEr Z4bZY6LxqGIHdWnRNwGp1oMIUrmnuUlaD2rYAoCiCHHowtkpfIWZojrYfVYIm/UD4GBCId FgB/XfwUoyJQkALDpUgsbnFKrmIVlIw= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-500-8OoR-esOP_-N48BToII9fQ-1; Fri, 12 Nov 2021 04:51:14 -0500 X-MC-Unique: 8OoR-esOP_-N48BToII9fQ-1 Received: by mail-ed1-f71.google.com with SMTP id l15-20020a056402124f00b003e57269ab87so1157911edw.6 for ; Fri, 12 Nov 2021 01:51:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=Q3tNE6PVDp1t883O718wrcDgXu0HBWeQVivnEf04yl8=; b=ybQCWxI5KEeYMB+Uu78hxvyU0yPBbCm1fP6LkWpQ3W1kklSlHo7EZz8b/m0CDvsjcj OrqIvOfiyBSEfRwgFqIU8wGF9yjAZ5mCtVIkkSOxBP6sRoz/q8aW3zB/Byoqt7+zoWGb edcXvakYbcj953egtdb722Olb/PwhD1SEV72Iov3AzSYqy2FXPkLjZeGu2v/FdeF7nyV hID7ThN8gZ6b1AnpyLE0h7ZMRcm/OyLZo7GyICSP/7aY4I7BA+9VXLVRJwGUpwn6euEf 5/cstN2KFrsKwLmR9F08z0G9kGILN8qNR41AwFr/+Q4LoNpTml3y/ybvhRa1DjacI/gW 8c7A== X-Gm-Message-State: AOAM53331Fk+i3w0r9u0z1BYxvfQza1vBPvnDgcpIIqljdOgQalTUyp9 wWS46fYs4PEL40zH5Yj2M5fTWvjoHPYGnkrPtha6QjCYTcu2/KZd2sGtK9GV+wB8APgny6HrXxO ptl2pyXbOhjKhPYXCMHUpNFYozyClcZouiDk= X-Received: by 2002:a05:6402:27d3:: with SMTP id c19mr19433498ede.2.1636710672947; Fri, 12 Nov 2021 01:51:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJzLDjahuFseIZ7e+RowWVF5yMybji+/f1TpttlCU7zwauHRgLTGqMPciQsWHxQYl16h4+En2Q== X-Received: by 2002:a05:6402:27d3:: with SMTP id c19mr19433437ede.2.1636710672630; Fri, 12 Nov 2021 01:51:12 -0800 (PST) Received: from fedora (g-server-2.ign.cz. [91.219.240.2]) by smtp.gmail.com with ESMTPSA id q14sm2845328edj.42.2021.11.12.01.51.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Nov 2021 01:51:12 -0800 (PST) From: Vitaly Kuznetsov To: Marc Zyngier Cc: kvm@vger.kernel.org, Paolo Bonzini , Sean Christopherson , Wanpeng Li , Jim Mattson , Eduardo Habkost , Andrew Jones , Huacai Chen , Aleksandar Markovic , Anup Patel , Paul Mackerras , Michael Ellerman , kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/5] KVM: arm64: Cap KVM_CAP_NR_VCPUS by KVM_CAP_MAX_VCPUS In-Reply-To: References: <20211111162746.100598-1-vkuznets@redhat.com> <20211111162746.100598-2-vkuznets@redhat.com> Date: Fri, 12 Nov 2021 10:51:10 +0100 Message-ID: <875ysxg0s1.fsf@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=vkuznets@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211112_015123_398736_0377037D X-CRM114-Status: GOOD ( 24.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Marc Zyngier writes: > Hi Vitaly, > > On 2021-11-11 16:27, Vitaly Kuznetsov wrote: >> It doesn't make sense to return the recommended maximum number of >> vCPUs which exceeds the maximum possible number of vCPUs. >> >> Signed-off-by: Vitaly Kuznetsov >> --- >> arch/arm64/kvm/arm.c | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c >> index 7838e9fb693e..391dc7a921d5 100644 >> --- a/arch/arm64/kvm/arm.c >> +++ b/arch/arm64/kvm/arm.c >> @@ -223,7 +223,12 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, >> long ext) >> r = 1; >> break; >> case KVM_CAP_NR_VCPUS: >> - r = num_online_cpus(); >> + if (kvm) >> + r = min_t(unsigned int, num_online_cpus(), >> + kvm->arch.max_vcpus); >> + else >> + r = min_t(unsigned int, num_online_cpus(), >> + kvm_arm_default_max_vcpus()); >> break; >> case KVM_CAP_MAX_VCPUS: >> case KVM_CAP_MAX_VCPU_ID: > > This looks odd. This means that depending on the phase userspace is > in while initialising the VM, KVM_CAP_NR_VCPUS can return one thing > or the other. > > For example, I create a VM on a 32 CPU system, NR_VCPUS says 32. > I create a GICv2 interrupt controller, it now says 8. > > That's a change in behaviour that is visible by userspace Yes, I realize this is a userspace visible change. The reason I suggest it is that logically, it seems very odd that the maximum recommended number of vCPUs (KVM_CAP_NR_VCPUS) can be higher, than the maximum supported number of vCPUs (KVM_CAP_MAX_VCPUS). All userspaces which use this information somehow should already contain some workaround for this case. (maybe it's a rare one and nobody hit it yet or maybe there are no userspaces using KVM_CAP_NR_VCPUS for anything besides complaining -- like QEMU). I'd like KVM to be consistent across architectures and have the same (similar) meaning for KVM_CAP_NR_VCPUS. > which I'm keen on avoiding. I'd rather have the kvm and !kvm cases > return the same thing. Forgive me my (ARM?) ignorance but what would it be then? If we go for min(num_online_cpus(), kvm_arm_default_max_vcpus()) in both cases, cat this can still go above KVM_CAP_MAX_VCPUS after vGIC is created? Thanks for the feedback! -- Vitaly _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vitaly Kuznetsov Date: Fri, 12 Nov 2021 09:51:10 +0000 Subject: Re: [PATCH 1/5] KVM: arm64: Cap KVM_CAP_NR_VCPUS by KVM_CAP_MAX_VCPUS Message-Id: <875ysxg0s1.fsf@redhat.com> List-Id: References: <20211111162746.100598-1-vkuznets@redhat.com> <20211111162746.100598-2-vkuznets@redhat.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Marc Zyngier Cc: kvm@vger.kernel.org, Paolo Bonzini , Sean Christopherson , Wanpeng Li , Jim Mattson , Eduardo Habkost , Andrew Jones , Huacai Chen , Aleksandar Markovic , Anup Patel , Paul Mackerras , Michael Ellerman , kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Marc Zyngier writes: > Hi Vitaly, > > On 2021-11-11 16:27, Vitaly Kuznetsov wrote: >> It doesn't make sense to return the recommended maximum number of >> vCPUs which exceeds the maximum possible number of vCPUs. >> >> Signed-off-by: Vitaly Kuznetsov >> --- >> arch/arm64/kvm/arm.c | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c >> index 7838e9fb693e..391dc7a921d5 100644 >> --- a/arch/arm64/kvm/arm.c >> +++ b/arch/arm64/kvm/arm.c >> @@ -223,7 +223,12 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, >> long ext) >> r = 1; >> break; >> case KVM_CAP_NR_VCPUS: >> - r = num_online_cpus(); >> + if (kvm) >> + r = min_t(unsigned int, num_online_cpus(), >> + kvm->arch.max_vcpus); >> + else >> + r = min_t(unsigned int, num_online_cpus(), >> + kvm_arm_default_max_vcpus()); >> break; >> case KVM_CAP_MAX_VCPUS: >> case KVM_CAP_MAX_VCPU_ID: > > This looks odd. This means that depending on the phase userspace is > in while initialising the VM, KVM_CAP_NR_VCPUS can return one thing > or the other. > > For example, I create a VM on a 32 CPU system, NR_VCPUS says 32. > I create a GICv2 interrupt controller, it now says 8. > > That's a change in behaviour that is visible by userspace Yes, I realize this is a userspace visible change. The reason I suggest it is that logically, it seems very odd that the maximum recommended number of vCPUs (KVM_CAP_NR_VCPUS) can be higher, than the maximum supported number of vCPUs (KVM_CAP_MAX_VCPUS). All userspaces which use this information somehow should already contain some workaround for this case. (maybe it's a rare one and nobody hit it yet or maybe there are no userspaces using KVM_CAP_NR_VCPUS for anything besides complaining -- like QEMU). I'd like KVM to be consistent across architectures and have the same (similar) meaning for KVM_CAP_NR_VCPUS. > which I'm keen on avoiding. I'd rather have the kvm and !kvm cases > return the same thing. Forgive me my (ARM?) ignorance but what would it be then? If we go for min(num_online_cpus(), kvm_arm_default_max_vcpus()) in both cases, cat this can still go above KVM_CAP_MAX_VCPUS after vGIC is created? Thanks for the feedback! -- Vitaly