* [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs @ 2022-01-18 4:19 Reiji Watanabe 2022-01-18 4:19 ` [PATCH v2 2/2] KVM: arm64: selftests: Introduce vcpu_width_config Reiji Watanabe 2022-02-08 14:41 ` [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs Marc Zyngier 0 siblings, 2 replies; 8+ messages in thread From: Reiji Watanabe @ 2022-01-18 4:19 UTC (permalink / raw) To: Marc Zyngier, kvmarm Cc: kvm, linux-arm-kernel, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata, Reiji Watanabe KVM allows userspace to configure either all 32bit or 64bit vCPUs for a guest. At vCPU reset, vcpu_allowed_register_width() checks if the vcpu's register width is consistent with all other vCPUs'. Since the checking is done even against vCPUs that are not initialized (KVM_ARM_VCPU_INIT has not been done) yet, the uninitialized vCPUs are erroneously treated as 64bit vCPU, which causes the function to incorrectly detect a mixed-width VM. Introduce a new flag (el1_reg_width) in kvm_arch to indicates that the guest needs to be configured with all 32bit or 64bit vCPUs, and initialize it at the first KVM_ARM_VCPU_INIT for the guest. Check vcpu's register width against the flag at the vcpu's KVM_ARM_VCPU_INIT (instead of against other vCPUs' register width). Fixes: 66e94d5cafd4 ("KVM: arm64: Prevent mixed-width VM creation") Signed-off-by: Reiji Watanabe <reijiw@google.com> --- arch/arm64/include/asm/kvm_host.h | 13 +++++++++++++ arch/arm64/kvm/arm.c | 30 ++++++++++++++++++++++++++++++ arch/arm64/kvm/reset.c | 8 -------- 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 2a5f7f38006f..c02b7caf2c82 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -102,6 +102,12 @@ struct kvm_s2_mmu { struct kvm_arch_memory_slot { }; +enum kvm_el1_reg_width { + EL1_WIDTH_UNINITIALIZED = 0, + EL1_32BIT, + EL1_64BIT, +}; + struct kvm_arch { struct kvm_s2_mmu mmu; @@ -137,6 +143,13 @@ struct kvm_arch { /* Memory Tagging Extension enabled for the guest */ bool mte_enabled; + + /* + * EL1 register width for the guest. + * This is set at the first KVM_ARM_VCPU_INIT for the guest based + * on whether the vcpu has KVM_ARM_VCPU_EL1_32BIT or not. + */ + enum kvm_el1_reg_width el1_reg_width; }; struct kvm_vcpu_fault_info { diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e4727dc771bf..54ae8bf9d187 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1058,6 +1058,32 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level, return -EINVAL; } +/* + * A guest can have either all 32bit or 64bit vcpus only. + * Either one the guest has is indicated in kvm->arch.el1_reg_width. + * Check if the vcpu's register width is consistent with + * kvm->arch.el1_reg_width. If kvm->arch.el1_reg_width is not set yet, + * set it based on the vcpu's KVM_ARM_VCPU_EL1_32BIT configuration. + */ +static int kvm_register_width_check_or_init(struct kvm_vcpu *vcpu) +{ + bool is32bit; + bool allowed = true; + struct kvm *kvm = vcpu->kvm; + + is32bit = vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT); + + mutex_lock(&kvm->lock); + + if (kvm->arch.el1_reg_width == EL1_WIDTH_UNINITIALIZED) + kvm->arch.el1_reg_width = is32bit ? EL1_32BIT : EL1_64BIT; + else + allowed = (is32bit == (kvm->arch.el1_reg_width == EL1_32BIT)); + + mutex_unlock(&kvm->lock); + return allowed ? 0 : -EINVAL; +} + static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu, const struct kvm_vcpu_init *init) { @@ -1097,6 +1123,10 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu, /* Now we know what it is, we can reset it. */ ret = kvm_reset_vcpu(vcpu); + + if (!ret) + ret = kvm_register_width_check_or_init(vcpu); + if (ret) { vcpu->arch.target = -1; bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES); diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 426bd7fbc3fd..dbf2939a6a96 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -168,9 +168,7 @@ static int kvm_vcpu_enable_ptrauth(struct kvm_vcpu *vcpu) static bool vcpu_allowed_register_width(struct kvm_vcpu *vcpu) { - struct kvm_vcpu *tmp; bool is32bit; - int i; is32bit = vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT); if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1) && is32bit) @@ -180,12 +178,6 @@ static bool vcpu_allowed_register_width(struct kvm_vcpu *vcpu) if (kvm_has_mte(vcpu->kvm) && is32bit) return false; - /* Check that the vcpus are either all 32bit or all 64bit */ - kvm_for_each_vcpu(i, tmp, vcpu->kvm) { - if (vcpu_has_feature(tmp, KVM_ARM_VCPU_EL1_32BIT) != is32bit) - return false; - } - return true; } base-commit: 37144b2c855f9311c72f292125061d4a52d02856 -- 2.34.1.703.g22d0c6ccf7-goog ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 2/2] KVM: arm64: selftests: Introduce vcpu_width_config 2022-01-18 4:19 [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs Reiji Watanabe @ 2022-01-18 4:19 ` Reiji Watanabe 2022-02-08 14:41 ` [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs Marc Zyngier 1 sibling, 0 replies; 8+ messages in thread From: Reiji Watanabe @ 2022-01-18 4:19 UTC (permalink / raw) To: Marc Zyngier, kvmarm Cc: kvm, linux-arm-kernel, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata, Reiji Watanabe, Andrew Jones Introduce a test for aarch64 that ensures non-mixed-width vCPUs (all 64bit vCPUs or all 32bit vcPUs) can be configured, and mixed-width vCPUs cannot be configured. Reviewed-by: Andrew Jones <drjones@redhat.com> Signed-off-by: Reiji Watanabe <reijiw@google.com> --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/vcpu_width_config.c | 125 ++++++++++++++++++ 3 files changed, 127 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/vcpu_width_config.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index 3cb5ac5da087..8795a83cc382 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -3,6 +3,7 @@ /aarch64/debug-exceptions /aarch64/get-reg-list /aarch64/psci_cpu_on_test +/aarch64/vcpu_width_config /aarch64/vgic_init /s390x/memop /s390x/resets diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 17342b575e85..259e01d0735a 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -95,6 +95,7 @@ TEST_GEN_PROGS_aarch64 += aarch64/arch_timer TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/psci_cpu_on_test +TEST_GEN_PROGS_aarch64 += aarch64/vcpu_width_config TEST_GEN_PROGS_aarch64 += aarch64/vgic_init TEST_GEN_PROGS_aarch64 += demand_paging_test TEST_GEN_PROGS_aarch64 += dirty_log_test diff --git a/tools/testing/selftests/kvm/aarch64/vcpu_width_config.c b/tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file mode 100644 index 000000000000..cd238e068236 --- /dev/null +++ b/tools/testing/selftests/kvm/aarch64/vcpu_width_config.c @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * vcpu_width_config - Test KVM_ARM_VCPU_INIT() with KVM_ARM_VCPU_EL1_32BIT. + * + * Copyright (c) 2022 Google LLC. + * + * This is a test that ensures that non-mixed-width vCPUs (all 64bit vCPUs + * or all 32bit vcPUs) can be configured and mixed-width vCPUs cannot be + * configured. + */ + +#define _GNU_SOURCE + +#include "kvm_util.h" +#include "processor.h" +#include "test_util.h" + + +/* + * Add a vCPU, run KVM_ARM_VCPU_INIT with @init1, and then + * add another vCPU, and run KVM_ARM_VCPU_INIT with @init2. + */ +int add_init_2vcpus(struct kvm_vcpu_init *init1, + struct kvm_vcpu_init *init2) +{ + struct kvm_vm *vm; + int ret; + + vm = vm_create(VM_MODE_DEFAULT, DEFAULT_GUEST_PHY_PAGES, O_RDWR); + + vm_vcpu_add(vm, 0); + ret = _vcpu_ioctl(vm, 0, KVM_ARM_VCPU_INIT, init1); + if (ret) + goto free_exit; + + vm_vcpu_add(vm, 1); + ret = _vcpu_ioctl(vm, 1, KVM_ARM_VCPU_INIT, init2); + +free_exit: + kvm_vm_free(vm); + return ret; +} + +/* + * Add two vCPUs, then run KVM_ARM_VCPU_INIT for one vCPU with @init1, + * and run KVM_ARM_VCPU_INIT for another vCPU with @init2. + */ +int add_2vcpus_init_2vcpus(struct kvm_vcpu_init *init1, + struct kvm_vcpu_init *init2) +{ + struct kvm_vm *vm; + int ret; + + vm = vm_create(VM_MODE_DEFAULT, DEFAULT_GUEST_PHY_PAGES, O_RDWR); + + vm_vcpu_add(vm, 0); + vm_vcpu_add(vm, 1); + + ret = _vcpu_ioctl(vm, 0, KVM_ARM_VCPU_INIT, init1); + if (ret) + goto free_exit; + + ret = _vcpu_ioctl(vm, 1, KVM_ARM_VCPU_INIT, init2); + +free_exit: + kvm_vm_free(vm); + return ret; +} + +/* + * Tests that two 64bit vCPUs can be configured, two 32bit vCPUs can be + * configured, and two mixed-witgh vCPUs cannot be configured. + * Each of those three cases, configure vCPUs in two different orders. + * The one is running KVM_CREATE_VCPU for 2 vCPUs, and then running + * KVM_ARM_VCPU_INIT for them. + * The other is running KVM_CREATE_VCPU and KVM_ARM_VCPU_INIT for a vCPU, + * and then run those commands for another vCPU. + */ +int main(void) +{ + struct kvm_vcpu_init init1, init2; + struct kvm_vm *vm; + int ret; + + if (kvm_check_cap(KVM_CAP_ARM_EL1_32BIT) <= 0) { + print_skip("KVM_CAP_ARM_EL1_32BIT is not supported"); + exit(KSFT_SKIP); + } + + /* Get the preferred target type and copy that to init2 */ + vm = vm_create(VM_MODE_DEFAULT, DEFAULT_GUEST_PHY_PAGES, O_RDWR); + vm_ioctl(vm, KVM_ARM_PREFERRED_TARGET, &init1); + kvm_vm_free(vm); + memcpy(&init2, &init1, sizeof(init2)); + + /* Test with 64bit vCPUs */ + ret = add_init_2vcpus(&init1, &init2); + TEST_ASSERT(ret == 0, + "Configuring 64bit EL1 vCPUs failed unexpectedly"); + ret = add_2vcpus_init_2vcpus(&init1, &init2); + TEST_ASSERT(ret == 0, + "Configuring 64bit EL1 vCPUs failed unexpectedly"); + + /* Test with 32bit vCPUs */ + init1.features[0] = (1 << KVM_ARM_VCPU_EL1_32BIT); + init2.features[0] = (1 << KVM_ARM_VCPU_EL1_32BIT); + ret = add_init_2vcpus(&init1, &init2); + TEST_ASSERT(ret == 0, + "Configuring 32bit EL1 vCPUs failed unexpectedly"); + ret = add_2vcpus_init_2vcpus(&init1, &init2); + TEST_ASSERT(ret == 0, + "Configuring 32bit EL1 vCPUs failed unexpectedly"); + + /* Test with mixed-width vCPUs */ + init1.features[0] = 0; + init2.features[0] = (1 << KVM_ARM_VCPU_EL1_32BIT); + ret = add_init_2vcpus(&init1, &init2); + TEST_ASSERT(ret != 0, + "Configuring mixed-width vCPUs worked unexpectedly"); + ret = add_2vcpus_init_2vcpus(&init1, &init2); + TEST_ASSERT(ret != 0, + "Configuring mixed-width vCPUs worked unexpectedly"); + + return 0; +} -- 2.34.1.703.g22d0c6ccf7-goog ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs 2022-01-18 4:19 [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs Reiji Watanabe 2022-01-18 4:19 ` [PATCH v2 2/2] KVM: arm64: selftests: Introduce vcpu_width_config Reiji Watanabe @ 2022-02-08 14:41 ` Marc Zyngier 2022-02-09 5:32 ` Reiji Watanabe 1 sibling, 1 reply; 8+ messages in thread From: Marc Zyngier @ 2022-02-08 14:41 UTC (permalink / raw) To: Reiji Watanabe Cc: kvmarm, kvm, linux-arm-kernel, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata On Tue, 18 Jan 2022 04:19:22 +0000, Reiji Watanabe <reijiw@google.com> wrote: > > KVM allows userspace to configure either all 32bit or 64bit vCPUs > for a guest. At vCPU reset, vcpu_allowed_register_width() checks > if the vcpu's register width is consistent with all other vCPUs'. > Since the checking is done even against vCPUs that are not initialized > (KVM_ARM_VCPU_INIT has not been done) yet, the uninitialized vCPUs > are erroneously treated as 64bit vCPU, which causes the function to > incorrectly detect a mixed-width VM. > > Introduce a new flag (el1_reg_width) in kvm_arch to indicates that > the guest needs to be configured with all 32bit or 64bit vCPUs, > and initialize it at the first KVM_ARM_VCPU_INIT for the guest. > Check vcpu's register width against the flag at the vcpu's > KVM_ARM_VCPU_INIT (instead of against other vCPUs' register width). > > Fixes: 66e94d5cafd4 ("KVM: arm64: Prevent mixed-width VM creation") > Signed-off-by: Reiji Watanabe <reijiw@google.com> > --- > arch/arm64/include/asm/kvm_host.h | 13 +++++++++++++ > arch/arm64/kvm/arm.c | 30 ++++++++++++++++++++++++++++++ > arch/arm64/kvm/reset.c | 8 -------- > 3 files changed, 43 insertions(+), 8 deletions(-) > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index 2a5f7f38006f..c02b7caf2c82 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -102,6 +102,12 @@ struct kvm_s2_mmu { > struct kvm_arch_memory_slot { > }; > > +enum kvm_el1_reg_width { > + EL1_WIDTH_UNINITIALIZED = 0, > + EL1_32BIT, > + EL1_64BIT, > +}; > + > struct kvm_arch { > struct kvm_s2_mmu mmu; > > @@ -137,6 +143,13 @@ struct kvm_arch { > > /* Memory Tagging Extension enabled for the guest */ > bool mte_enabled; > + > + /* > + * EL1 register width for the guest. > + * This is set at the first KVM_ARM_VCPU_INIT for the guest based > + * on whether the vcpu has KVM_ARM_VCPU_EL1_32BIT or not. > + */ > + enum kvm_el1_reg_width el1_reg_width; I really don't like that we need to keep track of yet another bit of state on top of the existing one. Duplicating state is a source of bugs, because you always end up checking the wrong one at the wrong time (and I have scars to prove it). > }; > > struct kvm_vcpu_fault_info { > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index e4727dc771bf..54ae8bf9d187 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -1058,6 +1058,32 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level, > return -EINVAL; > } > > +/* > + * A guest can have either all 32bit or 64bit vcpus only. That's not strictly true. All we are enforcing is that EL1 is either 32 or 64bit. > + * Either one the guest has is indicated in kvm->arch.el1_reg_width. > + * Check if the vcpu's register width is consistent with > + * kvm->arch.el1_reg_width. If kvm->arch.el1_reg_width is not set yet, > + * set it based on the vcpu's KVM_ARM_VCPU_EL1_32BIT configuration. > + */ > +static int kvm_register_width_check_or_init(struct kvm_vcpu *vcpu) > +{ > + bool is32bit; > + bool allowed = true; > + struct kvm *kvm = vcpu->kvm; > + > + is32bit = vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT); > + > + mutex_lock(&kvm->lock); > + > + if (kvm->arch.el1_reg_width == EL1_WIDTH_UNINITIALIZED) > + kvm->arch.el1_reg_width = is32bit ? EL1_32BIT : EL1_64BIT; > + else > + allowed = (is32bit == (kvm->arch.el1_reg_width == EL1_32BIT)); > + > + mutex_unlock(&kvm->lock); > + return allowed ? 0 : -EINVAL; > +} > + > static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu, > const struct kvm_vcpu_init *init) > { > @@ -1097,6 +1123,10 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu, > > /* Now we know what it is, we can reset it. */ > ret = kvm_reset_vcpu(vcpu); > + > + if (!ret) > + ret = kvm_register_width_check_or_init(vcpu); > + > if (ret) { > vcpu->arch.target = -1; > bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES); > diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c > index 426bd7fbc3fd..dbf2939a6a96 100644 > --- a/arch/arm64/kvm/reset.c > +++ b/arch/arm64/kvm/reset.c > @@ -168,9 +168,7 @@ static int kvm_vcpu_enable_ptrauth(struct kvm_vcpu *vcpu) > > static bool vcpu_allowed_register_width(struct kvm_vcpu *vcpu) > { > - struct kvm_vcpu *tmp; > bool is32bit; > - int i; > > is32bit = vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT); > if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1) && is32bit) > @@ -180,12 +178,6 @@ static bool vcpu_allowed_register_width(struct kvm_vcpu *vcpu) > if (kvm_has_mte(vcpu->kvm) && is32bit) > return false; > > - /* Check that the vcpus are either all 32bit or all 64bit */ > - kvm_for_each_vcpu(i, tmp, vcpu->kvm) { > - if (vcpu_has_feature(tmp, KVM_ARM_VCPU_EL1_32BIT) != is32bit) > - return false; > - } > - In [1], I suggested another approach that didn't require extra state, and moved the existing checks under the kvm lock. What was wrong with that approach? Thanks, M. [1] https://lore.kernel.org/r/875yqqtn5q.wl-maz@kernel.org -- Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs 2022-02-08 14:41 ` [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs Marc Zyngier @ 2022-02-09 5:32 ` Reiji Watanabe 2022-02-09 12:04 ` Marc Zyngier 0 siblings, 1 reply; 8+ messages in thread From: Reiji Watanabe @ 2022-02-09 5:32 UTC (permalink / raw) To: Marc Zyngier Cc: kvmarm, kvm, Linux ARM, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata Hi Marc, On Tue, Feb 8, 2022 at 6:41 AM Marc Zyngier <maz@kernel.org> wrote: > > On Tue, 18 Jan 2022 04:19:22 +0000, > Reiji Watanabe <reijiw@google.com> wrote: > > > > KVM allows userspace to configure either all 32bit or 64bit vCPUs > > for a guest. At vCPU reset, vcpu_allowed_register_width() checks > > if the vcpu's register width is consistent with all other vCPUs'. > > Since the checking is done even against vCPUs that are not initialized > > (KVM_ARM_VCPU_INIT has not been done) yet, the uninitialized vCPUs > > are erroneously treated as 64bit vCPU, which causes the function to > > incorrectly detect a mixed-width VM. > > > > Introduce a new flag (el1_reg_width) in kvm_arch to indicates that > > the guest needs to be configured with all 32bit or 64bit vCPUs, > > and initialize it at the first KVM_ARM_VCPU_INIT for the guest. > > Check vcpu's register width against the flag at the vcpu's > > KVM_ARM_VCPU_INIT (instead of against other vCPUs' register width). > > > > Fixes: 66e94d5cafd4 ("KVM: arm64: Prevent mixed-width VM creation") > > Signed-off-by: Reiji Watanabe <reijiw@google.com> > > --- > > arch/arm64/include/asm/kvm_host.h | 13 +++++++++++++ > > arch/arm64/kvm/arm.c | 30 ++++++++++++++++++++++++++++++ > > arch/arm64/kvm/reset.c | 8 -------- > > 3 files changed, 43 insertions(+), 8 deletions(-) > > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > > index 2a5f7f38006f..c02b7caf2c82 100644 > > --- a/arch/arm64/include/asm/kvm_host.h > > +++ b/arch/arm64/include/asm/kvm_host.h > > @@ -102,6 +102,12 @@ struct kvm_s2_mmu { > > struct kvm_arch_memory_slot { > > }; > > > > +enum kvm_el1_reg_width { > > + EL1_WIDTH_UNINITIALIZED = 0, > > + EL1_32BIT, > > + EL1_64BIT, > > +}; > > + > > struct kvm_arch { > > struct kvm_s2_mmu mmu; > > > > @@ -137,6 +143,13 @@ struct kvm_arch { > > > > /* Memory Tagging Extension enabled for the guest */ > > bool mte_enabled; > > + > > + /* > > + * EL1 register width for the guest. > > + * This is set at the first KVM_ARM_VCPU_INIT for the guest based > > + * on whether the vcpu has KVM_ARM_VCPU_EL1_32BIT or not. > > + */ > > + enum kvm_el1_reg_width el1_reg_width; > > I really don't like that we need to keep track of yet another bit of > state on top of the existing one. Duplicating state is a source of > bugs, because you always end up checking the wrong one at the wrong > time (and I have scars to prove it). > > > }; > > > > struct kvm_vcpu_fault_info { > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > > index e4727dc771bf..54ae8bf9d187 100644 > > --- a/arch/arm64/kvm/arm.c > > +++ b/arch/arm64/kvm/arm.c > > @@ -1058,6 +1058,32 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level, > > return -EINVAL; > > } > > > > +/* > > + * A guest can have either all 32bit or 64bit vcpus only. > > That's not strictly true. All we are enforcing is that EL1 is either > 32 or 64bit. I will fix the comment. > > > + * Either one the guest has is indicated in kvm->arch.el1_reg_width. > > + * Check if the vcpu's register width is consistent with > > + * kvm->arch.el1_reg_width. If kvm->arch.el1_reg_width is not set yet, > > + * set it based on the vcpu's KVM_ARM_VCPU_EL1_32BIT configuration. > > + */ > > +static int kvm_register_width_check_or_init(struct kvm_vcpu *vcpu) > > +{ > > + bool is32bit; > > + bool allowed = true; > > + struct kvm *kvm = vcpu->kvm; > > + > > + is32bit = vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT); > > + > > + mutex_lock(&kvm->lock); > > + > > + if (kvm->arch.el1_reg_width == EL1_WIDTH_UNINITIALIZED) > > + kvm->arch.el1_reg_width = is32bit ? EL1_32BIT : EL1_64BIT; > > + else > > + allowed = (is32bit == (kvm->arch.el1_reg_width == EL1_32BIT)); > > + > > + mutex_unlock(&kvm->lock); > > + return allowed ? 0 : -EINVAL; > > +} > > + > > static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu, > > const struct kvm_vcpu_init *init) > > { > > @@ -1097,6 +1123,10 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu, > > > > /* Now we know what it is, we can reset it. */ > > ret = kvm_reset_vcpu(vcpu); > > + > > + if (!ret) > > + ret = kvm_register_width_check_or_init(vcpu); > > + > > if (ret) { > > vcpu->arch.target = -1; > > bitmap_zero(vcpu->arch.features, KVM_VCPU_MAX_FEATURES); > > diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c > > index 426bd7fbc3fd..dbf2939a6a96 100644 > > --- a/arch/arm64/kvm/reset.c > > +++ b/arch/arm64/kvm/reset.c > > @@ -168,9 +168,7 @@ static int kvm_vcpu_enable_ptrauth(struct kvm_vcpu *vcpu) > > > > static bool vcpu_allowed_register_width(struct kvm_vcpu *vcpu) > > { > > - struct kvm_vcpu *tmp; > > bool is32bit; > > - int i; > > > > is32bit = vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT); > > if (!cpus_have_const_cap(ARM64_HAS_32BIT_EL1) && is32bit) > > @@ -180,12 +178,6 @@ static bool vcpu_allowed_register_width(struct kvm_vcpu *vcpu) > > if (kvm_has_mte(vcpu->kvm) && is32bit) > > return false; > > > > - /* Check that the vcpus are either all 32bit or all 64bit */ > > - kvm_for_each_vcpu(i, tmp, vcpu->kvm) { > > - if (vcpu_has_feature(tmp, KVM_ARM_VCPU_EL1_32BIT) != is32bit) > > - return false; > > - } > > - > > In [1], I suggested another approach that didn't require extra state, > and moved the existing checks under the kvm lock. What was wrong with > that approach? With that approach, even for a vcpu that has a broken set of features, which leads kvm_reset_vcpu() to fail for the vcpu, the vcpu->arch.features are checked by other vCPUs' vcpu_allowed_register_width() until the vcpu->arch.target is set to -1. Due to this, I would think some or possibly all vCPUs' kvm_reset_vcpu() may or may not fail (e.g. if userspace tries to configure vCPU#0 with 32bit EL1, and vCPU#1 and #2 with 64 bit EL1, KVM_ARM_VCPU_INIT for either vCPU#0, or both vCPU#1 and #2 should fail. But, with that approach, it doesn't always work that way. Instead, KVM_ARM_VCPU_INIT for all vCPUs could fail or KVM_ARM_VCPU_INIT for vCPU#0 and #1 could fail while the one for CPU#2 works). Also, even after the first KVM_RUN for vCPUs are already done, (the first) KVM_ARM_VCPU_INIT for another vCPU could cause the kvm_reset_vcpu() for those vCPUs to fail. I would think those behaviors are odd, and I wanted to avoid them. Thanks, Reiji > > Thanks, > > M. > > [1] https://lore.kernel.org/r/875yqqtn5q.wl-maz@kernel.org > > -- > Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs 2022-02-09 5:32 ` Reiji Watanabe @ 2022-02-09 12:04 ` Marc Zyngier 2022-02-10 5:31 ` Reiji Watanabe 0 siblings, 1 reply; 8+ messages in thread From: Marc Zyngier @ 2022-02-09 12:04 UTC (permalink / raw) To: Reiji Watanabe Cc: kvmarm, kvm, Linux ARM, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata Hi Reiji, On Wed, 09 Feb 2022 05:32:36 +0000, Reiji Watanabe <reijiw@google.com> wrote: > > Hi Marc, > > On Tue, Feb 8, 2022 at 6:41 AM Marc Zyngier <maz@kernel.org> wrote: > > > > In [1], I suggested another approach that didn't require extra state, > > and moved the existing checks under the kvm lock. What was wrong with > > that approach? > > With that approach, even for a vcpu that has a broken set of features, > which leads kvm_reset_vcpu() to fail for the vcpu, the vcpu->arch.features > are checked by other vCPUs' vcpu_allowed_register_width() until the > vcpu->arch.target is set to -1. > Due to this, I would think some or possibly all vCPUs' kvm_reset_vcpu() > may or may not fail (e.g. if userspace tries to configure vCPU#0 with > 32bit EL1, and vCPU#1 and #2 with 64 bit EL1, KVM_ARM_VCPU_INIT > for either vCPU#0, or both vCPU#1 and #2 should fail. But, with that > approach, it doesn't always work that way. Instead, KVM_ARM_VCPU_INIT > for all vCPUs could fail or KVM_ARM_VCPU_INIT for vCPU#0 and #1 could > fail while the one for CPU#2 works). > Also, even after the first KVM_RUN for vCPUs are already done, > (the first) KVM_ARM_VCPU_INIT for another vCPU could cause the > kvm_reset_vcpu() for those vCPUs to fail. > > I would think those behaviors are odd, and I wanted to avoid them. OK, fair enough. But then you need to remove most of the uses of KVM_ARM_VCPU_EL1_32BIT so that it is only used as a userspace interface and maybe not carried as part of the vcpu feature flag anymore. Also, we really should turn all these various bits in the kvm struct into a set of flags. I have a patch posted there[1] for this, feel free to pick it up. Thanks, M. [1] https://lore.kernel.org/r/20211004174849.2831548-2-maz@kernel.org -- Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs 2022-02-09 12:04 ` Marc Zyngier @ 2022-02-10 5:31 ` Reiji Watanabe 2022-02-10 10:31 ` Marc Zyngier 0 siblings, 1 reply; 8+ messages in thread From: Reiji Watanabe @ 2022-02-10 5:31 UTC (permalink / raw) To: Marc Zyngier Cc: kvmarm, kvm, Linux ARM, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata Hi Marc, On Wed, Feb 9, 2022 at 4:04 AM Marc Zyngier <maz@kernel.org> wrote: > > Hi Reiji, > > On Wed, 09 Feb 2022 05:32:36 +0000, > Reiji Watanabe <reijiw@google.com> wrote: > > > > Hi Marc, > > > > On Tue, Feb 8, 2022 at 6:41 AM Marc Zyngier <maz@kernel.org> wrote: > > > > > > In [1], I suggested another approach that didn't require extra state, > > > and moved the existing checks under the kvm lock. What was wrong with > > > that approach? > > > > With that approach, even for a vcpu that has a broken set of features, > > which leads kvm_reset_vcpu() to fail for the vcpu, the vcpu->arch.features > > are checked by other vCPUs' vcpu_allowed_register_width() until the > > vcpu->arch.target is set to -1. > > Due to this, I would think some or possibly all vCPUs' kvm_reset_vcpu() > > may or may not fail (e.g. if userspace tries to configure vCPU#0 with > > 32bit EL1, and vCPU#1 and #2 with 64 bit EL1, KVM_ARM_VCPU_INIT > > for either vCPU#0, or both vCPU#1 and #2 should fail. But, with that > > approach, it doesn't always work that way. Instead, KVM_ARM_VCPU_INIT > > for all vCPUs could fail or KVM_ARM_VCPU_INIT for vCPU#0 and #1 could > > fail while the one for CPU#2 works). > > Also, even after the first KVM_RUN for vCPUs are already done, > > (the first) KVM_ARM_VCPU_INIT for another vCPU could cause the > > kvm_reset_vcpu() for those vCPUs to fail. > > > > I would think those behaviors are odd, and I wanted to avoid them. > > OK, fair enough. But then you need to remove most of the uses of > KVM_ARM_VCPU_EL1_32BIT so that it is only used as a userspace > interface and Yes, I will. > maybe not carried as part of the vcpu feature flag anymore. At the first call of kvm_reset_vcpu() for the guest, the new kvm flag is not set yet. So, KVM_ARM_VCPU_EL1_32BIT will be needed by the function (unless we pass the flag as an argument for the function or by any other way). > Also, we really should turn all these various bits in the kvm struct > into a set of flags. I have a patch posted there[1] for this, feel > free to pick it up. Thank you for the suggestion. But, kvm->arch.el1_reg_width is not a binary because it needs to indicate an uninitialized state. So, it won't fit perfectly with kvm->arch.flags, which is introduced by [1] as it is. Of course it's feasible by using 2 bits of the flags though... Thanks, Reiji > > Thanks, > > M. > > [1] https://lore.kernel.org/r/20211004174849.2831548-2-maz@kernel.org > > -- > Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs 2022-02-10 5:31 ` Reiji Watanabe @ 2022-02-10 10:31 ` Marc Zyngier 2022-02-11 5:04 ` Reiji Watanabe 0 siblings, 1 reply; 8+ messages in thread From: Marc Zyngier @ 2022-02-10 10:31 UTC (permalink / raw) To: Reiji Watanabe Cc: kvmarm, kvm, Linux ARM, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata On Thu, 10 Feb 2022 05:31:49 +0000, Reiji Watanabe <reijiw@google.com> wrote: > > Hi Marc, > > On Wed, Feb 9, 2022 at 4:04 AM Marc Zyngier <maz@kernel.org> wrote: > > > > Hi Reiji, > > > > On Wed, 09 Feb 2022 05:32:36 +0000, > > Reiji Watanabe <reijiw@google.com> wrote: > > > > > > Hi Marc, > > > > > > On Tue, Feb 8, 2022 at 6:41 AM Marc Zyngier <maz@kernel.org> wrote: > > > > > > > > In [1], I suggested another approach that didn't require extra state, > > > > and moved the existing checks under the kvm lock. What was wrong with > > > > that approach? > > > > > > With that approach, even for a vcpu that has a broken set of features, > > > which leads kvm_reset_vcpu() to fail for the vcpu, the vcpu->arch.features > > > are checked by other vCPUs' vcpu_allowed_register_width() until the > > > vcpu->arch.target is set to -1. > > > Due to this, I would think some or possibly all vCPUs' kvm_reset_vcpu() > > > may or may not fail (e.g. if userspace tries to configure vCPU#0 with > > > 32bit EL1, and vCPU#1 and #2 with 64 bit EL1, KVM_ARM_VCPU_INIT > > > for either vCPU#0, or both vCPU#1 and #2 should fail. But, with that > > > approach, it doesn't always work that way. Instead, KVM_ARM_VCPU_INIT > > > for all vCPUs could fail or KVM_ARM_VCPU_INIT for vCPU#0 and #1 could > > > fail while the one for CPU#2 works). > > > Also, even after the first KVM_RUN for vCPUs are already done, > > > (the first) KVM_ARM_VCPU_INIT for another vCPU could cause the > > > kvm_reset_vcpu() for those vCPUs to fail. > > > > > > I would think those behaviors are odd, and I wanted to avoid them. > > > > OK, fair enough. But then you need to remove most of the uses of > > KVM_ARM_VCPU_EL1_32BIT so that it is only used as a userspace > > interface and > > Yes, I will. > > > maybe not carried as part of the vcpu feature flag anymore. > > At the first call of kvm_reset_vcpu() for the guest, the new kvm > flag is not set yet. So, KVM_ARM_VCPU_EL1_32BIT will be needed > by the function (unless we pass the flag as an argument for the > function or by any other way). Which is why I said 'maybe'. It's not a big deal if the flags stays, but I don't want it evaluated further down the line. It is also pretty similar to HCR_EL2.RW, which we already test with vcpu_el1_is_32bit(). Overall, we need to reduce that state to be as simple as possible. > > > Also, we really should turn all these various bits in the kvm struct > > into a set of flags. I have a patch posted there[1] for this, feel > > free to pick it up. > > Thank you for the suggestion. But, kvm->arch.el1_reg_width is not > a binary because it needs to indicate an uninitialized state. So, it > won't fit perfectly with kvm->arch.flags, which is introduced by [1] > as it is. Of course it's feasible by using 2 bits of the flags though... 2 bits is what I had in mind (one bit to indicate that it has already been initialised, another to carry the actual width). Thanks, M. -- Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs 2022-02-10 10:31 ` Marc Zyngier @ 2022-02-11 5:04 ` Reiji Watanabe 0 siblings, 0 replies; 8+ messages in thread From: Reiji Watanabe @ 2022-02-11 5:04 UTC (permalink / raw) To: Marc Zyngier Cc: kvmarm, kvm, Linux ARM, James Morse, Alexandru Elisei, Suzuki K Poulose, Paolo Bonzini, Will Deacon, Peter Shier, Ricardo Koller, Oliver Upton, Jing Zhang, Raghavendra Rao Anata Hi Marc, > > > Also, we really should turn all these various bits in the kvm struct > > > into a set of flags. I have a patch posted there[1] for this, feel > > > free to pick it up. > > > > Thank you for the suggestion. But, kvm->arch.el1_reg_width is not > > a binary because it needs to indicate an uninitialized state. So, it > > won't fit perfectly with kvm->arch.flags, which is introduced by [1] > > as it is. Of course it's feasible by using 2 bits of the flags though... > > 2 bits is what I had in mind (one bit to indicate that it has already > been initialised, another to carry the actual width). Understood. Then, I will take the patch and will work on v3. Thank you for all the comments! Regards, Reiji ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-02-11 5:04 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-01-18 4:19 [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs Reiji Watanabe 2022-01-18 4:19 ` [PATCH v2 2/2] KVM: arm64: selftests: Introduce vcpu_width_config Reiji Watanabe 2022-02-08 14:41 ` [PATCH v2 1/2] KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs Marc Zyngier 2022-02-09 5:32 ` Reiji Watanabe 2022-02-09 12:04 ` Marc Zyngier 2022-02-10 5:31 ` Reiji Watanabe 2022-02-10 10:31 ` Marc Zyngier 2022-02-11 5:04 ` Reiji Watanabe
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).