From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ED10C48BE5 for ; Fri, 11 Jun 2021 09:44:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ECF93613D0 for ; Fri, 11 Jun 2021 09:44:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231553AbhFKJqj (ORCPT ); Fri, 11 Jun 2021 05:46:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50107 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230321AbhFKJqh (ORCPT ); Fri, 11 Jun 2021 05:46:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623404680; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NMYoozdWQK4JS5PWCOrmwHkHgIFc9V+TLEIJ1okexV8=; b=MS2O/HvdalX/1S2wldtHO3fW5y0o3Za5RdccyyvjnH5m2M5O/12/pqbRLLZgKPXz//cpZR jnuIWzN46WavVUvui5esVwwuMrBcxQWGpGlTnXyRcu34dhmPqR5I8B7Etg6lA1p/CHeCeU Xud8SNraz/qxcIemIkaXJFAdnPbWJDs= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-523-00-XyXJxM0SnYgD-wb8RNA-1; Fri, 11 Jun 2021 05:44:38 -0400 X-MC-Unique: 00-XyXJxM0SnYgD-wb8RNA-1 Received: by mail-wm1-f71.google.com with SMTP id v2-20020a7bcb420000b0290146b609814dso4247136wmj.0 for ; Fri, 11 Jun 2021 02:44:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=NMYoozdWQK4JS5PWCOrmwHkHgIFc9V+TLEIJ1okexV8=; b=HTzJWbFbCkudIEI21dOZ3dA3gLNxfk90dzzV/+InUHqlPMAEltpRNJ9LZM2dpaK8+s ihYap3i9qd2VLiSCgioyMEtIglRhx8HAQ9A1NQ8vjicZva9jTDCvenXOQU3LeC3/SafX M9hQbtpd5YMlp6pHhmpvbTTINSZDd/BPz29ysEdYuM5VEXB49AzankqiprocWQ4ZftHA 67p2A0Fb28xMSTObuAB1uih9PTB/3+7nFD9m53pebf5pMbZwWKHT/0IDXVBnxmo2kBEQ 1+rD9ULoiuW8S+fSnWg9PM4csDWEE64ycbCVJ7p+yAj5YC5aCbt7QblxRGvb5FkjUWpC ToeQ== X-Gm-Message-State: AOAM530p157T7HDVBaarcVuf4r7vQU9v0ZpI97jEtdqtFdus1z0bkNVM Ae4tWDBbxFA2G95weUFEvHp4ekrb8P5Xk/NHldIXpEY06rd7ugNn2qCVZ4UY5RuHEvQBUWz398t hDIoBARtwayYO1mbPGzoEoLwv X-Received: by 2002:a05:6000:2c4:: with SMTP id o4mr3022549wry.267.1623404677692; Fri, 11 Jun 2021 02:44:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwF7t6FkjQ+uHMri4EB7+VpKoiqBIYBM/hLoewJameHohV7ZDP+tkBEdZpbP3/QlCLr8Q0qdA== X-Received: by 2002:a05:6000:2c4:: with SMTP id o4mr3022530wry.267.1623404677478; Fri, 11 Jun 2021 02:44:37 -0700 (PDT) Received: from vitty.brq.redhat.com (g-server-2.ign.cz. [91.219.240.2]) by smtp.gmail.com with ESMTPSA id w13sm7073285wrc.31.2021.06.11.02.44.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Jun 2021 02:44:37 -0700 (PDT) From: Vitaly Kuznetsov To: Maxim Levitsky , Paolo Bonzini , Vineeth Pillai , Lan Tianyu , Michael Kelley , Sean Christopherson , Tom Lendacky , Wanpeng Li , Jim Mattson , Joerg Roedel , Wei Liu , Stephen Hemminger , Haiyang Zhang Cc: "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "K. Y. Srinivasan" , x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-hyperv@vger.kernel.org Subject: Re: [PATCH v5 0/7] Hyper-V nested virt enlightenments for SVM In-Reply-To: <683fa50765b29f203cb4b0953542dc43226a7a2f.camel@redhat.com> References: <5af1ccce-a07d-5a13-107b-fc4c4553dd4d@redhat.com> <683fa50765b29f203cb4b0953542dc43226a7a2f.camel@redhat.com> Date: Fri, 11 Jun 2021 11:44:35 +0200 Message-ID: <878s3gybuk.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Maxim Levitsky writes: > On Thu, 2021-06-10 at 17:17 +0200, Paolo Bonzini wrote: >> On 03/06/21 17:14, Vineeth Pillai wrote: >> > This patch series enables the nested virtualization enlightenments for >> > SVM. This is very similar to the enlightenments for VMX except for the >> > fact that there is no enlightened VMCS. For SVM, VMCB is already an >> > architectural in-memory data structure. >> >=20 >> > Note: v5 is just a rebase on hyperv-next(5.13-rc1) and needed a rework >> > based on the patch series: (KVM: VMX: Clean up Hyper-V PV TLB flush) >> > https://lore.kernel.org/lkml/20210305183123.3978098-1-seanjc@google.co= m/ >> >=20 >> > The supported enlightenments are: >> >=20 >> > Enlightened TLB Flush: If this is enabled, ASID invalidations invalida= te >> > only gva -> hpa entries. To flush entries derived from NPT, hyper-v >> > provided hypercalls (HvFlushGuestPhysicalAddressSpace or >> > HvFlushGuestPhysicalAddressList) should be used. >> >=20 >> > Enlightened MSR bitmap(TLFS 16.5.3): "When enabled, L0 hypervisor does >> > not monitor the MSR bitmaps for changes. Instead, the L1 hypervisor mu= st >> > invalidate the corresponding clean field after making changes to one of >> > the MSR bitmaps." >> >=20 >> > Direct Virtual Flush(TLFS 16.8): The hypervisor exposes hypercalls >> > (HvFlushVirtualAddressSpace, HvFlushVirtualAddressSpaceEx, >> > HvFlushVirtualAddressList, and HvFlushVirtualAddressListEx) that allow >> > operating systems to more efficiently manage the virtual TLB. The L1 >> > hypervisor can choose to allow its guest to use those hypercalls and >> > delegate the responsibility to handle them to the L0 hypervisor. This >> > requires the use of a partition assist page." >> >=20 >> > L2 Windows boot time was measured with and without the patch. Time was >> > measured from power on to the login screen and was averaged over a >> > consecutive 5 trials: >> > Without the patch: 42 seconds >> > With the patch: 29 seconds >> > -- >> >=20 >> > Changes from v4 >> > - Rebased on top of 5.13-rc1 and reworked based on the changes in the >> > patch series: (KVM: VMX: Clean up Hyper-V PV TLB flush) >> >=20=20=20=20 >> > Changes from v3 >> > - Included definitions for software/hypervisor reserved fields in SVM >> > architectural data structures. >> > - Consolidated Hyper-V specific code into svm_onhyperv.[ch] to reduce >> > the "ifdefs". This change applies only to SVM, VMX is not touched a= nd >> > is not in the scope of this patch series. >> >=20 >> > Changes from v2: >> > - Refactored the Remote TLB Flush logic into separate hyperv specific >> > source files (kvm_onhyperv.[ch]). >> > - Reverted the VMCB Clean bits macro changes as it is no longer needed. >> >=20 >> > Changes from v1: >> > - Move the remote TLB flush related fields from kvm_vcpu_hv and kvm_hv >> > to kvm_vcpu_arch and kvm_arch. >> > - Modify the VMCB clean mask runtime based on whether L1 hypervisor >> > is running on Hyper-V or not. >> > - Detect Hyper-V nested enlightenments based on >> > HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS. >> > - Address other minor review comments. >> > --- >> >=20 >> > Vineeth Pillai (7): >> > hyperv: Detect Nested virtualization support for SVM >> > hyperv: SVM enlightened TLB flush support flag >> > KVM: x86: hyper-v: Move the remote TLB flush logic out of vmx >> > KVM: SVM: Software reserved fields >> > KVM: SVM: hyper-v: Remote TLB flush for SVM >> > KVM: SVM: hyper-v: Enlightened MSR-Bitmap support >> > KVM: SVM: hyper-v: Direct Virtual Flush support >> >=20 >> > arch/x86/include/asm/hyperv-tlfs.h | 9 ++ >> > arch/x86/include/asm/kvm_host.h | 9 ++ >> > arch/x86/include/asm/svm.h | 9 +- >> > arch/x86/include/uapi/asm/svm.h | 3 + >> > arch/x86/kernel/cpu/mshyperv.c | 10 ++- >> > arch/x86/kvm/Makefile | 9 ++ >> > arch/x86/kvm/kvm_onhyperv.c | 93 +++++++++++++++++++++ >> > arch/x86/kvm/kvm_onhyperv.h | 32 +++++++ >> > arch/x86/kvm/svm/svm.c | 14 ++++ >> > arch/x86/kvm/svm/svm.h | 22 ++++- >> > arch/x86/kvm/svm/svm_onhyperv.c | 41 +++++++++ >> > arch/x86/kvm/svm/svm_onhyperv.h | 129 +++++++++++++++++++++++++++= ++ >> > arch/x86/kvm/vmx/vmx.c | 105 +---------------------- >> > arch/x86/kvm/vmx/vmx.h | 9 -- >> > arch/x86/kvm/x86.c | 9 ++ >> > 15 files changed, 384 insertions(+), 119 deletions(-) >> > create mode 100644 arch/x86/kvm/kvm_onhyperv.c >> > create mode 100644 arch/x86/kvm/kvm_onhyperv.h >> > create mode 100644 arch/x86/kvm/svm/svm_onhyperv.c >> > create mode 100644 arch/x86/kvm/svm/svm_onhyperv.h >> >=20 >>=20 >> Queued, thanks. >>=20 >> Paolo >>=20 > > Hi! > > This patch series causes a build failure here: > > arch/x86/kvm/vmx/vmx.c: In function =E2=80=98hardware_setup=E2=80=99: > arch/x86/kvm/vmx/vmx.c:7752:34: error: =E2=80=98hv_remote_flush_tlb=E2=80= =99 undeclared (first use in this function) > 7752 | vmx_x86_ops.tlb_remote_flush =3D hv_remote_flush_tlb; > | ^~~~~~~~~~~~~~~~~~~ > arch/x86/kvm/vmx/vmx.c:7752:34: note: each undeclared identifier is repor= ted only once for each function it appears in > arch/x86/kvm/vmx/vmx.c:7754:5: error: =E2=80=98hv_remote_flush_tlb_with_r= ange=E2=80=99 undeclared (first use in this function) > 7754 | hv_remote_flush_tlb_with_range; > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > Also this: > > arch/x86/kvm/vmx/vmx.c: In function =E2=80=98hardware_setup=E2=80=99: > arch/x86/kvm/vmx/vmx.c:7752:34: error: =E2=80=98hv_remote_flush_tlb=E2=80= =99 undeclared (first use in this function) > 7752 | vmx_x86_ops.tlb_remote_flush =3D hv_remote_flush_tlb; > | ^~~~~~~~~~~~~~~~~~~ > arch/x86/kvm/vmx/vmx.c:7752:34: note: each undeclared identifier is repor= ted only once for each function it appears in > arch/x86/kvm/vmx/vmx.c:7754:5: error: =E2=80=98hv_remote_flush_tlb_with_r= ange=E2=80=99 undeclared (first use in this function) > 7754 | hv_remote_flush_tlb_with_range; > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >=20=20 Yea, reported already: https://lore.kernel.org/kvm/683fa50765b29f203cb4b0953542dc43226a7a2f.camel@= redhat.com/T/#mf732ba9567923d800329f896761e4ee104475894 > > Best regards, > Maxim Levitsky > --=20 Vitaly