From: Paolo Bonzini <pbonzini@redhat.com>
To: Tom Lendacky <thomas.lendacky@amd.com>,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
x86@kernel.org
Cc: Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
Sean Christopherson <seanjc@google.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Wanpeng Li <wanpengli@tencent.com>,
Borislav Petkov <bp@alien8.de>, Ingo Molnar <mingo@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
Brijesh Singh <brijesh.singh@amd.com>
Subject: Re: [PATCH v5 27/34] KVM: SVM: Add support for booting APs for an SEV-ES guest
Date: Mon, 14 Dec 2020 17:03:00 +0100 [thread overview]
Message-ID: <8ed48a0f-d490-d74d-d10a-968b561a4f2e@redhat.com> (raw)
In-Reply-To: <47d11ed1c1a48ab71858fc3cde766bf67a4612d1.1607620209.git.thomas.lendacky@amd.com>
On 10/12/20 18:10, Tom Lendacky wrote:
> From: Tom Lendacky <thomas.lendacky@amd.com>
>
> Typically under KVM, an AP is booted using the INIT-SIPI-SIPI sequence,
> where the guest vCPU register state is updated and then the vCPU is VMRUN
> to begin execution of the AP. For an SEV-ES guest, this won't work because
> the guest register state is encrypted.
>
> Following the GHCB specification, the hypervisor must not alter the guest
> register state, so KVM must track an AP/vCPU boot. Should the guest want
> to park the AP, it must use the AP Reset Hold exit event in place of, for
> example, a HLT loop.
>
> First AP boot (first INIT-SIPI-SIPI sequence):
> Execute the AP (vCPU) as it was initialized and measured by the SEV-ES
> support. It is up to the guest to transfer control of the AP to the
> proper location.
>
> Subsequent AP boot:
> KVM will expect to receive an AP Reset Hold exit event indicating that
> the vCPU is being parked and will require an INIT-SIPI-SIPI sequence to
> awaken it. When the AP Reset Hold exit event is received, KVM will place
> the vCPU into a simulated HLT mode. Upon receiving the INIT-SIPI-SIPI
> sequence, KVM will make the vCPU runnable. It is again up to the guest
> to then transfer control of the AP to the proper location.
>
> The GHCB specification also requires the hypervisor to save the address of
> an AP Jump Table so that, for example, vCPUs that have been parked by UEFI
> can be started by the OS. Provide support for the AP Jump Table set/get
> exit code.
>
> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
> ---
> arch/x86/include/asm/kvm_host.h | 2 ++
> arch/x86/kvm/svm/sev.c | 50 +++++++++++++++++++++++++++++++++
> arch/x86/kvm/svm/svm.c | 7 +++++
> arch/x86/kvm/svm/svm.h | 3 ++
> arch/x86/kvm/x86.c | 9 ++++++
> 5 files changed, 71 insertions(+)
>
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 048b08437c33..60a3b9d33407 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1286,6 +1286,8 @@ struct kvm_x86_ops {
>
> void (*migrate_timers)(struct kvm_vcpu *vcpu);
> void (*msr_filter_changed)(struct kvm_vcpu *vcpu);
> +
> + void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
> };
>
> struct kvm_x86_nested_ops {
> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index a7531de760b5..b47285384b1f 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
> @@ -17,6 +17,8 @@
> #include <linux/processor.h>
> #include <linux/trace_events.h>
>
> +#include <asm/trapnr.h>
> +
> #include "x86.h"
> #include "svm.h"
> #include "cpuid.h"
> @@ -1449,6 +1451,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
> if (!ghcb_sw_scratch_is_valid(ghcb))
> goto vmgexit_err;
> break;
> + case SVM_VMGEXIT_AP_HLT_LOOP:
> + case SVM_VMGEXIT_AP_JUMP_TABLE:
> case SVM_VMGEXIT_UNSUPPORTED_EVENT:
> break;
> default:
> @@ -1770,6 +1774,35 @@ int sev_handle_vmgexit(struct vcpu_svm *svm)
> control->exit_info_2,
> svm->ghcb_sa);
> break;
> + case SVM_VMGEXIT_AP_HLT_LOOP:
> + svm->ap_hlt_loop = true;
This value needs to be communicated to userspace. Let's get this right
from the beginning and use a new KVM_MP_STATE_* value instead (perhaps
reuse KVM_MP_STATE_STOPPED but for x86 #define it as
KVM_MP_STATE_AP_HOLD_RECEIVED?).
> @@ -68,6 +68,7 @@ struct kvm_sev_info {
> int fd; /* SEV device fd */
> unsigned long pages_locked; /* Number of pages locked */
> struct list_head regions_list; /* List of registered regions */
> + u64 ap_jump_table; /* SEV-ES AP Jump Table address */
Do you have any plans for migration of this value? How does the guest
ensure that the hypervisor does not screw with it?
Paolo
> + ret = kvm_emulate_halt(&svm->vcpu);
> + break;
> + case SVM_VMGEXIT_AP_JUMP_TABLE: {
> + struct kvm_sev_info *sev = &to_kvm_svm(svm->vcpu.kvm)->sev_info;
> +
> + switch (control->exit_info_1) {
> + case 0:
> + /* Set AP jump table address */
> + sev->ap_jump_table = control->exit_info_2;
> + break;
> + case 1:
> + /* Get AP jump table address */
> + ghcb_set_sw_exit_info_2(ghcb, sev->ap_jump_table);
> + break;
> + default:
> + pr_err("svm: vmgexit: unsupported AP jump table request - exit_info_1=%#llx\n",
> + control->exit_info_1);
> + ghcb_set_sw_exit_info_1(ghcb, 1);
> + ghcb_set_sw_exit_info_2(ghcb,
> + X86_TRAP_UD |
> + SVM_EVTINJ_TYPE_EXEPT |
> + SVM_EVTINJ_VALID);
> + }
> +
> + ret = 1;
> + break;
> + }
> case SVM_VMGEXIT_UNSUPPORTED_EVENT:
> vcpu_unimpl(&svm->vcpu,
> "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n",
> @@ -1790,3 +1823,20 @@ int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in)
> return kvm_sev_es_string_io(&svm->vcpu, size, port,
> svm->ghcb_sa, svm->ghcb_sa_len, in);
> }
> +
> +void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
> +{
> + struct vcpu_svm *svm = to_svm(vcpu);
> +
> + /* First SIPI: Use the values as initially set by the VMM */
> + if (!svm->ap_hlt_loop)
> + return;
> +
> + /*
> + * Subsequent SIPI: Return from an AP Reset Hold VMGEXIT, where
> + * the guest will set the CS and RIP. Set SW_EXIT_INFO_2 to a
> + * non-zero value.
> + */
> + ghcb_set_sw_exit_info_2(svm->ghcb, 1);
> + svm->ap_hlt_loop = false;
> +}
> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index 8d22ae25a0f8..2dbc20701ef5 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c
> @@ -4400,6 +4400,11 @@ static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu)
> (vmcb_is_intercept(&svm->vmcb->control, INTERCEPT_INIT));
> }
>
> +static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
> +{
> + sev_vcpu_deliver_sipi_vector(vcpu, vector);
> +}
> +
> static void svm_vm_destroy(struct kvm *kvm)
> {
> avic_vm_destroy(kvm);
> @@ -4541,6 +4546,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
> .apic_init_signal_blocked = svm_apic_init_signal_blocked,
>
> .msr_filter_changed = svm_msr_filter_changed,
> +
> + .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
> };
>
> static struct kvm_x86_init_ops svm_init_ops __initdata = {
> diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
> index b3f03dede6ac..5d570d5a6a2c 100644
> --- a/arch/x86/kvm/svm/svm.h
> +++ b/arch/x86/kvm/svm/svm.h
> @@ -68,6 +68,7 @@ struct kvm_sev_info {
> int fd; /* SEV device fd */
> unsigned long pages_locked; /* Number of pages locked */
> struct list_head regions_list; /* List of registered regions */
> + u64 ap_jump_table; /* SEV-ES AP Jump Table address */
> };
>
> struct kvm_svm {
> @@ -174,6 +175,7 @@ struct vcpu_svm {
> struct vmcb_save_area *vmsa;
> struct ghcb *ghcb;
> struct kvm_host_map ghcb_map;
> + bool ap_hlt_loop;
>
> /* SEV-ES scratch area support */
> void *ghcb_sa;
> @@ -574,5 +576,6 @@ void sev_hardware_teardown(void);
> void sev_free_vcpu(struct kvm_vcpu *vcpu);
> int sev_handle_vmgexit(struct vcpu_svm *svm);
> int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in);
> +void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
>
> #endif
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index ddd614a76744..4fd216b61a89 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -10144,6 +10144,15 @@ void kvm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
> {
> struct kvm_segment cs;
>
> + /*
> + * Guests with protected state can't have their state altered by KVM,
> + * call the vcpu_deliver_sipi_vector() x86 op for processing.
> + */
> + if (vcpu->arch.guest_state_protected) {
> + kvm_x86_ops.vcpu_deliver_sipi_vector(vcpu, vector);
> + return;
> + }
> +
> kvm_get_segment(vcpu, &cs, VCPU_SREG_CS);
> cs.selector = vector << 8;
> cs.base = vector << 12;
>
next prev parent reply other threads:[~2020-12-14 16:04 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-10 17:09 [PATCH v5 00/34] SEV-ES hypervisor support Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 01/34] x86/cpu: Add VM page flush MSR availablility as a CPUID feature Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 02/34] KVM: SVM: Remove the call to sev_platform_status() during setup Tom Lendacky
2020-12-14 12:29 ` Paolo Bonzini
2020-12-14 16:45 ` Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 03/34] KVM: SVM: Add support for SEV-ES capability in KVM Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 04/34] KVM: SVM: Add GHCB accessor functions for retrieving fields Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 05/34] KVM: SVM: Add support for the SEV-ES VMSA Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 06/34] KVM: x86: Mark GPRs dirty when written Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 07/34] KVM: SVM: Add required changes to support intercepts under SEV-ES Tom Lendacky
2020-12-14 15:33 ` Paolo Bonzini
2020-12-14 19:00 ` Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 08/34] KVM: SVM: Prevent debugging " Tom Lendacky
2020-12-14 15:35 ` Paolo Bonzini
2020-12-14 15:41 ` Paolo Bonzini
2020-12-14 19:10 ` Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 09/34] KVM: SVM: Do not allow instruction emulation " Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 10/34] KVM: SVM: Cannot re-initialize the VMCB after shutdown with SEV-ES Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 11/34] KVM: SVM: Prepare for SEV-ES exit handling in the sev.c file Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 12/34] KVM: SVM: Add initial support for a VMGEXIT VMEXIT Tom Lendacky
2020-12-14 15:45 ` Paolo Bonzini
2020-12-14 19:21 ` Tom Lendacky
2020-12-14 19:41 ` Sean Christopherson
2020-12-15 10:16 ` Paolo Bonzini
2020-12-10 17:09 ` [PATCH v5 13/34] KVM: SVM: Create trace events for VMGEXIT processing Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 14/34] KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x002 Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 15/34] KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x004 Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 16/34] KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x100 Tom Lendacky
2020-12-14 15:49 ` Paolo Bonzini
2020-12-14 19:21 ` Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 17/34] KVM: SVM: Create trace events for VMGEXIT MSR protocol processing Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 18/34] KVM: SVM: Support MMIO for an SEV-ES guest Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 19/34] KVM: SVM: Support string IO operations " Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 20/34] KVM: SVM: Add support for EFER write traps " Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 21/34] KVM: SVM: Add support for CR0 " Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 22/34] KVM: SVM: Add support for CR4 " Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 23/34] KVM: SVM: Add support for CR8 " Tom Lendacky
2020-12-10 17:09 ` [PATCH v5 24/34] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 25/34] KVM: SVM: Do not report support for SMM for an SEV-ES guest Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 26/34] KVM: SVM: Guest FPU state save/restore not needed for " Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 27/34] KVM: SVM: Add support for booting APs for an " Tom Lendacky
2020-12-14 16:03 ` Paolo Bonzini [this message]
2020-12-14 19:46 ` Tom Lendacky
2020-12-15 20:25 ` Tom Lendacky
2021-01-04 17:38 ` Tom Lendacky
2021-01-04 17:50 ` Paolo Bonzini
2020-12-14 16:05 ` Paolo Bonzini
2021-01-04 20:20 ` [PATCH v5.1 27/34] KVM: SVM: Add support for booting APs in " Tom Lendacky
2021-01-07 18:13 ` Paolo Bonzini
2021-01-07 19:53 ` Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 28/34] KVM: SVM: Add NMI support for " Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 29/34] KVM: SVM: Set the encryption mask for the SVM host save area Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 30/34] KVM: SVM: Update ASID allocation to support SEV-ES guests Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 31/34] KVM: SVM: Provide support for SEV-ES vCPU creation/loading Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 32/34] KVM: SVM: Provide support for SEV-ES vCPU loading Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 33/34] KVM: SVM: Provide an updated VMRUN invocation for SEV-ES guests Tom Lendacky
2020-12-10 17:10 ` [PATCH v5 34/34] KVM: SVM: Provide support to launch and run an SEV-ES guest Tom Lendacky
2020-12-16 16:55 ` Tom Lendacky
2020-12-14 18:13 ` [PATCH v5 00/34] SEV-ES hypervisor support Paolo Bonzini
2020-12-14 19:06 ` Tom Lendacky
2020-12-15 16:46 ` Tom Lendacky
2020-12-15 17:42 ` Paolo Bonzini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8ed48a0f-d490-d74d-d10a-968b561a4f2e@redhat.com \
--to=pbonzini@redhat.com \
--cc=bp@alien8.de \
--cc=brijesh.singh@amd.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).