From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E917C2BB40 for ; Mon, 14 Dec 2020 18:16:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3EF1722509 for ; Mon, 14 Dec 2020 18:16:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2440552AbgLNSPp (ORCPT ); Mon, 14 Dec 2020 13:15:45 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26019 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391382AbgLNSPF (ORCPT ); Mon, 14 Dec 2020 13:15:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607969615; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AwwzZdDyF1r6qcS1a83NpXAsbn9SW/LPeb1RFJju/b4=; b=ORK/hdJv2KLSE7P0fTClKEl14It18qK3jwu3CpI6x59RZ0qAc3zm6brJ8u3YScTHoYb7F3 6cV0UgjAFTJhli6uaOSjzyfByUwzSXMde6MNOZzvQEfjDeEC418erR1o/k4PB7uH4vNauU 8t5KaEdUWuGC/GmxndB7uhEWK7TdEkQ= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-169--PXK4H4dO3epZdSEM4PDGA-1; Mon, 14 Dec 2020 13:13:24 -0500 X-MC-Unique: -PXK4H4dO3epZdSEM4PDGA-1 Received: by mail-wm1-f69.google.com with SMTP id k128so7088817wme.7 for ; Mon, 14 Dec 2020 10:13:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=AwwzZdDyF1r6qcS1a83NpXAsbn9SW/LPeb1RFJju/b4=; b=hJxTfW3SEvs3GeUXRQqmZRnZIA3l+sSNw70D1Gt4DovP189JTZJKH6eOyTbRwPDc3o u9YW8t+WYgao7sY/nr+rV/zo2i7ZK+exL2v/Lc5myyMQn7K7T71URJi9E3nYzD9MVGj4 sZUQEX+JP2MK5ufQkPvLaLfipIOe6M8rzgnao1kEmMr6sMjhd3jd2k8gc2za9bldOAtt KkOSCEQzYlkqtI6dMC6q1P2zY5gmnScsyvpYC2NQ1L28DFYQdfFFkWXr7zOZiy+lfgCr dlghz2AEnLZtgvWTidHJ5IPOCIcgVslFIJjVC7EoFh0LGsTiUFSn4Iu3t/UtbyxZtidF 1EfQ== X-Gm-Message-State: AOAM530088SPJuvjHevnReLnEzj5JXA8LxAzJxSmmysrsExD/fjGZMTL y2DLCo8FLiCx87IpB3xej22RK/DoX0aGvzeZDetGGP9OZhcV/MofO8WEAssCSEmCtTO77+G1Xjl d9e4tvuqC7t5M3wyeDPbYXqaZ X-Received: by 2002:a1c:6008:: with SMTP id u8mr28240640wmb.173.1607969603260; Mon, 14 Dec 2020 10:13:23 -0800 (PST) X-Google-Smtp-Source: ABdhPJyuUM/UOHcT7yv0tgDWk8gjs5bzu4UDZt8mSBTZrr1Gppa0JPSQqRDV8wZEKahD+QUrFaqO9g== X-Received: by 2002:a1c:6008:: with SMTP id u8mr28240616wmb.173.1607969602964; Mon, 14 Dec 2020 10:13:22 -0800 (PST) Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id j59sm4170676wrj.13.2020.12.14.10.13.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 14 Dec 2020 10:13:22 -0800 (PST) Subject: Re: [PATCH v5 00/34] SEV-ES hypervisor support To: Tom Lendacky , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: Jim Mattson , Joerg Roedel , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Borislav Petkov , Ingo Molnar , Thomas Gleixner , Brijesh Singh , Sean Christopherson References: From: Paolo Bonzini Message-ID: Date: Mon, 14 Dec 2020 19:13:20 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/12/20 18:09, Tom Lendacky wrote: > From: Tom Lendacky > > This patch series provides support for running SEV-ES guests under KVM. > > Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the > SEV support to protect the guest register state from the hypervisor. See > "AMD64 Architecture Programmer's Manual Volume 2: System Programming", > section "15.35 Encrypted State (SEV-ES)" [1]. > > In order to allow a hypervisor to perform functions on behalf of a guest, > there is architectural support for notifying a guest's operating system > when certain types of VMEXITs are about to occur. This allows the guest to > selectively share information with the hypervisor to satisfy the requested > function. The notification is performed using a new exception, the VMM > Communication exception (#VC). The information is shared through the > Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction. > The GHCB format and the protocol for using it is documented in "SEV-ES > Guest-Hypervisor Communication Block Standardization" [2]. > > Under SEV-ES, a vCPU save area (VMSA) must be encrypted. SVM is updated to > build the initial VMSA and then encrypt it before running the guest. Once > encrypted, it must not be modified by the hypervisor. Modification of the > VMSA will result in the VMRUN instruction failing with a SHUTDOWN exit > code. KVM must support the VMGEXIT exit code in order to perform the > necessary functions required of the guest. The GHCB is used to exchange > the information needed by both the hypervisor and the guest. > > Register data from the GHCB is copied into the KVM register variables and > accessed as usual during handling of the exit. Upon return to the guest, > updated registers are copied back to the GHCB for the guest to act upon. > > There are changes to some of the intercepts that are needed under SEV-ES. > For example, CR0 writes cannot be intercepted, so the code needs to ensure > that the intercept is not enabled during execution or that the hypervisor > does not try to read the register as part of exit processing. Another > example is shutdown processing, where the vCPU cannot be directly reset. > > Support is added to handle VMGEXIT events and implement the GHCB protocol. > This includes supporting standard exit events, like a CPUID instruction > intercept, to new support, for things like AP processor booting. Much of > the existing SVM intercept support can be re-used by setting the exit > code information from the VMGEXIT and calling the appropriate intercept > handlers. > > Finally, to launch and run an SEV-ES guest requires changes to the vCPU > initialization, loading and execution. > > [1] https://www.amd.com/system/files/TechDocs/24593.pdf > [2] https://developer.amd.com/wp-content/resources/56421.pdf > > --- > > These patches are based on the KVM queue branch: > https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue > > dc924b062488 ("KVM: SVM: check CR4 changes against vcpu->arch") > > A version of the tree can also be found at: > https://github.com/AMDESE/linux/tree/sev-es-v5 > This tree has one addition patch that is not yet part of the queue > tree that is required to run any SEV guest: > [PATCH] KVM: x86: adjust SEV for commit 7e8e6eed75e > https://lore.kernel.org/kvm/20201130143959.3636394-1-pbonzini@redhat.com/ > > Changes from v4: > - Updated the tracking support for CR0/CR4 > > Changes from v3: > - Some krobot fixes. > - Some checkpatch cleanups. > > Changes from v2: > - Update the freeing of the VMSA page to account for the encrypted memory > cache coherency feature as well as the VM page flush feature. > - Update the GHCB dump function with a bit more detail. > - Don't check for RAX being present as part of a string IO operation. > - Include RSI when syncing from GHCB to support KVM hypercall arguments. > - Add GHCB usage field validation check. > > Changes from v1: > - Removed the VMSA indirection support: > - On LAUNCH_UPDATE_VMSA, sync traditional VMSA over to the new SEV-ES > VMSA area to be encrypted. > - On VMGEXIT VMEXIT, directly copy valid registers into vCPU arch > register array from GHCB. On VMRUN (following a VMGEXIT), directly > copy dirty vCPU arch registers to GHCB. > - Removed reg_read_override()/reg_write_override() KVM ops. > - Added VMGEXIT exit-reason validation. > - Changed kvm_vcpu_arch variable vmsa_encrypted to guest_state_protected > - Updated the tracking support for EFER/CR0/CR4/CR8 to minimize changes > to the x86.c code > - Updated __set_sregs to not set any register values (previously supported > setting the tracked values of EFER/CR0/CR4/CR8) > - Added support for reporting SMM capability at the VM-level. This allows > an SEV-ES guest to indicate SMM is not supported > - Updated FPU support to check for a guest FPU save area before using it. > Updated SVM to free guest FPU for an SEV-ES guest during KVM create_vcpu > op. > - Removed changes to the kvm_skip_emulated_instruction() > - Added VMSA validity checks before invoking LAUNCH_UPDATE_VMSA > - Minor code restructuring in areas for better readability > > Cc: Paolo Bonzini > Cc: Jim Mattson > Cc: Joerg Roedel > Cc: Sean Christopherson > Cc: Vitaly Kuznetsov > Cc: Wanpeng Li > Cc: Borislav Petkov > Cc: Ingo Molnar > Cc: Thomas Gleixner > Cc: Brijesh Singh I'm queuing everything except patch 27, there's time to include it later in 5.11. Regarding MSRs, take a look at the series I'm sending shortly (or perhaps in a couple hours). For now I'll keep it in kvm/queue, but the plan is to get acks quickly and/or just include it in 5.11. Please try the kvm/queue branch to see if I screwed up anything. Paolo