All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 00/25] KVM SGX virtualization support
@ 2021-03-09  1:38 Kai Huang
  2021-03-09  1:38 ` [PATCH v2 01/25] x86/cpufeatures: Make SGX_LC feature bit depend on SGX bit Kai Huang
                   ` (25 more replies)
  0 siblings, 26 replies; 69+ messages in thread
From: Kai Huang @ 2021-03-09  1:38 UTC (permalink / raw)
  To: kvm, x86, linux-sgx
  Cc: linux-kernel, seanjc, jarkko, luto, dave.hansen,
	rick.p.edgecombe, haitao.huang, pbonzini, bp, tglx, mingo, hpa,
	jethro, b.thiel, jmattson, joro, vkuznets, wanpengli, corbet

This series adds KVM SGX virtualization support. The first 14 patches starting
with x86/sgx or x86/cpu.. are necessary changes to x86 and SGX core/driver to
support KVM SGX virtualization, while the rest are patches to KVM subsystem.

This series is based against latest upstream kernel master branch.

You can also get the code from upstream branch of kvm-sgx repo on github:

        https://github.com/intel/kvm-sgx.git upstream

It also requires Qemu changes to create VM with SGX support. You can find Qemu
repo here:

	https://github.com/intel/qemu-sgx.git upstream

Please refer to README.md of above qemu-sgx repo for detail on how to create
guest with SGX support. At meantime, for your quick reference you can use below
command to create SGX guest:

	#qemu-system-x86_64 -smp 4 -m 2G -drive file=<your_vm_image>,if=virtio \
		-cpu host,+sgx_provisionkey \
		-sgx-epc id=epc1,memdev=mem1 \
		-object memory-backend-epc,id=mem1,size=64M,prealloc

Please note that the SGX relevant part is:

		-cpu host,+sgx_provisionkey \
		-sgx-epc id=epc1,memdev=mem1 \
		-object memory-backend-epc,id=mem1,size=64M,prealloc

And you can change other parameters of your qemu command based on your needs.

=========
Changelog:

(Changelog here is for global changes. Please see each patch's changelog for
 changes made to specific patch.)

v1->v2:

 - No big change in design, structural of patch series, etc.
 - Addressed Boris's comments regarding to suppressing both SGX1 and SGX2 in
   /proc/cpuinfo, and improvement in feat_ctl.c when enabling SGX (patch 2
   and 6).
 - Addressed Sean's comments for both x86 part patches and KVM patches (patch 3,
   5, 9, 12, 19, 21).
 - Addressed Dave's comments in RFC v6 series (patch 13).

RFC->v1:

 - Refined patch (x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()) to print
   error msg that EPC page is leaked when EREMOVE failed, requested by Dave.
 - Changelog history of all RFC series is removed in both this cover letter
   and each individual patch, since majority of x86 part patches already got
   Acked-by from Dave and Jarkko. And the changelogs are not quite useful from
   my perspective.

=========
KVM SGX virtualization Overview

- Virtual EPC

SGX enclave memory is special and is reserved specifically for enclave use.
In bare-metal SGX enclaves, the kernel allocates enclave pages, copies data
into the pages with privileged instructions, then allows the enclave to start.
In this scenario, only initialized pages already assigned to an enclave are
mapped to userspace.

In virtualized environments, the hypervisor still needs to do the physical
enclave page allocation.  The guest kernel is responsible for the data copying
(among other things).  This means that the job of starting an enclave is now
split between hypervisor and guest.

This series introduces a new misc device: /dev/sgx_vepc.  This device allows
the host to map *uninitialized* enclave memory into userspace, which can then
be passed into a guest.

While it might be *possible* to start a host-side enclave with /dev/sgx_enclave
and pass its memory into a guest, it would be wasteful and convoluted.

Implement the *raw* EPC allocation in the x86 core-SGX subsystem via
/dev/sgx_vepc rather than in KVM.  Doing so has two major advantages:

  - Does not require changes to KVM's uAPI, e.g. EPC gets handled as
    just another memory backend for guests.

  - EPC management is wholly contained in the SGX subsystem, e.g. SGX
    does not have to export any symbols, changes to reclaim flows don't
    need to be routed through KVM, SGX's dirty laundry doesn't have to
    get aired out for the world to see, and so on and so forth.

The virtual EPC pages allocated to guests are currently not reclaimable.
Reclaiming EPC page used by enclave requires a special reclaim mechanism
separate from normal page reclaim, and that mechanism is not supported
for virutal EPC pages.  Due to the complications of handling reclaim
conflicts between guest and host, reclaiming virtual EPC pages is 
significantly more complex than basic support for SGX virtualization.

- Support SGX virtualization without SGX Flexible Launch Control

SGX hardware supports two "launch control" modes to limit which enclaves can
run.  In the "locked" mode, the hardware prevents enclaves from running unless
they are blessed by a third party.  In the unlocked mode, the kernel is in
full control of which enclaves can run.  The bare-metal SGX code refuses to
launch enclaves unless it is in the unlocked mode.

This sgx_virt_epc driver does not have such a restriction.  This allows guests
which are OK with the locked mode to use SGX, even if the host kernel refuses
to.

- Support exposing SGX2

Due to the same reason above, SGX2 feature detection is added to core SGX code
to allow KVM to expose SGX2 to guest, even currently SGX driver doesn't support
SGX2, because SGX2 can work just fine in guest w/o any interaction to host SGX
driver.

- Restricit SGX guest access to provisioning key

To grant guest being able to fully use SGX, guest needs to be able to access
provisioning key.  The provisioning key is sensitive, and accessing to it should
be restricted. In bare-metal driver, allowing enclave to access provisioning key
is restricted by being able to open /dev/sgx_provision.

Add a new KVM_CAP_SGX_ATTRIBUTE to KVM uAPI to extend above mechanism to KVM
guests as well.  When userspace hypervisor creates a new VM, the new cap is only
added to VM when userspace hypervisior is able to open /dev/sgx_provision,
following the same role as in bare-metal driver.  KVM then traps ECREATE from
guest, and only allows ECREATE with provisioning key bit to run when guest
supports KVM_CAP_SGX_ATTRIBUTE.

Jarkko Sakkinen (1):
  x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()

Kai Huang (3):
  x86/cpufeatures: Make SGX_LC feature bit depend on SGX bit
  x86/sgx: Initialize virtual EPC driver even when SGX driver is
    disabled
  x86/sgx: Add helper to update SGX_LEPUBKEYHASHn MSRs

Sean Christopherson (21):
  x86/cpufeatures: Add SGX1 and SGX2 sub-features
  x86/sgx: Add SGX_CHILD_PRESENT hardware error code
  x86/sgx: Introduce virtual EPC for use by KVM guests
  x86/cpu/intel: Allow SGX virtualization without Launch Control support
  x86/sgx: Expose SGX architectural definitions to the kernel
  x86/sgx: Move ENCLS leaf definitions to sgx.h
  x86/sgx: Add SGX2 ENCLS leaf definitions (EAUG, EMODPR and EMODT)
  x86/sgx: Add encls_faulted() helper
  x86/sgx: Add helpers to expose ECREATE and EINIT to KVM
  x86/sgx: Move provisioning device creation out of SGX driver
  KVM: x86: Export kvm_mmu_gva_to_gpa_{read,write}() for SGX (VMX)
  KVM: x86: Define new #PF SGX error code bit
  KVM: x86: Add support for reverse CPUID lookup of scattered features
  KVM: x86: Add reverse-CPUID lookup support for scattered SGX features
  KVM: VMX: Add basic handling of VM-Exit from SGX enclave
  KVM: VMX: Frame in ENCLS handler for SGX virtualization
  KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce CPUID restrictions
  KVM: VMX: Add emulation of SGX Launch Control LE hash MSRs
  KVM: VMX: Add ENCLS[EINIT] handler to support SGX Launch Control (LC)
  KVM: VMX: Enable SGX virtualization for SGX1, SGX2 and LC
  KVM: x86: Add capability to grant VM access to privileged SGX
    attribute

 Documentation/virt/kvm/api.rst                |  23 +
 arch/x86/Kconfig                              |  12 +
 arch/x86/include/asm/cpufeatures.h            |   2 +
 arch/x86/include/asm/kvm_host.h               |   5 +
 .../cpu/sgx/arch.h => include/asm/sgx.h}      |  50 +-
 arch/x86/include/asm/vmx.h                    |   1 +
 arch/x86/include/uapi/asm/vmx.h               |   1 +
 arch/x86/kernel/cpu/cpuid-deps.c              |   3 +
 arch/x86/kernel/cpu/feat_ctl.c                |  71 ++-
 arch/x86/kernel/cpu/scattered.c               |   2 +
 arch/x86/kernel/cpu/sgx/Makefile              |   1 +
 arch/x86/kernel/cpu/sgx/driver.c              |  17 -
 arch/x86/kernel/cpu/sgx/encl.c                |  29 +-
 arch/x86/kernel/cpu/sgx/encls.h               |  30 +-
 arch/x86/kernel/cpu/sgx/ioctl.c               |  23 +-
 arch/x86/kernel/cpu/sgx/main.c                |  94 +++-
 arch/x86/kernel/cpu/sgx/sgx.h                 |  13 +-
 arch/x86/kernel/cpu/sgx/virt.c                | 370 ++++++++++++++
 arch/x86/kvm/Makefile                         |   2 +
 arch/x86/kvm/cpuid.c                          |  89 +++-
 arch/x86/kvm/cpuid.h                          |  50 +-
 arch/x86/kvm/vmx/nested.c                     |  28 +-
 arch/x86/kvm/vmx/nested.h                     |   5 +
 arch/x86/kvm/vmx/sgx.c                        | 481 ++++++++++++++++++
 arch/x86/kvm/vmx/sgx.h                        |  34 ++
 arch/x86/kvm/vmx/vmcs12.c                     |   1 +
 arch/x86/kvm/vmx/vmcs12.h                     |   4 +-
 arch/x86/kvm/vmx/vmx.c                        | 109 +++-
 arch/x86/kvm/vmx/vmx.h                        |   2 +
 arch/x86/kvm/x86.c                            |  23 +
 include/uapi/linux/kvm.h                      |   1 +
 tools/testing/selftests/sgx/defines.h         |   2 +-
 32 files changed, 1460 insertions(+), 118 deletions(-)
 rename arch/x86/{kernel/cpu/sgx/arch.h => include/asm/sgx.h} (89%)
 create mode 100644 arch/x86/kernel/cpu/sgx/virt.c
 create mode 100644 arch/x86/kvm/vmx/sgx.c
 create mode 100644 arch/x86/kvm/vmx/sgx.h

-- 
2.29.2


^ permalink raw reply	[flat|nested] 69+ messages in thread

end of thread, other threads:[~2021-03-18  0:05 UTC | newest]

Thread overview: 69+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-09  1:38 [PATCH v2 00/25] KVM SGX virtualization support Kai Huang
2021-03-09  1:38 ` [PATCH v2 01/25] x86/cpufeatures: Make SGX_LC feature bit depend on SGX bit Kai Huang
2021-03-12 21:05   ` Sean Christopherson
2021-03-09  1:39 ` [PATCH v2 02/25] x86/cpufeatures: Add SGX1 and SGX2 sub-features Kai Huang
2021-03-12 21:10   ` Sean Christopherson
2021-03-09  1:39 ` [PATCH v2 03/25] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page() Kai Huang
2021-03-10  9:35   ` Kai Huang
2021-03-11  2:01   ` [PATCH v3 " Kai Huang
2021-03-12 21:21     ` Sean Christopherson
2021-03-13 10:45       ` Jarkko Sakkinen
2021-03-15  7:12         ` Kai Huang
2021-03-15 13:18           ` Jarkko Sakkinen
2021-03-15 13:19             ` Jarkko Sakkinen
2021-03-15 20:29               ` Kai Huang
2021-03-15 22:59                 ` Jarkko Sakkinen
2021-03-15 23:50                   ` Kai Huang
2021-03-15 23:11                 ` Jarkko Sakkinen
2021-03-09  1:39 ` [PATCH v2 04/25] x86/sgx: Add SGX_CHILD_PRESENT hardware error code Kai Huang
2021-03-09  1:39 ` [PATCH v2 05/25] x86/sgx: Introduce virtual EPC for use by KVM guests Kai Huang
2021-03-09  1:39 ` [PATCH v2 06/25] x86/cpu/intel: Allow SGX virtualization without Launch Control support Kai Huang
2021-03-12 21:33   ` Sean Christopherson
2021-03-09  1:39 ` [PATCH v2 07/25] x86/sgx: Initialize virtual EPC driver even when SGX driver is disabled Kai Huang
2021-03-12 21:44   ` Sean Christopherson
2021-03-13 19:05     ` Jarkko Sakkinen
2021-03-13 19:07       ` Jarkko Sakkinen
2021-03-14 15:25         ` Jarkko Sakkinen
2021-03-14 15:27           ` Jarkko Sakkinen
2021-03-15  3:13             ` Kai Huang
2021-03-15 13:04               ` Jarkko Sakkinen
2021-03-15 13:51                 ` Jarkko Sakkinen
2021-03-15 20:48                   ` Kai Huang
2021-03-15 23:05                     ` Jarkko Sakkinen
2021-03-15 23:08                       ` Jarkko Sakkinen
2021-03-15 23:49                         ` Kai Huang
2021-03-16 12:44                           ` Jarkko Sakkinen
2021-03-16  1:13                         ` Sean Christopherson
2021-03-16 12:46                           ` Jarkko Sakkinen
2021-03-18  0:04                             ` Kai Huang
2021-03-09  1:39 ` [PATCH v2 08/25] x86/sgx: Expose SGX architectural definitions to the kernel Kai Huang
2021-03-12 21:58   ` Sean Christopherson
2021-03-15  3:36     ` Kai Huang
2021-03-09  1:39 ` [PATCH v2 09/25] x86/sgx: Move ENCLS leaf definitions to sgx.h Kai Huang
2021-03-09  1:39 ` [PATCH v2 10/25] x86/sgx: Add SGX2 ENCLS leaf definitions (EAUG, EMODPR and EMODT) Kai Huang
2021-03-09  1:39 ` [PATCH v2 11/25] x86/sgx: Add encls_faulted() helper Kai Huang
2021-03-09  1:39 ` [PATCH v2 12/25] x86/sgx: Add helper to update SGX_LEPUBKEYHASHn MSRs Kai Huang
2021-03-09  1:39 ` [PATCH v2 13/25] x86/sgx: Add helpers to expose ECREATE and EINIT to KVM Kai Huang
2021-03-15  4:08   ` Kai Huang
2021-03-09  1:39 ` [PATCH v2 14/25] x86/sgx: Move provisioning device creation out of SGX driver Kai Huang
2021-03-09  1:40 ` [PATCH v2 15/25] KVM: x86: Export kvm_mmu_gva_to_gpa_{read,write}() for SGX (VMX) Kai Huang
2021-03-09  1:40 ` [PATCH v2 16/25] KVM: x86: Define new #PF SGX error code bit Kai Huang
2021-03-09  1:40 ` [PATCH v2 17/25] KVM: x86: Add support for reverse CPUID lookup of scattered features Kai Huang
2021-03-09  1:40 ` [PATCH v2 18/25] KVM: x86: Add reverse-CPUID lookup support for scattered SGX features Kai Huang
2021-03-09  1:40 ` [PATCH v2 19/25] KVM: VMX: Add basic handling of VM-Exit from SGX enclave Kai Huang
2021-03-09  1:40 ` [PATCH v2 20/25] KVM: VMX: Frame in ENCLS handler for SGX virtualization Kai Huang
2021-03-09  1:40 ` [PATCH v2 21/25] KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce CPUID restrictions Kai Huang
2021-03-09  1:40 ` [PATCH v2 22/25] KVM: VMX: Add emulation of SGX Launch Control LE hash MSRs Kai Huang
2021-03-09  1:40 ` [PATCH v2 23/25] KVM: VMX: Add ENCLS[EINIT] handler to support SGX Launch Control (LC) Kai Huang
2021-03-09  1:40 ` [PATCH v2 24/25] KVM: VMX: Enable SGX virtualization for SGX1, SGX2 and LC Kai Huang
2021-03-09  1:41 ` [PATCH v2 25/25] KVM: x86: Add capability to grant VM access to privileged SGX attribute Kai Huang
2021-03-09  9:30 ` [PATCH v2 00/25] KVM SGX virtualization support Borislav Petkov
2021-03-09 18:08   ` Kai Huang
2021-03-09 18:49   ` Paolo Bonzini
2021-03-12 22:04     ` Sean Christopherson
2021-03-13  4:30       ` Kai Huang
2021-03-10  9:27   ` Kai Huang
2021-03-10 13:29     ` Borislav Petkov
2021-03-11  2:05       ` Kai Huang
2021-03-10 18:01   ` Jarkko Sakkinen
2021-03-10 20:44     ` Kai Huang

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.