LKML Archive on
 help / color / Atom feed
From: Jarkko Sakkinen <>
	Jarkko Sakkinen <>
Subject: [PATCH v21 27/28] docs: x86/sgx: Document kernel internals
Date: Sat, 13 Jul 2019 20:08:03 +0300
Message-ID: <> (raw)
In-Reply-To: <>

From: Sean Christopherson <>

Document some of the more tricky parts of the kernel implementation

Signed-off-by: Sean Christopherson <>
Co-developed-by: Jarkko Sakkinen <>
Signed-off-by: Jarkko Sakkinen <>
 Documentation/x86/sgx/2.Kernel-internals.rst | 76 ++++++++++++++++++++
 Documentation/x86/sgx/index.rst              |  1 +
 2 files changed, 77 insertions(+)
 create mode 100644 Documentation/x86/sgx/2.Kernel-internals.rst

diff --git a/Documentation/x86/sgx/2.Kernel-internals.rst b/Documentation/x86/sgx/2.Kernel-internals.rst
new file mode 100644
index 000000000000..5c90a65936f2
--- /dev/null
+++ b/Documentation/x86/sgx/2.Kernel-internals.rst
@@ -0,0 +1,76 @@
+.. SPDX-License-Identifier: GPL-2.0
+Kernel Internals
+CPU configuration
+Because SGX has an ever evolving and expanding feature set, it's possible for
+a BIOS or VMM to configure a system in such a way that not all CPUs are equal,
+e.g. where Launch Control is only enabled on a subset of CPUs.  Linux does
+*not* support such a heterogeneous system configuration, nor does it even
+attempt to play nice in the face of a misconfigured system.  With the exception
+of Launch Control's hash MSRs, which can vary per CPU, Linux assumes that all
+CPUs have a configuration that is identical to the boot CPU.
+EPC management
+Because the kernel can't arbitrarily read EPC memory or share RO backing pages
+between enclaves, traditional memory models such as CoW and fork() do not work
+with enclaves.  In other words, the architectural rules of EPC forces it to be
+treated as MAP_SHARED at all times.
+The inability to employ traditional memory models also means that EPC memory
+must be isolated from normal memory pools, e.g. attempting to use EPC memory
+for normal mappings would result in faults and/or perceived data corruption.
+Furthermore, EPC is not enumerated by as normal memory, e.g. BIOS enumerates
+EPC as reserved memory in the e820 tables, or not at all.  As a result, EPC
+memory is directly managed by the SGX subsystem, e.g. SGX employs VM_PFNMAP to
+manually insert/zap/swap page table entries, and exposes EPC to userspace via
+a well known device, /dev/sgx/enclave.
+The net effect is that all enclave VMAs must be MAP_SHARED and are backed by
+a single file, /dev/sgx/enclave.
+EPC oversubscription
+SGX allows to have larger enclaves than amount of available EPC by providing a
+subset of leaf instruction for swapping EPC pages to the system memory.  The
+details of these instructions are discussed in the architecture document. Due
+to the unique requirements for swapping EPC pages, and because EPC pages do not
+have associated page structures, management of the EPC is not handled by the
+standard memory subsystem.
+SGX directly handles swapping of EPC pages, including a thread to initiate the
+reclaiming process and a rudimentary LRU mechanism. When the amount of free EPC
+pages goes below a low watermark the swapping thread starts reclaiming pages.
+The pages that have not been recently accessed (i.e. do not have the A bit set)
+are selected as victim pages. Each enclave holds an shmem file as a backing
+storage for reclaimed pages.
+Launch Control
+The current kernel implementation supports only writable MSRs. The launch is
+performed by setting the MSRs to the hash of the public key modulus of the
+enclave signer and a token with the valid bit set to zero. Because kernel makes
+ultimately all the launch decisions token are not needed for anything.  We
+don't need or have a launch enclave for generating them as the MSRs must always
+be writable.
+The use of provisioning must be controlled because it allows to get access to
+the provisioning keys to attest to a remote party that the software is running
+inside a legit enclave. This could be used by a malware network to ensure that
+its nodes are running inside legit enclaves.
+The driver introduces a special device file /dev/sgx/provision and a special
+ioctl SGX_IOC_ENCLAVE_SET_ATTRIBUTE to accomplish this. A file descriptor
+pointing to /dev/sgx/provision is passed to ioctl from which kernel authorizes
+the PROVISION_KEY attribute to the enclave.
diff --git a/Documentation/x86/sgx/index.rst b/Documentation/x86/sgx/index.rst
index c5dfef62e612..5d660e83d984 100644
--- a/Documentation/x86/sgx/index.rst
+++ b/Documentation/x86/sgx/index.rst
@@ -14,3 +14,4 @@ potentially malicious.
    :maxdepth: 1
+   2.Kernel-internals

  parent reply index

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-13 17:07 [PATCH v21 00/28] Intel SGX foundations Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 01/28] x86/cpufeatures: Add Intel-defined SGX feature bit Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 02/28] x86/cpufeatures: Add SGX sub-features (as Linux-defined bits) Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 03/28] x86/msr: Add IA32_FEATURE_CONTROL.SGX_ENABLE definition Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 04/28] x86/cpufeatures: Add Intel-defined SGX_LC feature bit Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 05/28] x86/msr: Add SGX Launch Control MSR definitions Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 06/28] x86/mm: x86/sgx: Add new 'PF_SGX' page fault error code bit Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 07/28] x86/mm: x86/sgx: Signal SIGSEGV for userspace #PFs w/ PF_SGX Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 08/28] x86/cpu/intel: Detect SGX support and update caps appropriately Jarkko Sakkinen
2019-07-24 19:35   ` Sean Christopherson
2019-08-02 20:48     ` Jarkko Sakkinen
2019-08-07 15:17     ` Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 09/28] x86/sgx: Add ENCLS architectural error codes Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 10/28] x86/sgx: Add SGX1 and SGX2 architectural data structures Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 11/28] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 12/28] x86/sgx: Enumerate and track EPC sections Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 13/28] x86/sgx: Add functions to allocate and free EPC pages Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 14/28] x86/sgx: Add sgx_einit() for initializing enclaves Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 15/28] mm: Introduce vm_ops->may_mprotect() Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 16/28] x86/sgx: Add the Linux SGX Enclave Driver Jarkko Sakkinen
2019-07-29 11:17   ` Ayoun, Serge
2019-08-07 15:15     ` Jarkko Sakkinen
2019-08-07 15:17       ` Jarkko Sakkinen
2019-08-07 16:45         ` Jethro Beekman
2019-08-08 15:40       ` Sean Christopherson
2019-08-09 15:02         ` Jarkko Sakkinen
2019-08-09 15:24           ` Sean Christopherson
2019-08-05 16:16   ` Sean Christopherson
2019-08-05 21:39     ` Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 17/28] x86/sgx: Add provisioning Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 18/28] x86/sgx: Add swapping code to the core and SGX driver Jarkko Sakkinen
2019-08-07  6:33   ` Jethro Beekman
2019-08-07 19:12     ` Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 19/28] x86/sgx: ptrace() support for the " Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 20/28] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 21/28] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 22/28] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen
2019-07-13 17:07 ` [PATCH v21 23/28] x86/vdso: Add __vdso_sgx_enter_enclave() to wrap SGX enclave transitions Jarkko Sakkinen
2019-07-17 22:07   ` Xing, Cedric
2019-07-13 17:08 ` [PATCH v21 24/28] selftests/x86: Add a selftest for SGX Jarkko Sakkinen
2019-07-17 22:37   ` Xing, Cedric
2019-08-02 20:46     ` Jarkko Sakkinen
2019-08-16 15:43     ` Jarkko Sakkinen
2019-08-16 15:51       ` Jarkko Sakkinen
2019-08-16 16:56     ` Jarkko Sakkinen
2019-07-13 17:08 ` [PATCH v21 25/28] x86/sgx: Update MAINTAINERS Jarkko Sakkinen
2019-07-13 17:08 ` [PATCH v21 26/28] docs: x86/sgx: Add Architecture documentation Jarkko Sakkinen
2019-07-13 17:08 ` Jarkko Sakkinen [this message]
2019-07-13 17:08 ` [PATCH v21 28/28] docs: x86/sgx: Document the enclave API Jarkko Sakkinen
2019-07-14 14:36 ` [PATCH v21 00/28] Intel SGX foundations Jarkko Sakkinen
2019-08-07  6:40   ` Jethro Beekman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on

Archives are clonable:
	git clone --mirror lkml/git/0.git
	git clone --mirror lkml/git/1.git
	git clone --mirror lkml/git/2.git
	git clone --mirror lkml/git/3.git
	git clone --mirror lkml/git/4.git
	git clone --mirror lkml/git/5.git
	git clone --mirror lkml/git/6.git
	git clone --mirror lkml/git/7.git
	git clone --mirror lkml/git/8.git
	git clone --mirror lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ \
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:

AGPL code for this site: git clone