From: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-sgx@vger.kernel.org
Cc: akpm@linux-foundation.org, dave.hansen@intel.com,
sean.j.christopherson@intel.com, nhorman@redhat.com,
npmccallum@redhat.com, haitao.huang@intel.com,
andriy.shevchenko@linux.intel.com, tglx@linutronix.de,
kai.svahn@intel.com, bp@alien8.de, josh@joshtriplett.org,
luto@kernel.org, kai.huang@intel.com, rientjes@google.com,
cedric.xing@intel.com, puiterwijk@redhat.com,
Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
linux-doc@vger.kernel.org, Randy Dunlap <rdunlap@infradead.org>
Subject: [PATCH v28 12/22] docs: x86/sgx: Document SGX micro architecture and kernel internals
Date: Wed, 4 Mar 2020 01:35:59 +0200 [thread overview]
Message-ID: <20200303233609.713348-13-jarkko.sakkinen@linux.intel.com> (raw)
In-Reply-To: <20200303233609.713348-1-jarkko.sakkinen@linux.intel.com>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=a, Size: 9110 bytes --]
Document Intel SGX micro architecture and kernel internals. The motivation
is to make the core ideas approachable by keeping a fairly high abstraction
level. Fine-grained micro architecture details can be looked up from Intel
SDM Volume 3D.
Cc: linux-doc@vger.kernel.org
Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
---
Documentation/x86/index.rst | 1 +
Documentation/x86/sgx.rst | 192 ++++++++++++++++++++++++++++++++++++
2 files changed, 193 insertions(+)
create mode 100644 Documentation/x86/sgx.rst
diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst
index a8de2fbc1caa..971f30a7d166 100644
--- a/Documentation/x86/index.rst
+++ b/Documentation/x86/index.rst
@@ -31,3 +31,4 @@ x86-specific Documentation
usb-legacy-support
i386/index
x86_64/index
+ sgx
diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst
new file mode 100644
index 000000000000..740b09323f18
--- /dev/null
+++ b/Documentation/x86/sgx.rst
@@ -0,0 +1,192 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============
+Architecture
+============
+
+*Software Guard eXtensions (SGX)* is a set of instructions that enable ring-3
+applications to set aside private regions of code and data. These regions are
+called enclaves. An enclave can be entered to a fixed set of entry points. Only
+a CPU running inside the enclave can access its code and data.
+
+The support can be determined by
+
+ ``grep sgx /proc/cpuinfo``
+
+Enclave Page Cache
+==================
+
+SGX utilizes an *Enclave Page Cache (EPC)* to store pages that are associated
+with an enclave. It is contained in a BIOS reserved region of physical memory.
+Unlike pages used for regular memory, pages can only be accessed outside the
+enclave for different purposes with the instructions **ENCLS**, **ENCLV** and
+**ENCLU**.
+
+Direct memory accesses to an enclave can be only done by a CPU executing inside
+the enclave. An enclave can be entered with **ENCLU[EENTER]** to a fixed set of
+entry points. However, a CPU executing inside the enclave can do outside memory
+accesses.
+
+Page Types
+----------
+
+**SGX Enclave Control Structure (SECS)**
+ Enclave's address range, attributes and other global data are defined
+ by this structure.
+
+**Regular (REG)**
+ Regular EPC pages contain the code and data of an enclave.
+
+**Thread Control Structure (TCS)**
+ Thread Control Structure pages define the entry points to an enclave and
+ track the execution state of an enclave thread.
+
+**Version Array (VA)**
+ Version Array pages contain 512 slots, each of which can contain a version
+ number for a page evicted from the EPC.
+
+Enclave Page Cache Map
+----------------------
+
+The processor tracks EPC pages via the *Enclave Page Cache Map (EPCM)*. EPCM
+contains an entry for each EPC page, which describes the owning enclave, access
+rights and page type among the other things.
+
+The permissions from EPCM is consulted if and only if walking the kernel page
+tables succeeds. The total permissions are thus a conjunction between page table
+and EPCM permissions.
+
+For all intents and purposes the SGX architecture allows the processor to
+invalidate all EPCM entries at will, i.e. requires that software be prepared to
+handle an EPCM fault at any time. The contents of EPC are encrypted with an
+ephemeral key, which is lost on power transitions.
+
+EPC management
+==============
+
+EPC pages do not have ``struct page`` instances. They are IO memory from kernel
+perspective. The consequence is that they are always mapped as shared memory.
+Kernel defines ``/dev/sgx/enclave`` that can be mapped as ``MAP_SHARED`` to
+define the address range for an enclave.
+
+EPC Over-subscription
+=====================
+
+When the amount of free EPC pages goes below a low watermark the swapping thread
+starts reclaiming pages. The pages that do not have the **A** bit set are
+selected as victim pages.
+
+Launch Control
+==============
+
+SGX provides a launch control mechanism. After all enclave pages have been
+copied, kernel executes **ENCLS[EINIT]**, which initializes the enclave. Only
+after this the CPU can execute inside the enclave.
+
+This leaf function takes an RSA-3072 signature of the enclave measurement and an
+optional cryptographic token. Linux does not take advantage of launch tokens.
+The instruction checks that the signature is signed with the key defined in
+**IA32_SGXLEPUBKEYHASH?** MSRs and the measurement is correct. If so, the
+enclave is allowed to be executed.
+
+MSRs can be configured by the BIOS to be either readable or writable. Linux
+supports only writable configuration in order to give full control to the kernel
+on launch control policy. Readable configuration requires the use of previously
+mentioned launch tokens.
+
+The current kernel implementation supports only writable MSRs. The launch is
+performed by setting the MSRs to the hash of the enclave signer's public key.
+The alternative would be to have *a launch enclave* that would be signed with
+the key set into MSRs, which would then generate launch tokens for other
+enclaves. This would only make sense with read-only MSRs, and thus the option
+has been discarded.
+
+Attestation
+===========
+
+Local Attestation
+-----------------
+
+In local attestation an enclave creates a **REPORT** data structure with
+**ENCLS[EREPORT]**, which describes the origin of an enclave. In particular, it
+contains a AES-CMAC of the enclave contents signed with a report key unique to
+each processor. All enclaves have access to this key.
+
+This mechanism can also be used in addition as a communication channel as the
+**REPORT** data structure includes a 64-byte field for variable information.
+
+Remote Attestation
+------------------
+
+For remote attestation (aka provisioning) there are multiple options available:
+
+* EPID based scheme, which requires the use of Intel managed attestation
+ service.
+* ECDSA based scheme, which allows a 3rd party to act as an attestation service.
+
+Intel provides an open source *quoting enclave (QE)* and *provisioning
+certification enclave (PCE)* for the ECDSA based scheme. PCE acts as the
+CA for the local QE's.
+
+Intel also provides a proprietary binary version of the PCE. This is a
+necessity when the software needs to prove to be running inside a legit enclave
+on real hardware.
+
+The use of remote attestation must be strictly controlled because it allows to
+get access to the provisioning keys to attest to a remote party that the
+software is running inside a legitimate enclave on real hardware. This could be
+potentially used by malware, and thus must be protected.
+
+Enclaves can attest their identity when **ATTRIBUTES.PROVISIONKEY** is set in
+SECS. This attribute authorizes **ENCLS[EGETKEY]** to access provisioning keys.
+
+References
+----------
+
+"Intel® Software Guard Extensions: EPID Provisioning and Attestation Services"
+ https://software.intel.com/sites/default/files/managed/57/0e/ww10-2016-sgx-provisioning-and-attestation-final.pdf
+
+"Supporting Third Party Attestation for Intel® SGX with Intel® Data Center
+Attestation Primitives"
+ https://software.intel.com/sites/default/files/managed/f1/b8/intel-sgx-support-for-third-party-attestation.pdf
+
+Usage Models
+============
+
+Shared Library
+--------------
+
+Sensitive data and the code that acts on it is partitioned from the application
+into a separate library. The library is then linked as a DSO which can be loaded
+into an enclave. The application can then make individual function calls into
+the enclave through special SGX instructions. A run-time within the enclave is
+configured to marshal function parameters into and out of the enclave and to
+call the correct library function.
+
+Application Container
+---------------------
+
+An application may be loaded into a container enclave which is specially
+configured with a library OS and run-time which permits the application to run.
+The enclave run-time and library OS work together to execute the application
+when a thread enters the enclave.
+
+================
+Kernel internals
+================
+
+An enclave is created by opening ``/dev/sgx/enclave`` and calling a set of ioctl
+calls, which reserve a fixed range of memory addresses for the enclave and
+initialize its memory contents.
+
+An enclave can be made visible with ``mmap()`` calls. Permissions are capped by
+enclave page permissions given during the building phase because CPU disallows a
+PTE have higher permissions than the enclave page that it contains.
+
+Enclaves can be forked or sent through UDS sockets, which allows an enclave
+consumer and a builder to be separate processes with a different set of
+privileges.
+
+The backing memory is implemented with a private shemm file, which is not
+accounted. This makes it advicable to not allow all processes in a system
+to build enclaves.
--
2.25.0
next prev parent reply other threads:[~2020-03-03 23:38 UTC|newest]
Thread overview: 110+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-03 23:35 [PATCH v28 00/22] Intel SGX foundations Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 01/22] x86/sgx: Update MAINTAINERS Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 02/22] x86/cpufeatures: x86/msr: Add Intel SGX hardware bits Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 03/22] x86/cpufeatures: x86/msr: Intel SGX Launch Control " Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 04/22] x86/mm: x86/sgx: Signal SIGSEGV with PF_SGX Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 05/22] x86/sgx: Add SGX microarchitectural data structures Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 06/22] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen
2020-03-09 21:14 ` Sean Christopherson
2020-03-03 23:35 ` [PATCH v28 07/22] x86/cpu/intel: Detect SGX supprt Jarkko Sakkinen
2020-03-09 21:56 ` Sean Christopherson
2020-03-11 17:03 ` Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 08/22] x86/sgx: Enumerate and track EPC sections Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 09/22] x86/sgx: Add functions to allocate and free EPC pages Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 10/22] mm: Introduce vm_ops->may_mprotect() Jarkko Sakkinen
2020-03-03 23:35 ` [PATCH v28 11/22] x86/sgx: Linux Enclave Driver Jarkko Sakkinen
2020-03-05 17:40 ` Sean Christopherson
2020-03-05 18:24 ` Jethro Beekman
2020-03-05 19:04 ` Sean Christopherson
2020-03-06 19:00 ` Jarkko Sakkinen
2020-03-19 18:22 ` Dr. Greg
2020-03-06 18:58 ` Jarkko Sakkinen
2020-03-03 23:35 ` Jarkko Sakkinen [this message]
2020-03-03 23:36 ` [PATCH v28 13/22] selftests/x86: Recurse into subdirectories Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 14/22] selftests/x86: Add a selftest for SGX Jarkko Sakkinen
2020-03-04 19:27 ` Nathaniel McCallum
2020-03-05 11:33 ` Jarkko Sakkinen
2020-03-06 15:42 ` Dr. Greg
2020-03-06 19:07 ` Jarkko Sakkinen
2020-03-07 17:42 ` Dr. Greg
2020-03-10 13:08 ` Jarkko Sakkinen
2020-03-11 13:28 ` Jarkko Sakkinen
2020-03-11 16:40 ` Sean Christopherson
2020-03-13 19:24 ` Jarkko Sakkinen
2020-03-04 19:44 ` Nathaniel McCallum
2020-03-04 19:51 ` Nathaniel McCallum
2020-03-06 5:32 ` Dr. Greg
2020-03-06 19:04 ` Jarkko Sakkinen
2020-03-10 19:29 ` Haitao Huang
2020-03-11 9:13 ` Dr. Greg
2020-03-11 17:15 ` Haitao Huang
2020-03-17 1:07 ` Dr. Greg
2020-03-03 23:36 ` [PATCH v28 15/22] x86/sgx: Add provisioning Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 16/22] x86/sgx: Add a page reclaimer Jarkko Sakkinen
2020-03-05 19:03 ` Sean Christopherson
2020-03-06 18:47 ` Jarkko Sakkinen
2020-03-12 18:38 ` Sean Christopherson
2020-03-15 0:27 ` Jarkko Sakkinen
2020-03-15 1:17 ` Jarkko Sakkinen
2020-03-09 21:16 ` Sean Christopherson
2020-03-03 23:36 ` [PATCH v28 17/22] x86/sgx: ptrace() support for the SGX driver Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 18/22] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 19/22] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 20/22] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen
2020-03-03 23:36 ` [PATCH v28 21/22] x86/vdso: Implement a vDSO for Intel SGX enclave call Jarkko Sakkinen
2020-03-11 17:30 ` Nathaniel McCallum
2020-03-11 17:38 ` Jethro Beekman
2020-03-11 19:15 ` Nathaniel McCallum
2020-03-13 15:48 ` Nathaniel McCallum
2020-03-13 16:46 ` Sean Christopherson
2020-03-13 18:32 ` Nathaniel McCallum
2020-03-13 18:44 ` Sean Christopherson
2020-03-13 20:14 ` Nathaniel McCallum
2020-03-13 22:08 ` Sean Christopherson
2020-03-14 14:10 ` Nathaniel McCallum
2020-03-18 23:40 ` Sean Christopherson
2020-03-19 0:38 ` Xing, Cedric
2020-03-19 1:03 ` Sean Christopherson
2020-03-20 13:55 ` Nathaniel McCallum
2020-03-15 1:25 ` Jarkko Sakkinen
2020-03-15 17:53 ` Nathaniel McCallum
2020-03-16 13:31 ` Jethro Beekman
2020-03-16 13:57 ` Nathaniel McCallum
2020-03-16 13:59 ` Jethro Beekman
2020-03-16 14:03 ` Nathaniel McCallum
2020-03-16 17:17 ` Sean Christopherson
2020-03-16 21:27 ` Jarkko Sakkinen
2020-03-16 21:29 ` Jarkko Sakkinen
2020-03-16 22:55 ` Sean Christopherson
2020-03-16 23:56 ` Xing, Cedric
2020-03-18 22:01 ` Jarkko Sakkinen
2020-03-18 22:18 ` Jarkko Sakkinen
2020-03-16 13:56 ` Jarkko Sakkinen
2020-03-16 14:01 ` Nathaniel McCallum
2020-03-16 21:38 ` Jarkko Sakkinen
2020-03-16 22:53 ` Sean Christopherson
2020-03-16 23:50 ` Xing, Cedric
2020-03-16 23:59 ` Sean Christopherson
2020-03-17 0:18 ` Xing, Cedric
2020-03-17 0:27 ` Sean Christopherson
2020-03-17 16:37 ` Nathaniel McCallum
2020-03-17 16:50 ` Nathaniel McCallum
2020-03-17 21:40 ` Xing, Cedric
2020-03-17 22:09 ` Sean Christopherson
2020-03-17 22:36 ` Xing, Cedric
2020-03-17 23:57 ` Sean Christopherson
2020-03-17 22:23 ` Xing, Cedric
2020-03-18 13:01 ` Nathaniel McCallum
2020-03-20 15:53 ` Nathaniel McCallum
2020-03-17 16:28 ` Nathaniel McCallum
2020-03-18 22:58 ` Jarkko Sakkinen
2020-03-18 22:39 ` Jarkko Sakkinen
2020-03-11 19:30 ` Nathaniel McCallum
2020-03-13 0:52 ` Sean Christopherson
2020-03-13 16:07 ` Nathaniel McCallum
2020-03-13 16:33 ` Sean Christopherson
2020-03-03 23:36 ` [PATCH v28 22/22] selftests/x86: Add vDSO selftest for SGX Jarkko Sakkinen
2020-03-04 19:24 ` [PATCH v28 00/22] Intel SGX foundations Nathaniel McCallum
2020-03-17 16:00 ` Jordan Hand
2020-03-18 21:56 ` Jarkko Sakkinen
2020-03-19 17:16 ` Dr. Greg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200303233609.713348-13-jarkko.sakkinen@linux.intel.com \
--to=jarkko.sakkinen@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=andriy.shevchenko@linux.intel.com \
--cc=bp@alien8.de \
--cc=cedric.xing@intel.com \
--cc=dave.hansen@intel.com \
--cc=haitao.huang@intel.com \
--cc=josh@joshtriplett.org \
--cc=kai.huang@intel.com \
--cc=kai.svahn@intel.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=luto@kernel.org \
--cc=nhorman@redhat.com \
--cc=npmccallum@redhat.com \
--cc=puiterwijk@redhat.com \
--cc=rdunlap@infradead.org \
--cc=rientjes@google.com \
--cc=sean.j.christopherson@intel.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).