From: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
To: x86@kernel.org, linux-sgx@vger.kernel.org
Cc: linux-kernel@vger.kernel.org,
Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
linux-doc@vger.kernel.org, Randy Dunlap <rdunlap@infradead.org>,
Sean Christopherson <sean.j.christopherson@intel.com>,
akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com,
asapek@google.com, bp@alien8.de, cedric.xing@intel.com,
chenalexchen@google.com, conradparker@google.com,
cyhanish@google.com, dave.hansen@intel.com,
haitao.huang@intel.com, josh@joshtriplett.org,
kai.huang@intel.com, kai.svahn@intel.com, kmoy@google.com,
ludloff@google.com, luto@kernel.org, nhorman@redhat.com,
npmccallum@redhat.com, puiterwijk@redhat.com,
rientjes@google.com, tglx@linutronix.de, yaozhangx@google.com
Subject: [PATCH v33 20/21] docs: x86/sgx: Document SGX micro architecture and kernel internals
Date: Thu, 18 Jun 2020 01:08:42 +0300 [thread overview]
Message-ID: <20200617220844.57423-21-jarkko.sakkinen@linux.intel.com> (raw)
In-Reply-To: <20200617220844.57423-1-jarkko.sakkinen@linux.intel.com>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 9339 bytes --]
Document the Intel SGX kernel architecture. The fine-grained micro
architecture details can be looked up from Intel SDM Volume 3D.
Cc: linux-doc@vger.kernel.org
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
---
Documentation/x86/index.rst | 1 +
Documentation/x86/sgx.rst | 197 ++++++++++++++++++++++++++++++++++++
2 files changed, 198 insertions(+)
create mode 100644 Documentation/x86/sgx.rst
diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst
index 265d9e9a093b..807290bf357c 100644
--- a/Documentation/x86/index.rst
+++ b/Documentation/x86/index.rst
@@ -30,3 +30,4 @@ x86-specific Documentation
usb-legacy-support
i386/index
x86_64/index
+ sgx
diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst
new file mode 100644
index 000000000000..249843666963
--- /dev/null
+++ b/Documentation/x86/sgx.rst
@@ -0,0 +1,197 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============
+Architecture
+============
+
+*Software Guard eXtensions (SGX)* is a set of instructions that enable ring-3
+applications to set aside private regions of code and data. These regions are
+called enclaves. An enclave can be entered to a fixed set of entry points. Only
+a CPU running inside the enclave can access its code and data.
+
+The support can be determined by
+
+ ``grep sgx /proc/cpuinfo``
+
+Enclave Page Cache
+==================
+
+SGX utilizes an *Enclave Page Cache (EPC)* to store pages that are associated
+with an enclave. It is contained in a BIOS reserved region of physical memory.
+Unlike pages used for regular memory, pages can only be accessed outside the
+enclave for different purposes with the instructions **ENCLS**, **ENCLV** and
+**ENCLU**.
+
+Direct memory accesses to an enclave can be only done by a CPU executing inside
+the enclave. An enclave can be entered with **ENCLU[EENTER]** to a fixed set of
+entry points. However, a CPU executing inside the enclave can do outside memory
+accesses.
+
+Page Types
+----------
+
+**SGX Enclave Control Structure (SECS)**
+ Enclave's address range, attributes and other global data are defined
+ by this structure.
+
+**Regular (REG)**
+ Regular EPC pages contain the code and data of an enclave.
+
+**Thread Control Structure (TCS)**
+ Thread Control Structure pages define the entry points to an enclave and
+ track the execution state of an enclave thread.
+
+**Version Array (VA)**
+ Version Array pages contain 512 slots, each of which can contain a version
+ number for a page evicted from the EPC.
+
+Enclave Page Cache Map
+----------------------
+
+The processor tracks EPC pages via the *Enclave Page Cache Map (EPCM)*. EPCM
+contains an entry for each EPC page, which describes the owning enclave, access
+rights and page type among the other things.
+
+The permissions from EPCM is consulted if and only if walking the kernel page
+tables succeeds. The total permissions are thus a conjunction between page table
+and EPCM permissions.
+
+For all intents and purposes the SGX architecture allows the processor to
+invalidate all EPCM entries at will, i.e. requires that software be prepared to
+handle an EPCM fault at any time. The contents of EPC are encrypted with an
+ephemeral key, which is lost on power transitions.
+
+EPC management
+==============
+
+EPC pages do not have ``struct page`` instances. They are IO memory from kernel
+perspective. The consequence is that they are always mapped as shared memory.
+Kernel defines ``/dev/sgx/enclave`` that can be mapped as ``MAP_SHARED`` to
+define the address range for an enclave.
+
+EPC Over-subscription
+=====================
+
+When the amount of free EPC pages goes below a low watermark the swapping thread
+starts reclaiming pages. The pages that do not have the **A** bit set are
+selected as victim pages.
+
+Launch Control
+==============
+
+SGX provides a launch control mechanism. After all enclave pages have been
+copied, kernel executes **ENCLS[EINIT]**, which initializes the enclave. Only
+after this the CPU can execute inside the enclave.
+
+This leaf function takes an RSA-3072 signature of the enclave measurement and an
+optional cryptographic token. Linux does not take advantage of launch tokens.
+The instruction checks that the signature is signed with the key defined in
+**IA32_SGXLEPUBKEYHASH?** MSRs and the measurement is correct. If so, the
+enclave is allowed to be executed.
+
+MSRs can be configured by the BIOS to be either readable or writable. Linux
+supports only writable configuration in order to give full control to the kernel
+on launch control policy. Readable configuration requires the use of previously
+mentioned launch tokens.
+
+The current kernel implementation supports only writable MSRs. The launch is
+performed by setting the MSRs to the hash of the enclave signer's public key.
+The alternative would be to have *a launch enclave* that would be signed with
+the key set into MSRs, which would then generate launch tokens for other
+enclaves. This would only make sense with read-only MSRs, and thus the option
+has been discarded.
+
+Attestation
+===========
+
+Local Attestation
+-----------------
+
+In local attestation an enclave creates a **REPORT** data structure with
+**ENCLS[EREPORT]**, which describes the origin of an enclave. In particular, it
+contains a AES-CMAC of the enclave contents signed with a report key unique to
+each processor. All enclaves have access to this key.
+
+This mechanism can also be used in addition as a communication channel as the
+**REPORT** data structure includes a 64-byte field for variable information.
+
+Remote Attestation
+------------------
+
+Provisioning Certification Enclave (PCE), the root of trust for other enclaves,
+generates a signing key from a fused key called Provisioning Certification Key.
+PCE can then use this key to certify an attestation key of a QE, e.g. we get the
+chain of trust down to the hardware if the Intel signed PCE is used.
+
+To use the needed keys, ATTRIBUTE.PROVISIONKEY is required but should be only
+allowed for those who actually need it so that only the trusted parties can
+certify QE's.
+
+A device file called /dev/sgx/provision exists to provide file descriptors that
+act as privilege tokens for building provisioning enclaves. These can be
+associated with enclaves with the ioctl SGX_IOC_ENCLAVE_SET_ATTRIBUTE.
+
+Encryption engines
+==================
+
+In order to conceal the enclave data while it is out of the CPU package,
+memory controller has to be extended with an encryption engine. MC can then
+route incoming requests coming from CPU cores running in enclave mode to the
+encryption engine.
+
+In CPUs prior to Icelake, Memory Encryption Engine (MEE) is used to
+encrypt pages leaving the CPU caches. MEE uses a n-ary Merkle tree with root in
+SRAM to maintain integrity of the encrypted data. This provides integrity and
+anti-replay protection but does not scale to large memory sizes because the time
+required to update the Merkle tree grows logarithmically in relation to the
+memory size.
+
+CPUs starting from Icelake use Total Memory Encryption (TME) in the place of
+MEE. TME throws away the Merkle tree, which means losing integrity and
+anti-replay protection but also enables variable size memory pools for EPC.
+Using this attack for benefit would require an interposer on the system bus.
+
+Backing storage
+===============
+
+Backing storage is shared and not accounted. It is implemented as a private
+shmem file. Providing a backing storage in some form from user space is not
+possible - accounting would go to invalid state as reclaimed pages would get
+accounted to the processes of which behalf the kernel happened to be acting on.
+
+Access control
+==============
+
+`mmap()` permissions are capped by the enclave permissions. A direct
+consequence of this is that all the pages for an address range must be added
+before `mmap()` can be applied. Effectively an enclave page with minimum
+permission in the address range sets the permission cap for the mapping
+operation.
+
+Usage Models
+============
+
+Shared Library
+--------------
+
+Sensitive data and the code that acts on it is partitioned from the application
+into a separate library. The library is then linked as a DSO which can be loaded
+into an enclave. The application can then make individual function calls into
+the enclave through special SGX instructions. A run-time within the enclave is
+configured to marshal function parameters into and out of the enclave and to
+call the correct library function.
+
+Application Container
+---------------------
+
+An application may be loaded into a container enclave which is specially
+configured with a library OS and run-time which permits the application to run.
+The enclave run-time and library OS work together to execute the application
+when a thread enters the enclave.
+
+References
+==========
+
+"Supporting Third Party Attestation for Intel® SGX with Intel® Data Center
+Attestation Primitives"
+ https://software.intel.com/sites/default/files/managed/f1/b8/intel-sgx-support-for-third-party-attestation.pdf
--
2.25.1
next prev parent reply other threads:[~2020-06-17 22:13 UTC|newest]
Thread overview: 93+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-17 22:08 [PATCH v33 00/21] Intel SGX foundations Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 01/21] x86/cpufeatures: x86/msr: Add Intel SGX hardware bits Jarkko Sakkinen
2020-06-22 17:37 ` Borislav Petkov
2020-06-25 1:25 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 02/21] x86/cpufeatures: x86/msr: Add Intel SGX Launch Control " Jarkko Sakkinen
2020-06-24 13:04 ` Borislav Petkov
2020-06-24 14:34 ` Sean Christopherson
2020-06-25 1:28 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 03/21] x86/mm: x86/sgx: Signal SIGSEGV with PF_SGX Jarkko Sakkinen
2020-06-25 8:59 ` Borislav Petkov
2020-06-25 15:34 ` Sean Christopherson
2020-06-25 16:49 ` Borislav Petkov
2020-06-25 20:52 ` Jarkko Sakkinen
2020-06-25 21:11 ` Borislav Petkov
2020-06-26 13:34 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 04/21] x86/sgx: Add SGX microarchitectural data structures Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 05/21] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 06/21] x86/cpu/intel: Detect SGX support Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 07/21] x86/cpu/intel: Add nosgx kernel parameter Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 08/21] x86/sgx: Initialize metadata for Enclave Page Cache (EPC) sections Jarkko Sakkinen
2020-06-25 10:14 ` Borislav Petkov
2020-06-25 20:11 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 09/21] x86/sgx: Add __sgx_alloc_epc_page() and sgx_free_epc_page() Jarkko Sakkinen
2020-06-25 17:06 ` Borislav Petkov
2020-06-25 20:55 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 10/21] mm: Introduce vm_ops->may_mprotect() Jarkko Sakkinen
2020-06-25 17:14 ` Borislav Petkov
2020-06-25 17:30 ` Matthew Wilcox
2020-06-25 18:06 ` Sean Christopherson
2020-06-25 22:40 ` Jarkko Sakkinen
2020-06-25 22:26 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 11/21] x86/sgx: Linux Enclave Driver Jarkko Sakkinen
2020-06-25 17:23 ` Borislav Petkov
2020-06-25 18:34 ` Sean Christopherson
2020-06-25 18:45 ` Borislav Petkov
2020-06-26 14:19 ` Jarkko Sakkinen
2020-06-25 20:21 ` Jarkko Sakkinen
2020-06-25 20:25 ` Borislav Petkov
2020-06-26 13:40 ` Jarkko Sakkinen
2020-06-25 18:53 ` Borislav Petkov
2020-06-26 14:17 ` Jarkko Sakkinen
2020-06-26 9:14 ` Borislav Petkov
2020-06-26 14:16 ` Sean Christopherson
2020-06-26 14:20 ` Borislav Petkov
2020-07-03 23:04 ` Jarkko Sakkinen
2020-07-03 3:09 ` Jarkko Sakkinen
2020-06-26 15:34 ` Borislav Petkov
2020-07-04 0:13 ` Jarkko Sakkinen
2020-10-26 21:26 ` Dave Hansen
2020-10-27 1:52 ` Jarkko Sakkinen
2020-10-27 10:05 ` Borislav Petkov
2020-10-27 15:20 ` Dave Hansen
2020-10-27 15:37 ` Borislav Petkov
2020-06-27 17:43 ` Borislav Petkov
2020-06-29 15:27 ` Sean Christopherson
2020-06-29 15:37 ` Borislav Petkov
2020-07-04 1:43 ` Jarkko Sakkinen
2020-07-07 1:38 ` Sean Christopherson
2020-07-07 3:29 ` Jarkko Sakkinen
2020-07-04 1:42 ` Jarkko Sakkinen
2020-07-02 3:59 ` Sean Christopherson
2020-07-04 3:31 ` Jarkko Sakkinen
2020-09-02 3:06 ` Haitao Huang
2020-09-02 16:10 ` Sean Christopherson
2020-09-02 18:40 ` Haitao Huang
2020-09-04 12:01 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 12/21] x86/sgx: Allow a limited use of ATTRIBUTE.PROVISIONKEY for attestation Jarkko Sakkinen
2020-06-29 16:02 ` Borislav Petkov
2020-06-29 22:04 ` Sean Christopherson
2020-06-30 8:49 ` Borislav Petkov
2020-06-30 14:20 ` Sean Christopherson
2020-06-30 17:13 ` Andy Lutomirski
2020-07-02 20:47 ` Dr. Greg
2020-07-03 2:43 ` Jarkko Sakkinen
2020-07-03 2:38 ` Jarkko Sakkinen
2020-07-03 2:32 ` Jarkko Sakkinen
2020-07-03 2:55 ` Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 13/21] x86/sgx: Add a page reclaimer Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 14/21] x86/sgx: ptrace() support for the SGX driver Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 15/21] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen
2020-06-29 17:10 ` Borislav Petkov
2020-06-30 6:00 ` Sean Christopherson
2020-06-30 8:41 ` Borislav Petkov
2020-06-30 14:55 ` Sean Christopherson
2020-06-30 16:48 ` Andy Lutomirski
2020-06-30 17:23 ` Sean Christopherson
2020-07-02 12:52 ` Thomas Gleixner
2020-06-17 22:08 ` [PATCH v33 16/21] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 17/21] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 18/21] x86/vdso: Implement a vDSO for Intel SGX enclave call Jarkko Sakkinen
2020-06-17 22:08 ` [PATCH v33 19/21] selftests/x86: Add a selftest for SGX Jarkko Sakkinen
2020-06-17 22:08 ` Jarkko Sakkinen [this message]
2020-06-17 22:08 ` [PATCH v33 21/21] x86/sgx: Update MAINTAINERS Jarkko Sakkinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200617220844.57423-21-jarkko.sakkinen@linux.intel.com \
--to=jarkko.sakkinen@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=andriy.shevchenko@linux.intel.com \
--cc=asapek@google.com \
--cc=bp@alien8.de \
--cc=cedric.xing@intel.com \
--cc=chenalexchen@google.com \
--cc=conradparker@google.com \
--cc=cyhanish@google.com \
--cc=dave.hansen@intel.com \
--cc=haitao.huang@intel.com \
--cc=josh@joshtriplett.org \
--cc=kai.huang@intel.com \
--cc=kai.svahn@intel.com \
--cc=kmoy@google.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=ludloff@google.com \
--cc=luto@kernel.org \
--cc=nhorman@redhat.com \
--cc=npmccallum@redhat.com \
--cc=puiterwijk@redhat.com \
--cc=rdunlap@infradead.org \
--cc=rientjes@google.com \
--cc=sean.j.christopherson@intel.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=yaozhangx@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).