From: Randy Dunlap <rdunlap@infradead.org>
To: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
x86@kernel.org, platform-driver-x86@vger.kernel.org
Cc: dave.hansen@intel.com, sean.j.christopherson@intel.com,
nhorman@redhat.com, npmccallum@redhat.com,
Jonathan Corbet <corbet@lwn.net>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
open list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v11 12/13] intel_sgx: driver documentation
Date: Fri, 8 Jun 2018 14:41:07 -0700 [thread overview]
Message-ID: <d88234f6-d9f9-561e-b1ab-4bfaddc3dd72@infradead.org> (raw)
In-Reply-To: <20180608171216.26521-13-jarkko.sakkinen@linux.intel.com>
On 06/08/2018 10:09 AM, Jarkko Sakkinen wrote:
> Documentation of the features of the Software Guard eXtensions usable
> for the Linux kernel and how the driver internals uses these features.
> In addition, contains documentation for the ioctl API.
>
> Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
Hi,
I have a few corrections below...
> ---
> Documentation/index.rst | 1 +
> Documentation/x86/intel_sgx.rst | 195 ++++++++++++++++++++++++++++++++
> 2 files changed, 196 insertions(+)
> create mode 100644 Documentation/x86/intel_sgx.rst
>
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 3b99ab931d41..b9fb92928e8c 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -100,6 +100,7 @@ implementation.
> :maxdepth: 2
>
> sh/index
> + x86/index
>
> Korean translations
> -------------------
> diff --git a/Documentation/x86/intel_sgx.rst b/Documentation/x86/intel_sgx.rst
> new file mode 100644
> index 000000000000..ecbe544eb2cb
> --- /dev/null
> +++ b/Documentation/x86/intel_sgx.rst
> @@ -0,0 +1,195 @@
> +===================
> +Intel(R) SGX driver
> +===================
> +
> +Introduction
> +============
> +
> +Intel(R) SGX is a set of CPU instructions that can be used by applications to
> +set aside private regions of code and data. The code outside the enclave is
> +disallowed to access the memory inside the enclave by the CPU access control.
> +In a way you can think that SGX provides inverted sandbox. It protects the
> +application from a malicious host.
> +
> +You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``:
> +
> + ``cat /proc/cpuinfo | grep sgx``
> +
> +Overview of SGX
> +===============
> +
> +SGX has a set of data structures to maintain information about the enclaves and
> +their security properties. BIOS reserves a fixed size region of physical memory
> +for these structures by setting Processor Reserved Memory Range Registers
> +(PRMRR).
> +
> +This memory range is protected from outside access by the CPU and all the data
> +coming in and out of the CPU package is encrypted by a key that is generated for
> +each boot cycle.
> +
> +Enclaves execute in ring-3 in a special enclave submode using pages from the
> +reserved memory range. A fixed logical address range for the enclave is reserved
> +by ENCLS(ECREATE), a leaf instruction used to create enclaves. It is referred in
> +the documentation commonly as the ELRANGE.
> +
> +Every memory access to the ELRANGE is asserted by the CPU. If the CPU is not
> +executing in the enclave mode inside the enclave, #GP is raised. On the other
> +hand enclave code can make memory accesses both inside and outside of the
> +ELRANGE.
> +
> +Enclave can only execute code inside the ELRANGE. Instructions that may cause
> +VMEXIT, IO instructions and instructions that require a privilege change are
> +prohibited inside the enclave. Interrupts and exceptions always cause enclave
> +to exit and jump to an address outside the enclave given when the enclave is
> +entered by using the leaf instruction ENCLS(EENTER).
> +
> +Data types
> +----------
> +
> +The protected memory range contains the following data:
> +
> +* **Enclave Page Cache (EPC):** protected pages
> +* **Enclave Page Cache Map (EPCM):** a database that describes the state of the
> + pages and link them to an enclave.
> +
> +EPC has a number of different types of pages:
> +
> +* **SGX Enclave Control Structure (SECS)**: describes the global
> + properties of an enclave.
> +* **Regular (REG):** code and data pages in the ELRANGE.
> +* **Thread Control Structure (TCS):** pages that define entry points inside an
> + enclave. The enclave can only be entered through these entry points and each
> + can host a single hardware thread at a time.
> +* **Version Array (VA)**: 64-bit version numbers for pages that have been
> + swapped outside the enclave. Each page contains 512 version numbers.
> +
> +Launch control
> +--------------
> +
> +To launch an enclave, two structures must be provided for ENCLS(EINIT):
> +
> +1. **SIGSTRUCT:** signed measurement of the enclave binary.
> +2. **EINITTOKEN:** a cryptographic token CMAC-signed with a AES256-key called
> + *launch key*, which is re-generated for each boot cycle.
> +
> +The CPU holds a SHA256 hash of a 3072-bit RSA public key inside
> +IA32_SGXLEPUBKEYHASHn MSRs. Enclaves with a SIGSTRUCT that is signed with this
> +key do not require a valid EINITTOKEN and can be authorized with special
> +privileges. One of those privileges is ability to acquire the launch key with
> +ENCLS(EGETKEY).
> +
> +**IA32_FEATURE_CONTROL[17]** is used by to BIOS configure whether
by the BIOS to configure whether
> +IA32_SGXLEPUBKEYHASH MSRs are read-only or read-write before locking the
> +feature control register and handing over control to the operating system.
> +
> +Enclave construction
> +--------------------
> +
> +The construction is started by filling out the SECS that contains enclave
> +address range, privileged attributes and measurement of TCS and REG pages (pages
> +that will be mapped to the address range) among the other things. This structure
> +is passed out to the ENCLS(ECREATE) together with a physical address of a page
> +in EPC that will hold the SECS.
> +
> +Then pages are added with ENCLS(EADD) and measured with ENCLS(EEXTEND). Finally
"measured"? what does that mean?
> +enclave is initialized with ENCLS(EINIT). ENCLS(INIT) checks that the SIGSTRUCT
> +is signed with the contained public key and that the supplied EINITTOKEN is
> +valid (CMAC'd with the launch key). If these hold, the enclave is successfully
> +initialized.
> +
> +Swapping pages
> +--------------
> +
> +Enclave pages can be swapped out with ENCLS(EWB) to the unprotected memory. In
> +addition to the EPC page, ENCLS(EWB) takes in a VA page and address for PCMD
> +structure (Page Crypto MetaData) as input. The VA page will seal a version
> +number for the page. PCMD is 128 byte structure that contains tracking
> +information for the page, most importantly its MAC. With these structures the
> +enclave is sealed and rollback protected while it resides in the unprotected
> +memory.
> +
> +Before the page can be swapped out it must not have any active TLB references.
> +By using ENCLS(EBLOCK) instructions no new TLB entries can be created to it.
> +After this the a counter called *epoch* associated hardware threads inside the
huh?
> +enclave is increased with ENCLS(ETRACK). After all the threads from the previous
> +epoch have exited the page can be safely swapped out.
> +
> +An enclave memory access to a swapped out pages will cause #PF. #PF handler can
> +fault the page back by using ENCLS(ELDU).
> +
> +Kernel internals
> +================
> +
> +Requirements
> +------------
> +
> +Because SGX has an ever evolving and expanding feature set, it's possible for
> +a BIOS or VMM to configure a system in such a way that not all cpus are equal,
CPUs
> +e.g. where Launch Control is only enabled on a subset of cpus. Linux does
CPUs.
> +*not* support such a heterogenous system configuration, nor does it even
heterogeneous
> +attempt to play nice in the face of a misconfigured system. With the exception
> +of Launch Control's hash MSRs, which can vary per cpu, Linux assumes that all
CPU,
> +cpus have a configuration that is identical to the boot cpu.
CPUs CPU.
> +
> +
> +Roles and responsibilities
> +--------------------------
> +
> +SGX introduces system resources, e.g. EPC memory, that must be accessible to
> +multiple entities, e.g. the native kernel driver (to expose SGX to userspace)
> +and KVM (to expose SGX to VMs), ideally without introducing any dependencies
> +between each SGX entity. To that end, the kernel owns and manages the shared
> +system resources, i.e. the EPC and Launch Control MSRs, and defines functions
> +that provide appropriate access to the shared resources. SGX support for
> +userpace and VMs is left to the SGX platform driver and KVM respectively.
userspace
> +
> +Launching enclaves
> +------------------
> +
> +For privileged enclaves the launch is performed simply by submitting the
> +SIGSTRUCT for that enclave to ENCLS(EINIT). For unprivileged enclaves the
> +driver hosts a process in ring-3 that hosts a launch enclave signed with a key
> +supplied for kbuild.
> +
> +The current implementation of the launch enclave generates a token for any
> +enclave. In the future it could be potentially extended to have ways to
> +configure policy what can be lauched.
launched.
> +
> +The driver will fail to initialize if it cannot start its own launch enclave.
> +A user space application can submit a SIGSTRUCT instance through the ioctl API.
> +The kernel will take care of the rest.
> +
> +This design assures that the Linux kernel has always full control, which
> +enclaves get to launch and which do not, even if the public key MSRs are
> +read-only. Having launch intrinsics inside the kernel also enables easy
> +development of enclaves without necessarily needing any heavy weight SDK.
> +Having a low-barrier to implement enclaves could make sense for example for
low barrier
> +system daemons where amount of dependecies ought to be minimized.
dependencies
> +
> +EPC management
> +--------------
> +
> +Due to the unique requirements for swapping EPC pages, and because EPC pages
> +(currently) do not have associated page structures, management of the EPC is
> +not handled by the standard Linux swapper. SGX directly handles swapping
> +of EPC pages, including a kthread to initiate reclaim and a rudimentary LRU
> +mechanism. Consumsers of EPC pages, e.g. the SGX driver, are required to
Consumers
> +implement function callbacks that can be invoked by the kernel to age,
> +swap, and/or forcefully reclaim a target EPC page. In effect, the kernel
> +controls what happens and when, while the consumers (driver, KVM, etc..) do
> +the actual work.
> +
> +SGX uapi
> +========
> +
> +.. kernel-doc:: drivers/platform/x86/intel_sgx/sgx_ioctl.c
> + :functions: sgx_ioc_enclave_create
> + sgx_ioc_enclave_add_page
> + sgx_ioc_enclave_init
> +
> +.. kernel-doc:: arch/x86/include/uapi/asm/sgx.h
> +
> +References
> +==========
> +
> +* System Programming Manual: 39.1.4 Intel® SGX Launch Control Configuration
>
--
~Randy
next prev parent reply other threads:[~2018-06-08 21:41 UTC|newest]
Thread overview: 104+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-08 17:09 [PATCH v11 00/13] Intel SGX1 support Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 01/13] compiler.h, kasan: add __SANITIZE_ADDRESS__ check for __no_kasan_or_inline Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 02/13] x86, sgx: updated MAINTAINERS Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 03/13] x86, sgx: add SGX definitions to cpufeature Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 04/13] x86, sgx: add SGX definitions to msr-index.h Jarkko Sakkinen
2018-06-08 17:25 ` Dave Hansen
2018-06-19 13:18 ` Jarkko Sakkinen
2018-06-19 14:01 ` Dave Hansen
2018-06-21 17:22 ` Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 05/13] x86, cpufeatures: add Intel-defined SGX leaf CPUID_12_EAX Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 06/13] crypto: aesni: add minimal build option for SGX LE Jarkko Sakkinen
2018-06-08 17:27 ` Dave Hansen
2018-06-11 15:24 ` Sean Christopherson
2018-06-08 17:09 ` [PATCH v11 07/13] x86, sgx: detect Intel SGX Jarkko Sakkinen
2018-06-08 17:36 ` Dave Hansen
2018-06-18 21:36 ` [intel-sgx-kernel-dev] " Andy Lutomirski
2018-06-25 7:39 ` Jarkko Sakkinen
2018-06-19 13:33 ` Jarkko Sakkinen
2018-06-11 11:35 ` Neil Horman
2018-06-19 13:34 ` Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 08/13] x86, sgx: added ENCLS wrappers Jarkko Sakkinen
2018-06-08 17:43 ` Dave Hansen
2018-06-19 13:25 ` Jarkko Sakkinen
2018-06-20 13:12 ` Sean Christopherson
2018-06-25 9:16 ` Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 09/13] x86, sgx: basic routines for enclave page cache Jarkko Sakkinen
2018-06-08 18:21 ` Jethro Beekman
2018-06-18 21:33 ` [intel-sgx-kernel-dev] " Andy Lutomirski
2018-06-25 7:36 ` Jarkko Sakkinen
2018-06-19 14:08 ` Jarkko Sakkinen
2018-06-19 15:44 ` Jethro Beekman
2018-06-08 18:24 ` Dave Hansen
2018-06-19 14:57 ` Jarkko Sakkinen
2018-06-19 15:19 ` Neil Horman
2018-06-19 15:32 ` Dave Hansen
2018-06-25 9:01 ` Jarkko Sakkinen
2018-06-19 15:59 ` Sean Christopherson
2018-06-25 9:14 ` Jarkko Sakkinen
2018-06-10 5:32 ` [intel-sgx-kernel-dev] " Andy Lutomirski
2018-06-11 15:12 ` Sean Christopherson
2018-06-20 13:21 ` Sean Christopherson
2018-06-25 9:21 ` Jarkko Sakkinen
2018-06-25 16:14 ` Neil Horman
2018-06-20 15:26 ` Sean Christopherson
2018-06-25 9:21 ` Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 10/13] intel_sgx: driver for Intel Software Guard Extensions Jarkko Sakkinen
2018-06-08 19:35 ` Dave Hansen
2018-06-19 13:29 ` Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 11/13] intel_sgx: ptrace() support Jarkko Sakkinen
2018-06-08 18:34 ` Dave Hansen
2018-06-11 15:02 ` Sean Christopherson
2018-06-19 13:38 ` Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 12/13] intel_sgx: driver documentation Jarkko Sakkinen
2018-06-08 18:32 ` Jethro Beekman
2018-06-19 13:30 ` Jarkko Sakkinen
2018-06-08 21:41 ` Randy Dunlap [this message]
2018-06-19 13:31 ` Jarkko Sakkinen
2018-06-08 17:09 ` [PATCH v11 13/13] intel_sgx: in-kernel launch enclave Jarkko Sakkinen
2018-06-08 18:50 ` [intel-sgx-kernel-dev] " Andy Lutomirski
2018-06-19 15:05 ` Jarkko Sakkinen
2018-06-10 5:39 ` Andy Lutomirski
2018-06-11 5:17 ` Andy Lutomirski
2018-06-11 11:52 ` Neil Horman
2018-06-12 4:55 ` Andy Lutomirski
2018-06-12 17:45 ` Neil Horman
2018-06-18 21:58 ` Andy Lutomirski
2018-06-19 13:17 ` Neil Horman
2018-06-20 16:28 ` Nathaniel McCallum
2018-06-20 18:16 ` Jethro Beekman
2018-06-20 18:39 ` Jethro Beekman
2018-06-20 21:01 ` Sean Christopherson
2018-06-21 12:32 ` Nathaniel McCallum
2018-06-21 15:29 ` Neil Horman
2018-06-21 19:11 ` Nathaniel McCallum
2018-06-21 21:20 ` Sean Christopherson
2018-06-25 21:00 ` Nathaniel McCallum
2018-06-25 22:35 ` Sean Christopherson
2018-06-21 22:48 ` Andy Lutomirski
2018-06-25 21:06 ` Nathaniel McCallum
2018-06-25 23:40 ` Andy Lutomirski
2018-06-25 9:41 ` Jarkko Sakkinen
2018-06-25 15:45 ` Andy Lutomirski
2018-06-25 21:28 ` Nathaniel McCallum
2018-06-26 8:43 ` Jarkko Sakkinen
2018-06-26 15:01 ` Nathaniel McCallum
2018-06-27 15:31 ` Jarkko Sakkinen
2018-06-21 12:12 ` Nathaniel McCallum
2018-06-25 9:27 ` Jarkko Sakkinen
2018-06-25 21:26 ` Nathaniel McCallum
2018-06-20 7:23 ` Jarkko Sakkinen
2018-06-12 10:50 ` [PATCH v11 00/13] Intel SGX1 support Pavel Machek
2018-06-19 14:59 ` Jarkko Sakkinen
2018-06-19 20:04 ` Pavel Machek
2018-06-19 20:23 ` Peter Zijlstra
2018-06-19 21:48 ` Josh Triplett
2018-12-09 20:06 ` Pavel Machek
2018-12-10 7:47 ` Josh Triplett
2018-12-10 8:27 ` Pavel Machek
2018-12-10 23:12 ` Josh Triplett
2018-12-11 18:10 ` Dave Hansen
2018-12-11 18:31 ` Sean Christopherson
2018-06-19 20:36 ` Peter Zijlstra
2018-06-21 12:55 ` Ingo Molnar
2018-06-25 9:44 ` Jarkko Sakkinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d88234f6-d9f9-561e-b1ab-4bfaddc3dd72@infradead.org \
--to=rdunlap@infradead.org \
--cc=corbet@lwn.net \
--cc=dave.hansen@intel.com \
--cc=hpa@zytor.com \
--cc=jarkko.sakkinen@linux.intel.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=nhorman@redhat.com \
--cc=npmccallum@redhat.com \
--cc=platform-driver-x86@vger.kernel.org \
--cc=sean.j.christopherson@intel.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).