Linux-Sgx Archive on lore.kernel.org
 help / color / Atom feed
From: Randy Dunlap <rdunlap@infradead.org>
To: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
	linux-kernel@vger.kernel.org, x86@kernel.org,
	linux-sgx@vger.kernel.org
Cc: akpm@linux-foundation.org, dave.hansen@intel.com,
	sean.j.christopherson@intel.com, nhorman@redhat.com,
	npmccallum@redhat.com, serge.ayoun@intel.com,
	shay.katz-zamir@intel.com, haitao.huang@intel.com,
	andriy.shevchenko@linux.intel.com, tglx@linutronix.de,
	kai.svahn@intel.com, bp@alien8.de, josh@joshtriplett.org,
	luto@kernel.org, kai.huang@intel.com, rientjes@google.com,
	cedric.xing@intel.com
Subject: Re: [PATCH v22 24/24] docs: x86/sgx: Document kernel internals
Date: Fri, 27 Sep 2019 10:07:10 -0700
Message-ID: <97dbd2a8-f1e5-a3f9-dac7-f1f4d6b6cd4c@infradead.org> (raw)
In-Reply-To: <20190903142655.21943-25-jarkko.sakkinen@linux.intel.com>

On 9/3/19 7:26 AM, Jarkko Sakkinen wrote:
> From: Sean Christopherson <sean.j.christopherson@intel.com>
> 
> Document some of the more tricky parts of the kernel implementation
> internals.
> 
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> Co-developed-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
> Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>

Hi,
Some edits for you to consider.

> ---
>  Documentation/x86/sgx/2.Kernel-internals.rst | 76 ++++++++++++++++++++
>  Documentation/x86/sgx/index.rst              |  1 +
>  2 files changed, 77 insertions(+)
>  create mode 100644 Documentation/x86/sgx/2.Kernel-internals.rst
> 
> diff --git a/Documentation/x86/sgx/2.Kernel-internals.rst b/Documentation/x86/sgx/2.Kernel-internals.rst
> new file mode 100644
> index 000000000000..5c90a65936f2
> --- /dev/null
> +++ b/Documentation/x86/sgx/2.Kernel-internals.rst
> @@ -0,0 +1,76 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +================
> +Kernel Internals
> +================
> +
> +CPU configuration
> +=================
> +
> +Because SGX has an ever evolving and expanding feature set, it's possible for
> +a BIOS or VMM to configure a system in such a way that not all CPUs are equal,
> +e.g. where Launch Control is only enabled on a subset of CPUs.  Linux does
> +*not* support such a heterogeneous system configuration, nor does it even
> +attempt to play nice in the face of a misconfigured system.  With the exception
> +of Launch Control's hash MSRs, which can vary per CPU, Linux assumes that all
> +CPUs have a configuration that is identical to the boot CPU.
> +
> +EPC management
> +==============
> +
> +Because the kernel can't arbitrarily read EPC memory or share RO backing pages
> +between enclaves, traditional memory models such as CoW and fork() do not work
> +with enclaves.  In other words, the architectural rules of EPC forces it to be

                                                                  force

> +treated as MAP_SHARED at all times.
> +
> +The inability to employ traditional memory models also means that EPC memory
> +must be isolated from normal memory pools, e.g. attempting to use EPC memory
> +for normal mappings would result in faults and/or perceived data corruption.
> +Furthermore, EPC is not enumerated by as normal memory, e.g. BIOS enumerates

                           enumerated as

> +EPC as reserved memory in the e820 tables, or not at all.  As a result, EPC
> +memory is directly managed by the SGX subsystem, e.g. SGX employs VM_PFNMAP to
> +manually insert/zap/swap page table entries, and exposes EPC to userspace via
> +a well known device, /dev/sgx/enclave.
> +
> +The net effect is that all enclave VMAs must be MAP_SHARED and are backed by
> +a single file, /dev/sgx/enclave.
> +
> +EPC oversubscription
> +====================
> +
> +SGX allows to have larger enclaves than amount of available EPC by providing a

                                      than the amount of

> +subset of leaf instruction for swapping EPC pages to the system memory.  The

                  instructions  {I think}

> +details of these instructions are discussed in the architecture document. Due
> +to the unique requirements for swapping EPC pages, and because EPC pages do not
> +have associated page structures, management of the EPC is not handled by the
> +standard memory subsystem.
> +
> +SGX directly handles swapping of EPC pages, including a thread to initiate the
> +reclaiming process and a rudimentary LRU mechanism. When the amount of free EPC
> +pages goes below a low watermark the swapping thread starts reclaiming pages.
> +The pages that have not been recently accessed (i.e. do not have the A bit set)
> +are selected as victim pages. Each enclave holds an shmem file as a backing
> +storage for reclaimed pages.
> +
> +Launch Control
> +==============
> +
> +The current kernel implementation supports only writable MSRs. The launch is
> +performed by setting the MSRs to the hash of the public key modulus of the
> +enclave signer and a token with the valid bit set to zero. Because kernel makes

                                                              Because the kernel

> +ultimately all the launch decisions token are not needed for anything.  We

   ultimately makes all the launch decisions, tokens are not


> +don't need or have a launch enclave for generating them as the MSRs must always
> +be writable.
> +
> +Provisioning
> +============
> +
> +The use of provisioning must be controlled because it allows to get access to
> +the provisioning keys to attest to a remote party that the software is running
> +inside a legit enclave. This could be used by a malware network to ensure that

            legitimate

> +its nodes are running inside legit enclaves.

                                legitimate

> +
> +The driver introduces a special device file /dev/sgx/provision and a special
> +ioctl SGX_IOC_ENCLAVE_SET_ATTRIBUTE to accomplish this. A file descriptor
> +pointing to /dev/sgx/provision is passed to ioctl from which kernel authorizes
> +the PROVISION_KEY attribute to the enclave.



-- 
~Randy

  reply index

Thread overview: 102+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-03 14:26 [PATCH v22 00/24] Intel SGX foundations Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 01/24] x86/cpufeatures: x86/msr: Add Intel SGX hardware bits Jarkko Sakkinen
2019-09-24 15:28   ` Borislav Petkov
2019-09-24 16:11     ` Sean Christopherson
2019-09-24 16:25       ` Borislav Petkov
2019-09-03 14:26 ` [PATCH v22 02/24] x86/cpufeatures: x86/msr: Intel SGX Launch Control " Jarkko Sakkinen
2019-09-24 15:52   ` Borislav Petkov
2019-09-24 20:22     ` Sean Christopherson
2019-09-25  8:51       ` Borislav Petkov
2019-09-25 17:18         ` Sean Christopherson
2019-09-25 18:31           ` Borislav Petkov
2019-09-25 19:08             ` Sean Christopherson
2019-09-27 16:11           ` Jarkko Sakkinen
2019-09-25 14:09     ` Jarkko Sakkinen
2019-09-25 14:10       ` Jarkko Sakkinen
2019-09-25 14:38         ` Jarkko Sakkinen
2019-09-25 15:19       ` Borislav Petkov
2019-09-25 16:49         ` Sean Christopherson
2019-09-25 17:28           ` Borislav Petkov
2019-09-25 18:18             ` Sean Christopherson
2019-09-03 14:26 ` [PATCH v22 03/24] x86/mm: x86/sgx: Signal SIGSEGV with PF_SGX Jarkko Sakkinen
2019-09-24 16:04   ` Borislav Petkov
2019-09-25 14:16     ` Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 04/24] x86/cpu/intel: Detect SGX supprt Jarkko Sakkinen
2019-09-24 16:13   ` Borislav Petkov
2019-09-24 17:43     ` Sean Christopherson
2019-09-24 18:21       ` Borislav Petkov
2019-09-25 14:46         ` Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 05/24] x86/sgx: Add ENCLS architectural error codes Jarkko Sakkinen
2019-09-27 10:20   ` Borislav Petkov
2019-09-27 16:08     ` Jarkko Sakkinen
2019-09-27 17:20       ` Sean Christopherson
2019-10-01 20:23         ` Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 06/24] x86/sgx: Add SGX microarchitectural data structures Jarkko Sakkinen
2019-09-27 16:27   ` Borislav Petkov
2019-10-01 19:10     ` Jarkko Sakkinen
2019-10-01 20:39     ` Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 07/24] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen
2019-10-04  9:45   ` Borislav Petkov
2019-10-04 18:56     ` Jarkko Sakkinen
2019-10-08  4:04     ` Sean Christopherson
2019-10-08  7:18       ` Borislav Petkov
2019-10-08 13:35         ` Sean Christopherson
2019-10-08 14:56           ` Borislav Petkov
2019-09-03 14:26 ` [PATCH v22 08/24] x86/sgx: Enumerate and track EPC sections Jarkko Sakkinen
2019-10-05  9:26   ` Borislav Petkov
2019-10-07 11:58     ` Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 09/24] x86/sgx: Add functions to allocate and free EPC pages Jarkko Sakkinen
2019-10-05 16:44   ` Borislav Petkov
2019-10-07 14:50     ` Sean Christopherson
2019-10-08  9:09       ` Borislav Petkov
2019-10-08 13:31         ` Sean Christopherson
2019-10-07 17:55     ` Jarkko Sakkinen
2019-10-07 18:09       ` Borislav Petkov
2019-09-03 14:26 ` [PATCH v22 10/24] x86/sgx: Add sgx_einit() for wrapping ENCLS[EINIT] Jarkko Sakkinen
2019-10-08 17:30   ` Borislav Petkov
2019-10-08 17:45     ` Sean Christopherson
2019-10-08 17:46       ` Sean Christopherson
2019-10-08 17:53         ` Borislav Petkov
2019-09-03 14:26 ` [PATCH v22 11/24] mm: Introduce vm_ops->may_mprotect() Jarkko Sakkinen
2019-10-08 17:41   ` Borislav Petkov
2019-09-03 14:26 ` [PATCH v22 12/24] x86/sgx: Linux Enclave Driver Jarkko Sakkinen
2019-10-08 17:59   ` Borislav Petkov
2019-10-08 18:17     ` Sean Christopherson
2019-10-08 19:19       ` Borislav Petkov
2019-09-03 14:26 ` [PATCH v22 13/24] x86/sgx: Add provisioning Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 14/24] x86/sgx: Add a page reclaimer Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 15/24] x86/sgx: ptrace() support for the SGX driver Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 16/24] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen
2019-10-02 23:18   ` Jarkko Sakkinen
2019-10-02 23:45     ` Jarkko Sakkinen
2019-10-04  0:03     ` Sean Christopherson
2019-10-04 18:49       ` Jarkko Sakkinen
2019-10-04  0:15     ` Sean Christopherson
2019-10-04 18:52       ` Jarkko Sakkinen
2019-10-05 15:54         ` Sean Christopherson
2019-10-07  7:57           ` Jarkko Sakkinen
2019-10-07  8:10             ` Jarkko Sakkinen
2019-10-07 12:04               ` Jarkko Sakkinen
2019-10-08  4:54                 ` Sean Christopherson
2019-10-05 18:39         ` Sean Christopherson
2019-10-07  8:01           ` Jarkko Sakkinen
2019-10-06 23:38         ` Jarkko Sakkinen
2019-10-06 23:40           ` Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 17/24] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 18/24] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 19/24] x86/vdso: Add __vdso_sgx_enter_enclave() to wrap SGX enclave transitions Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 20/24] selftests/x86: Add a selftest for SGX Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 21/24] selftests/x86: Recurse into subdirectories Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 22/24] x86/sgx: Update MAINTAINERS Jarkko Sakkinen
2019-09-03 14:26 ` [PATCH v22 23/24] docs: x86/sgx: Document microarchitecture Jarkko Sakkinen
2019-09-27 18:15   ` Randy Dunlap
2019-09-03 14:26 ` [PATCH v22 24/24] docs: x86/sgx: Document kernel internals Jarkko Sakkinen
2019-09-27 17:07   ` Randy Dunlap [this message]
2019-10-01 19:34     ` Jarkko Sakkinen
2019-09-13 20:38 ` [PATCH v22 00/24] Intel SGX foundations Dave Hansen
2019-09-14 13:41   ` Jarkko Sakkinen
2019-09-14 15:32     ` Dave Hansen
2019-09-16  5:23       ` Jarkko Sakkinen
2019-09-24 17:20         ` Andy Lutomirski
2019-09-25 14:32           ` Jarkko Sakkinen
2019-10-02 23:42             ` Jarkko Sakkinen

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=97dbd2a8-f1e5-a3f9-dac7-f1f4d6b6cd4c@infradead.org \
    --to=rdunlap@infradead.org \
    --cc=akpm@linux-foundation.org \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=bp@alien8.de \
    --cc=cedric.xing@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=haitao.huang@intel.com \
    --cc=jarkko.sakkinen@linux.intel.com \
    --cc=josh@joshtriplett.org \
    --cc=kai.huang@intel.com \
    --cc=kai.svahn@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-sgx@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=nhorman@redhat.com \
    --cc=npmccallum@redhat.com \
    --cc=rientjes@google.com \
    --cc=sean.j.christopherson@intel.com \
    --cc=serge.ayoun@intel.com \
    --cc=shay.katz-zamir@intel.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Sgx Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-sgx/0 linux-sgx/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-sgx linux-sgx/ https://lore.kernel.org/linux-sgx \
		linux-sgx@vger.kernel.org
	public-inbox-index linux-sgx

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-sgx


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git