From: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-sgx@vger.kernel.org Cc: akpm@linux-foundation.org, dave.hansen@intel.com, sean.j.christopherson@intel.com, nhorman@redhat.com, npmccallum@redhat.com, haitao.huang@intel.com, andriy.shevchenko@linux.intel.com, tglx@linutronix.de, kai.svahn@intel.com, bp@alien8.de, josh@joshtriplett.org, luto@kernel.org, kai.huang@intel.com, rientjes@google.com, cedric.xing@intel.com, puiterwijk@redhat.com, Serge Ayoun <serge.ayoun@intel.com>, Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> Subject: [PATCH v28 08/22] x86/sgx: Enumerate and track EPC sections Date: Wed, 4 Mar 2020 01:35:55 +0200 Message-ID: <20200303233609.713348-9-jarkko.sakkinen@linux.intel.com> (raw) In-Reply-To: <20200303233609.713348-1-jarkko.sakkinen@linux.intel.com> From: Sean Christopherson <sean.j.christopherson@intel.com> Enumerate Enclave Page Cache (EPC) sections via CPUID and add the data structures necessary to track EPC pages so that they can be allocated, freed and managed. As a system may have multiple EPC sections, invoke CPUID on SGX sub-leafs until an invalid leaf is encountered. For simplicity, support a maximum of eight EPC sections. Existing client hardware supports only a single section, while upcoming server hardware will support at most eight sections. Bounding the number of sections also allows the section ID to be embedded along with a page's offset in a single unsigned long, enabling easy retrieval of both the VA and PA for a given page. Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com> Co-developed-by: Serge Ayoun <serge.ayoun@intel.com> Signed-off-by: Serge Ayoun <serge.ayoun@intel.com> Co-developed-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> --- arch/x86/Kconfig | 14 +++ arch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/sgx/Makefile | 3 + arch/x86/kernel/cpu/sgx/main.c | 151 ++++++++++++++++++++++++++++++ arch/x86/kernel/cpu/sgx/reclaim.c | 82 ++++++++++++++++ arch/x86/kernel/cpu/sgx/sgx.h | 70 ++++++++++++++ 6 files changed, 321 insertions(+) create mode 100644 arch/x86/kernel/cpu/sgx/Makefile create mode 100644 arch/x86/kernel/cpu/sgx/main.c create mode 100644 arch/x86/kernel/cpu/sgx/reclaim.c create mode 100644 arch/x86/kernel/cpu/sgx/sgx.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index beea77046f9b..8bbb4313fae3 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1950,6 +1950,20 @@ config X86_INTEL_TSX_MODE_AUTO side channel attacks- equals the tsx=auto command line parameter. endchoice +config INTEL_SGX + bool "Intel SGX" + depends on X86_64 && CPU_SUP_INTEL + select SRCU + select MMU_NOTIFIER + help + Intel(R) SGX is a set of CPU instructions that can be used by + applications to set aside private regions of code and data, referred + to as enclaves. An enclave's private memory can only be accessed by + code running within the enclave. Accesses from outside the enclave, + including other enclaves, are disallowed by hardware. + + If unsure, say N. + config EFI bool "EFI runtime service support" depends on ACPI diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile index 7dc4ad68eb41..45534fb81007 100644 --- a/arch/x86/kernel/cpu/Makefile +++ b/arch/x86/kernel/cpu/Makefile @@ -46,6 +46,7 @@ obj-$(CONFIG_X86_MCE) += mce/ obj-$(CONFIG_MTRR) += mtrr/ obj-$(CONFIG_MICROCODE) += microcode/ obj-$(CONFIG_X86_CPU_RESCTRL) += resctrl/ +obj-$(CONFIG_INTEL_SGX) += sgx/ obj-$(CONFIG_X86_LOCAL_APIC) += perfctr-watchdog.o diff --git a/arch/x86/kernel/cpu/sgx/Makefile b/arch/x86/kernel/cpu/sgx/Makefile new file mode 100644 index 000000000000..2dec75916a5e --- /dev/null +++ b/arch/x86/kernel/cpu/sgx/Makefile @@ -0,0 +1,3 @@ +obj-y += \ + main.o \ + reclaim.o diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c new file mode 100644 index 000000000000..38424c1e8341 --- /dev/null +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) +// Copyright(c) 2016-17 Intel Corporation. + +#include <linux/freezer.h> +#include <linux/highmem.h> +#include <linux/kthread.h> +#include <linux/pagemap.h> +#include <linux/ratelimit.h> +#include <linux/sched/signal.h> +#include <linux/slab.h> +#include "encls.h" + +struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS]; +int sgx_nr_epc_sections; + +static void __init sgx_free_epc_section(struct sgx_epc_section *section) +{ + struct sgx_epc_page *page; + + while (!list_empty(§ion->page_list)) { + page = list_first_entry(§ion->page_list, + struct sgx_epc_page, list); + list_del(&page->list); + kfree(page); + } + + while (!list_empty(§ion->unsanitized_page_list)) { + page = list_first_entry(§ion->unsanitized_page_list, + struct sgx_epc_page, list); + list_del(&page->list); + kfree(page); + } + + memunmap(section->va); +} + +static bool __init sgx_alloc_epc_section(u64 addr, u64 size, + unsigned long index, + struct sgx_epc_section *section) +{ + unsigned long nr_pages = size >> PAGE_SHIFT; + struct sgx_epc_page *page; + unsigned long i; + + section->va = memremap(addr, size, MEMREMAP_WB); + if (!section->va) + return false; + + section->pa = addr; + spin_lock_init(§ion->lock); + INIT_LIST_HEAD(§ion->page_list); + INIT_LIST_HEAD(§ion->unsanitized_page_list); + + for (i = 0; i < nr_pages; i++) { + page = kzalloc(sizeof(*page), GFP_KERNEL); + if (!page) + goto err_out; + + page->desc = (addr + (i << PAGE_SHIFT)) | index; + list_add_tail(&page->list, §ion->unsanitized_page_list); + } + + return true; + +err_out: + sgx_free_epc_section(section); + return false; +} + +static void __init sgx_page_cache_teardown(void) +{ + int i; + + for (i = 0; i < sgx_nr_epc_sections; i++) + sgx_free_epc_section(&sgx_epc_sections[i]); +} + +/** + * A section metric is concatenated in a way that @low bits 12-31 define the + * bits 12-31 of the metric and @high bits 0-19 define the bits 32-51 of the + * metric. + */ +static inline u64 __init sgx_calc_section_metric(u64 low, u64 high) +{ + return (low & GENMASK_ULL(31, 12)) + + ((high & GENMASK_ULL(19, 0)) << 32); +} + +static bool __init sgx_page_cache_init(void) +{ + u32 eax, ebx, ecx, edx, type; + u64 pa, size; + int i; + + for (i = 0; i <= ARRAY_SIZE(sgx_epc_sections); i++) { + cpuid_count(SGX_CPUID, i + SGX_CPUID_FIRST_VARIABLE_SUB_LEAF, + &eax, &ebx, &ecx, &edx); + + type = eax & SGX_CPUID_SUB_LEAF_TYPE_MASK; + if (type == SGX_CPUID_SUB_LEAF_INVALID) + break; + + if (type != SGX_CPUID_SUB_LEAF_EPC_SECTION) { + pr_err_once("Unknown EPC section type: %u\n", type); + break; + } + + if (i == ARRAY_SIZE(sgx_epc_sections)) { + pr_warn("No free slot for an EPC section\n"); + break; + } + + pa = sgx_calc_section_metric(eax, ebx); + size = sgx_calc_section_metric(ecx, edx); + + pr_info("EPC section 0x%llx-0x%llx\n", pa, pa + size - 1); + + if (!sgx_alloc_epc_section(pa, size, i, &sgx_epc_sections[i])) { + pr_err("No free memory for an EPC section\n"); + break; + } + + sgx_nr_epc_sections++; + } + + if (!sgx_nr_epc_sections) { + pr_err("There are zero EPC sections.\n"); + return false; + } + + return true; +} + +static void __init sgx_init(void) +{ + if (!boot_cpu_has(X86_FEATURE_SGX)) + return; + + if (!sgx_page_cache_init()) + return; + + if (!sgx_page_reclaimer_init()) + goto err_page_cache; + + return; + +err_page_cache: + sgx_page_cache_teardown(); +} + +arch_initcall(sgx_init); diff --git a/arch/x86/kernel/cpu/sgx/reclaim.c b/arch/x86/kernel/cpu/sgx/reclaim.c new file mode 100644 index 000000000000..215371588a25 --- /dev/null +++ b/arch/x86/kernel/cpu/sgx/reclaim.c @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) +// Copyright(c) 2016-19 Intel Corporation. + +#include <linux/freezer.h> +#include <linux/highmem.h> +#include <linux/kthread.h> +#include <linux/pagemap.h> +#include <linux/ratelimit.h> +#include <linux/slab.h> +#include <linux/sched/mm.h> +#include <linux/sched/signal.h> +#include "encls.h" + +struct task_struct *ksgxswapd_tsk; + +static void sgx_sanitize_section(struct sgx_epc_section *section) +{ + struct sgx_epc_page *page; + LIST_HEAD(secs_list); + int ret; + + while (!list_empty(§ion->unsanitized_page_list)) { + if (kthread_should_stop()) + return; + + spin_lock(§ion->lock); + + page = list_first_entry(§ion->unsanitized_page_list, + struct sgx_epc_page, list); + + ret = __eremove(sgx_epc_addr(page)); + if (!ret) + list_move(&page->list, §ion->page_list); + else + list_move_tail(&page->list, &secs_list); + + spin_unlock(§ion->lock); + + cond_resched(); + } +} + +static int ksgxswapd(void *p) +{ + int i; + + set_freezable(); + + /* + * Reset all pages to uninitialized state. Pages could be in initialized + * on kmemexec. + */ + for (i = 0; i < sgx_nr_epc_sections; i++) + sgx_sanitize_section(&sgx_epc_sections[i]); + + /* + * 2nd round for the SECS pages as they cannot be removed when they + * still hold child pages. + */ + for (i = 0; i < sgx_nr_epc_sections; i++) { + sgx_sanitize_section(&sgx_epc_sections[i]); + + /* Should never happen. */ + if (!list_empty(&sgx_epc_sections[i].unsanitized_page_list)) + WARN(1, "EPC section %d has unsanitized pages.\n", i); + } + + return 0; +} + +bool __init sgx_page_reclaimer_init(void) +{ + struct task_struct *tsk; + + tsk = kthread_run(ksgxswapd, NULL, "ksgxswapd"); + if (IS_ERR(tsk)) + return false; + + ksgxswapd_tsk = tsk; + + return true; +} diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h new file mode 100644 index 000000000000..aad30980be32 --- /dev/null +++ b/arch/x86/kernel/cpu/sgx/sgx.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ +#ifndef _X86_SGX_H +#define _X86_SGX_H + +#include <linux/bitops.h> +#include <linux/err.h> +#include <linux/io.h> +#include <linux/rwsem.h> +#include <linux/types.h> +#include <asm/asm.h> +#include "arch.h" + +#undef pr_fmt +#define pr_fmt(fmt) "sgx: " fmt + +struct sgx_epc_page { + unsigned long desc; + struct list_head list; +}; + +/** + * struct sgx_epc_section + * + * The firmware can define multiple chunks of EPC to the different areas of the + * physical memory e.g. for memory areas of the each node. This structure is + * used to store EPC pages for one EPC section and virtual memory area where + * the pages have been mapped. + */ +struct sgx_epc_section { + unsigned long pa; + void *va; + struct list_head page_list; + struct list_head unsanitized_page_list; + spinlock_t lock; +}; + +/** + * enum sgx_epc_page_desc - bits and masks for an EPC page's descriptor + * %SGX_EPC_SECTION_MASK: SGX allows to have multiple EPC sections in the + * physical memory. The existing and near-future + * hardware defines at most eight sections, hence + * three bits to hold a section. + */ +enum sgx_epc_page_desc { + SGX_EPC_SECTION_MASK = GENMASK_ULL(3, 0), + /* bits 12-63 are reserved for the physical page address of the page */ +}; + +#define SGX_MAX_EPC_SECTIONS (SGX_EPC_SECTION_MASK + 1) + +extern struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS]; + +static inline struct sgx_epc_section *sgx_epc_section(struct sgx_epc_page *page) +{ + return &sgx_epc_sections[page->desc & SGX_EPC_SECTION_MASK]; +} + +static inline void *sgx_epc_addr(struct sgx_epc_page *page) +{ + struct sgx_epc_section *section = sgx_epc_section(page); + + return section->va + (page->desc & PAGE_MASK) - section->pa; +} + +extern int sgx_nr_epc_sections; +extern struct task_struct *ksgxswapd_tsk; + +bool __init sgx_page_reclaimer_init(void); + +#endif /* _X86_SGX_H */ -- 2.25.0
next prev parent reply index Thread overview: 109+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-03-03 23:35 [PATCH v28 00/22] Intel SGX foundations Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 01/22] x86/sgx: Update MAINTAINERS Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 02/22] x86/cpufeatures: x86/msr: Add Intel SGX hardware bits Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 03/22] x86/cpufeatures: x86/msr: Intel SGX Launch Control " Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 04/22] x86/mm: x86/sgx: Signal SIGSEGV with PF_SGX Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 05/22] x86/sgx: Add SGX microarchitectural data structures Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 06/22] x86/sgx: Add wrappers for ENCLS leaf functions Jarkko Sakkinen 2020-03-09 21:14 ` Sean Christopherson 2020-03-03 23:35 ` [PATCH v28 07/22] x86/cpu/intel: Detect SGX supprt Jarkko Sakkinen 2020-03-09 21:56 ` Sean Christopherson 2020-03-11 17:03 ` Jarkko Sakkinen 2020-03-03 23:35 ` Jarkko Sakkinen [this message] 2020-03-03 23:35 ` [PATCH v28 09/22] x86/sgx: Add functions to allocate and free EPC pages Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 10/22] mm: Introduce vm_ops->may_mprotect() Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 11/22] x86/sgx: Linux Enclave Driver Jarkko Sakkinen 2020-03-05 17:40 ` Sean Christopherson 2020-03-05 18:24 ` Jethro Beekman 2020-03-05 19:04 ` Sean Christopherson 2020-03-06 19:00 ` Jarkko Sakkinen 2020-03-19 18:22 ` Dr. Greg 2020-03-06 18:58 ` Jarkko Sakkinen 2020-03-03 23:35 ` [PATCH v28 12/22] docs: x86/sgx: Document SGX micro architecture and kernel internals Jarkko Sakkinen 2020-03-03 23:36 ` [PATCH v28 13/22] selftests/x86: Recurse into subdirectories Jarkko Sakkinen 2020-03-03 23:36 ` [PATCH v28 14/22] selftests/x86: Add a selftest for SGX Jarkko Sakkinen 2020-03-04 19:27 ` Nathaniel McCallum 2020-03-05 11:33 ` Jarkko Sakkinen 2020-03-06 15:42 ` Dr. Greg 2020-03-06 19:07 ` Jarkko Sakkinen 2020-03-07 17:42 ` Dr. Greg 2020-03-10 13:08 ` Jarkko Sakkinen 2020-03-11 13:28 ` Jarkko Sakkinen [not found] ` <20200311164047.GG21852@linux.intel.com> 2020-03-13 19:24 ` Jarkko Sakkinen 2020-03-04 19:44 ` Nathaniel McCallum 2020-03-04 19:51 ` Nathaniel McCallum 2020-03-06 5:32 ` Dr. Greg 2020-03-06 19:04 ` Jarkko Sakkinen 2020-03-10 19:29 ` Haitao Huang 2020-03-11 9:13 ` Dr. Greg 2020-03-11 17:15 ` Haitao Huang 2020-03-17 1:07 ` Dr. Greg 2020-03-03 23:36 ` [PATCH v28 15/22] x86/sgx: Add provisioning Jarkko Sakkinen 2020-03-03 23:36 ` [PATCH v28 16/22] x86/sgx: Add a page reclaimer Jarkko Sakkinen 2020-03-05 19:03 ` Sean Christopherson 2020-03-06 18:47 ` Jarkko Sakkinen 2020-03-12 18:38 ` Sean Christopherson 2020-03-15 0:27 ` Jarkko Sakkinen 2020-03-15 1:17 ` Jarkko Sakkinen 2020-03-09 21:16 ` Sean Christopherson 2020-03-03 23:36 ` [PATCH v28 17/22] x86/sgx: ptrace() support for the SGX driver Jarkko Sakkinen 2020-03-03 23:36 ` [PATCH v28 18/22] x86/vdso: Add support for exception fixup in vDSO functions Jarkko Sakkinen 2020-03-03 23:36 ` [PATCH v28 19/22] x86/fault: Add helper function to sanitize error code Jarkko Sakkinen 2020-03-03 23:36 ` [PATCH v28 20/22] x86/traps: Attempt to fixup exceptions in vDSO before signaling Jarkko Sakkinen 2020-03-03 23:36 ` [PATCH v28 21/22] x86/vdso: Implement a vDSO for Intel SGX enclave call Jarkko Sakkinen 2020-03-11 17:30 ` Nathaniel McCallum 2020-03-11 17:38 ` Jethro Beekman 2020-03-11 19:15 ` Nathaniel McCallum 2020-03-13 15:48 ` Nathaniel McCallum 2020-03-13 16:46 ` Sean Christopherson 2020-03-13 18:32 ` Nathaniel McCallum 2020-03-13 18:44 ` Sean Christopherson 2020-03-13 20:14 ` Nathaniel McCallum 2020-03-13 22:08 ` Sean Christopherson 2020-03-14 14:10 ` Nathaniel McCallum 2020-03-18 23:40 ` Sean Christopherson 2020-03-19 0:38 ` Xing, Cedric 2020-03-19 1:03 ` Sean Christopherson 2020-03-20 13:55 ` Nathaniel McCallum 2020-03-15 1:25 ` Jarkko Sakkinen 2020-03-15 17:53 ` Nathaniel McCallum 2020-03-16 13:31 ` Jethro Beekman 2020-03-16 13:57 ` Nathaniel McCallum 2020-03-16 13:59 ` Jethro Beekman 2020-03-16 14:03 ` Nathaniel McCallum 2020-03-16 17:17 ` Sean Christopherson 2020-03-16 21:27 ` Jarkko Sakkinen 2020-03-16 21:29 ` Jarkko Sakkinen 2020-03-16 22:55 ` Sean Christopherson 2020-03-16 23:56 ` Xing, Cedric 2020-03-18 22:01 ` Jarkko Sakkinen 2020-03-18 22:18 ` Jarkko Sakkinen 2020-03-16 13:56 ` Jarkko Sakkinen 2020-03-16 14:01 ` Nathaniel McCallum 2020-03-16 21:38 ` Jarkko Sakkinen 2020-03-16 22:53 ` Sean Christopherson 2020-03-16 23:50 ` Xing, Cedric 2020-03-16 23:59 ` Sean Christopherson 2020-03-17 0:18 ` Xing, Cedric 2020-03-17 0:27 ` Sean Christopherson 2020-03-17 16:37 ` Nathaniel McCallum 2020-03-17 16:50 ` Nathaniel McCallum 2020-03-17 21:40 ` Xing, Cedric 2020-03-17 22:09 ` Sean Christopherson 2020-03-17 22:36 ` Xing, Cedric 2020-03-17 23:57 ` Sean Christopherson 2020-03-17 22:23 ` Xing, Cedric 2020-03-18 13:01 ` Nathaniel McCallum 2020-03-20 15:53 ` Nathaniel McCallum 2020-03-17 16:28 ` Nathaniel McCallum 2020-03-18 22:58 ` Jarkko Sakkinen 2020-03-18 22:39 ` Jarkko Sakkinen 2020-03-11 19:30 ` Nathaniel McCallum 2020-03-13 0:52 ` Sean Christopherson 2020-03-13 16:07 ` Nathaniel McCallum 2020-03-13 16:33 ` Sean Christopherson 2020-03-03 23:36 ` [PATCH v28 22/22] selftests/x86: Add vDSO selftest for SGX Jarkko Sakkinen 2020-03-04 19:24 ` [PATCH v28 00/22] Intel SGX foundations Nathaniel McCallum 2020-03-17 16:00 ` Jordan Hand 2020-03-18 21:56 ` Jarkko Sakkinen 2020-03-19 17:16 ` Dr. Greg
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200303233609.713348-9-jarkko.sakkinen@linux.intel.com \ --to=jarkko.sakkinen@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=andriy.shevchenko@linux.intel.com \ --cc=bp@alien8.de \ --cc=cedric.xing@intel.com \ --cc=dave.hansen@intel.com \ --cc=haitao.huang@intel.com \ --cc=josh@joshtriplett.org \ --cc=kai.huang@intel.com \ --cc=kai.svahn@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-sgx@vger.kernel.org \ --cc=luto@kernel.org \ --cc=nhorman@redhat.com \ --cc=npmccallum@redhat.com \ --cc=puiterwijk@redhat.com \ --cc=rientjes@google.com \ --cc=sean.j.christopherson@intel.com \ --cc=serge.ayoun@intel.com \ --cc=tglx@linutronix.de \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-Sgx Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-sgx/0 linux-sgx/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-sgx linux-sgx/ https://lore.kernel.org/linux-sgx \ linux-sgx@vger.kernel.org public-inbox-index linux-sgx Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-sgx AGPL code for this site: git clone https://public-inbox.org/public-inbox.git