From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB653C433E0 for ; Thu, 6 Aug 2020 17:39:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2A184206C3 for ; Thu, 6 Aug 2020 17:39:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="F9hnk0aG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729128AbgHFRjc (ORCPT ); Thu, 6 Aug 2020 13:39:32 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:35464 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725535AbgHFRjW (ORCPT ); Thu, 6 Aug 2020 13:39:22 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 076DIMIR007771; Thu, 6 Aug 2020 13:27:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : in-reply-to : references : date : message-id : mime-version : content-type; s=corp-2020-01-29; bh=2BinGB7+lQzYG5K9qR+HHSrCX3Nf4pK8wQY3yuXLDmY=; b=F9hnk0aGIjJ353o+MR+GOr9pY7si5THOUECiy1UEga7fNdxM4MVZkH7qhkdNpcEYYyGl VvAlIKu+7w51hsA/l7qMZSjpOdO2gGRsHwWbF4eVmB/kW52tnm6dgjxSccc7XqIjbJxn xYr4AWgfYH4UGyYeI8twI1kCF3zIwWXKq7OmAVOo/8uqCOg7CD1cCeU2iqMHoOWJTEXk ywYD+epMbJgwJz+/SMLkaVP6g0+wdaJdUrk0pNkWLefiZDHjvS0VuDrqfnAVwQk+eJ81 qIWKl1jfFOSyZJ5NuIKu6o6Fh2jtRUZCiKDphZJR/8kzu4rbbhS6sPJuv+DjwW75hEEZ BA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 32r6ep2wdw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 06 Aug 2020 13:27:36 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 076DJMe5031411; Thu, 6 Aug 2020 13:27:36 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 32qy8ncesp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 06 Aug 2020 13:27:35 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 076DRWl3006056; Thu, 6 Aug 2020 13:27:32 GMT Received: from starbug-mbp.localdomain (/79.97.215.145) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Aug 2020 06:27:32 -0700 Received: by starbug-mbp.localdomain (Postfix, from userid 501) id 0153EF04DBB; Thu, 6 Aug 2020 14:27:26 +0100 (IST) From: Darren Kenny To: Jarkko Sakkinen , x86@kernel.org, linux-sgx@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Sean Christopherson , Jethro Beekman , Serge Ayoun , Jarkko Sakkinen , akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, asapek@google.com, bp@alien8.de, cedric.xing@intel.com, chenalexchen@google.com, conradparker@google.com, cyhanish@google.com, dave.hansen@intel.com, haitao.huang@intel.com, josh@joshtriplett.org, kai.huang@intel.com, kai.svahn@intel.com, kmoy@google.com, ludloff@google.com, luto@kernel.org, nhorman@redhat.com, npmccallum@redhat.com, puiterwijk@redhat.com, rientjes@google.com, tglx@linutronix.de, yaozhangx@google.com Subject: Re: [PATCH v36 08/24] x86/sgx: Initialize metadata for Enclave Page Cache (EPC) sections In-Reply-To: <20200716135303.276442-9-jarkko.sakkinen@linux.intel.com> References: <20200716135303.276442-1-jarkko.sakkinen@linux.intel.com> <20200716135303.276442-9-jarkko.sakkinen@linux.intel.com> Date: Thu, 06 Aug 2020 14:27:26 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9704 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=5 malwarescore=0 spamscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008060095 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9704 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=5 bulkscore=0 malwarescore=0 clxscore=1011 mlxscore=0 priorityscore=1501 adultscore=0 impostorscore=0 lowpriorityscore=0 phishscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008060095 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org On Thursday, 2020-07-16 at 16:52:47 +03, Jarkko Sakkinen wrote: > From: Sean Christopherson > > Enumerate Enclave Page Cache (EPC) sections via CPUID and add the data > structures necessary to track EPC pages so that they can be easily borrowed > for different uses. > > Embed section index to the first eight bits of the EPC page descriptor. > Existing client hardware supports only a single section, while upcoming > server hardware will support at most eight sections. Thus, eight bits > should be enough for long term needs. > > Acked-by: Jethro Beekman > Signed-off-by: Sean Christopherson > Co-developed-by: Serge Ayoun > Signed-off-by: Serge Ayoun > Co-developed-by: Jarkko Sakkinen > Signed-off-by: Jarkko Sakkinen Reviewed-by: Darren Kenny > --- > arch/x86/Kconfig | 17 +++ > arch/x86/kernel/cpu/Makefile | 1 + > arch/x86/kernel/cpu/sgx/Makefile | 2 + > arch/x86/kernel/cpu/sgx/main.c | 216 +++++++++++++++++++++++++++++++ > arch/x86/kernel/cpu/sgx/sgx.h | 52 ++++++++ > 5 files changed, 288 insertions(+) > create mode 100644 arch/x86/kernel/cpu/sgx/Makefile > create mode 100644 arch/x86/kernel/cpu/sgx/main.c > create mode 100644 arch/x86/kernel/cpu/sgx/sgx.h > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > index 883da0abf779..0dea7fdd7a00 100644 > --- a/arch/x86/Kconfig > +++ b/arch/x86/Kconfig > @@ -1926,6 +1926,23 @@ config X86_INTEL_TSX_MODE_AUTO > side channel attacks- equals the tsx=auto command line parameter. > endchoice > > +config INTEL_SGX > + bool "Intel SGX" > + depends on X86_64 && CPU_SUP_INTEL > + depends on CRYPTO=y > + depends on CRYPTO_SHA256=y > + select SRCU > + select MMU_NOTIFIER > + help > + Intel(R) Software Guard eXtensions (SGX) is a set of CPU instructions > + that can be used by applications to set aside private regions of code > + and data, referred to as enclaves. An enclave's private memory can > + only be accessed by code running within the enclave. Accesses from > + outside the enclave, including other enclaves, are disallowed by > + hardware. > + > + If unsure, say N. > + > config EFI > bool "EFI runtime service support" > depends on ACPI > diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile > index dba6a83bc349..b00f801601f3 100644 > --- a/arch/x86/kernel/cpu/Makefile > +++ b/arch/x86/kernel/cpu/Makefile > @@ -49,6 +49,7 @@ obj-$(CONFIG_X86_MCE) += mce/ > obj-$(CONFIG_MTRR) += mtrr/ > obj-$(CONFIG_MICROCODE) += microcode/ > obj-$(CONFIG_X86_CPU_RESCTRL) += resctrl/ > +obj-$(CONFIG_INTEL_SGX) += sgx/ > > obj-$(CONFIG_X86_LOCAL_APIC) += perfctr-watchdog.o > > diff --git a/arch/x86/kernel/cpu/sgx/Makefile b/arch/x86/kernel/cpu/sgx/Makefile > new file mode 100644 > index 000000000000..79510ce01b3b > --- /dev/null > +++ b/arch/x86/kernel/cpu/sgx/Makefile > @@ -0,0 +1,2 @@ > +obj-y += \ > + main.o > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c > new file mode 100644 > index 000000000000..c5831e3db14a > --- /dev/null > +++ b/arch/x86/kernel/cpu/sgx/main.c > @@ -0,0 +1,216 @@ > +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) > +// Copyright(c) 2016-17 Intel Corporation. > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include "encls.h" > + > +struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS]; > +static int sgx_nr_epc_sections; > +static struct task_struct *ksgxswapd_tsk; > + > +static void sgx_sanitize_section(struct sgx_epc_section *section) > +{ > + struct sgx_epc_page *page; > + LIST_HEAD(secs_list); > + int ret; > + > + while (!list_empty(§ion->unsanitized_page_list)) { > + if (kthread_should_stop()) > + return; > + > + spin_lock(§ion->lock); > + > + page = list_first_entry(§ion->unsanitized_page_list, > + struct sgx_epc_page, list); > + > + ret = __eremove(sgx_get_epc_addr(page)); > + if (!ret) > + list_move(&page->list, §ion->page_list); > + else > + list_move_tail(&page->list, &secs_list); > + > + spin_unlock(§ion->lock); > + > + cond_resched(); > + } > +} > + > +static int ksgxswapd(void *p) > +{ > + int i; > + > + set_freezable(); > + > + /* > + * Reset all pages to uninitialized state. Pages could be in initialized > + * on kmemexec. > + */ > + for (i = 0; i < sgx_nr_epc_sections; i++) > + sgx_sanitize_section(&sgx_epc_sections[i]); > + > + /* > + * 2nd round for the SECS pages as they cannot be removed when they > + * still hold child pages. > + */ > + for (i = 0; i < sgx_nr_epc_sections; i++) { > + sgx_sanitize_section(&sgx_epc_sections[i]); > + > + /* Should never happen. */ > + if (!list_empty(&sgx_epc_sections[i].unsanitized_page_list)) > + WARN(1, "EPC section %d has unsanitized pages.\n", i); > + } > + > + return 0; > +} > + > +static bool __init sgx_page_reclaimer_init(void) > +{ > + struct task_struct *tsk; > + > + tsk = kthread_run(ksgxswapd, NULL, "ksgxswapd"); > + if (IS_ERR(tsk)) > + return false; > + > + ksgxswapd_tsk = tsk; > + > + return true; > +} > + > +static void __init sgx_free_epc_section(struct sgx_epc_section *section) > +{ > + struct sgx_epc_page *page; > + > + while (!list_empty(§ion->page_list)) { > + page = list_first_entry(§ion->page_list, > + struct sgx_epc_page, list); > + list_del(&page->list); > + kfree(page); > + } > + > + while (!list_empty(§ion->unsanitized_page_list)) { > + page = list_first_entry(§ion->unsanitized_page_list, > + struct sgx_epc_page, list); > + list_del(&page->list); > + kfree(page); > + } > + > + memunmap(section->va); > +} > + > +static bool __init sgx_setup_epc_section(u64 addr, u64 size, > + unsigned long index, > + struct sgx_epc_section *section) > +{ > + unsigned long nr_pages = size >> PAGE_SHIFT; > + struct sgx_epc_page *page; > + unsigned long i; > + > + section->va = memremap(addr, size, MEMREMAP_WB); > + if (!section->va) > + return false; > + > + section->pa = addr; > + spin_lock_init(§ion->lock); > + INIT_LIST_HEAD(§ion->page_list); > + INIT_LIST_HEAD(§ion->unsanitized_page_list); > + > + for (i = 0; i < nr_pages; i++) { > + page = kzalloc(sizeof(*page), GFP_KERNEL); > + if (!page) > + goto err_out; > + > + page->desc = (addr + (i << PAGE_SHIFT)) | index; > + list_add_tail(&page->list, §ion->unsanitized_page_list); > + } > + > + return true; > + > +err_out: > + sgx_free_epc_section(section); > + return false; > +} > + > +static void __init sgx_page_cache_teardown(void) > +{ > + int i; > + > + for (i = 0; i < sgx_nr_epc_sections; i++) > + sgx_free_epc_section(&sgx_epc_sections[i]); > +} > + > +/** > + * A section metric is concatenated in a way that @low bits 12-31 define the > + * bits 12-31 of the metric and @high bits 0-19 define the bits 32-51 of the > + * metric. > + */ > +static inline u64 __init sgx_calc_section_metric(u64 low, u64 high) > +{ > + return (low & GENMASK_ULL(31, 12)) + > + ((high & GENMASK_ULL(19, 0)) << 32); > +} > + > +static bool __init sgx_page_cache_init(void) > +{ > + u32 eax, ebx, ecx, edx, type; > + u64 pa, size; > + int i; > + > + for (i = 0; i < ARRAY_SIZE(sgx_epc_sections); i++) { > + cpuid_count(SGX_CPUID, i + SGX_CPUID_FIRST_VARIABLE_SUB_LEAF, > + &eax, &ebx, &ecx, &edx); > + > + type = eax & SGX_CPUID_SUB_LEAF_TYPE_MASK; > + if (type == SGX_CPUID_SUB_LEAF_INVALID) > + break; > + > + if (type != SGX_CPUID_SUB_LEAF_EPC_SECTION) { > + pr_err_once("Unknown EPC section type: %u\n", type); > + break; > + } > + > + pa = sgx_calc_section_metric(eax, ebx); > + size = sgx_calc_section_metric(ecx, edx); > + > + pr_info("EPC section 0x%llx-0x%llx\n", pa, pa + size - 1); > + > + if (!sgx_setup_epc_section(pa, size, i, &sgx_epc_sections[i])) { > + pr_err("No free memory for an EPC section\n"); > + break; > + } > + > + sgx_nr_epc_sections++; > + } > + > + if (!sgx_nr_epc_sections) { > + pr_err("There are zero EPC sections.\n"); > + return false; > + } > + > + return true; > +} > + > +static void __init sgx_init(void) > +{ > + if (!boot_cpu_has(X86_FEATURE_SGX)) > + return; > + > + if (!sgx_page_cache_init()) > + return; > + > + if (!sgx_page_reclaimer_init()) > + goto err_page_cache; > + > + return; > + > +err_page_cache: > + sgx_page_cache_teardown(); > +} > + > +device_initcall(sgx_init); > diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h > new file mode 100644 > index 000000000000..dff4f5f16d09 > --- /dev/null > +++ b/arch/x86/kernel/cpu/sgx/sgx.h > @@ -0,0 +1,52 @@ > +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ > +#ifndef _X86_SGX_H > +#define _X86_SGX_H > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include "arch.h" > + > +#undef pr_fmt > +#define pr_fmt(fmt) "sgx: " fmt > + > +struct sgx_epc_page { > + unsigned long desc; > + struct list_head list; > +}; > + > +/* > + * The firmware can define multiple chunks of EPC to the different areas of the > + * physical memory e.g. for memory areas of the each node. This structure is > + * used to store EPC pages for one EPC section and virtual memory area where > + * the pages have been mapped. > + */ > +struct sgx_epc_section { > + unsigned long pa; > + void *va; > + struct list_head page_list; > + struct list_head unsanitized_page_list; > + spinlock_t lock; > +}; > + > +#define SGX_EPC_SECTION_MASK GENMASK(7, 0) > +#define SGX_MAX_EPC_SECTIONS (SGX_EPC_SECTION_MASK + 1) > + > +extern struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS]; > + > +static inline struct sgx_epc_section *sgx_get_epc_section(struct sgx_epc_page *page) > +{ > + return &sgx_epc_sections[page->desc & SGX_EPC_SECTION_MASK]; > +} > + > +static inline void *sgx_get_epc_addr(struct sgx_epc_page *page) > +{ > + struct sgx_epc_section *section = sgx_get_epc_section(page); > + > + return section->va + (page->desc & PAGE_MASK) - section->pa; > +} > + > +#endif /* _X86_SGX_H */ > -- > 2.25.1