From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D691DC4742C for ; Tue, 3 Nov 2020 06:50:33 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1709C22277 for ; Tue, 3 Nov 2020 06:50:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="EsTSPVC9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1709C22277 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 8A4831631FAD1; Mon, 2 Nov 2020 22:50:32 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=198.145.29.99; helo=mail.kernel.org; envelope-from=gregkh@linuxfoundation.org; receiver= Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 3132015CEE860 for ; Mon, 2 Nov 2020 22:50:30 -0800 (PST) Received: from localhost (83-86-74-64.cable.dynamic.v4.ziggo.nl [83.86.74.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0DB9822277; Tue, 3 Nov 2020 06:50:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604386229; bh=MAxrnbt5VIvXOwsvpicABX0MkmmlreTcaHNcZhiy7CA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EsTSPVC9j10ih+q2EMTVwZOtdEAKNetrRW9ikYC4jlFJLRmZmVAyccU2hmWR1dfRo xKq0jVFsgjWZJtFD6+CK7uJCJr0rUzMJPiUqcRVXHxZXU73KRvLO+wozvcpayScpkP ExKkmRj244rGFoo2ONBt0S+ylaBIu8qbUKYtgJCk= Date: Tue, 3 Nov 2020 07:50:24 +0100 From: Greg KH To: ira.weiny@intel.com Subject: Re: [PATCH V2 05/10] x86/pks: Add PKS kernel API Message-ID: <20201103065024.GC75930@kroah.com> References: <20201102205320.1458656-1-ira.weiny@intel.com> <20201102205320.1458656-6-ira.weiny@intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20201102205320.1458656-6-ira.weiny@intel.com> Message-ID-Hash: WEQG7HMF7SESNV4RSIMZC3OZIEE4FBNE X-Message-ID-Hash: WEQG7HMF7SESNV4RSIMZC3OZIEE4FBNE X-MailFrom: gregkh@linuxfoundation.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Andy Lutomirski , Peter Zijlstra , Dave Hansen , Fenghua Yu , x86@kernel.org, Andrew Morton , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Mon, Nov 02, 2020 at 12:53:15PM -0800, ira.weiny@intel.com wrote: > From: Fenghua Yu > > PKS allows kernel users to define domains of page mappings which have > additional protections beyond the paging protections. > > Add an API to allocate, use, and free a protection key which identifies > such a domain. Export 5 new symbols pks_key_alloc(), pks_mknoaccess(), > pks_mkread(), pks_mkrdwr(), and pks_key_free(). Add 2 new macros; > PAGE_KERNEL_PKEY(key) and _PAGE_PKEY(pkey). > > Update the protection key documentation to cover pkeys on supervisor > pages. > > Co-developed-by: Ira Weiny > Signed-off-by: Ira Weiny > Signed-off-by: Fenghua Yu > > --- > Changes from V1 > Per Dave Hansen > Add flags to pks_key_alloc() to help future proof the > interface if/when the key space is exhausted. > > Changes from RFC V3 > Per Dave Hansen > Put WARN_ON_ONCE in pks_key_free() > s/pks_mknoaccess/pks_mk_noaccess/ > s/pks_mkread/pks_mk_readonly/ > s/pks_mkrdwr/pks_mk_readwrite/ > Change return pks_key_alloc() to EOPNOTSUPP when not > supported or configured > Per Peter Zijlstra > Remove unneeded preempt disable/enable > --- > Documentation/core-api/protection-keys.rst | 102 ++++++++++++++--- > arch/x86/include/asm/pgtable_types.h | 12 ++ > arch/x86/include/asm/pkeys.h | 11 ++ > arch/x86/include/asm/pkeys_common.h | 4 + > arch/x86/mm/pkeys.c | 126 +++++++++++++++++++++ > include/linux/pgtable.h | 4 + > include/linux/pkeys.h | 24 ++++ > 7 files changed, 265 insertions(+), 18 deletions(-) > > diff --git a/Documentation/core-api/protection-keys.rst b/Documentation/core-api/protection-keys.rst > index ec575e72d0b2..c4e6c480562f 100644 > --- a/Documentation/core-api/protection-keys.rst > +++ b/Documentation/core-api/protection-keys.rst > @@ -4,25 +4,33 @@ > Memory Protection Keys > ====================== > > -Memory Protection Keys for Userspace (PKU aka PKEYs) is a feature > -which is found on Intel's Skylake (and later) "Scalable Processor" > -Server CPUs. It will be available in future non-server Intel parts > -and future AMD processors. > - > -For anyone wishing to test or use this feature, it is available in > -Amazon's EC2 C5 instances and is known to work there using an Ubuntu > -17.04 image. > - > Memory Protection Keys provides a mechanism for enforcing page-based > protections, but without requiring modification of the page tables > -when an application changes protection domains. It works by > -dedicating 4 previously ignored bits in each page table entry to a > -"protection key", giving 16 possible keys. > +when an application changes protection domains. > + > +PKeys Userspace (PKU) is a feature which is found on Intel's Skylake "Scalable > +Processor" Server CPUs and later. And It will be available in future > +non-server Intel parts and future AMD processors. > + > +Future Intel processors will support Protection Keys for Supervisor pages > +(PKS). > + > +For anyone wishing to test or use user space pkeys, it is available in Amazon's > +EC2 C5 instances and is known to work there using an Ubuntu 17.04 image. > + > +pkeys work by dedicating 4 previously Reserved bits in each page table entry to > +a "protection key", giving 16 possible keys. User and Supervisor pages are > +treated separately. > + > +Protections for each page are controlled with per CPU registers for each type > +of page User and Supervisor. Each of these 32 bit register stores two separate > +bits (Access Disable and Write Disable) for each key. > > -There is also a new user-accessible register (PKRU) with two separate > -bits (Access Disable and Write Disable) for each key. Being a CPU > -register, PKRU is inherently thread-local, potentially giving each > -thread a different set of protections from every other thread. > +For Userspace the register is user-accessible (rdpkru/wrpkru). For > +Supervisor, the register (MSR_IA32_PKRS) is accessible only to the kernel. > + > +Being a CPU register, pkeys are inherently thread-local, potentially giving > +each thread an independent set of protections from every other thread. > > There are two new instructions (RDPKRU/WRPKRU) for reading and writing > to the new register. The feature is only available in 64-bit mode, > @@ -30,8 +38,11 @@ even though there is theoretically space in the PAE PTEs. These > permissions are enforced on data access only and have no effect on > instruction fetches. > > -Syscalls > -======== > +For kernel space rdmsr/wrmsr are used to access the kernel MSRs. > + > + > +Syscalls for user space keys > +============================ > > There are 3 system calls which directly interact with pkeys:: > > @@ -98,3 +109,58 @@ with a read():: > The kernel will send a SIGSEGV in both cases, but si_code will be set > to SEGV_PKERR when violating protection keys versus SEGV_ACCERR when > the plain mprotect() permissions are violated. > + > + > +Kernel API for PKS support > +========================== > + > +The following interface is used to allocate, use, and free a pkey which defines > +a 'protection domain' within the kernel. Setting a pkey value in a supervisor > +mapping adds that mapping to the protection domain. > + > + int pks_key_alloc(const char * const pkey_user, int flags); > + #define PAGE_KERNEL_PKEY(pkey) > + #define _PAGE_KEY(pkey) > + void pks_mk_noaccess(int pkey); > + void pks_mk_readonly(int pkey); > + void pks_mk_readwrite(int pkey); > + void pks_key_free(int pkey); > + > +pks_key_alloc() allocates keys dynamically to allow better use of the limited > +key space. 'flags' alter the allocation based on the users need. Currently > +they can request an exclusive key. > + > +Callers of pks_key_alloc() _must_ be prepared for it to fail and take > +appropriate action. This is due mainly to the fact that PKS may not be > +available on all arch's. Failure to check the return of pks_key_alloc() and > +using any of the rest of the API is undefined. > + > +Kernel users must set the PTE permissions in the page table entries for the > +mappings they want to protect. This can be done with PAGE_KERNEL_PKEY() or > +_PAGE_KEY(). > + > +The pks_mk*() family of calls allows kernel users the ability to change the > +protections for the domain identified by the pkey specified. 3 states are > +available pks_mk_noaccess(), pks_mk_readonly(), and pks_mk_readwrite() which > +set the access to none, read, and read/write respectively. > + > +Finally, pks_key_free() allows a user to return the key to the allocator for > +use by others. > + > +The interface maintains pks_mk_noaccess() (Access Disabled (AD=1)) for all keys > +not currently allocated. Therefore, the user can depend on access being > +disabled when pks_key_alloc() returns a key and the user should remove mappings > +from the domain (remove the pkey from the PTE) prior to calling pks_key_free(). > + > +It should be noted that the underlying WRMSR(MSR_IA32_PKRS) is not serializing > +but still maintains ordering properties similar to WRPKRU. Thus it is safe to > +immediately use a mapping when the pks_mk*() functions returns. > + > +The current SDM section on PKRS needs updating but should be the same as that > +of WRPKRU. So to quote from the WRPKRU text: > + > + WRPKRU will never execute transiently. Memory accesses > + affected by PKRU register will not execute (even transiently) > + until all prior executions of WRPKRU have completed execution > + and updated the PKRU register. > + > diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h > index 816b31c68550..c9fdfbdcbbfb 100644 > --- a/arch/x86/include/asm/pgtable_types.h > +++ b/arch/x86/include/asm/pgtable_types.h > @@ -73,6 +73,12 @@ > _PAGE_PKEY_BIT2 | \ > _PAGE_PKEY_BIT3) > > +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS > +#define _PAGE_PKEY(pkey) (_AT(pteval_t, pkey) << _PAGE_BIT_PKEY_BIT0) > +#else > +#define _PAGE_PKEY(pkey) (_AT(pteval_t, 0)) > +#endif > + > #if defined(CONFIG_X86_64) || defined(CONFIG_X86_PAE) > #define _PAGE_KNL_ERRATUM_MASK (_PAGE_DIRTY | _PAGE_ACCESSED) > #else > @@ -229,6 +235,12 @@ enum page_cache_mode { > #define PAGE_KERNEL_IO __pgprot_mask(__PAGE_KERNEL_IO) > #define PAGE_KERNEL_IO_NOCACHE __pgprot_mask(__PAGE_KERNEL_IO_NOCACHE) > > +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS > +#define PAGE_KERNEL_PKEY(pkey) __pgprot_mask(__PAGE_KERNEL | _PAGE_PKEY(pkey)) > +#else > +#define PAGE_KERNEL_PKEY(pkey) PAGE_KERNEL > +#endif > + > #endif /* __ASSEMBLY__ */ > > /* xwr */ > diff --git a/arch/x86/include/asm/pkeys.h b/arch/x86/include/asm/pkeys.h > index 4526245b03e5..f84351b4ac7c 100644 > --- a/arch/x86/include/asm/pkeys.h > +++ b/arch/x86/include/asm/pkeys.h > @@ -3,6 +3,7 @@ > #define _ASM_X86_PKEYS_H > > #include > +#include > > #define ARCH_DEFAULT_PKEY 0 > > @@ -138,4 +139,14 @@ static inline int vma_pkey(struct vm_area_struct *vma) > > u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags); > > +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS > +int pks_key_alloc(const char *const pkey_user, int flags); > +void pks_key_free(int pkey); > + > +void pks_mk_noaccess(int pkey); > +void pks_mk_readonly(int pkey); > +void pks_mk_readwrite(int pkey); > + > +#endif /* CONFIG_ARCH_HAS_SUPERVISOR_PKEYS */ > + > #endif /*_ASM_X86_PKEYS_H */ > diff --git a/arch/x86/include/asm/pkeys_common.h b/arch/x86/include/asm/pkeys_common.h > index 801a75615209..cd492c23b28c 100644 > --- a/arch/x86/include/asm/pkeys_common.h > +++ b/arch/x86/include/asm/pkeys_common.h > @@ -26,6 +26,10 @@ > PKR_AD_KEY(10) | PKR_AD_KEY(11) | PKR_AD_KEY(12) | \ > PKR_AD_KEY(13) | PKR_AD_KEY(14) | PKR_AD_KEY(15)) > > +/* PKS supports 16 keys. Key 0 is reserved for the kernel. */ > +#define PKS_KERN_DEFAULT_KEY 0 > +#define PKS_NUM_KEYS 16 > + > #ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS > void write_pkrs(u32 new_pkrs); > #else > diff --git a/arch/x86/mm/pkeys.c b/arch/x86/mm/pkeys.c > index 76a62419c446..0dc77409957a 100644 > --- a/arch/x86/mm/pkeys.c > +++ b/arch/x86/mm/pkeys.c > @@ -3,6 +3,9 @@ > * Intel Memory Protection Keys management > * Copyright (c) 2015, Intel Corporation. > */ > +#undef pr_fmt > +#define pr_fmt(fmt) "x86/pkeys: " fmt > + > #include /* debugfs_create_u32() */ > #include /* mm_struct, vma, etc... */ > #include /* PKEY_* */ > @@ -231,6 +234,7 @@ u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) > > return pk_reg; > } > +EXPORT_SYMBOL_GPL(update_pkey_val); > > DEFINE_PER_CPU(u32, pkrs_cache); > > @@ -262,3 +266,125 @@ void write_pkrs(u32 new_pkrs) > } > put_cpu_ptr(pkrs); > } > +EXPORT_SYMBOL_GPL(write_pkrs); > + > +/** > + * Do not call this directly, see pks_mk*() below. > + * > + * @pkey: Key for the domain to change > + * @protection: protection bits to be used > + * > + * Protection utilizes the same protection bits specified for User pkeys > + * PKEY_DISABLE_ACCESS > + * PKEY_DISABLE_WRITE > + * > + */ > +static inline void pks_update_protection(int pkey, unsigned long protection) > +{ > + current->thread.saved_pkrs = update_pkey_val(current->thread.saved_pkrs, > + pkey, protection); > + write_pkrs(current->thread.saved_pkrs); > +} > + > +/** > + * PKS access control functions > + * > + * Change the access of the domain specified by the pkey. These are global > + * updates. They only affects the current running thread. It is undefined and > + * a bug for users to call this without having allocated a pkey and using it as > + * pkey here. > + * > + * pks_mk_noaccess() > + * Disable all access to the domain > + * pks_mk_readonly() > + * Make the domain Read only > + * pks_mk_readwrite() > + * Make the domain Read/Write > + * > + * @pkey the pkey for which the access should change. > + * > + */ > +void pks_mk_noaccess(int pkey) > +{ > + pks_update_protection(pkey, PKEY_DISABLE_ACCESS); > +} > +EXPORT_SYMBOL_GPL(pks_mk_noaccess); > + > +void pks_mk_readonly(int pkey) > +{ > + pks_update_protection(pkey, PKEY_DISABLE_WRITE); > +} > +EXPORT_SYMBOL_GPL(pks_mk_readonly); > + > +void pks_mk_readwrite(int pkey) > +{ > + pks_update_protection(pkey, 0); > +} > +EXPORT_SYMBOL_GPL(pks_mk_readwrite); > + > +static const char pks_key_user0[] = "kernel"; > + > +/* Store names of allocated keys for debug. Key 0 is reserved for the kernel. */ > +static const char *pks_key_users[PKS_NUM_KEYS] = { > + pks_key_user0 > +}; > + > +/* > + * Each key is represented by a bit. Bit 0 is set for key 0 and reserved for > + * its use. We use ulong for the bit operations but only 16 bits are used. > + */ > +static unsigned long pks_key_allocation_map = 1 << PKS_KERN_DEFAULT_KEY; > + > +/* > + * pks_key_alloc - Allocate a PKS key > + * @pkey_user: String stored for debugging of key exhaustion. The caller is > + * responsible to maintain this memory until pks_key_free(). > + * @flags: Flags to modify behavior: see pks_alloc_flags > + * > + * Returns: pkey if success > + * -EOPNOTSUPP if pks is not supported or not enabled > + * -ENOSPC if no keys are available (even for sharing) > + */ > +int pks_key_alloc(const char * const pkey_user, int flags) > +{ > + int nr; > + > + if (!cpu_feature_enabled(X86_FEATURE_PKS)) > + return -EOPNOTSUPP; > + > + while (1) { > + nr = find_first_zero_bit(&pks_key_allocation_map, PKS_NUM_KEYS); > + if (nr >= PKS_NUM_KEYS) { > + pr_info("Cannot allocate supervisor key for %s.\n", > + pkey_user); > + return -ENOSPC; > + } > + if (!test_and_set_bit_lock(nr, &pks_key_allocation_map)) > + break; > + } > + > + /* for debugging key exhaustion */ > + pks_key_users[nr] = pkey_user; > + > + return nr; > +} > +EXPORT_SYMBOL_GPL(pks_key_alloc); > + > +/* > + * pks_key_free - Free a previously allocate PKS key > + * @pkey: Key to be free'ed > + */ > +void pks_key_free(int pkey) > +{ > + if (!cpu_feature_enabled(X86_FEATURE_PKS)) > + return; > + > + if (WARN_ON_ONCE(pkey >= PKS_NUM_KEYS || pkey <= PKS_KERN_DEFAULT_KEY)) > + return; > + > + /* Restore to default of no access */ > + pks_mk_noaccess(pkey); > + pks_key_users[pkey] = NULL; > + __clear_bit(pkey, &pks_key_allocation_map); > +} > +EXPORT_SYMBOL_GPL(pks_key_free); > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h > index 38c33eabea89..cd72d73e8e1c 100644 > --- a/include/linux/pgtable.h > +++ b/include/linux/pgtable.h > @@ -1402,6 +1402,10 @@ static inline bool arch_has_pfn_modify_check(void) > # define PAGE_KERNEL_EXEC PAGE_KERNEL > #endif > > +#ifndef PAGE_KERNEL_PKEY > +#define PAGE_KERNEL_PKEY(pkey) PAGE_KERNEL > +#endif > + > /* > * Page Table Modification bits for pgtbl_mod_mask. > * > diff --git a/include/linux/pkeys.h b/include/linux/pkeys.h > index 2955ba976048..0959a4c0ca64 100644 > --- a/include/linux/pkeys.h > +++ b/include/linux/pkeys.h > @@ -50,4 +50,28 @@ static inline void copy_init_pkru_to_fpregs(void) > > #endif /* ! CONFIG_ARCH_HAS_PKEYS */ > > +#define PKS_FLAG_EXCLUSIVE 0x00 > + > +#ifndef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS > +static inline int pks_key_alloc(const char * const pkey_user, int flags) > +{ > + return -EOPNOTSUPP; > +} > +static inline void pks_key_free(int pkey) > +{ > +} > +static inline void pks_mk_noaccess(int pkey) > +{ > + WARN_ON_ONCE(1); So for panic-on-warn systems, this is ok to reboot the box? Are you sure, that feels odd... greg k-h _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org