From: "Kirill A. Shutemov" <kirill@shutemov.name> To: Peter Zijlstra <peterz@infradead.org> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>, Andrew Morton <akpm@linux-foundation.org>, x86@kernel.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>, Borislav Petkov <bp@alien8.de>, Andy Lutomirski <luto@amacapital.net>, David Howells <dhowells@redhat.com>, Kees Cook <keescook@chromium.org>, Dave Hansen <dave.hansen@intel.com>, Kai Huang <kai.huang@linux.intel.com>, Jacob Pan <jacob.jun.pan@linux.intel.com>, Alison Schofield <alison.schofield@intel.com>, linux-mm@kvack.org, kvm@vger.kernel.org, keyrings@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH, RFC 18/62] x86/mm: Implement syncing per-KeyID direct mappings Date: Mon, 17 Jun 2019 17:43:28 +0300 Message-ID: <20190617144328.oqwx5rb5yfm2ziws@box> (raw) In-Reply-To: <20190617092755.GA3419@hirez.programming.kicks-ass.net> On Mon, Jun 17, 2019 at 11:27:55AM +0200, Peter Zijlstra wrote: > On Sat, Jun 15, 2019 at 01:43:09AM +0300, Kirill A. Shutemov wrote: > > On Fri, Jun 14, 2019 at 11:51:32AM +0200, Peter Zijlstra wrote: > > > On Wed, May 08, 2019 at 05:43:38PM +0300, Kirill A. Shutemov wrote: > > > > For MKTME we use per-KeyID direct mappings. This allows kernel to have > > > > access to encrypted memory. > > > > > > > > sync_direct_mapping() sync per-KeyID direct mappings with a canonical > > > > one -- KeyID-0. > > > > > > > > The function tracks changes in the canonical mapping: > > > > - creating or removing chunks of the translation tree; > > > > - changes in mapping flags (i.e. protection bits); > > > > - splitting huge page mapping into a page table; > > > > - replacing page table with a huge page mapping; > > > > > > > > The function need to be called on every change to the direct mapping: > > > > hotplug, hotremove, changes in permissions bits, etc. > > > > > > And yet I don't see anything in pageattr.c. > > > > You're right. I've hooked up the sync in the wrong place. > > > > > > Also, this seems like an expensive scheme; if you know where the changes > > > where, a more fine-grained update would be faster. > > > > Do we have any hot enough pageattr users that makes it crucial? > > > > I'll look into this anyway. > > The graphics people would be the most agressive users of this I'd think. > They're the ones that yelled when I broke it last ;-) I think something like this should do (I'll fold it in after testing): diff --git a/arch/x86/include/asm/mktme.h b/arch/x86/include/asm/mktme.h index 6c973cb1e64c..b30386d84281 100644 --- a/arch/x86/include/asm/mktme.h +++ b/arch/x86/include/asm/mktme.h @@ -68,7 +68,7 @@ static inline void arch_free_page(struct page *page, int order) free_encrypted_page(page, order); } -int sync_direct_mapping(void); +int sync_direct_mapping(unsigned long start, unsigned long end); int mktme_get_alg(int keyid); @@ -86,7 +86,7 @@ static inline bool mktme_enabled(void) static inline void mktme_disable(void) {} -static inline int sync_direct_mapping(void) +static inline int sync_direct_mapping(unsigned long start, unsigned long end) { return 0; } diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index f50a38d86cc4..f8123aeb24a6 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -761,7 +761,7 @@ __kernel_physical_mapping_init(unsigned long paddr_start, pgd_changed = true; } - ret = sync_direct_mapping(); + ret = sync_direct_mapping(vaddr_start, vaddr_end); WARN_ON(ret); if (pgd_changed) @@ -1209,7 +1209,7 @@ kernel_physical_mapping_remove(unsigned long start, unsigned long end) end = (unsigned long)__va(end); remove_pagetable(start, end, true, NULL); - ret = sync_direct_mapping(); + ret = sync_direct_mapping(start, end); WARN_ON(ret); } @@ -1315,7 +1315,6 @@ void mark_rodata_ro(void) unsigned long text_end = PFN_ALIGN(&__stop___ex_table); unsigned long rodata_end = PFN_ALIGN(&__end_rodata); unsigned long all_end; - int ret; printk(KERN_INFO "Write protecting the kernel read-only data: %luk\n", (end - start) >> 10); @@ -1349,8 +1348,6 @@ void mark_rodata_ro(void) free_kernel_image_pages((void *)text_end, (void *)rodata_start); free_kernel_image_pages((void *)rodata_end, (void *)_sdata); - ret = sync_direct_mapping(); - WARN_ON(ret); debug_checkwx(); } diff --git a/arch/x86/mm/mktme.c b/arch/x86/mm/mktme.c index 9d2bb534f2ba..c099e1da055b 100644 --- a/arch/x86/mm/mktme.c +++ b/arch/x86/mm/mktme.c @@ -76,7 +76,7 @@ static void init_page_mktme(void) { static_branch_enable(&mktme_enabled_key); - sync_direct_mapping(); + sync_direct_mapping(PAGE_OFFSET, PAGE_OFFSET + direct_mapping_size); } struct page_ext_operations page_mktme_ops = { @@ -596,15 +596,13 @@ static int sync_direct_mapping_p4d(unsigned long keyid, return ret; } -static int sync_direct_mapping_keyid(unsigned long keyid) +static int sync_direct_mapping_keyid(unsigned long keyid, + unsigned long addr, unsigned long end) { pgd_t *src_pgd, *dst_pgd; - unsigned long addr, end, next; + unsigned long next; int ret = 0; - addr = PAGE_OFFSET; - end = PAGE_OFFSET + direct_mapping_size; - dst_pgd = pgd_offset_k(addr + keyid * direct_mapping_size); src_pgd = pgd_offset_k(addr); @@ -643,7 +641,7 @@ static int sync_direct_mapping_keyid(unsigned long keyid) * * The function is nop until MKTME is enabled. */ -int sync_direct_mapping(void) +int sync_direct_mapping(unsigned long start, unsigned long end) { int i, ret = 0; @@ -651,7 +649,7 @@ int sync_direct_mapping(void) return 0; for (i = 1; !ret && i <= mktme_nr_keyids; i++) - ret = sync_direct_mapping_keyid(i); + ret = sync_direct_mapping_keyid(i, start, end); flush_tlb_all(); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 6a9a77a403c9..eafbe0d8c44f 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -347,6 +347,28 @@ static void cpa_flush(struct cpa_data *data, int cache) BUG_ON(irqs_disabled() && !early_boot_irqs_disabled); + if (mktme_enabled()) { + unsigned long start, end; + + start = *cpa->vaddr; + end = *cpa->vaddr + cpa->numpages * PAGE_SIZE; + + /* Sync all direct mapping for an array */ + if (cpa->flags & CPA_ARRAY) { + start = PAGE_OFFSET; + end = PAGE_OFFSET + direct_mapping_size; + } + + /* + * Sync per-KeyID direct mappings with the canonical one + * (KeyID-0). + * + * sync_direct_mapping() does full TLB flush. + */ + sync_direct_mapping(start, end); + return; + } + if (cache && !static_cpu_has(X86_FEATURE_CLFLUSH)) { cpa_flush_all(cache); return; -- Kirill A. Shutemov
next prev parent reply index Thread overview: 153+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-08 14:43 [PATCH, RFC 00/62] Intel MKTME enabling Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 01/62] mm: Do no merge VMAs with different encryption KeyIDs Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 02/62] mm: Add helpers to setup zero page mappings Kirill A. Shutemov 2019-05-29 7:21 ` Mike Rapoport 2019-05-08 14:43 ` [PATCH, RFC 03/62] mm/ksm: Do not merge pages with different KeyIDs Kirill A. Shutemov 2019-05-10 18:07 ` Dave Hansen 2019-05-13 14:27 ` Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 04/62] mm/page_alloc: Unify alloc_hugepage_vma() Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 05/62] mm/page_alloc: Handle allocation for encrypted memory Kirill A. Shutemov 2019-05-29 7:21 ` Mike Rapoport 2019-05-29 12:47 ` Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 06/62] mm/khugepaged: Handle encrypted pages Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 07/62] x86/mm: Mask out KeyID bits from page table entry pfn Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 08/62] x86/mm: Introduce variables to store number, shift and mask of KeyIDs Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 09/62] x86/mm: Preserve KeyID on pte_modify() and pgprot_modify() Kirill A. Shutemov 2019-06-14 9:15 ` Peter Zijlstra 2019-06-14 13:03 ` Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 10/62] x86/mm: Detect MKTME early Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 11/62] x86/mm: Add a helper to retrieve KeyID for a page Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 12/62] x86/mm: Add a helper to retrieve KeyID for a VMA Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 13/62] x86/mm: Add hooks to allocate and free encrypted pages Kirill A. Shutemov 2019-06-14 9:34 ` Peter Zijlstra 2019-06-14 11:04 ` Peter Zijlstra 2019-06-14 13:28 ` Kirill A. Shutemov 2019-06-14 13:43 ` Peter Zijlstra 2019-06-14 22:41 ` Kirill A. Shutemov 2019-06-17 9:25 ` Peter Zijlstra 2019-06-14 13:14 ` Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 14/62] x86/mm: Map zero pages into encrypted mappings correctly Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 15/62] x86/mm: Rename CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 16/62] x86/mm: Allow to disable MKTME after enumeration Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 17/62] x86/mm: Calculate direct mapping size Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 18/62] x86/mm: Implement syncing per-KeyID direct mappings Kirill A. Shutemov 2019-06-14 9:51 ` Peter Zijlstra 2019-06-14 22:43 ` Kirill A. Shutemov 2019-06-17 9:27 ` Peter Zijlstra 2019-06-17 14:43 ` Kirill A. Shutemov [this message] 2019-06-17 14:51 ` Peter Zijlstra 2019-06-17 15:17 ` Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 19/62] x86/mm: Handle encrypted memory in page_to_virt() and __pa() Kirill A. Shutemov 2019-06-14 11:10 ` Peter Zijlstra 2019-05-08 14:43 ` [PATCH, RFC 20/62] mm/page_ext: Export lookup_page_ext() symbol Kirill A. Shutemov 2019-06-14 11:12 ` Peter Zijlstra 2019-06-14 22:44 ` Kirill A. Shutemov 2019-06-17 9:30 ` Peter Zijlstra 2019-06-17 11:01 ` Kai Huang 2019-06-17 11:13 ` Huang, Kai 2019-05-08 14:43 ` [PATCH, RFC 21/62] mm/rmap: Clear vma->anon_vma on unlink_anon_vmas() Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 22/62] x86/pconfig: Set a valid encryption algorithm for all MKTME commands Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 23/62] keys/mktme: Introduce a Kernel Key Service for MKTME Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 24/62] keys/mktme: Preparse the MKTME key payload Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 25/62] keys/mktme: Instantiate and destroy MKTME keys Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 26/62] keys/mktme: Move the MKTME payload into a cache aligned structure Kirill A. Shutemov 2019-06-14 11:35 ` Peter Zijlstra 2019-06-14 17:10 ` Alison Schofield 2019-05-08 14:43 ` [PATCH, RFC 27/62] keys/mktme: Strengthen the entropy of CPU generated MKTME keys Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 28/62] keys/mktme: Set up PCONFIG programming targets for " Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 29/62] keys/mktme: Program MKTME keys into the platform hardware Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 30/62] keys/mktme: Set up a percpu_ref_count for MKTME keys Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 31/62] keys/mktme: Require CAP_SYS_RESOURCE capability " Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 32/62] keys/mktme: Store MKTME payloads if cmdline parameter allows Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 33/62] acpi: Remove __init from acpi table parsing functions Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 34/62] acpi/hmat: Determine existence of an ACPI HMAT Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 35/62] keys/mktme: Require ACPI HMAT to register the MKTME Key Service Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 36/62] acpi/hmat: Evaluate topology presented in ACPI HMAT for MKTME Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 37/62] keys/mktme: Do not allow key creation in unsafe topologies Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 38/62] keys/mktme: Support CPU hotplug for MKTME key service Kirill A. Shutemov 2019-05-08 14:43 ` [PATCH, RFC 39/62] keys/mktme: Find new PCONFIG targets during memory hotplug Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 40/62] keys/mktme: Program new PCONFIG targets with MKTME keys Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 41/62] keys/mktme: Support memory hotplug for " Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 42/62] mm: Generalize the mprotect implementation to support extensions Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 43/62] syscall/x86: Wire up a system call for MKTME encryption keys Kirill A. Shutemov 2019-05-29 7:21 ` Mike Rapoport 2019-05-29 18:12 ` Alison Schofield 2019-05-08 14:44 ` [PATCH, RFC 44/62] x86/mm: Set KeyIDs in encrypted VMAs for MKTME Kirill A. Shutemov 2019-06-14 11:44 ` Peter Zijlstra 2019-06-14 17:33 ` Alison Schofield 2019-06-14 18:26 ` Dave Hansen 2019-06-14 18:46 ` Alison Schofield 2019-06-14 19:11 ` Dave Hansen 2019-06-17 9:10 ` Peter Zijlstra 2019-05-08 14:44 ` [PATCH, RFC 45/62] mm: Add the encrypt_mprotect() system call " Kirill A. Shutemov 2019-06-14 11:47 ` Peter Zijlstra 2019-06-14 17:35 ` Alison Schofield 2019-06-14 11:51 ` Peter Zijlstra 2019-06-15 0:32 ` Alison Schofield 2019-06-17 9:08 ` Peter Zijlstra 2019-06-17 15:07 ` Andy Lutomirski 2019-06-17 15:28 ` Dave Hansen 2019-06-17 15:46 ` Andy Lutomirski 2019-06-17 18:27 ` Dave Hansen 2019-06-17 19:12 ` Andy Lutomirski 2019-06-17 21:36 ` Dave Hansen 2019-06-18 0:48 ` Kai Huang 2019-06-18 1:50 ` Andy Lutomirski 2019-06-18 2:11 ` Kai Huang 2019-06-18 4:24 ` Andy Lutomirski 2019-06-18 14:19 ` Dave Hansen 2019-06-18 0:05 ` Kai Huang 2019-06-18 0:15 ` Andy Lutomirski 2019-06-18 1:35 ` Kai Huang 2019-06-18 1:43 ` Andy Lutomirski 2019-06-18 2:23 ` Kai Huang 2019-06-18 9:12 ` Peter Zijlstra 2019-06-18 14:09 ` Dave Hansen 2019-06-18 16:15 ` Kirill A. Shutemov 2019-06-18 16:22 ` Dave Hansen 2019-06-18 16:36 ` Andy Lutomirski 2019-06-18 16:48 ` Dave Hansen 2019-06-18 14:13 ` Dave Hansen 2019-06-17 23:59 ` Kai Huang 2019-06-18 1:34 ` Lendacky, Thomas 2019-06-18 1:40 ` Andy Lutomirski 2019-06-18 2:02 ` Lendacky, Thomas 2019-06-18 4:19 ` Andy Lutomirski 2019-05-08 14:44 ` [PATCH, RFC 46/62] x86/mm: Keep reference counts on encrypted VMAs " Kirill A. Shutemov 2019-06-14 11:54 ` Peter Zijlstra 2019-06-14 18:39 ` Alison Schofield 2019-05-08 14:44 ` [PATCH, RFC 47/62] mm: Restrict MKTME memory encryption to anonymous VMAs Kirill A. Shutemov 2019-06-14 11:55 ` Peter Zijlstra 2019-06-15 0:07 ` Alison Schofield 2019-05-08 14:44 ` [PATCH, RFC 48/62] selftests/x86/mktme: Test the MKTME APIs Kirill A. Shutemov 2019-05-08 17:09 ` Alison Schofield 2019-05-08 14:44 ` [PATCH, RFC 49/62] mm, x86: export several MKTME variables Kirill A. Shutemov 2019-06-14 11:56 ` Peter Zijlstra 2019-06-17 3:14 ` Kai Huang 2019-06-17 7:46 ` Peter Zijlstra 2019-06-17 8:39 ` Kai Huang 2019-06-17 11:25 ` Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 50/62] kvm, x86, mmu: setup MKTME keyID to spte for given PFN Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 51/62] iommu/vt-d: Support MKTME in DMA remapping Kirill A. Shutemov 2019-06-14 12:04 ` Peter Zijlstra 2019-05-08 14:44 ` [PATCH, RFC 52/62] x86/mm: introduce common code for mem encryption Kirill A. Shutemov 2019-05-08 16:58 ` Christoph Hellwig 2019-05-08 20:52 ` Jacob Pan 2019-05-08 21:21 ` Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 53/62] x86/mm: Use common code for DMA memory encryption Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 54/62] x86/mm: Disable MKTME on incompatible platform configurations Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 55/62] x86/mm: Disable MKTME if not all system memory supports encryption Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 56/62] x86: Introduce CONFIG_X86_INTEL_MKTME Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 57/62] x86/mktme: Overview of Multi-Key Total Memory Encryption Kirill A. Shutemov 2019-05-29 7:21 ` Mike Rapoport 2019-05-29 18:13 ` Alison Schofield 2019-07-14 18:16 ` Randy Dunlap 2019-07-15 9:02 ` Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 58/62] x86/mktme: Document the MKTME provided security mitigations Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 59/62] x86/mktme: Document the MKTME kernel configuration requirements Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 60/62] x86/mktme: Document the MKTME Key Service API Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 61/62] x86/mktme: Document the MKTME API for anonymous memory encryption Kirill A. Shutemov 2019-05-08 14:44 ` [PATCH, RFC 62/62] x86/mktme: Demonstration program using the MKTME APIs Kirill A. Shutemov 2019-05-29 7:30 ` [PATCH, RFC 00/62] Intel MKTME enabling Mike Rapoport 2019-05-29 18:20 ` Alison Schofield 2019-06-14 12:15 ` Peter Zijlstra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190617144328.oqwx5rb5yfm2ziws@box \ --to=kirill@shutemov.name \ --cc=akpm@linux-foundation.org \ --cc=alison.schofield@intel.com \ --cc=bp@alien8.de \ --cc=dave.hansen@intel.com \ --cc=dhowells@redhat.com \ --cc=hpa@zytor.com \ --cc=jacob.jun.pan@linux.intel.com \ --cc=kai.huang@linux.intel.com \ --cc=keescook@chromium.org \ --cc=keyrings@vger.kernel.org \ --cc=kirill.shutemov@linux.intel.com \ --cc=kvm@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=luto@amacapital.net \ --cc=mingo@redhat.com \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-mm Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \ linux-mm@kvack.org public-inbox-index linux-mm Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kvack.linux-mm AGPL code for this site: git clone https://public-inbox.org/public-inbox.git