From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
To: Ingo Molnar <mingo@redhat.com>,
x86@kernel.org, Thomas Gleixner <tglx@linutronix.de>,
"H. Peter Anvin" <hpa@zytor.com>,
Tom Lendacky <thomas.lendacky@amd.com>
Cc: Dave Hansen <dave.hansen@intel.com>,
Kai Huang <kai.huang@linux.intel.com>,
Jacob Pan <jacob.jun.pan@linux.intel.com>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Subject: [PATCHv5 16/19] x86/mm: Calculate direct mapping size
Date: Tue, 17 Jul 2018 14:20:26 +0300 [thread overview]
Message-ID: <20180717112029.42378-17-kirill.shutemov@linux.intel.com> (raw)
In-Reply-To: <20180717112029.42378-1-kirill.shutemov@linux.intel.com>
The kernel needs to have a way to access encrypted memory. We have two
option on how approach it:
- Create temporary mappings every time kernel needs access to encrypted
memory. That's basically brings highmem and its overhead back.
- Create multiple direct mappings, one per-KeyID. In this setup we
don't need to create temporary mappings on the fly -- encrypted
memory is permanently available in kernel address space.
We take the second approach as it has lower overhead.
It's worth noting that with per-KeyID direct mappings compromised kernel
would give access to decrypted data right away without additional tricks
to get memory mapped with the correct KeyID.
Per-KeyID mappings require a lot more virtual address space. On 4-level
machine with 64 KeyIDs we max out 46-bit virtual address space dedicated
for direct mapping with 1TiB of RAM. Given that we round up any
calculation on direct mapping size to 1TiB, we effectively claim all
46-bit address space for direct mapping on such machine regardless of
RAM size.
Increased usage of virtual address space has implications for KASLR:
we have less space for randomization. With 64 TiB claimed for direct
mapping with 4-level we left with 27 TiB of entropy to place
page_offset_base, vmalloc_base and vmemmap_base.
5-level paging provides much wider virtual address space and KASLR
doesn't suffer significantly from per-KeyID direct mappings.
It's preferred to run MKTME with 5-level paging.
A direct mapping for each KeyID will be put next to each other in the
virtual address space. We need to have a way to find boundaries of
direct mapping for particular KeyID.
The new variable direct_mapping_size specifies the size of direct
mapping. With the value, it's trivial to find direct mapping for
KeyID-N: PAGE_OFFSET + N * direct_mapping_size.
Size of direct mapping is calculated during KASLR setup. If KALSR is
disabled it happens during MKTME initialization.
With MKTME size of direct mapping has to be power-of-2. It makes
implementation of __pa() efficient.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
Documentation/x86/x86_64/mm.txt | 4 +++
arch/x86/include/asm/page_64.h | 2 ++
arch/x86/include/asm/setup.h | 6 ++++
arch/x86/kernel/head64.c | 4 +++
arch/x86/kernel/setup.c | 3 ++
arch/x86/mm/init_64.c | 58 +++++++++++++++++++++++++++++++++
arch/x86/mm/kaslr.c | 11 +++++--
7 files changed, 85 insertions(+), 3 deletions(-)
diff --git a/Documentation/x86/x86_64/mm.txt b/Documentation/x86/x86_64/mm.txt
index 5432a96d31ff..c5b92904090f 100644
--- a/Documentation/x86/x86_64/mm.txt
+++ b/Documentation/x86/x86_64/mm.txt
@@ -61,6 +61,10 @@ The direct mapping covers all memory in the system up to the highest
memory address (this means in some cases it can also include PCI memory
holes).
+With MKTME, we have multiple direct mappings. One per-KeyID. They are put
+next to each other. PAGE_OFFSET + N * direct_mapping_size can be used to
+find direct mapping for KeyID-N.
+
vmalloc space is lazily synchronized into the different PML4/PML5 pages of
the processes using the page fault handler, with init_top_pgt as
reference.
diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h
index 939b1cff4a7b..f57fc3cc2246 100644
--- a/arch/x86/include/asm/page_64.h
+++ b/arch/x86/include/asm/page_64.h
@@ -14,6 +14,8 @@ extern unsigned long phys_base;
extern unsigned long page_offset_base;
extern unsigned long vmalloc_base;
extern unsigned long vmemmap_base;
+extern unsigned long direct_mapping_size;
+extern unsigned long direct_mapping_mask;
static inline unsigned long __phys_addr_nodebug(unsigned long x)
{
diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h
index ae13bc974416..bcac5080cca5 100644
--- a/arch/x86/include/asm/setup.h
+++ b/arch/x86/include/asm/setup.h
@@ -59,6 +59,12 @@ extern void x86_ce4100_early_setup(void);
static inline void x86_ce4100_early_setup(void) { }
#endif
+#ifdef CONFIG_MEMORY_PHYSICAL_PADDING
+void calculate_direct_mapping_size(void);
+#else
+static inline void calculate_direct_mapping_size(void) { }
+#endif
+
#ifndef _SETUP
#include <asm/espfix.h>
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 8047379e575a..79b92518ceee 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -59,6 +59,10 @@ EXPORT_SYMBOL(vmalloc_base);
unsigned long vmemmap_base __ro_after_init = __VMEMMAP_BASE_L4;
EXPORT_SYMBOL(vmemmap_base);
#endif
+unsigned long direct_mapping_size __ro_after_init = -1UL;
+EXPORT_SYMBOL(direct_mapping_size);
+unsigned long direct_mapping_mask __ro_after_init = -1UL;
+EXPORT_SYMBOL(direct_mapping_mask);
#define __head __section(.head.text)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 2f86d883dd95..09ddbd142e3c 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1053,6 +1053,9 @@ void __init setup_arch(char **cmdline_p)
*/
init_cache_modes();
+ /* direct_mapping_size has to be initialized before KASLR and MKTME */
+ calculate_direct_mapping_size();
+
/*
* Define random base addresses for memory sections after max_pfn is
* defined and before each memory section base is used.
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index a688617c727e..d3a93ca69c67 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -1399,6 +1399,64 @@ unsigned long memory_block_size_bytes(void)
return memory_block_size_probed;
}
+#ifdef CONFIG_MEMORY_PHYSICAL_PADDING
+void __init calculate_direct_mapping_size(void)
+{
+ unsigned long available_va;
+
+ /* 1/4 of virtual address space is didicated for direct mapping */
+ available_va = 1UL << (__VIRTUAL_MASK_SHIFT - 1);
+
+ /* How much memory the system has? */
+ direct_mapping_size = max_pfn << PAGE_SHIFT;
+ direct_mapping_size = round_up(direct_mapping_size, 1UL << 40);
+
+ if (!mktme_nr_keyids)
+ goto out;
+
+ /*
+ * For MKTME we need direct_mapping_size to be power-of-2.
+ * It makes __pa() implementation efficient.
+ */
+ direct_mapping_size = roundup_pow_of_two(direct_mapping_size);
+
+ /*
+ * Not enough virtual address space to address all physical memory with
+ * MKTME enabled. Even without padding.
+ *
+ * Disable MKTME instead.
+ */
+ if (direct_mapping_size > available_va / (mktme_nr_keyids + 1)) {
+ pr_err("x86/mktme: Disabled. Not enough virtual address space\n");
+ pr_err("x86/mktme: Consider switching to 5-level paging\n");
+ mktme_disable();
+ goto out;
+ }
+
+ /*
+ * Virtual address space is divided between per-KeyID direct mappings.
+ */
+ available_va /= mktme_nr_keyids + 1;
+out:
+ /* Add padding, if there's enough virtual address space */
+ direct_mapping_size += (1UL << 40) * CONFIG_MEMORY_PHYSICAL_PADDING;
+ if (mktme_nr_keyids)
+ direct_mapping_size = roundup_pow_of_two(direct_mapping_size);
+
+ if (direct_mapping_size > available_va)
+ direct_mapping_size = available_va;
+
+ /*
+ * For MKTME, make sure direct_mapping_size is still power-of-2
+ * after adding padding and calculate mask that is used in __pa().
+ */
+ if (mktme_nr_keyids) {
+ direct_mapping_size = rounddown_pow_of_two(direct_mapping_size);
+ direct_mapping_mask = direct_mapping_size - 1;
+ }
+}
+#endif
+
#ifdef CONFIG_SPARSEMEM_VMEMMAP
/*
* Initialise the sparsemem vmemmap using huge-pages at the PMD level.
diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c
index 4408cd9a3bef..bf044ff50ec0 100644
--- a/arch/x86/mm/kaslr.c
+++ b/arch/x86/mm/kaslr.c
@@ -101,10 +101,15 @@ void __init kernel_randomize_memory(void)
* add padding if needed (especially for memory hotplug support).
*/
BUG_ON(kaslr_regions[0].base != &page_offset_base);
- memory_tb = DIV_ROUND_UP(max_pfn << PAGE_SHIFT, 1UL << TB_SHIFT) +
- CONFIG_MEMORY_PHYSICAL_PADDING;
- /* Adapt phyiscal memory region size based on available memory */
+ /*
+ * Calculate space required to map all physical memory.
+ * In case of MKTME, we map physical memory multiple times, one for
+ * each KeyID. If MKTME is disabled mktme_nr_keyids is 0.
+ */
+ memory_tb = (direct_mapping_size * (mktme_nr_keyids + 1)) >> TB_SHIFT;
+
+ /* Adapt physical memory region size based on available memory */
if (memory_tb < kaslr_regions[0].size_tb)
kaslr_regions[0].size_tb = memory_tb;
--
2.18.0
next prev parent reply other threads:[~2018-07-17 11:21 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-17 11:20 [PATCHv5 00/19] MKTME enabling Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 01/19] mm: Do no merge VMAs with different encryption KeyIDs Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 02/19] mm: Do not use zero page in encrypted pages Kirill A. Shutemov
2018-07-18 17:36 ` Dave Hansen
2018-07-19 7:16 ` Kirill A. Shutemov
2018-07-19 13:58 ` Dave Hansen
2018-07-20 12:16 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 03/19] mm/ksm: Do not merge pages with different KeyIDs Kirill A. Shutemov
2018-07-18 17:38 ` Dave Hansen
2018-07-19 7:32 ` Kirill A. Shutemov
2018-07-19 14:02 ` Dave Hansen
2018-07-20 12:23 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 04/19] mm/page_alloc: Unify alloc_hugepage_vma() Kirill A. Shutemov
2018-07-18 17:43 ` Dave Hansen
2018-07-17 11:20 ` [PATCHv5 05/19] mm/page_alloc: Handle allocation for encrypted memory Kirill A. Shutemov
2018-07-18 23:03 ` Dave Hansen
2018-07-19 8:27 ` Kirill A. Shutemov
2018-07-19 14:05 ` Dave Hansen
2018-07-20 12:25 ` Kirill A. Shutemov
2018-07-26 14:25 ` Michal Hocko
2018-07-17 11:20 ` [PATCHv5 06/19] mm/khugepaged: Handle encrypted pages Kirill A. Shutemov
2018-07-18 23:11 ` Dave Hansen
2018-07-19 8:59 ` Kirill A. Shutemov
2018-07-19 14:13 ` Dave Hansen
2018-07-20 12:29 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 07/19] x86/mm: Mask out KeyID bits from page table entry pfn Kirill A. Shutemov
2018-07-18 23:13 ` Dave Hansen
2018-07-19 9:54 ` Kirill A. Shutemov
2018-07-19 14:19 ` Dave Hansen
2018-07-20 12:31 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 08/19] x86/mm: Introduce variables to store number, shift and mask of KeyIDs Kirill A. Shutemov
2018-07-18 23:19 ` Dave Hansen
2018-07-19 10:21 ` Kirill A. Shutemov
2018-07-19 12:37 ` Thomas Gleixner
2018-07-19 13:12 ` Kirill A. Shutemov
2018-07-19 13:18 ` Thomas Gleixner
2018-07-19 13:23 ` Kirill A. Shutemov
2018-07-19 13:40 ` Thomas Gleixner
2018-07-20 12:34 ` Kirill A. Shutemov
2018-07-20 13:17 ` Thomas Gleixner
2018-07-20 13:40 ` Kirill A. Shutemov
2018-07-19 14:23 ` Dave Hansen
2018-07-20 12:34 ` Kirill A. Shutemov
2018-07-31 0:08 ` Kai Huang
2018-07-17 11:20 ` [PATCHv5 09/19] x86/mm: Preserve KeyID on pte_modify() and pgprot_modify() Kirill A. Shutemov
2018-07-18 23:30 ` Dave Hansen
2018-07-20 12:42 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 10/19] x86/mm: Implement page_keyid() using page_ext Kirill A. Shutemov
2018-07-18 23:38 ` Dave Hansen
2018-07-23 9:45 ` Kirill A. Shutemov
2018-07-23 17:22 ` Alison Schofield
2018-07-17 11:20 ` [PATCHv5 11/19] x86/mm: Implement vma_keyid() Kirill A. Shutemov
2018-07-18 23:40 ` Dave Hansen
2018-07-23 9:47 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 12/19] x86/mm: Implement prep_encrypted_page() and arch_free_page() Kirill A. Shutemov
2018-07-18 23:53 ` Dave Hansen
2018-07-23 9:50 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 13/19] x86/mm: Rename CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 14/19] x86/mm: Allow to disable MKTME after enumeration Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 15/19] x86/mm: Detect MKTME early Kirill A. Shutemov
2018-07-17 11:20 ` Kirill A. Shutemov [this message]
2018-07-17 11:20 ` [PATCHv5 17/19] x86/mm: Implement sync_direct_mapping() Kirill A. Shutemov
2018-07-19 0:01 ` Dave Hansen
2018-07-23 10:04 ` Kirill A. Shutemov
2018-07-23 12:25 ` Dave Hansen
2018-07-17 11:20 ` [PATCHv5 18/19] x86/mm: Handle encrypted memory in page_to_virt() and __pa() Kirill A. Shutemov
2018-07-18 22:21 ` Thomas Gleixner
2018-07-23 10:12 ` Kirill A. Shutemov
2018-07-26 17:26 ` Dave Hansen
2018-07-27 13:49 ` Kirill A. Shutemov
2018-07-17 11:20 ` [PATCHv5 19/19] x86: Introduce CONFIG_X86_INTEL_MKTME Kirill A. Shutemov
2018-08-15 7:48 ` Pavel Machek
2018-08-17 9:24 ` Kirill A. Shutemov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180717112029.42378-17-kirill.shutemov@linux.intel.com \
--to=kirill.shutemov@linux.intel.com \
--cc=dave.hansen@intel.com \
--cc=hpa@zytor.com \
--cc=jacob.jun.pan@linux.intel.com \
--cc=kai.huang@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@redhat.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).