From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31DDDC433E0 for ; Fri, 5 Feb 2021 15:17:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B024E650E7 for ; Fri, 5 Feb 2021 15:17:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B024E650E7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61F706B0078; Fri, 5 Feb 2021 10:16:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E6C86B0082; Fri, 5 Feb 2021 10:16:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13CDA6B0080; Fri, 5 Feb 2021 10:16:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0006.hostedemail.com [216.40.44.6]) by kanga.kvack.org (Postfix) with ESMTP id C475F6B007D for ; Fri, 5 Feb 2021 10:16:51 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 742E0180AD81A for ; Fri, 5 Feb 2021 15:16:51 +0000 (UTC) X-FDA: 77784566622.10.show88_570f002275e5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 2D00D16A0C3 for ; Fri, 5 Feb 2021 15:16:51 +0000 (UTC) X-HE-Tag: show88_570f002275e5 X-Filterd-Recvd-Size: 7647 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 5 Feb 2021 15:16:50 +0000 (UTC) IronPort-SDR: pZj+Zcyr6SfK3bn1Urk78+TaxGamE0/qbUy33W6qMl/hfsVgK7I/E3Zw5jPXZVVRJ3MKRvkbax 3CJWiLMNE/sg== X-IronPort-AV: E=McAfee;i="6000,8403,9885"; a="168561534" X-IronPort-AV: E=Sophos;i="5.81,155,1610438400"; d="scan'208";a="168561534" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2021 07:16:48 -0800 IronPort-SDR: zgKDaClIwIvBd6Pnmc+DLs9UeYhat+eFSlon/Anj46ldVGWyPnhUNAGAj12ZX7mu6BCs7KF2T7 G8Tq5C34EqbA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,155,1610438400"; d="scan'208";a="508569469" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga004.jf.intel.com with ESMTP; 05 Feb 2021 07:16:45 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 15AB03F9; Fri, 5 Feb 2021 17:16:41 +0200 (EET) From: "Kirill A. Shutemov" To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: x86@kernel.org, Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , "H . J . Lu" , Andi Kleen , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [RFC 8/9] x86/mm: Make LAM_U48 and mappings above 47-bits mutually exclusive Date: Fri, 5 Feb 2021 18:16:29 +0300 Message-Id: <20210205151631.43511-10-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210205151631.43511-1-kirill.shutemov@linux.intel.com> References: <20210205151631.43511-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: LAM_U48 steals bits above 47-bit for tags and makes it impossible for userspace to use full address space on 5-level paging machine. Make these features mutually exclusive: whichever gets enabled first blocks the othe one. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/elf.h | 3 ++- arch/x86/include/asm/mmu_context.h | 13 +++++++++++++ arch/x86/kernel/sys_x86_64.c | 5 +++-- arch/x86/mm/hugetlbpage.c | 6 ++++-- arch/x86/mm/mmap.c | 9 ++++++++- 5 files changed, 30 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index b9a5d488f1a5..8ba405dfb861 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -326,7 +326,8 @@ static inline int mmap_is_ia32(void) extern unsigned long task_size_32bit(void); extern unsigned long task_size_64bit(int full_addr_space); extern unsigned long get_mmap_base(int is_legacy); -extern bool mmap_address_hint_valid(unsigned long addr, unsigned long le= n); +extern bool mmap_address_hint_valid(struct mm_struct *mm, + unsigned long addr, unsigned long len); =20 #ifdef CONFIG_X86_32 =20 diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mm= u_context.h index d98016b83755..e338beb031b2 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -214,4 +214,17 @@ static inline bool arch_vma_access_permitted(struct = vm_area_struct *vma, =20 unsigned long __get_current_cr3_fast(void); =20 +#ifdef CONFIG_X86_5LEVEL +static inline bool full_va_allowed(struct mm_struct *mm) +{ + /* LAM_U48 steals VA bits abouve 47-bit for tags */ + return mm->context.lam !=3D LAM_U48; +} +#else +static inline bool full_va_allowed(struct mm_struct *mm) +{ + return false; +} +#endif + #endif /* _ASM_X86_MMU_CONTEXT_H */ diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c index 504fa5425bce..b5d0afee9b0f 100644 --- a/arch/x86/kernel/sys_x86_64.c +++ b/arch/x86/kernel/sys_x86_64.c @@ -21,6 +21,7 @@ =20 #include #include +#include =20 /* * Align a virtual address to avoid aliasing in the I$ on AMD F15h. @@ -189,7 +190,7 @@ arch_get_unmapped_area_topdown(struct file *filp, con= st unsigned long addr0, /* requesting a specific address */ if (addr) { addr &=3D PAGE_MASK; - if (!mmap_address_hint_valid(addr, len)) + if (!mmap_address_hint_valid(mm, addr, len)) goto get_unmapped_area; =20 vma =3D find_vma(mm, addr); @@ -210,7 +211,7 @@ arch_get_unmapped_area_topdown(struct file *filp, con= st unsigned long addr0, * !in_32bit_syscall() check to avoid high addresses for x32 * (and make it no op on native i386). */ - if (addr > DEFAULT_MAP_WINDOW && !in_32bit_syscall()) + if (addr > DEFAULT_MAP_WINDOW && !in_32bit_syscall() && full_va_allowed= (mm)) info.high_limit +=3D TASK_SIZE_MAX - DEFAULT_MAP_WINDOW; =20 info.align_mask =3D 0; diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index a0d023cb4292..9fdc8db42365 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 #if 0 /* This is just for testing */ struct page * @@ -103,6 +104,7 @@ static unsigned long hugetlb_get_unmapped_area_topdow= n(struct file *file, unsigned long pgoff, unsigned long flags) { struct hstate *h =3D hstate_file(file); + struct mm_struct *mm =3D current->mm; struct vm_unmapped_area_info info; =20 info.flags =3D VM_UNMAPPED_AREA_TOPDOWN; @@ -114,7 +116,7 @@ static unsigned long hugetlb_get_unmapped_area_topdow= n(struct file *file, * If hint address is above DEFAULT_MAP_WINDOW, look for unmapped area * in the full address space. */ - if (addr > DEFAULT_MAP_WINDOW && !in_32bit_syscall()) + if (addr > DEFAULT_MAP_WINDOW && !in_32bit_syscall() && full_va_allowed= (mm)) info.high_limit +=3D TASK_SIZE_MAX - DEFAULT_MAP_WINDOW; =20 info.align_mask =3D PAGE_MASK & ~huge_page_mask(h); @@ -161,7 +163,7 @@ hugetlb_get_unmapped_area(struct file *file, unsigned= long addr, =20 if (addr) { addr &=3D huge_page_mask(h); - if (!mmap_address_hint_valid(addr, len)) + if (!mmap_address_hint_valid(mm, addr, len)) goto get_unmapped_area; =20 vma =3D find_vma(mm, addr); diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c index c90c20904a60..f9ca824729de 100644 --- a/arch/x86/mm/mmap.c +++ b/arch/x86/mm/mmap.c @@ -21,6 +21,7 @@ #include #include #include +#include =20 #include "physaddr.h" =20 @@ -35,6 +36,8 @@ unsigned long task_size_32bit(void) =20 unsigned long task_size_64bit(int full_addr_space) { + if (!full_va_allowed(current->mm)) + return DEFAULT_MAP_WINDOW; return full_addr_space ? TASK_SIZE_MAX : DEFAULT_MAP_WINDOW; } =20 @@ -206,11 +209,15 @@ const char *arch_vma_name(struct vm_area_struct *vm= a) * the failure of such a fixed mapping request, so the restriction is no= t * applied. */ -bool mmap_address_hint_valid(unsigned long addr, unsigned long len) +bool mmap_address_hint_valid(struct mm_struct *mm, + unsigned long addr, unsigned long len) { if (TASK_SIZE - len < addr) return false; =20 + if (addr + len > DEFAULT_MAP_WINDOW && !full_va_allowed(mm)) + return false; + return (addr > DEFAULT_MAP_WINDOW) =3D=3D (addr + len > DEFAULT_MAP_WIN= DOW); } =20 --=20 2.26.2