From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1947FC433DB for ; Fri, 5 Feb 2021 15:17:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 94B6B650E7 for ; Fri, 5 Feb 2021 15:17:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 94B6B650E7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E2A8C6B007D; Fri, 5 Feb 2021 10:16:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D76FC6B0080; Fri, 5 Feb 2021 10:16:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AF986B0080; Fri, 5 Feb 2021 10:16:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 26FF16B007E for ; Fri, 5 Feb 2021 10:16:52 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id EA3F4824999B for ; Fri, 5 Feb 2021 15:16:51 +0000 (UTC) X-FDA: 77784566622.08.wool44_5117f6c275e5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id CE5421819E76F for ; Fri, 5 Feb 2021 15:16:51 +0000 (UTC) X-HE-Tag: wool44_5117f6c275e5 X-Filterd-Recvd-Size: 7463 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Fri, 5 Feb 2021 15:16:50 +0000 (UTC) IronPort-SDR: NDo8plnyAsEVNkclG1FUYMVWOPfXPIAz8GTDcRAyBUnBp+cQOjIR0MUJEFxBYdyZkX4QuGc+QH b9r6rACr2X7Q== X-IronPort-AV: E=McAfee;i="6000,8403,9885"; a="242947489" X-IronPort-AV: E=Sophos;i="5.81,155,1610438400"; d="scan'208";a="242947489" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2021 07:16:48 -0800 IronPort-SDR: L6EeRbz108IYTNk9eBnTyBrwHatHHod7ZI3K9FukzPCs+cX2c3y9djaf9hX7BzNKfU83xCVG+5 gmDjzeP2MZlA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,155,1610438400"; d="scan'208";a="358274762" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga007.fm.intel.com with ESMTP; 05 Feb 2021 07:16:45 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 2207C411; Fri, 5 Feb 2021 17:16:41 +0200 (EET) From: "Kirill A. Shutemov" To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: x86@kernel.org, Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , "H . J . Lu" , Andi Kleen , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [RFC 9/9] x86/mm: Implement PR_SET/GET_TAGGED_ADDR_CTRL with LAM Date: Fri, 5 Feb 2021 18:16:30 +0300 Message-Id: <20210205151631.43511-11-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210205151631.43511-1-kirill.shutemov@linux.intel.com> References: <20210205151631.43511-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Provide prctl() interface to enabled LAM for user addresses. Depending how many tag bits requested it may result in enabling LAM_U57 or LAM_U48. If LAM_U48 is enabled, the process is no longer able to use full address space on 5-level paging machine and gets limited to 47-bit VA. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/processor.h | 10 +++ arch/x86/kernel/process_64.c | 145 +++++++++++++++++++++++++++++++ 2 files changed, 155 insertions(+) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proc= essor.h index 82a08b585818..49fac2cc4329 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -810,6 +810,16 @@ extern void start_thread(struct pt_regs *regs, unsig= ned long new_ip, extern int get_tsc_mode(unsigned long adr); extern int set_tsc_mode(unsigned int val); =20 +#ifdef CONFIG_X86_64 +long set_tagged_addr_ctrl(unsigned long flags, + int __user *nr_bits, int __user *offset); +long get_tagged_addr_ctrl(int __user *nr_bits, int __user *offset); +#define SET_TAGGED_ADDR_CTRL(flags, nr_bits, offset) \ + set_tagged_addr_ctrl(flags, nr_bits, offset) +#define GET_TAGGED_ADDR_CTRL(nr_bits, offset) \ + get_tagged_addr_ctrl(nr_bits, offset) +#endif + DECLARE_PER_CPU(u64, msr_misc_features_shadow); =20 #ifdef CONFIG_CPU_SUP_AMD diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index df342bedea88..99b87f0e1bc7 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -837,3 +837,148 @@ unsigned long KSTK_ESP(struct task_struct *task) { return task_pt_regs(task)->sp; } + +/* + * Control the relaxed ABI allowing tagged user addresses into the kerne= l. + */ +static unsigned int tagged_addr_disabled; + +static bool lam_u48_allowed(void) +{ + struct mm_struct *mm =3D current->mm; + + if (!full_va_allowed(mm)) + return true; + + return find_vma(mm, DEFAULT_MAP_WINDOW) =3D=3D NULL; +} + +#define LAM_U48_BITS 15 +#define LAM_U57_BITS 6 + +long set_tagged_addr_ctrl(unsigned long flags, + int __user *nr_bits, int __user *offset) +{ + int val; + + if (in_32bit_syscall()) + return -EINVAL; + if (flags & ~PR_TAGGED_ADDR_ENABLE) + return -EINVAL; + if (!boot_cpu_has(X86_FEATURE_LAM)) + return -ENOTSUPP; + + /* Disable LAM */ + if (!(flags & PR_TAGGED_ADDR_ENABLE)) { + clear_thread_flag(TIF_LAM_U48); + clear_thread_flag(TIF_LAM_U57); + + /* Update CR3 */ + switch_mm(current->mm, current->mm, current); + + return 0; + } + + /* + * nr_bits =3D=3D NULL || offset =3D=3D NULL assumes ARM TBI (nr_bits =3D= =3D 8, + * offset =3D=3D 56). LAM cannot provide this. + */ + if (!nr_bits || !offset) + return -EINVAL; + + /* + * Do not allow the enabling of the tagged address ABI if globally + * disabled via sysctl abi.tagged_addr_disabled. + */ + if (tagged_addr_disabled) + return -EINVAL; + + if (get_user(val, nr_bits)) + return -EFAULT; + if (val > LAM_U48_BITS || val < 1) + return -EINVAL; + if (val > LAM_U57_BITS && !lam_u48_allowed()) + return -EINVAL; + + val =3D val > LAM_U57_BITS ? LAM_U48_BITS : LAM_U57_BITS; + if (put_user(val, nr_bits) || put_user(63 - val, offset)) + return -EFAULT; + + if (val =3D=3D LAM_U57_BITS) { + clear_thread_flag(TIF_LAM_U48); + set_thread_flag(TIF_LAM_U57); + if (current->mm->context.lam =3D=3D LAM_NONE) + current->mm->context.lam =3D LAM_U57; + } else { + clear_thread_flag(TIF_LAM_U57); + set_thread_flag(TIF_LAM_U48); + + /* + * Do not allow to create a mapping above 47 bit. + * + * It's one way road: once a thread of the process enabled + * LAM_U48, no thread can ever create mapping above 47 bit. + * Even the LAM got disabled later. + */ + current->mm->context.lam =3D LAM_U48; + } + + /* Update CR3 */ + switch_mm(current->mm, current->mm, current); + + return 0; +} + +long get_tagged_addr_ctrl(int __user *nr_bits, int __user *offset) +{ + if (in_32bit_syscall()) + return -EINVAL; + + if (test_thread_flag(TIF_LAM_U57)) { + if (nr_bits && put_user(LAM_U57_BITS, nr_bits)) + return -EFAULT; + if (offset && put_user(63 - LAM_U57_BITS, offset)) + return -EFAULT; + } else if (test_thread_flag(TIF_LAM_U48)) { + if (nr_bits && put_user(LAM_U48_BITS, nr_bits)) + return -EFAULT; + if (offset && put_user(63 - LAM_U48_BITS, offset)) + return -EFAULT; + } else { + int max_bits =3D lam_u48_allowed() ? LAM_U48_BITS : LAM_U57_BITS; + + /* Report maximum tag size */ + if (nr_bits && put_user(max_bits, nr_bits)) + return -EFAULT; + return 0; + } + + return PR_TAGGED_ADDR_ENABLE; +} + +/* + * Global sysctl to disable the tagged user addresses support. This cont= rol + * only prevents the tagged address ABI enabling via prctl() and does no= t + * disable it for tasks that already opted in to the relaxed ABI. + */ + +static struct ctl_table tagged_addr_sysctl_table[] =3D { + { + .procname =3D "tagged_addr_disabled", + .mode =3D 0644, + .data =3D &tagged_addr_disabled, + .maxlen =3D sizeof(int), + .proc_handler =3D proc_dointvec_minmax, + .extra1 =3D SYSCTL_ZERO, + .extra2 =3D SYSCTL_ONE, + }, + { } +}; + +static int __init tagged_addr_init(void) +{ + if (!register_sysctl("abi", tagged_addr_sysctl_table)) + return -EINVAL; + return 0; +} +core_initcall(tagged_addr_init); --=20 2.26.2