From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79550C4727E for ; Thu, 1 Oct 2020 23:11:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2443B206C1 for ; Thu, 1 Oct 2020 23:11:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="JBmbjsh0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2443B206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BD83D6B005D; Thu, 1 Oct 2020 19:11:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B5FD76B0062; Thu, 1 Oct 2020 19:11:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B4616B00C6; Thu, 1 Oct 2020 19:11:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0001.hostedemail.com [216.40.44.1]) by kanga.kvack.org (Postfix) with ESMTP id 700D86B005D for ; Thu, 1 Oct 2020 19:11:51 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 117B33632 for ; Thu, 1 Oct 2020 23:11:51 +0000 (UTC) X-FDA: 77324906022.05.pies43_630d7252719f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id EAA4D18019540 for ; Thu, 1 Oct 2020 23:11:50 +0000 (UTC) X-HE-Tag: pies43_630d7252719f X-Filterd-Recvd-Size: 6637 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Thu, 1 Oct 2020 23:11:50 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id a2so125830wrp.8 for ; Thu, 01 Oct 2020 16:11:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=gnytaCb94xVQ7L/CojXY6Um8kg/YstpjCzFIZv8WTx8=; b=JBmbjsh06TRX2MzOuMJtipf1JeMyyK4+o+s0GwITV5q225JHL6TRmNrSwW/SwVnwpd 9pFyGXO4cBlZ1O9c8OKx/TtXAOlLIhePYK0EWrzxHQ1NYlTuc6OvKJFzu6h3x8YydlpQ /l5My7D/fx4LoDdyhfi5NYiUjs0f9ICkJfzi55QZC4HSGST1/gorGgLCHPIUS2/IamAc /5sZKV4Osw8xaa2Ls1j3ItngPHnHoTBUOKKLc8UvZN7E6VwTR9lH+CoBkL8bBT3gxoJR tfv/2Tir2xiYX4nerOc7jdK75KBUkrOWpW0WLXtVQ8Uc4LdDlduG0EtFfp3fb+hDeHqn HOHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gnytaCb94xVQ7L/CojXY6Um8kg/YstpjCzFIZv8WTx8=; b=JHAApivzLKyvYTQuRSDJ6R4d6x8GRWRJOsb8+/7a3Qg+34kPcwTvGuZ7OnjzV4B5qq ehQPkRvHG2NPGYVkz/Do9q6kZhcxR98E78JcJ2Gatq8kjsqg5oO81UfERADwxy5q0qCX ITLAdsVpdLp8iFUYt1MgIxk9w+E7rew/VbpJOwL1tM5Cx4+OSpAxj7ES8NQFZcS5Imch Xe+EeyGd/GF59Kzf5+CMTM0Myj4ndQR3yVJpBSerjjA5fIJn9a1OMqc8PtYVeyqBZLfr oJFmCiyZzggWh81TVxlZhMPj2ECYO1QPg88isu9Uli6hqvvVtX5XcI9gHZGE+uYdFYyr glDw== X-Gm-Message-State: AOAM533CFspcr2GYKrQpH5toGkwW1WRl4adPBuxqYrFjGeyLbsatgnMX K/cRzR21X8G8B/3GmyDa/XhStCafnDnY1Z8u X-Google-Smtp-Source: ABdhPJzJbo5CtXwA6Gi/sqrbZmN6s9dyBVmdhWMApSx7WyyHOrMaayQuq65zZXh83F1UYFJ51megLIxRwmYFJ7Gw X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:adf:cc8c:: with SMTP id p12mr11956869wrj.92.1601593909486; Thu, 01 Oct 2020 16:11:49 -0700 (PDT) Date: Fri, 2 Oct 2020 01:10:28 +0200 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.28.0.709.gb0816b6eb0-goog Subject: [PATCH v4 27/39] arm64: kasan: Enable in-kernel MTE From: Andrey Konovalov To: Dmitry Vyukov , Vincenzo Frascino , Catalin Marinas , kasan-dev@googlegroups.com Cc: Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Elena Petrova , Branislav Rankov , Kevin Brodsky , Will Deacon , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) feature and requires it to be enabled. The Tag Checking operation causes a synchronous data abort as a consequence of a tag check fault when MTE is configured in synchronous mode. Enable MTE in Synchronous mode in EL1 to provide a more immediate way of tag check failure detection in the kernel. As part of this change enable match-all tag for EL1 to allow the kernel to access user pages without faulting. This is required because the kernel does not have knowledge of the tags set by the user in a page. Note: For MTE, the TCF bit field in SCTLR_EL1 affects only EL1 in a similar way as TCF0 affects EL0. Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov --- Change-Id: I4d67497268bb7f0c2fc5dcacefa1e273df4af71d --- arch/arm64/kernel/cpufeature.c | 7 +++++++ arch/arm64/mm/proc.S | 23 ++++++++++++++++++++--- 2 files changed, 27 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index add9da5d8ea3..eca06b8c74db 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1718,6 +1718,13 @@ static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) cleared_zero_page = true; mte_clear_page_tags(lm_alias(empty_zero_page)); } + + /* Enable in-kernel MTE only if KASAN_HW_TAGS is enabled */ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) { + /* Enable MTE Sync Mode for EL1 */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + isb(); + } } #endif /* CONFIG_ARM64_MTE */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 23c326a06b2d..6c1a6621d769 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -40,9 +40,15 @@ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA #ifdef CONFIG_KASAN_SW_TAGS -#define TCR_KASAN_FLAGS TCR_TBI1 +#define TCR_KASAN_SW_FLAGS TCR_TBI1 #else -#define TCR_KASAN_FLAGS 0 +#define TCR_KASAN_SW_FLAGS 0 +#endif + +#ifdef CONFIG_KASAN_HW_TAGS +#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 +#else +#define TCR_KASAN_HW_FLAGS 0 #endif /* @@ -427,6 +433,10 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x5, MAIR_EL1_SET #ifdef CONFIG_ARM64_MTE + mte_tcr .req x20 + + mov mte_tcr, #0 + /* * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported * (ID_AA64PFR1_EL1[11:8] > 1). @@ -447,6 +457,9 @@ SYM_FUNC_START(__cpu_setup) /* clear any pending tag check faults in TFSR*_EL1 */ msr_s SYS_TFSR_EL1, xzr msr_s SYS_TFSRE0_EL1, xzr + + /* set the TCR_EL1 bits */ + mov_q mte_tcr, TCR_KASAN_HW_FLAGS 1: #endif msr mair_el1, x5 @@ -456,7 +469,11 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS + TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS +#ifdef CONFIG_ARM64_MTE + orr x10, x10, mte_tcr + .unreq mte_tcr +#endif tcr_clear_errata_bits x10, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52 -- 2.28.0.709.gb0816b6eb0-goog