From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB64BC43381 for ; Fri, 15 Jan 2021 15:46:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B5D6623128 for ; Fri, 15 Jan 2021 15:46:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733201AbhAOPqQ (ORCPT ); Fri, 15 Jan 2021 10:46:16 -0500 Received: from foss.arm.com ([217.140.110.172]:43290 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725910AbhAOPqQ (ORCPT ); Fri, 15 Jan 2021 10:46:16 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 457CBD6E; Fri, 15 Jan 2021 07:45:30 -0800 (PST) Received: from C02TD0UTHF1T.local (unknown [10.57.41.13]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 70C153F70D; Fri, 15 Jan 2021 07:45:24 -0800 (PST) Date: Fri, 15 Jan 2021 15:45:20 +0000 From: Mark Rutland To: Vincenzo Frascino Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov Subject: Re: [PATCH v3 4/4] arm64: mte: Optimize mte_assign_mem_tag_range() Message-ID: <20210115154520.GD44111@C02TD0UTHF1T.local> References: <20210115120043.50023-1-vincenzo.frascino@arm.com> <20210115120043.50023-5-vincenzo.frascino@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210115120043.50023-5-vincenzo.frascino@arm.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jan 15, 2021 at 12:00:43PM +0000, Vincenzo Frascino wrote: > mte_assign_mem_tag_range() is called on production KASAN HW hot > paths. It makes sense to optimize it in an attempt to reduce the > overhead. > > Optimize mte_assign_mem_tag_range() based on the indications provided at > [1]. ... what exactly is the optimization? I /think/ you're just trying to have it inlined, but you should mention that explicitly. > > [1] https://lore.kernel.org/r/CAAeHK+wCO+J7D1_T89DG+jJrPLk3X9RsGFKxJGd0ZcUFjQT-9Q@mail.gmail.com/ > > Cc: Catalin Marinas > Cc: Will Deacon > Signed-off-by: Vincenzo Frascino > --- > arch/arm64/include/asm/mte.h | 26 +++++++++++++++++++++++++- > arch/arm64/lib/mte.S | 15 --------------- > 2 files changed, 25 insertions(+), 16 deletions(-) > > diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h > index 1a715963d909..9730f2b07b79 100644 > --- a/arch/arm64/include/asm/mte.h > +++ b/arch/arm64/include/asm/mte.h > @@ -49,7 +49,31 @@ long get_mte_ctrl(struct task_struct *task); > int mte_ptrace_copy_tags(struct task_struct *child, long request, > unsigned long addr, unsigned long data); > > -void mte_assign_mem_tag_range(void *addr, size_t size); > +static inline void mte_assign_mem_tag_range(void *addr, size_t size) > +{ > + u64 _addr = (u64)addr; > + u64 _end = _addr + size; > + > + /* > + * This function must be invoked from an MTE enabled context. > + * > + * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and > + * size must be non-zero and MTE_GRANULE_SIZE aligned. > + */ > + do { > + /* > + * 'asm volatile' is required to prevent the compiler to move > + * the statement outside of the loop. > + */ > + asm volatile(__MTE_PREAMBLE "stg %0, [%0]" > + : > + : "r" (_addr) > + : "memory"); > + > + _addr += MTE_GRANULE_SIZE; > + } while (_addr < _end); Is there any chance that this can be used for the last bytes of the virtual address space? This might need to change to `_addr == _end` if that is possible, otherwise it'll terminate early in that case. > +} What does the code generation look like for this, relative to the assembly version? Thanks, Mark. > + > > #else /* CONFIG_ARM64_MTE */ > > diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S > index 9e1a12e10053..a0a650451510 100644 > --- a/arch/arm64/lib/mte.S > +++ b/arch/arm64/lib/mte.S > @@ -150,18 +150,3 @@ SYM_FUNC_START(mte_restore_page_tags) > ret > SYM_FUNC_END(mte_restore_page_tags) > > -/* > - * Assign allocation tags for a region of memory based on the pointer tag > - * x0 - source pointer > - * x1 - size > - * > - * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and > - * size must be non-zero and MTE_GRANULE_SIZE aligned. > - */ > -SYM_FUNC_START(mte_assign_mem_tag_range) > -1: stg x0, [x0] > - add x0, x0, #MTE_GRANULE_SIZE > - subs x1, x1, #MTE_GRANULE_SIZE > - b.gt 1b > - ret > -SYM_FUNC_END(mte_assign_mem_tag_range) > -- > 2.30.0 > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0747CC433DB for ; Fri, 15 Jan 2021 15:47:42 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B71BB2339D for ; Fri, 15 Jan 2021 15:47:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B71BB2339D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uUV4wRJzQJj/qNmkNXPecpGXFzhGowLS0VEri5v+5j8=; b=WXa0l9VyIYhks33e38lKsUNHI 4Lhq9fRhU1mvcKGBKL+LsigaAXpQZP49nNoEO+ZStOojTYXegBdg0P6rVzDA8C0+Bx3lZK5nhLdKg C9uPF1MfUg9jZq4fcgVnMk8ankWvxAi/ilA/k9rdoAIL+FZgzMALJhov/T/mR2QLg8jT0DA7Sa/cP JgVByL+TCQIJez2Vp9r9r7U3/REFEuYphF3hnGN73iP1yya4bmK4jqJjuPgtDs/2gwediA3bRcGZX dux9+9VdTPCDx8BO0z9sIAs0N8Ue9B/XfguOnJnrK9jESPGmkyKfmU0NMFuSUPWck7TpBQPba4myE bHZxVH2bw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l0RI6-0006Wm-De; Fri, 15 Jan 2021 15:45:34 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l0RI3-0006WM-UR for linux-arm-kernel@lists.infradead.org; Fri, 15 Jan 2021 15:45:32 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 457CBD6E; Fri, 15 Jan 2021 07:45:30 -0800 (PST) Received: from C02TD0UTHF1T.local (unknown [10.57.41.13]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 70C153F70D; Fri, 15 Jan 2021 07:45:24 -0800 (PST) Date: Fri, 15 Jan 2021 15:45:20 +0000 From: Mark Rutland To: Vincenzo Frascino Subject: Re: [PATCH v3 4/4] arm64: mte: Optimize mte_assign_mem_tag_range() Message-ID: <20210115154520.GD44111@C02TD0UTHF1T.local> References: <20210115120043.50023-1-vincenzo.frascino@arm.com> <20210115120043.50023-5-vincenzo.frascino@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210115120043.50023-5-vincenzo.frascino@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210115_104532_080275_FA871802 X-CRM114-Status: GOOD ( 25.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Marco Elver , Catalin Marinas , Evgenii Stepanov , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, Alexander Potapenko , linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Will Deacon , Dmitry Vyukov Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Jan 15, 2021 at 12:00:43PM +0000, Vincenzo Frascino wrote: > mte_assign_mem_tag_range() is called on production KASAN HW hot > paths. It makes sense to optimize it in an attempt to reduce the > overhead. > > Optimize mte_assign_mem_tag_range() based on the indications provided at > [1]. ... what exactly is the optimization? I /think/ you're just trying to have it inlined, but you should mention that explicitly. > > [1] https://lore.kernel.org/r/CAAeHK+wCO+J7D1_T89DG+jJrPLk3X9RsGFKxJGd0ZcUFjQT-9Q@mail.gmail.com/ > > Cc: Catalin Marinas > Cc: Will Deacon > Signed-off-by: Vincenzo Frascino > --- > arch/arm64/include/asm/mte.h | 26 +++++++++++++++++++++++++- > arch/arm64/lib/mte.S | 15 --------------- > 2 files changed, 25 insertions(+), 16 deletions(-) > > diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h > index 1a715963d909..9730f2b07b79 100644 > --- a/arch/arm64/include/asm/mte.h > +++ b/arch/arm64/include/asm/mte.h > @@ -49,7 +49,31 @@ long get_mte_ctrl(struct task_struct *task); > int mte_ptrace_copy_tags(struct task_struct *child, long request, > unsigned long addr, unsigned long data); > > -void mte_assign_mem_tag_range(void *addr, size_t size); > +static inline void mte_assign_mem_tag_range(void *addr, size_t size) > +{ > + u64 _addr = (u64)addr; > + u64 _end = _addr + size; > + > + /* > + * This function must be invoked from an MTE enabled context. > + * > + * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and > + * size must be non-zero and MTE_GRANULE_SIZE aligned. > + */ > + do { > + /* > + * 'asm volatile' is required to prevent the compiler to move > + * the statement outside of the loop. > + */ > + asm volatile(__MTE_PREAMBLE "stg %0, [%0]" > + : > + : "r" (_addr) > + : "memory"); > + > + _addr += MTE_GRANULE_SIZE; > + } while (_addr < _end); Is there any chance that this can be used for the last bytes of the virtual address space? This might need to change to `_addr == _end` if that is possible, otherwise it'll terminate early in that case. > +} What does the code generation look like for this, relative to the assembly version? Thanks, Mark. > + > > #else /* CONFIG_ARM64_MTE */ > > diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S > index 9e1a12e10053..a0a650451510 100644 > --- a/arch/arm64/lib/mte.S > +++ b/arch/arm64/lib/mte.S > @@ -150,18 +150,3 @@ SYM_FUNC_START(mte_restore_page_tags) > ret > SYM_FUNC_END(mte_restore_page_tags) > > -/* > - * Assign allocation tags for a region of memory based on the pointer tag > - * x0 - source pointer > - * x1 - size > - * > - * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and > - * size must be non-zero and MTE_GRANULE_SIZE aligned. > - */ > -SYM_FUNC_START(mte_assign_mem_tag_range) > -1: stg x0, [x0] > - add x0, x0, #MTE_GRANULE_SIZE > - subs x1, x1, #MTE_GRANULE_SIZE > - b.gt 1b > - ret > -SYM_FUNC_END(mte_assign_mem_tag_range) > -- > 2.30.0 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel