From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B644C43461 for ; Mon, 14 Sep 2020 15:20:31 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ED9F520829 for ; Mon, 14 Sep 2020 15:20:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Lgxf+ASb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED9F520829 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9VV6j7oxOGkWy0ho3yzJiZ6rOhU5dgw1WSohk4ouEN8=; b=Lgxf+ASbCJWE+7rAzPp/IY5XVB rl9QC65AB3XSZ18kHk8ZBgFqrDUUafCeFPu5R+HU7buJdi4pbCZ+WFoTNFcnvlQ9jCdOZHSxXxbxO rHmjplVeqnM9sxwtm7usQDRqKXTaI+aPIdwTpNSk8Iqga0uGPAwqKyOEqXWwwgSV/9fu4NNFCR60u J9NTEeY2dSUKo+qqnQTvLvrKrbMLzWJo012gsWfU3STCbyWH02MiVqh0O1hEs++bT+3UlqKnpVJyp tUnJU5CdzVSkAtvvI9XjH/pEyR0ZVhcDxmFQN/i69LByUAaUMb0/LC/USHav0rrKgEKV+MSGMWP1l C4WFgQ5Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kHqFG-0004Ja-Cr; Mon, 14 Sep 2020 15:18:18 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kHqF8-0004Ec-4Z for linux-arm-kernel@lists.infradead.org; Mon, 14 Sep 2020 15:18:14 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 64CBA106F; Mon, 14 Sep 2020 08:18:08 -0700 (PDT) Received: from seattle-bionic.arm.com.Home (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8531A3F718; Mon, 14 Sep 2020 08:18:07 -0700 (PDT) From: Oliver Swede To: catalin.marinas@arm.com, will@kernel.org Subject: [PATCH v5 01/14] arm64: Allow passing fault address to fixup handlers Date: Mon, 14 Sep 2020 15:17:47 +0000 Message-Id: <20200914151800.2270-2-oli.swede@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200914151800.2270-1-oli.swede@arm.com> References: <20200914151800.2270-1-oli.swede@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200914_111810_294420_8BCC6B22 X-CRM114-Status: GOOD ( 22.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Sam Tebbs Extend fixup_exception() to optionally place the faulting address in a register when returning to a fixup handler. Since A64 instructions must be 4-byte-aligned, we can mimic the IA-64 implementation and encode a flag in the lower bits of the offset field to indicate handlers which expect an address. This will allow us to use more efficient offset addressing modes in usercopy routines, rather than updating the base register on every access just for the sake of inferring where a fault occurred in order to compute the return value upon failure. The choice of x15 is somewhat arbitrary, but with the consideration that as the highest-numbered temporary register with no possible 'special' role in the ABI, it is most likely not used by hand-written assembly code, and thus a minimally-invasive option for imported routines. Signed-off-by: Sam Tebbs [ rm: split into separate patch, use UL(), expand commit message ] Signed-off-by: Robin Murphy [ os: move insertion to condition block for rebase onto bpf changes] Signed-off-by: Oliver Swede --- arch/arm64/include/asm/assembler.h | 9 +++++++++ arch/arm64/include/asm/extable.h | 11 ++++++++++- arch/arm64/mm/extable.c | 27 +++++++++++++++++---------- arch/arm64/mm/fault.c | 2 +- 4 files changed, 37 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 54d181177656..438382a277c8 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -129,6 +130,14 @@ alternative_endif .popsection .endm +/* + * Emit an entry into the exception table. + * The fixup handler will receive the faulting address in x15 + */ + .macro _asm_extable_faultaddr, from, to + _asm_extable \from, \to + FIXUP_WITH_ADDR + .endm + #define USER(l, x...) \ 9999: x; \ _asm_extable 9999b, l diff --git a/arch/arm64/include/asm/extable.h b/arch/arm64/include/asm/extable.h index 840a35ed92ec..2cfb3dce9a10 100644 --- a/arch/arm64/include/asm/extable.h +++ b/arch/arm64/include/asm/extable.h @@ -2,6 +2,12 @@ #ifndef __ASM_EXTABLE_H #define __ASM_EXTABLE_H +#include + +#define FIXUP_WITH_ADDR UL(1) + +#ifndef __ASSEMBLY__ + /* * The exception table consists of pairs of relative offsets: the first * is the relative offset to an instruction that is allowed to fault, @@ -34,5 +40,8 @@ int arm64_bpf_fixup_exception(const struct exception_table_entry *ex, } #endif /* !CONFIG_BPF_JIT */ -extern int fixup_exception(struct pt_regs *regs); +extern int fixup_exception(struct pt_regs *regs, unsigned long addr); + +#endif + #endif diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c index eee1732ab6cd..97802e1f178d 100644 --- a/arch/arm64/mm/extable.c +++ b/arch/arm64/mm/extable.c @@ -6,19 +6,26 @@ #include #include -int fixup_exception(struct pt_regs *regs) +int fixup_exception(struct pt_regs *regs, unsigned long addr) { const struct exception_table_entry *fixup; fixup = search_exception_tables(instruction_pointer(regs)); - if (!fixup) - return 0; - if (IS_ENABLED(CONFIG_BPF_JIT) && - regs->pc >= BPF_JIT_REGION_START && - regs->pc < BPF_JIT_REGION_END) - return arm64_bpf_fixup_exception(fixup, regs); - - regs->pc = (unsigned long)&fixup->fixup + fixup->fixup; - return 1; + if (fixup) { + if (IS_ENABLED(CONFIG_BPF_JIT) && + regs->pc >= BPF_JIT_REGION_START && + regs->pc < BPF_JIT_REGION_END) { + return arm64_bpf_fixup_exception(fixup, regs); + } else { + unsigned long offset = fixup->fixup; + if (offset & FIXUP_WITH_ADDR) { + regs->regs[15] = addr; + offset &= ~FIXUP_WITH_ADDR; + } + regs->pc = (unsigned long)&fixup->fixup + offset; + return 1; + } + } + return 0; } diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index f07333e86c2f..b9c263d18510 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -303,7 +303,7 @@ static void __do_kernel_fault(unsigned long addr, unsigned int esr, * Are we prepared to handle this kernel fault? * We are almost certainly not prepared to handle instruction faults. */ - if (!is_el1_instruction_abort(esr) && fixup_exception(regs)) + if (!is_el1_instruction_abort(esr) && fixup_exception(regs, addr)) return; if (WARN_RATELIMIT(is_spurious_el1_translation_fault(addr, esr, regs), -- 2.17.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel