From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C815C433F5 for ; Mon, 30 May 2022 14:06:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ASdWYvX0DkTMpyl4qStkrwNhutkLnPBcNa1wQmSxhgE=; b=APH0YfH2rq4XPG fX6cvx9w+CM1/Sn5lTGMDwLRtlX8E2mSHsRNGI/d6lCwowW3sO3sgVVyu7rc4Lgwr0xTH/8QVjcM7 kCYvAZ3YQRiL8hQGzxKz29ycEZYR8yFM7kl5PCxO7RWi8UJ4KNM/KRjt3pO+2WRJS4QwaX/dCbRMy 0B20Nje/eN9ZBMADAjR0/eINnPrHCrW5vsFeFvLRg/VINeZRuJTExjgzeWAoc+snWffJcq1HmqDMY PB6wIUrMgpjr+3JM+W1N6xXDrNnQG5HM8F1jlAQj4qv7pPdjR1wtNO4+c794RYTQJ/h82cJwDzqMX d3vmXbB+25HXG5biXNpw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nvg0n-0072O9-E2; Mon, 30 May 2022 14:04:50 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nvfgP-006qGY-Rm for linux-arm-kernel@lists.infradead.org; Mon, 30 May 2022 13:43:47 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5CEA7B80DB9; Mon, 30 May 2022 13:43:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 52701C385B8; Mon, 30 May 2022 13:43:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1653918222; bh=mkH+eDG1BW8aP4WRDNcQ5PhFkB9evQgyXJcnlgsnMxU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dGCGnNIMJ4ny52+lERu/tcrYDmwrxD3BTlqtmRV6EA/AxF95INPS+Izf6boBuelEK DO7yE1zl2ddmZEWjdy6YFbO4gSH3HUVdSoIrav/BpzYkR1QVJzjQ4JrQodAkbAGmLM uYH8ZdeHwmX8lhXaxySKbJoKaOAVju26xtqtJDGhDh1dAhfW6a72NNBKawpLd3/eSC 4xSKRiOcGUoWS4Y2sb6HxHoEjIZY1StWo/TzhVw39A1Lt34OGTOiJYlpsS4kGeGUHW EylRSIGNFSSBl3jpaBOJdJHJQufcUG02R4Xm5p2V1W5FimN0rlJbIxdO0yhQQkvQ7r x4J43Bro0AZVw== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Ard Biesheuvel , Russell King , Sasha Levin , linux@armlinux.org.uk, linus.walleij@linaro.org, nico@fluxnic.net, keithpac@amazon.com, arnd@arndb.de, linux-arm-kernel@lists.infradead.org Subject: [PATCH AUTOSEL 5.15 100/109] ARM: 9201/1: spectre-bhb: rely on linker to emit cross-section literal loads Date: Mon, 30 May 2022 09:38:16 -0400 Message-Id: <20220530133825.1933431-100-sashal@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220530133825.1933431-1-sashal@kernel.org> References: <20220530133825.1933431-1-sashal@kernel.org> MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220530_064346_282024_E95B6C40 X-CRM114-Status: GOOD ( 18.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel [ Upstream commit ad12c2f1587c6ec9b52ff226f438955bfae6ad89 ] The assembler does not permit 'LDR PC, ' when the symbol lives in a different section, which is why we have been relying on rather fragile open-coded arithmetic to load the address of the vector_swi routine into the program counter using a single LDR instruction in the SWI slot in the vector table. The literal was moved to a different section to in commit 19accfd373847 ("ARM: move vector stubs") to ensure that the vector stubs page does not need to be mapped readable for user space, which is the case for the vector page itself, as it carries the kuser helpers as well. So the cross-section literal load is open-coded, and this relies on the address of vector_swi to be at the very start of the vector stubs page, and we won't notice if we got it wrong until booting the kernel and see it break. Fortunately, it was guaranteed to break, so this was fragile but not problematic. Now that we have added two other variants of the vector table, we have 3 occurrences of the same trick, and so the size of our ISA/compiler/CPU validation space has tripled, in a way that may cause regressions to only be observed once booting the image in question on a CPU that exercises a particular vector table. So let's switch to true cross section references, and let the linker fix them up like it fixes up all the other cross section references in the vector page. Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King (Oracle) Signed-off-by: Sasha Levin --- arch/arm/kernel/entry-armv.S | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 68261a83b7ad..01c5b3aee22f 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -1069,10 +1069,15 @@ ENDPROC(vector_bhb_bpiall_\name) .endm .section .stubs, "ax", %progbits - @ This must be the first word + @ These need to remain at the start of the section so that + @ they are in range of the 'SWI' entries in the vector tables + @ located 4k down. +.L__vector_swi: .word vector_swi #ifdef CONFIG_HARDEN_BRANCH_HISTORY +.L__vector_bhb_loop8_swi: .word vector_bhb_loop8_swi +.L__vector_bhb_bpiall_swi: .word vector_bhb_bpiall_swi #endif @@ -1215,10 +1220,11 @@ vector_addrexcptn: .globl vector_fiq .section .vectors, "ax", %progbits -.L__vectors_start: W(b) vector_rst W(b) vector_und - W(ldr) pc, .L__vectors_start + 0x1000 +ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_swi ) +THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_swi ) + W(ldr) pc, . W(b) vector_pabt W(b) vector_dabt W(b) vector_addrexcptn @@ -1227,10 +1233,11 @@ vector_addrexcptn: #ifdef CONFIG_HARDEN_BRANCH_HISTORY .section .vectors.bhb.loop8, "ax", %progbits -.L__vectors_bhb_loop8_start: W(b) vector_rst W(b) vector_bhb_loop8_und - W(ldr) pc, .L__vectors_bhb_loop8_start + 0x1004 +ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_bhb_loop8_swi ) +THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_bhb_loop8_swi ) + W(ldr) pc, . W(b) vector_bhb_loop8_pabt W(b) vector_bhb_loop8_dabt W(b) vector_addrexcptn @@ -1238,10 +1245,11 @@ vector_addrexcptn: W(b) vector_bhb_loop8_fiq .section .vectors.bhb.bpiall, "ax", %progbits -.L__vectors_bhb_bpiall_start: W(b) vector_rst W(b) vector_bhb_bpiall_und - W(ldr) pc, .L__vectors_bhb_bpiall_start + 0x1008 +ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_bhb_bpiall_swi ) +THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_bhb_bpiall_swi ) + W(ldr) pc, . W(b) vector_bhb_bpiall_pabt W(b) vector_bhb_bpiall_dabt W(b) vector_addrexcptn -- 2.35.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53D53C4167B for ; Mon, 30 May 2022 14:21:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240820AbiE3OUb (ORCPT ); Mon, 30 May 2022 10:20:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240733AbiE3OQW (ORCPT ); Mon, 30 May 2022 10:16:22 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF2C58D695; Mon, 30 May 2022 06:43:48 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id D2D45B80AE8; Mon, 30 May 2022 13:43:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 52701C385B8; Mon, 30 May 2022 13:43:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1653918222; bh=mkH+eDG1BW8aP4WRDNcQ5PhFkB9evQgyXJcnlgsnMxU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dGCGnNIMJ4ny52+lERu/tcrYDmwrxD3BTlqtmRV6EA/AxF95INPS+Izf6boBuelEK DO7yE1zl2ddmZEWjdy6YFbO4gSH3HUVdSoIrav/BpzYkR1QVJzjQ4JrQodAkbAGmLM uYH8ZdeHwmX8lhXaxySKbJoKaOAVju26xtqtJDGhDh1dAhfW6a72NNBKawpLd3/eSC 4xSKRiOcGUoWS4Y2sb6HxHoEjIZY1StWo/TzhVw39A1Lt34OGTOiJYlpsS4kGeGUHW EylRSIGNFSSBl3jpaBOJdJHJQufcUG02R4Xm5p2V1W5FimN0rlJbIxdO0yhQQkvQ7r x4J43Bro0AZVw== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Ard Biesheuvel , Russell King , Sasha Levin , linux@armlinux.org.uk, linus.walleij@linaro.org, nico@fluxnic.net, keithpac@amazon.com, arnd@arndb.de, linux-arm-kernel@lists.infradead.org Subject: [PATCH AUTOSEL 5.15 100/109] ARM: 9201/1: spectre-bhb: rely on linker to emit cross-section literal loads Date: Mon, 30 May 2022 09:38:16 -0400 Message-Id: <20220530133825.1933431-100-sashal@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220530133825.1933431-1-sashal@kernel.org> References: <20220530133825.1933431-1-sashal@kernel.org> MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Ard Biesheuvel [ Upstream commit ad12c2f1587c6ec9b52ff226f438955bfae6ad89 ] The assembler does not permit 'LDR PC, ' when the symbol lives in a different section, which is why we have been relying on rather fragile open-coded arithmetic to load the address of the vector_swi routine into the program counter using a single LDR instruction in the SWI slot in the vector table. The literal was moved to a different section to in commit 19accfd373847 ("ARM: move vector stubs") to ensure that the vector stubs page does not need to be mapped readable for user space, which is the case for the vector page itself, as it carries the kuser helpers as well. So the cross-section literal load is open-coded, and this relies on the address of vector_swi to be at the very start of the vector stubs page, and we won't notice if we got it wrong until booting the kernel and see it break. Fortunately, it was guaranteed to break, so this was fragile but not problematic. Now that we have added two other variants of the vector table, we have 3 occurrences of the same trick, and so the size of our ISA/compiler/CPU validation space has tripled, in a way that may cause regressions to only be observed once booting the image in question on a CPU that exercises a particular vector table. So let's switch to true cross section references, and let the linker fix them up like it fixes up all the other cross section references in the vector page. Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King (Oracle) Signed-off-by: Sasha Levin --- arch/arm/kernel/entry-armv.S | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 68261a83b7ad..01c5b3aee22f 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -1069,10 +1069,15 @@ ENDPROC(vector_bhb_bpiall_\name) .endm .section .stubs, "ax", %progbits - @ This must be the first word + @ These need to remain at the start of the section so that + @ they are in range of the 'SWI' entries in the vector tables + @ located 4k down. +.L__vector_swi: .word vector_swi #ifdef CONFIG_HARDEN_BRANCH_HISTORY +.L__vector_bhb_loop8_swi: .word vector_bhb_loop8_swi +.L__vector_bhb_bpiall_swi: .word vector_bhb_bpiall_swi #endif @@ -1215,10 +1220,11 @@ vector_addrexcptn: .globl vector_fiq .section .vectors, "ax", %progbits -.L__vectors_start: W(b) vector_rst W(b) vector_und - W(ldr) pc, .L__vectors_start + 0x1000 +ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_swi ) +THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_swi ) + W(ldr) pc, . W(b) vector_pabt W(b) vector_dabt W(b) vector_addrexcptn @@ -1227,10 +1233,11 @@ vector_addrexcptn: #ifdef CONFIG_HARDEN_BRANCH_HISTORY .section .vectors.bhb.loop8, "ax", %progbits -.L__vectors_bhb_loop8_start: W(b) vector_rst W(b) vector_bhb_loop8_und - W(ldr) pc, .L__vectors_bhb_loop8_start + 0x1004 +ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_bhb_loop8_swi ) +THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_bhb_loop8_swi ) + W(ldr) pc, . W(b) vector_bhb_loop8_pabt W(b) vector_bhb_loop8_dabt W(b) vector_addrexcptn @@ -1238,10 +1245,11 @@ vector_addrexcptn: W(b) vector_bhb_loop8_fiq .section .vectors.bhb.bpiall, "ax", %progbits -.L__vectors_bhb_bpiall_start: W(b) vector_rst W(b) vector_bhb_bpiall_und - W(ldr) pc, .L__vectors_bhb_bpiall_start + 0x1008 +ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_bhb_bpiall_swi ) +THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_bhb_bpiall_swi ) + W(ldr) pc, . W(b) vector_bhb_bpiall_pabt W(b) vector_bhb_bpiall_dabt W(b) vector_addrexcptn -- 2.35.1