From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B412DC433F5 for ; Mon, 6 Dec 2021 16:55:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JAoEqCQSX+e5Sg34j52KmKs/HZO8BhARDEeLZ6yiDPg=; b=ydhEvZaFLVqWLl 5aejukb8DtljIL/TMcvcAZRPry9OOXRDb7l+qa3aPb1361Z9WfE5tksyvGUEHLKyB6D0i0f0hSwID jMFr6sS8uCjbT6tY2DrhDX+UTK2h8U++PJRacSck3R98RMCzctShGnOmbvEA9M4r/9TbiJ1mX0tJi 1UZN/VabI1K+CCGzuk7i34AvH69g9rWeblKaxQZf5aXvflxNJJU3jR+iFjvM0VZ3RJTGZYpf+UbgS mRMs7IDQoNfcbO3m/OE6CiLRXL40KCQhDh7v1fSxkTeQmkBdimsu/ptOR3kWIw/X6t8plRMTf+HqJ vQWYNYakfY9KpypnbsMQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muHF4-004tIz-RO; Mon, 06 Dec 2021 16:53:31 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muH9Y-004qkQ-SG for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 16:47:50 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 92657B8117A; Mon, 6 Dec 2021 16:47:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B83C2C341C9; Mon, 6 Dec 2021 16:47:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1638809266; bh=i2j5W/7PduSe9218Yfcjy7D+7m93YWJEr/WsHJhSjwk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q4mOq2UoEOegRYq3/GJaXrN5/0H5j4eXa23vH8RflDur+pCkVVet6JDNT8pH8RJZI a4cEkwr2z9ywx0GGAmngKtghHWlQ/Vy7aHATjkLtDUWYVirTwK+Effk2hTG9+Y80iM erxzwFySpyEj6rdtoov5uPqYq7TAVo+F0pYSRIFoLa3wgKKqx0KTNVXSPLcsAyRX8w gHmptBuc6x9y+KbL/v0TnGnb5HFy8B91/0eLvNbDicRE3ZBAAA0/Cs6ccFwpylDBlg Fq0ySaM+xCcROlTjaQkp3OSmyWuicOR+ZEQiGjoY7cVYwB3PZJ3lDDH77RYtIqnPqQ +JD1XZtV4MW4w== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk Cc: Ard Biesheuvel , Nicolas Pitre , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Tony Lindgren , Marc Zyngier , Vladimir Murzin , Jesse Taube Subject: [PATCH v4 09/15] ARM: module: implement support for PC-relative group relocations Date: Mon, 6 Dec 2021 17:46:53 +0100 Message-Id: <20211206164659.1495084-10-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211206164659.1495084-1-ardb@kernel.org> References: <20211206164659.1495084-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5153; h=from:subject; bh=i2j5W/7PduSe9218Yfcjy7D+7m93YWJEr/WsHJhSjwk=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhrj52Qev8hLq8wuNPu8aoXWeDBlUyr6N4S1qrGEgl Q4fI2eqJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYa4+dgAKCRDDTyI5ktmPJP/PC/ 4vWaFkpqAns90m8Cuf8eBKeXuB0kerA7MraYNc09LgplyA5bnMyVOZGrX1SxETLNla7epB1aVebN3l wp6flWnBPnQfsvTG+XqTTVzKEWKH8Cts8ygRjHq6qwr0qJXrRRe43eUMo6nLWA83kCprcNi6BOF3eh 4bJBaKjAxeyGOjcwTncjFz1Uay3pV7vyMZKVKfHNmBfsYg0VZV7R9FnyQMN3WE3IZsshtPAQr4+fJv a5i2mklvLMLyXqNeO8KuRoH58eIoKEJRmUH4wh89znrqgCbjEcSJbjumCu+9vHztHNNzC7AjZMQw9H +qPX+aEfCEGLfO516sjHbuyvAUz2MJEgbEjySaZdBHzS7K/2+Kdcn9ehyCc/Sl35GqUZtUr8uERl0o t1e7lyEpCINpowAGJhbqMKbV3C9MToBe4h5491U1ATchI7Bbi1pBkdI56YoPI76v6AxCZLMfrTxAZj Ga+6qFNeqbJ/HbnCoqREQTlLBzfYPDaV0vxHH+U1x1Sls= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_084749_221609_C0ED8EF9 X-CRM114-Status: GOOD ( 22.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add support for the R_ARM_ALU_PC_Gn_NC and R_ARM_LDR_PC_G2 group relocations [0] so we can use them in modules. These will be used to load the current task pointer from a global variable without having to rely on a literal pool entry to carry the address of this variable, which may have a significant negative impact on cache utilization for variables that are used often and in many different places, as each occurrence will result in a literal pool entry and therefore a line in the D-cache. [0] 'ELF for the ARM architecture' https://github.com/ARM-software/abi-aa/releases Acked-by: Linus Walleij Acked-by: Nicolas Pitre Signed-off-by: Ard Biesheuvel Tested-by: Marc Zyngier Tested-by: Vladimir Murzin # ARMv7M --- arch/arm/include/asm/elf.h | 3 + arch/arm/kernel/module.c | 85 ++++++++++++++++++++ 2 files changed, 88 insertions(+) diff --git a/arch/arm/include/asm/elf.h b/arch/arm/include/asm/elf.h index b8102a6ddf16..d68101655b74 100644 --- a/arch/arm/include/asm/elf.h +++ b/arch/arm/include/asm/elf.h @@ -61,6 +61,9 @@ typedef struct user_fp elf_fpregset_t; #define R_ARM_MOVT_ABS 44 #define R_ARM_MOVW_PREL_NC 45 #define R_ARM_MOVT_PREL 46 +#define R_ARM_ALU_PC_G0_NC 57 +#define R_ARM_ALU_PC_G1_NC 59 +#define R_ARM_LDR_PC_G2 63 #define R_ARM_THM_CALL 10 #define R_ARM_THM_JUMP24 30 diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c index beac45e89ba6..4d33a7acf617 100644 --- a/arch/arm/kernel/module.c +++ b/arch/arm/kernel/module.c @@ -68,6 +68,42 @@ bool module_exit_section(const char *name) strstarts(name, ".ARM.exidx.exit"); } +/* + * This implements the partitioning algorithm for group relocations as + * documented in the ARM AArch32 ELF psABI (IHI 0044). + * + * A single PC-relative symbol reference is divided in up to 3 add or subtract + * operations, where the final one could be incorporated into a load/store + * instruction with immediate offset. E.g., + * + * ADD Rd, PC, #... or ADD Rd, PC, #... + * ADD Rd, Rd, #... ADD Rd, Rd, #... + * LDR Rd, [Rd, #...] ADD Rd, Rd, #... + * + * The latter has a guaranteed range of only 16 MiB (3x8 == 24 bits), so it is + * of limited use in the kernel. However, the ADD/ADD/LDR combo has a range of + * -/+ 256 MiB, (2x8 + 12 == 28 bits), which means it has sufficient range for + * any in-kernel symbol reference (unless module PLTs are being used). + * + * The main advantage of this approach over the typical pattern using a literal + * load is that literal loads may miss in the D-cache, and generally lead to + * lower cache efficiency for variables that are referenced often from many + * different places in the code. + */ +static u32 get_group_rem(u32 group, u32 *offset) +{ + u32 val = *offset; + u32 shift; + do { + shift = val ? (31 - __fls(val)) & ~1 : 32; + *offset = val; + if (!val) + break; + val &= 0xffffff >> shift; + } while (group--); + return shift; +} + int apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, unsigned int symindex, unsigned int relindex, struct module *module) @@ -82,6 +118,7 @@ apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, unsigned int symindex, unsigned long loc; Elf32_Sym *sym; const char *symname; + u32 shift, group = 1; s32 offset; u32 tmp; #ifdef CONFIG_THUMB2_KERNEL @@ -212,6 +249,54 @@ apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, unsigned int symindex, *(u32 *)loc = __opcode_to_mem_arm(tmp); break; + case R_ARM_ALU_PC_G0_NC: + group = 0; + fallthrough; + case R_ARM_ALU_PC_G1_NC: + tmp = __mem_to_opcode_arm(*(u32 *)loc); + offset = ror32(tmp & 0xff, (tmp & 0xf00) >> 7); + if (tmp & BIT(22)) + offset = -offset; + offset += sym->st_value - loc; + if (offset < 0) { + offset = -offset; + tmp = (tmp & ~BIT(23)) | BIT(22); // SUB opcode + } else { + tmp = (tmp & ~BIT(22)) | BIT(23); // ADD opcode + } + + shift = get_group_rem(group, &offset); + if (shift < 24) { + offset >>= 24 - shift; + offset |= (shift + 8) << 7; + } + *(u32 *)loc = __opcode_to_mem_arm((tmp & ~0xfff) | offset); + break; + + case R_ARM_LDR_PC_G2: + tmp = __mem_to_opcode_arm(*(u32 *)loc); + offset = tmp & 0xfff; + if (~tmp & BIT(23)) // U bit cleared? + offset = -offset; + offset += sym->st_value - loc; + if (offset < 0) { + offset = -offset; + tmp &= ~BIT(23); // clear U bit + } else { + tmp |= BIT(23); // set U bit + } + get_group_rem(2, &offset); + + if (offset > 0xfff) { + pr_err("%s: section %u reloc %u sym '%s': relocation %u out of range (%#lx -> %#x)\n", + module->name, relindex, i, symname, + ELF32_R_TYPE(rel->r_info), loc, + sym->st_value); + return -ENOEXEC; + } + *(u32 *)loc = __opcode_to_mem_arm((tmp & ~0xfff) | offset); + break; + #ifdef CONFIG_THUMB2_KERNEL case R_ARM_THM_CALL: case R_ARM_THM_JUMP24: -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel