[PATCH v4 0/3] modversions: Fix CRC mangling under CONFIG_RELOCATABLE=y

[PATCH v4 0/3] modversions: Fix CRC mangling under CONFIG_RELOCATABLE=y

[Ard Biesheuvel]

This series is a followup to the single patch 'modversions: treat symbol
CRCs as 32 bit quantities on 64 bit archs', of which three versions have
been sent out so far [0][1][2]

Given the recent issues regarding modversions, I have added some more
people to cc this time.

As pointed out by Michael, GNU ld behaves a bit differently between arm64
and PowerPC64, and where the former gets rid of all runtime relocations
related to CRCs, the latter is not as easily convinced.

Patch #1 fixes the issue where CRCs are corrupted by the runtime relocation
routines for 32-bit PowerPC, for which the original fix was effectively
reverted by commit 0e0ed6406e61 ("powerpc/modules: Module CRC relocation fix
causes perf issues")

Patch #2 adds handling of R_PPC64_ADDR32 relocations against the NULL .dynsym
symbol entry to the PPC64 runtime relocation routines, so it is prepared to
deal with CRCs being emitted as 32-bit quantities.

Patch #3 is the original patch from the v1 and v2 submissions.

Changes since v3:
- rebased onto v4.10-rc
- add Suzuki's ack to patch #1
- update patch #2 to work around the binutils-powerpc behavior to only
  account for R_PPC64_RELATIVE relocations in the RELACOUNT entry in
  .dynamic

Changes since v2:
- added #1 and #2
- updated #3 to deal with CRC entries being emitted from assembler
- added Rusty's ack (#3)

Branch can be found here:
https://git.kernel.org/cgit/linux/kernel/git/ardb/linux.git/log/?h=kcrctab-reloc

[0] http://marc.info/?l=linux-kernel&m=147652300207369&w=2
[1] http://marc.info/?l=linux-kernel&m=147695629614409&w=2
[2] http://marc.info/?l=linux-kernel&m=147758564705776&w=2

Ard Biesheuvel (3):
  powerpc/reloc32: fix corrupted modversion CRCs
  powerpc/reloc64: add support for 32-bit CRC pseudo-symbols
  modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

 arch/powerpc/include/asm/module.h |  4 --
 arch/powerpc/kernel/module_64.c   |  8 ---
 arch/powerpc/kernel/reloc_32.S    | 36 ++++++++++--
 arch/powerpc/kernel/reloc_64.S    | 60 ++++++++++++--------
 arch/powerpc/kernel/vmlinux.lds.S |  1 +
 arch/powerpc/relocs_check.sh      |  5 +-
 include/asm-generic/export.h      |  7 +--
 include/linux/export.h            |  8 +++
 include/linux/module.h            | 14 ++---
 kernel/module.c                   | 47 ++++++---------
 10 files changed, 105 insertions(+), 85 deletions(-)

-- 
2.7.4

[PATCH v4 1/3] powerpc/reloc32: fix corrupted modversion CRCs

[Ard Biesheuvel]

Commit 0e0ed6406e61 ("powerpc/modules: Module CRC relocation fix causes
perf issues") fixed an issue with relocatable PIE kernels in a way that
essentially reintroduced the issue again for 32-bit builds.

Since the chosen approach does is not applicable to 32-bit, fix the
issue by updating the runtime relocation routine to ignore the load
offset for the interval [__start___kcrctab, __stop___kcrctab_gpl_future),
which is where the CRCs reside. This ensures that the values of the CRC
pseudo-symbols are no longer made dependent on the runtime load offset.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/powerpc/kernel/reloc_32.S | 36 +++++++++++++++++---
 1 file changed, 32 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/kernel/reloc_32.S b/arch/powerpc/kernel/reloc_32.S
index f366fedb0872..150686b9febb 100644
--- a/arch/powerpc/kernel/reloc_32.S
+++ b/arch/powerpc/kernel/reloc_32.S
@@ -87,12 +87,12 @@ eodyn:				/* End of Dyn Table scan */
 	 * Work out the current offset from the link time address of .rela
 	 * section.
 	 *  cur_offset[r7] = rela.run[r9] - rela.link [r7]
-	 *  _stext.link[r12] = _stext.run[r10] - cur_offset[r7]
-	 *  final_offset[r3] = _stext.final[r3] - _stext.link[r12]
+	 *  _stext.link[r11] = _stext.run[r10] - cur_offset[r7]
+	 *  final_offset[r3] = _stext.final[r3] - _stext.link[r11]
 	 */
 	subf	r7, r7, r9	/* cur_offset */
-	subf	r12, r7, r10
-	subf	r3, r12, r3	/* final_offset */
+	subf	r11, r7, r10
+	subf	r3, r11, r3	/* final_offset */
 
 	subf	r8, r6, r8	/* relaz -= relaent */
 	/*
@@ -101,6 +101,21 @@ eodyn:				/* End of Dyn Table scan */
 	 * r13	- points to the symbol table
 	 */
 
+#ifdef CONFIG_MODVERSIONS
+	/*
+	 * Treat R_PPC_RELATIVE relocations differently when they target the
+	 * interval [__start___kcrctab, __stop___kcrctab_gpl_future): in this
+	 * case, the relocated quantities are CRC pseudo-symbols, which should
+	 * be preserved as-is, rather than be modified to take the runtime
+	 * offset into account.
+	 */
+	lwz	r10, (p_kcrc_start - 0b)(r12)
+	lwz	r11, (p_kcrc_stop - 0b)(r12)
+	subf	r12, r7, r12			/* link time addr of 0b */
+	add	r10, r10, r12
+	add	r11, r11, r12
+#endif
+
 	/*
 	 * Check if we have a relocation based on symbol
 	 * r5 will hold the value of the symbol.
@@ -135,7 +150,15 @@ get_type:
 	bne	hi16
 	lwz	r4, 0(r9)	/* r_offset */
 	lwz	r0, 8(r9)	/* r_addend */
+#ifdef CONFIG_MODVERSIONS
+	cmplw	r4, r10
+	blt	do_add
+	cmplw	r4, r11
+	blt	skip_add
+do_add:
+#endif
 	add	r0, r0, r3	/* final addend */
+skip_add:
 	stwx	r0, r4, r7	/* memory[r4+r7]) = (u32)r0 */
 	b	nxtrela		/* continue */
 
@@ -207,3 +230,8 @@ p_dyn:		.long	__dynamic_start - 0b
 p_rela:		.long	__rela_dyn_start - 0b
 p_sym:		.long	__dynamic_symtab - 0b
 p_st:		.long	_stext - 0b
+
+#ifdef CONFIG_MODVERSIONS
+p_kcrc_start:	.long	__start___kcrctab - 0b
+p_kcrc_stop:	.long	__stop___kcrctab_gpl_future - 0b
+#endif
-- 
2.7.4

[PATCH v4 2/3] powerpc/reloc64: add support for 32-bit CRC pseudo-symbols

[Ard Biesheuvel]

In preparation of modifying the core modversions code to emit the CRCs
as 32-bit quantities, ensure that 64-bit PowerPC will be able to deal
with this when CONFIG_RELOCATABLE=y, in which case the CRCs will be
emitted into the final ELF binary as R_PPC64_ADDR32 relocations.

Since 32-bit relocations cannot be used to relocate memory addresses on
64-bit architectures, and since the CRC pseudo-symbol references are
emitted as anonymous relocations (i.e., against the NULL symbol in the
.dynsym section) with the final value recorded in the addend (*), we
can disregard any relocations where the symbol index != 0.

* Note that unsatisfied CRC pseudo-symbol references are emitted as
  R_PPC64_ADDR32 relocations against named symbols that are typed as
  weak undefined in the .dynsym symbol table. These can simply be
  ignored (as before), considering that zero CRCs are interpreted as
  missing, and the module code deals with that accordingly.

As it turns out, binutils for powerpc does not account for any relocations
beyond R_PPC64_RELATIVE ones in the RELACOUNT field of the .dynamic section,
which is unfortunate, since we need to do extra work to figure out the size
of the relocation array. So with a little help from the linker scripts,
grab an end pointer rather than a count, and iterate over the entire section.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/powerpc/kernel/reloc_64.S    | 60 ++++++++++++--------
 arch/powerpc/kernel/vmlinux.lds.S |  1 +
 arch/powerpc/relocs_check.sh      |  5 +-
 3 files changed, 40 insertions(+), 26 deletions(-)

diff --git a/arch/powerpc/kernel/reloc_64.S b/arch/powerpc/kernel/reloc_64.S
index d88736fbece6..e50f5d778ea2 100644
--- a/arch/powerpc/kernel/reloc_64.S
+++ b/arch/powerpc/kernel/reloc_64.S
@@ -12,8 +12,8 @@
 #include <asm/ppc_asm.h>
 
 RELA = 7
-RELACOUNT = 0x6ffffff9
 R_PPC64_RELATIVE = 22
+R_PPC64_ADDR32 = 1
 
 /*
  * r3 = desired final address of kernel
@@ -29,29 +29,27 @@ _GLOBAL(relocate)
 	add	r9,r9,r12	/* r9 has runtime addr of .rela.dyn section */
 	ld	r10,(p_st - 0b)(r12)
 	add	r10,r10,r12	/* r10 has runtime addr of _stext */
+	ld	r8,(p_rela_end - 0b)(r12)
+	add	r8,r8,r12	/* r8 has addr of end of .rela.dyn section */
 
 	/*
-	 * Scan the dynamic section for the RELA and RELACOUNT entries.
+	 * Scan the dynamic section for the RELA entry.
+	 * NOTE: the RELACOUNT entry only covers R_PPC64_RELATIVE relocations,
+	 *       so we cannot use it here.
 	 */
 	li	r7,0
-	li	r8,0
 1:	ld	r6,0(r11)	/* get tag */
 	cmpdi	r6,0
-	beq	4f		/* end of list */
+	beq	3f		/* end of list */
 	cmpdi	r6,RELA
-	bne	2f
-	ld	r7,8(r11)	/* get RELA pointer in r7 */
-	b	3f
-2:	addis	r6,r6,(-RELACOUNT)@ha
-	cmpdi	r6,RELACOUNT@l
-	bne	3f
-	ld	r8,8(r11)	/* get RELACOUNT value in r8 */
-3:	addi	r11,r11,16
+	beq	2f
+	addi	r11,r11,16
 	b	1b
-4:	cmpdi	r7,0		/* check we have both RELA and RELACOUNT */
-	cmpdi	cr1,r8,0
-	beq	6f
-	beq	cr1,6f
+2:	ld	r7,8(r11)	/* get RELA pointer in r7 */
+3:	cmpdi	r7,0		/* check we have both RELA and a non-empty */
+	cmpd	cr1,r8,r9	/* .rela.dyn section			   */
+	beq	7f
+	beq	cr1,7f
 
 	/*
 	 * Work out linktime address of _stext and hence the
@@ -63,26 +61,40 @@ _GLOBAL(relocate)
 	subf	r7,r7,r9	/* cur_offset */
 	subf	r10,r7,r10
 	subf	r3,r10,r3	/* final_offset */
+	b	4f
 
 	/*
 	 * Run through the list of relocations and process the
-	 * R_PPC64_RELATIVE ones.
+	 * R_PPC64_RELATIVE and R_PPC64_ADDR32 ones.
 	 */
-	mtctr	r8
-5:	ld	r0,8(9)		/* ELF64_R_TYPE(reloc->r_info) */
+3:	addi	r9,r9,24
+4:	cmpd	r9,r8
+	beq	7f
+5:	ld	r0,8(9)		/* reloc->r_info (type *and* symbol index) */
+	ld	r6,0(r9)	/* reloc->r_offset */
 	cmpdi	r0,R_PPC64_RELATIVE
 	bne	6f
-	ld	r6,0(r9)	/* reloc->r_offset */
 	ld	r0,16(r9)	/* reloc->r_addend */
 	add	r0,r0,r3
 	stdx	r0,r7,r6
-	addi	r9,r9,24
-	bdnz	5b
+	b	3b
+
+	/*
+	 * CRCs of exported symbols are emitted as 32-bit relocations against
+	 * the NULL .dynsym entry, with the CRC value recorded in the addend.
+	 */
+6:	cmpdi	r0,R_PPC64_ADDR32
+	bne	3b
+	ld	r0,16(r9)	/* reloc->r_addend */
+	stwx	r0,r7,r6
+	b	3b
+
+7:	blr
 
-6:	blr
 
 .balign 8
 p_dyn:	.llong	__dynamic_start - 0b
 p_rela:	.llong	__rela_dyn_start - 0b
+p_rela_end:
+	.llong	__rela_dyn_end - 0b
 p_st:	.llong	_stext - 0b
-
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index 7394b770ae1f..654728fc860d 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -241,6 +241,7 @@ SECTIONS
 	{
 		__rela_dyn_start = .;
 		*(.rela*)
+		__rela_dyn_end = .;
 	}
 #endif
 	/* .exit.data is discarded at runtime, not link time,
diff --git a/arch/powerpc/relocs_check.sh b/arch/powerpc/relocs_check.sh
index ec2d5c835170..f9636b4e4548 100755
--- a/arch/powerpc/relocs_check.sh
+++ b/arch/powerpc/relocs_check.sh
@@ -30,7 +30,7 @@ bad_relocs=$(
 	# On PPC64:
 	#	R_PPC64_RELATIVE, R_PPC64_NONE
 	#	R_PPC64_ADDR64 mach_<name>
-	#	R_PPC64_ADDR64 __crc_<name>
+	#	R_PPC64_ADDR32 __crc_<name>
 	# On PPC:
 	#	R_PPC_RELATIVE, R_PPC_ADDR16_HI,
 	#	R_PPC_ADDR16_HA,R_PPC_ADDR16_LO,
@@ -43,7 +43,8 @@ R_PPC_ADDR16_HA
 R_PPC_RELATIVE
 R_PPC_NONE' |
 	grep -E -v '\<R_PPC64_ADDR64[[:space:]]+mach_' |
-	grep -E -v '\<R_PPC64_ADDR64[[:space:]]+__crc_'
+	grep -E -v '\<R_PPC64_ADDR32[[:space:]]+__crc_' |
+	grep -E -v '\<R_PPC64_ADDR32[[:space:]]+\*ABS\*'
 )
 
 if [ -z "$bad_relocs" ]; then
-- 
2.7.4

[PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Ard Biesheuvel]

The modversion symbol CRCs are emitted as ELF symbols, which allows us to
easily populate the kcrctab sections by relying on the linker to associate
each kcrctab slot with the correct value.

This has a couple of downsides:
- Given that the CRCs are treated as memory addresses, we waste 4 bytes
  for each CRC on 64 bit architectures,
- On architectures that support runtime relocation, a R_<arch>_RELATIVE
  relocation entry is emitted for each CRC value, which identifies it as
  a quantity that requires fixing up based on the actual runtime load
  offset of the kernel. This results in corrupted CRCs unless we
  explicitly undo the fixup (and this is currently being handled in the
  core module code)
- Such runtime relocation entries take up 24 bytes of __init space each,
  resulting in a x8 overhead in [uncompressed] kernel size for CRCs.

Switching to explicit 32 bit values on 64 bit architectures fixes most
of these issues, given that 32 bit values are not treated as quantities
that require fixing up based on the actual runtime load offset. Note that
on some ELF64 architectures [such as PPC64], these 32-bit values are still
emitted as runtime relocatable quantities, even if the value resolves to
a build time constant. However, these can easily be distinguished from
variables that do need fixing up, and the CRCs can be emitted correctly
in the arch specific runtime relocation routines right off the bat.

So redefine all CRC fields and variables as u32, and redefine the
__CRC_SYMBOL() macro for 64 bit builds to emit the CRC reference using
inline assembler (which is necessary since 64-bit C code cannot use
32-bit types to hold memory addresses, even if they are ultimately
resolved using values that do not exceed 0xffffffff.

Note that this mostly reverts commit d4703aefdbc8 ("module: handle ppc64
relocating kcrctabs when CONFIG_RELOCATABLE=y")

Acked-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/powerpc/include/asm/module.h |  4 --
 arch/powerpc/kernel/module_64.c   |  8 ----
 include/asm-generic/export.h      |  7 +--
 include/linux/export.h            |  8 ++++
 include/linux/module.h            | 14 +++---
 kernel/module.c                   | 47 +++++++-------------
 6 files changed, 33 insertions(+), 55 deletions(-)

diff --git a/arch/powerpc/include/asm/module.h b/arch/powerpc/include/asm/module.h
index cc12c61ef315..53885512b8d3 100644
--- a/arch/powerpc/include/asm/module.h
+++ b/arch/powerpc/include/asm/module.h
@@ -90,9 +90,5 @@ static inline int module_finalize_ftrace(struct module *mod, const Elf_Shdr *sec
 }
 #endif
 
-#if defined(CONFIG_MODVERSIONS) && defined(CONFIG_PPC64)
-#define ARCH_RELOCATES_KCRCTAB
-#define reloc_start PHYSICAL_START
-#endif
 #endif /* __KERNEL__ */
 #endif	/* _ASM_POWERPC_MODULE_H */
diff --git a/arch/powerpc/kernel/module_64.c b/arch/powerpc/kernel/module_64.c
index bb1807184bad..0b0f89685b67 100644
--- a/arch/powerpc/kernel/module_64.c
+++ b/arch/powerpc/kernel/module_64.c
@@ -286,14 +286,6 @@ static void dedotify_versions(struct modversion_info *vers,
 	for (end = (void *)vers + size; vers < end; vers++)
 		if (vers->name[0] == '.') {
 			memmove(vers->name, vers->name+1, strlen(vers->name));
-#ifdef ARCH_RELOCATES_KCRCTAB
-			/* The TOC symbol has no CRC computed. To avoid CRC
-			 * check failing, we must force it to the expected
-			 * value (see CRC check in module.c).
-			 */
-			if (!strcmp(vers->name, "TOC."))
-				vers->crc = -(unsigned long)reloc_start;
-#endif
 		}
 }
 
diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 63554e9f6e0c..e3508a3d6e53 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -9,18 +9,15 @@
 #ifndef KSYM_ALIGN
 #define KSYM_ALIGN 8
 #endif
-#ifndef KCRC_ALIGN
-#define KCRC_ALIGN 8
-#endif
 #else
 #define __put .long
 #ifndef KSYM_ALIGN
 #define KSYM_ALIGN 4
 #endif
+#endif
 #ifndef KCRC_ALIGN
 #define KCRC_ALIGN 4
 #endif
-#endif
 
 #ifdef CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX
 #define KSYM(name) _##name
@@ -52,7 +49,7 @@ KSYM(__kstrtab_\name):
 	.section ___kcrctab\sec+\name,"a"
 	.balign KCRC_ALIGN
 KSYM(__kcrctab_\name):
-	__put KSYM(__crc_\name)
+	.long KSYM(__crc_\name)
 	.weak KSYM(__crc_\name)
 	.previous
 #endif
diff --git a/include/linux/export.h b/include/linux/export.h
index 2a0f61fbc731..a000d421526d 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -41,6 +41,7 @@ extern struct module __this_module;
 
 #if defined(__KERNEL__) && !defined(__GENKSYMS__)
 #ifdef CONFIG_MODVERSIONS
+#ifndef CONFIG_64BIT
 /* Mark the CRC weak since genksyms apparently decides not to
  * generate a checksums for some symbols */
 #define __CRC_SYMBOL(sym, sec)						\
@@ -50,6 +51,13 @@ extern struct module __this_module;
 	__attribute__((section("___kcrctab" sec "+" #sym), used))	\
 	= (unsigned long) &__crc_##sym;
 #else
+#define __CRC_SYMBOL(sym, sec)						\
+	asm("	.section \"___kcrctab" sec "+" #sym "\", \"a\"	\n"	\
+	    "	.weak	" VMLINUX_SYMBOL_STR(__crc_##sym) "	\n"	\
+	    "	.long	" VMLINUX_SYMBOL_STR(__crc_##sym) "	\n"	\
+	    "	.previous					\n");
+#endif
+#else
 #define __CRC_SYMBOL(sym, sec)
 #endif
 
diff --git a/include/linux/module.h b/include/linux/module.h
index 7c84273d60b9..4b65b9df3de2 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -346,7 +346,7 @@ struct module {
 
 	/* Exported symbols */
 	const struct kernel_symbol *syms;
-	const unsigned long *crcs;
+	const u32 *crcs;
 	unsigned int num_syms;
 
 	/* Kernel parameters. */
@@ -359,18 +359,18 @@ struct module {
 	/* GPL-only exported symbols. */
 	unsigned int num_gpl_syms;
 	const struct kernel_symbol *gpl_syms;
-	const unsigned long *gpl_crcs;
+	const u32 *gpl_crcs;
 
 #ifdef CONFIG_UNUSED_SYMBOLS
 	/* unused exported symbols. */
 	const struct kernel_symbol *unused_syms;
-	const unsigned long *unused_crcs;
+	const u32 *unused_crcs;
 	unsigned int num_unused_syms;
 
 	/* GPL-only, unused exported symbols. */
 	unsigned int num_unused_gpl_syms;
 	const struct kernel_symbol *unused_gpl_syms;
-	const unsigned long *unused_gpl_crcs;
+	const u32 *unused_gpl_crcs;
 #endif
 
 #ifdef CONFIG_MODULE_SIG
@@ -382,7 +382,7 @@ struct module {
 
 	/* symbols that will be GPL-only in the near future. */
 	const struct kernel_symbol *gpl_future_syms;
-	const unsigned long *gpl_future_crcs;
+	const u32 *gpl_future_crcs;
 	unsigned int num_gpl_future_syms;
 
 	/* Exception table */
@@ -523,7 +523,7 @@ struct module *find_module(const char *name);
 
 struct symsearch {
 	const struct kernel_symbol *start, *stop;
-	const unsigned long *crcs;
+	const u32 *crcs;
 	enum {
 		NOT_GPL_ONLY,
 		GPL_ONLY,
@@ -539,7 +539,7 @@ struct symsearch {
  */
 const struct kernel_symbol *find_symbol(const char *name,
 					struct module **owner,
-					const unsigned long **crc,
+					const u32 **crc,
 					bool gplok,
 					bool warn);
 
diff --git a/kernel/module.c b/kernel/module.c
index 5088784c0cf9..43d1478975a1 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -389,16 +389,16 @@ extern const struct kernel_symbol __start___ksymtab_gpl[];
 extern const struct kernel_symbol __stop___ksymtab_gpl[];
 extern const struct kernel_symbol __start___ksymtab_gpl_future[];
 extern const struct kernel_symbol __stop___ksymtab_gpl_future[];
-extern const unsigned long __start___kcrctab[];
-extern const unsigned long __start___kcrctab_gpl[];
-extern const unsigned long __start___kcrctab_gpl_future[];
+extern const u32 __start___kcrctab[];
+extern const u32 __start___kcrctab_gpl[];
+extern const u32 __start___kcrctab_gpl_future[];
 #ifdef CONFIG_UNUSED_SYMBOLS
 extern const struct kernel_symbol __start___ksymtab_unused[];
 extern const struct kernel_symbol __stop___ksymtab_unused[];
 extern const struct kernel_symbol __start___ksymtab_unused_gpl[];
 extern const struct kernel_symbol __stop___ksymtab_unused_gpl[];
-extern const unsigned long __start___kcrctab_unused[];
-extern const unsigned long __start___kcrctab_unused_gpl[];
+extern const u32 __start___kcrctab_unused[];
+extern const u32 __start___kcrctab_unused_gpl[];
 #endif
 
 #ifndef CONFIG_MODVERSIONS
@@ -497,7 +497,7 @@ struct find_symbol_arg {
 
 	/* Output */
 	struct module *owner;
-	const unsigned long *crc;
+	const u32 *crc;
 	const struct kernel_symbol *sym;
 };
 
@@ -563,7 +563,7 @@ static bool find_symbol_in_section(const struct symsearch *syms,
  * (optional) module which owns it.  Needs preempt disabled or module_mutex. */
 const struct kernel_symbol *find_symbol(const char *name,
 					struct module **owner,
-					const unsigned long **crc,
+					const u32 **crc,
 					bool gplok,
 					bool warn)
 {
@@ -1249,23 +1249,11 @@ static int try_to_force_load(struct module *mod, const char *reason)
 }
 
 #ifdef CONFIG_MODVERSIONS
-/* If the arch applies (non-zero) relocations to kernel kcrctab, unapply it. */
-static unsigned long maybe_relocated(unsigned long crc,
-				     const struct module *crc_owner)
-{
-#ifdef ARCH_RELOCATES_KCRCTAB
-	if (crc_owner == NULL)
-		return crc - (unsigned long)reloc_start;
-#endif
-	return crc;
-}
-
 static int check_version(Elf_Shdr *sechdrs,
 			 unsigned int versindex,
 			 const char *symname,
 			 struct module *mod,
-			 const unsigned long *crc,
-			 const struct module *crc_owner)
+			 const u32 *crc)
 {
 	unsigned int i, num_versions;
 	struct modversion_info *versions;
@@ -1286,10 +1274,10 @@ static int check_version(Elf_Shdr *sechdrs,
 		if (strcmp(versions[i].name, symname) != 0)
 			continue;
 
-		if (versions[i].crc == maybe_relocated(*crc, crc_owner))
+		if (versions[i].crc == *crc)
 			return 1;
-		pr_debug("Found checksum %lX vs module %lX\n",
-		       maybe_relocated(*crc, crc_owner), versions[i].crc);
+		pr_debug("Found checksum %X vs module %lX\n",
+		       *crc, versions[i].crc);
 		goto bad_version;
 	}
 
@@ -1307,7 +1295,7 @@ static inline int check_modstruct_version(Elf_Shdr *sechdrs,
 					  unsigned int versindex,
 					  struct module *mod)
 {
-	const unsigned long *crc;
+	const u32 *crc;
 
 	/*
 	 * Since this should be found in kernel (which can't be removed), no
@@ -1321,8 +1309,7 @@ static inline int check_modstruct_version(Elf_Shdr *sechdrs,
 	}
 	preempt_enable();
 	return check_version(sechdrs, versindex,
-			     VMLINUX_SYMBOL_STR(module_layout), mod, crc,
-			     NULL);
+			     VMLINUX_SYMBOL_STR(module_layout), mod, crc);
 }
 
 /* First part is kernel version, which we ignore if module has crcs. */
@@ -1340,8 +1327,7 @@ static inline int check_version(Elf_Shdr *sechdrs,
 				unsigned int versindex,
 				const char *symname,
 				struct module *mod,
-				const unsigned long *crc,
-				const struct module *crc_owner)
+				const u32 *crc)
 {
 	return 1;
 }
@@ -1368,7 +1354,7 @@ static const struct kernel_symbol *resolve_symbol(struct module *mod,
 {
 	struct module *owner;
 	const struct kernel_symbol *sym;
-	const unsigned long *crc;
+	const u32 *crc;
 	int err;
 
 	/*
@@ -1383,8 +1369,7 @@ static const struct kernel_symbol *resolve_symbol(struct module *mod,
 	if (!sym)
 		goto unlock;
 
-	if (!check_version(info->sechdrs, info->index.vers, name, mod, crc,
-			   owner)) {
+	if (!check_version(info->sechdrs, info->index.vers, name, mod, crc)) {
 		sym = ERR_PTR(-EINVAL);
 		goto getname;
 	}
-- 
2.7.4

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Linus Torvalds]

On Tue, Jan 17, 2017 at 11:26 AM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> The modversion symbol CRCs are emitted as ELF symbols, which allows us to
> easily populate the kcrctab sections by relying on the linker to associate
> each kcrctab slot with the correct value.
>
> This has a couple of downsides:

So the whole relocation of the crc is obviously completely crazy, but
you don't actually seem to *change* that. You just work around it, and
you make the symbols 32-bit. The latter part I agree with
whole-heartedly, btw.

But do we really have to accept this relocation insanity?

So I don't actually disagree with this patch 3/3 (turning the whole
crc array into an array of "u32" is clearly the right thing to do),
but the two other patches look oddly broken.

Why are those CRC's relocatable to begin with? Shouldn't they be
absolute values? Why do they get those idiotic relocation things? They
seem to be absolute on x86-64 (just doing a 'nm vmlinux', so I might
be missing something), why aren't they on ppc?

Is there something wrong with our generation script? Can we possibly
do something like

  -       printf("%s__crc_%s = 0x%08lx ;\n", mod_prefix, name, crc);
  +       printf("%s__crc_%s = ABSOLUTE(0x%08lx) ;\n", mod_prefix, name, crc);

in genksyms.c to get rid of the crazty relocation entries?

             Linus

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Ard Biesheuvel]

On 17 January 2017 at 23:47, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Tue, Jan 17, 2017 at 11:26 AM, Ard Biesheuvel
> <ard.biesheuvel@linaro.org> wrote:
>> The modversion symbol CRCs are emitted as ELF symbols, which allows us to
>> easily populate the kcrctab sections by relying on the linker to associate
>> each kcrctab slot with the correct value.
>>
>> This has a couple of downsides:
>
> So the whole relocation of the crc is obviously completely crazy, but
> you don't actually seem to *change* that. You just work around it, and
> you make the symbols 32-bit. The latter part I agree with
> whole-heartedly, btw.
>
> But do we really have to accept this relocation insanity?
>
> So I don't actually disagree with this patch 3/3 (turning the whole
> crc array into an array of "u32" is clearly the right thing to do),
> but the two other patches look oddly broken.
>
> Why are those CRC's relocatable to begin with? Shouldn't they be
> absolute values? Why do they get those idiotic relocation things? They
> seem to be absolute on x86-64 (just doing a 'nm vmlinux', so I might
> be missing something), why aren't they on ppc?
>

On powerpc and arm64, those __crc_xxx symbols are absolute as well,
but oddly enough, that does not mean they are not subject to runtime
relocation under -pie, which is how arm64 and powerpc create their
relocatable kernel images. The difference with x86 is that they
invented their own tooling to do runtime relocation, based on
--emit-relocs and filtering based on symbol names, so they don't rely
on ELF relocations at all for runtime relocation of the core kernel.

On ppc64:

$ nm vmlinux |grep __crc_ |head
000000009d37922d a __crc___ablkcipher_walk_complete
00000000c4309a46 a __crc_ablkcipher_walk_done
0000000038e1d7e1 a __crc_ablkcipher_walk_phys
00000000a55b33a4 a __crc_abort_creds
000000005776482e a __crc_access_process_vm
000000001215a9fb a __crc_account_page_dirtied
00000000b25ee3c8 a __crc_account_page_redirty
00000000ccab9422 a __crc_ack_all_badblocks
0000000027013bae a __crc_acomp_request_alloc
0000000013236c1b a __crc_acomp_request_free

[note the lowercase 'a', this is due to my attempt at emitting them as HIDDEN()]

On arm64, patch 3/3 is sufficient, because the linker infers from the
size of the symbol reference that it is not a memory address, and
resolves the relocation at link time.

> Is there something wrong with our generation script? Can we possibly
> do something like
>
>   -       printf("%s__crc_%s = 0x%08lx ;\n", mod_prefix, name, crc);
>   +       printf("%s__crc_%s = ABSOLUTE(0x%08lx) ;\n", mod_prefix, name, crc);
>
> in genksyms.c to get rid of the crazty relocation entries?
>

Nope, no difference at all, given that the symbols were ABSOLUTE to
begin with. Emitting them as HIDDEN() does not help either, nor does
passing -Bsymbolic on the linker command line.

So on powerpc, the linker insists on emitting the relocation into the
runtime relocation section [*], regardless of whether it is ABSOLUTE()
or HIDDEN(), or whether -Bsymbolic is being passed. My suspicion is
that, due to the fact that PIE handling is closely related to shared
library handling, the linker defers the resolution of relocations
against __crc_xxx symbols to runtime because they are preemptible
under ELF rules for shared libraries, i.e., an application linking to
a shared library is able to override symbols that the shared library
defines, and the shared library *must* update its internal references
to point to the new version of the symbol. Of course, this makes no
sense at all for PIE binaries, and on arm64, the toolchain does the
right thing in this regard when passing the -Bsymbolic parameter. On
powerpc, however, the linker *insists* on exposing these relocations
to the runtime loader, even if they are marked hidden (which is
supposed to inhibit this behavior).

I have also tried using relative references (which always get resolved
at link time), e.g.,

diff --git a/include/linux/export.h b/include/linux/export.h
index a000d421526d..df3f6762b3c0 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -54,7 +54,9 @@ extern struct module __this_module;
 #define __CRC_SYMBOL(sym, sec)                                         \
        asm("   .section \"___kcrctab" sec "+" #sym "\", \"a\"  \n"     \
            "   .weak   " VMLINUX_SYMBOL_STR(__crc_##sym) "     \n"     \
-           "   .long   " VMLINUX_SYMBOL_STR(__crc_##sym) "     \n"     \
+           "   .globl  " VMLINUX_SYMBOL_STR(__crc_rel_##sym) " \n"     \
+           VMLINUX_SYMBOL_STR(__crc_rel_##sym)":
 \n"     \
+           "   .long   " VMLINUX_SYMBOL_STR(__crc_##sym) " - . \n"     \
            "   .previous                                       \n");
 #endif
 #else
diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c
index 06121ce524a7..8dd9f1da8898 100644
--- a/scripts/genksyms/genksyms.c
+++ b/scripts/genksyms/genksyms.c
@@ -693,7 +693,8 @@ void export_symbol(const char *name)
                        fputs(">\n", debugfile);

                /* Used as a linker script. */
-               printf("%s__crc_%s = 0x%08lx ;\n", mod_prefix, name, crc);
+               printf("%s__crc_%s = %s__crc_rel_%s + 0x%08lx;\n", mod_prefix,
+                      name, mod_prefix, name, crc);
        }
 }

but this doesn't work either: the __crc_xxx symbols that are emitted
during partial linking evaluate the __crc_rel_xxx references at
partial link time, which means the resulting relative relocations
refer to the actual CRC value rather than the modified CRC value. The
only way to make /this/ work, afaik, is to hack the build scripts so
that the __crc_xxx = assignments occur in the scope of the final
linker script, rather than during partial linking (but only for
vmlinux, not for modules). All of this complicates the common path
used by all architectures, so I don't think we should go down this
road.

So I don't see any other way to work around this for powerpc (other
than creating some build time tooling to process the 32-bit
relocations for the CRC symbols in the vmlinux ELF binary) Hopefully,
Michael will be able to confirm that this v4 of 2/3 works correctly,
then we can discuss whether to go ahead with this or not.

-- 
Ard.

[*] and sadly, it only counts R_PPC64_RELATIVE relocations in its
.dynamic section's RELACOUNT, which requires additional handling in
patch 2/3

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Ard Biesheuvel]

On 18 January 2017 at 11:37, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 17 January 2017 at 23:47, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>> On Tue, Jan 17, 2017 at 11:26 AM, Ard Biesheuvel
>> <ard.biesheuvel@linaro.org> wrote:
>>> The modversion symbol CRCs are emitted as ELF symbols, which allows us to
>>> easily populate the kcrctab sections by relying on the linker to associate
>>> each kcrctab slot with the correct value.
>>>
>>> This has a couple of downsides:
>>
>> So the whole relocation of the crc is obviously completely crazy, but
>> you don't actually seem to *change* that. You just work around it, and
>> you make the symbols 32-bit. The latter part I agree with
>> whole-heartedly, btw.
>>
>> But do we really have to accept this relocation insanity?
>>
>> So I don't actually disagree with this patch 3/3 (turning the whole
>> crc array into an array of "u32" is clearly the right thing to do),
>> but the two other patches look oddly broken.
>>
>> Why are those CRC's relocatable to begin with? Shouldn't they be
>> absolute values? Why do they get those idiotic relocation things? They
>> seem to be absolute on x86-64 (just doing a 'nm vmlinux', so I might
>> be missing something), why aren't they on ppc?
>>
>
> On powerpc and arm64, those __crc_xxx symbols are absolute as well,
> but oddly enough, that does not mean they are not subject to runtime
> relocation under -pie, which is how arm64 and powerpc create their
> relocatable kernel images.

It turns out that this odd treatment of absolute symbols (i.e.,
symbols having section number SHN_ABS) is a known issue in GNU ld

https://sourceware.org/ml/binutils/2012-05/msg00019.html


> The difference with x86 is that they
> invented their own tooling to do runtime relocation, based on
> --emit-relocs and filtering based on symbol names, so they don't rely
> on ELF relocations at all for runtime relocation of the core kernel.
>
> On ppc64:
>
> $ nm vmlinux |grep __crc_ |head
> 000000009d37922d a __crc___ablkcipher_walk_complete
> 00000000c4309a46 a __crc_ablkcipher_walk_done
> 0000000038e1d7e1 a __crc_ablkcipher_walk_phys
> 00000000a55b33a4 a __crc_abort_creds
> 000000005776482e a __crc_access_process_vm
> 000000001215a9fb a __crc_account_page_dirtied
> 00000000b25ee3c8 a __crc_account_page_redirty
> 00000000ccab9422 a __crc_ack_all_badblocks
> 0000000027013bae a __crc_acomp_request_alloc
> 0000000013236c1b a __crc_acomp_request_free
>
> [note the lowercase 'a', this is due to my attempt at emitting them as HIDDEN()]
>
> On arm64, patch 3/3 is sufficient, because the linker infers from the
> size of the symbol reference that it is not a memory address, and
> resolves the relocation at link time.
>
>> Is there something wrong with our generation script? Can we possibly
>> do something like
>>
>>   -       printf("%s__crc_%s = 0x%08lx ;\n", mod_prefix, name, crc);
>>   +       printf("%s__crc_%s = ABSOLUTE(0x%08lx) ;\n", mod_prefix, name, crc);
>>
>> in genksyms.c to get rid of the crazty relocation entries?
>>
>
> Nope, no difference at all, given that the symbols were ABSOLUTE to
> begin with. Emitting them as HIDDEN() does not help either, nor does
> passing -Bsymbolic on the linker command line.
>
> So on powerpc, the linker insists on emitting the relocation into the
> runtime relocation section [*], regardless of whether it is ABSOLUTE()
> or HIDDEN(), or whether -Bsymbolic is being passed. My suspicion is
> that, due to the fact that PIE handling is closely related to shared
> library handling, the linker defers the resolution of relocations
> against __crc_xxx symbols to runtime because they are preemptible
> under ELF rules for shared libraries, i.e., an application linking to
> a shared library is able to override symbols that the shared library
> defines, and the shared library *must* update its internal references
> to point to the new version of the symbol. Of course, this makes no
> sense at all for PIE binaries, and on arm64, the toolchain does the
> right thing in this regard when passing the -Bsymbolic parameter. On
> powerpc, however, the linker *insists* on exposing these relocations
> to the runtime loader, even if they are marked hidden (which is
> supposed to inhibit this behavior).
>
> I have also tried using relative references (which always get resolved
> at link time), e.g.,
>
> diff --git a/include/linux/export.h b/include/linux/export.h
> index a000d421526d..df3f6762b3c0 100644
> --- a/include/linux/export.h
> +++ b/include/linux/export.h
> @@ -54,7 +54,9 @@ extern struct module __this_module;
>  #define __CRC_SYMBOL(sym, sec)                                         \
>         asm("   .section \"___kcrctab" sec "+" #sym "\", \"a\"  \n"     \
>             "   .weak   " VMLINUX_SYMBOL_STR(__crc_##sym) "     \n"     \
> -           "   .long   " VMLINUX_SYMBOL_STR(__crc_##sym) "     \n"     \
> +           "   .globl  " VMLINUX_SYMBOL_STR(__crc_rel_##sym) " \n"     \
> +           VMLINUX_SYMBOL_STR(__crc_rel_##sym)":
>  \n"     \
> +           "   .long   " VMLINUX_SYMBOL_STR(__crc_##sym) " - . \n"     \
>             "   .previous                                       \n");
>  #endif
>  #else
> diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c
> index 06121ce524a7..8dd9f1da8898 100644
> --- a/scripts/genksyms/genksyms.c
> +++ b/scripts/genksyms/genksyms.c
> @@ -693,7 +693,8 @@ void export_symbol(const char *name)
>                         fputs(">\n", debugfile);
>
>                 /* Used as a linker script. */
> -               printf("%s__crc_%s = 0x%08lx ;\n", mod_prefix, name, crc);
> +               printf("%s__crc_%s = %s__crc_rel_%s + 0x%08lx;\n", mod_prefix,
> +                      name, mod_prefix, name, crc);
>         }
>  }
>
> but this doesn't work either: the __crc_xxx symbols that are emitted
> during partial linking evaluate the __crc_rel_xxx references at
> partial link time, which means the resulting relative relocations
> refer to the actual CRC value rather than the modified CRC value. The
> only way to make /this/ work, afaik, is to hack the build scripts so
> that the __crc_xxx = assignments occur in the scope of the final
> linker script, rather than during partial linking (but only for
> vmlinux, not for modules). All of this complicates the common path
> used by all architectures, so I don't think we should go down this
> road.
>
> So I don't see any other way to work around this for powerpc (other
> than creating some build time tooling to process the 32-bit
> relocations for the CRC symbols in the vmlinux ELF binary) Hopefully,
> Michael will be able to confirm that this v4 of 2/3 works correctly,
> then we can discuss whether to go ahead with this or not.
>
> --
> Ard.
>
> [*] and sadly, it only counts R_PPC64_RELATIVE relocations in its
> .dynamic section's RELACOUNT, which requires additional handling in
> patch 2/3

Re: [PATCH v4 2/3] powerpc/reloc64: add support for 32-bit CRC pseudo-symbols

[Ard Biesheuvel]

On 17 January 2017 at 19:26, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> In preparation of modifying the core modversions code to emit the CRCs
> as 32-bit quantities, ensure that 64-bit PowerPC will be able to deal
> with this when CONFIG_RELOCATABLE=y, in which case the CRCs will be
> emitted into the final ELF binary as R_PPC64_ADDR32 relocations.
>
> Since 32-bit relocations cannot be used to relocate memory addresses on
> 64-bit architectures, and since the CRC pseudo-symbol references are
> emitted as anonymous relocations (i.e., against the NULL symbol in the
> .dynsym section) with the final value recorded in the addend (*), we
> can disregard any relocations where the symbol index != 0.
>
> * Note that unsatisfied CRC pseudo-symbol references are emitted as
>   R_PPC64_ADDR32 relocations against named symbols that are typed as
>   weak undefined in the .dynsym symbol table. These can simply be
>   ignored (as before), considering that zero CRCs are interpreted as
>   missing, and the module code deals with that accordingly.
>
> As it turns out, binutils for powerpc does not account for any relocations
> beyond R_PPC64_RELATIVE ones in the RELACOUNT field of the .dynamic section,
> which is unfortunate, since we need to do extra work to figure out the size
> of the relocation array. So with a little help from the linker scripts,
> grab an end pointer rather than a count, and iterate over the entire section.
>

While this is the case of RELACOUNT, RELASZ appears to behave as
expected. For instance, on a random vmlinux file (with patch 3/3
applied), I get

Dynamic Section:
  HASH                 0xc000000000cf5a40
  STRTAB               0xc000000000cf4d80
  SYMTAB               0xc000000000cf3e98
  STRSZ                0x0000000000000bae
  SYMENT               0x0000000000000018
  DEBUG                0x0000000000000000
  RELA                 0xc000000000cf5ee8
  RELASZ               0x00000000003308b8
  RELAENT              0x0000000000000018
  TEXTREL              0x0000000000000000
  FLAGS_1              0x0000000008000000
  RELACOUNT            0x0000000000020444

where RELACOUNT is the number of just the R_PPC64_RELATIVE
relocations, whereas RELASZ covers all of them (which is why RELASZ /
RELAENT != RELACOUNT)

So if preferred, I can respin this patch to retrieve RELASZ from the
dynamic section rather than the end of the .rela section from the
linker script, which would reduce the delta of this patch somewhat.


> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  arch/powerpc/kernel/reloc_64.S    | 60 ++++++++++++--------
>  arch/powerpc/kernel/vmlinux.lds.S |  1 +
>  arch/powerpc/relocs_check.sh      |  5 +-
>  3 files changed, 40 insertions(+), 26 deletions(-)
>
> diff --git a/arch/powerpc/kernel/reloc_64.S b/arch/powerpc/kernel/reloc_64.S
> index d88736fbece6..e50f5d778ea2 100644
> --- a/arch/powerpc/kernel/reloc_64.S
> +++ b/arch/powerpc/kernel/reloc_64.S
> @@ -12,8 +12,8 @@
>  #include <asm/ppc_asm.h>
>
>  RELA = 7
> -RELACOUNT = 0x6ffffff9
>  R_PPC64_RELATIVE = 22
> +R_PPC64_ADDR32 = 1
>
>  /*
>   * r3 = desired final address of kernel
> @@ -29,29 +29,27 @@ _GLOBAL(relocate)
>         add     r9,r9,r12       /* r9 has runtime addr of .rela.dyn section */
>         ld      r10,(p_st - 0b)(r12)
>         add     r10,r10,r12     /* r10 has runtime addr of _stext */
> +       ld      r8,(p_rela_end - 0b)(r12)
> +       add     r8,r8,r12       /* r8 has addr of end of .rela.dyn section */
>
>         /*
> -        * Scan the dynamic section for the RELA and RELACOUNT entries.
> +        * Scan the dynamic section for the RELA entry.
> +        * NOTE: the RELACOUNT entry only covers R_PPC64_RELATIVE relocations,
> +        *       so we cannot use it here.
>          */
>         li      r7,0
> -       li      r8,0
>  1:     ld      r6,0(r11)       /* get tag */
>         cmpdi   r6,0
> -       beq     4f              /* end of list */
> +       beq     3f              /* end of list */
>         cmpdi   r6,RELA
> -       bne     2f
> -       ld      r7,8(r11)       /* get RELA pointer in r7 */
> -       b       3f
> -2:     addis   r6,r6,(-RELACOUNT)@ha
> -       cmpdi   r6,RELACOUNT@l
> -       bne     3f
> -       ld      r8,8(r11)       /* get RELACOUNT value in r8 */
> -3:     addi    r11,r11,16
> +       beq     2f
> +       addi    r11,r11,16
>         b       1b
> -4:     cmpdi   r7,0            /* check we have both RELA and RELACOUNT */
> -       cmpdi   cr1,r8,0
> -       beq     6f
> -       beq     cr1,6f
> +2:     ld      r7,8(r11)       /* get RELA pointer in r7 */
> +3:     cmpdi   r7,0            /* check we have both RELA and a non-empty */
> +       cmpd    cr1,r8,r9       /* .rela.dyn section                       */
> +       beq     7f
> +       beq     cr1,7f
>
>         /*
>          * Work out linktime address of _stext and hence the
> @@ -63,26 +61,40 @@ _GLOBAL(relocate)
>         subf    r7,r7,r9        /* cur_offset */
>         subf    r10,r7,r10
>         subf    r3,r10,r3       /* final_offset */
> +       b       4f
>
>         /*
>          * Run through the list of relocations and process the
> -        * R_PPC64_RELATIVE ones.
> +        * R_PPC64_RELATIVE and R_PPC64_ADDR32 ones.
>          */
> -       mtctr   r8
> -5:     ld      r0,8(9)         /* ELF64_R_TYPE(reloc->r_info) */
> +3:     addi    r9,r9,24
> +4:     cmpd    r9,r8
> +       beq     7f
> +5:     ld      r0,8(9)         /* reloc->r_info (type *and* symbol index) */
> +       ld      r6,0(r9)        /* reloc->r_offset */
>         cmpdi   r0,R_PPC64_RELATIVE
>         bne     6f
> -       ld      r6,0(r9)        /* reloc->r_offset */
>         ld      r0,16(r9)       /* reloc->r_addend */
>         add     r0,r0,r3
>         stdx    r0,r7,r6
> -       addi    r9,r9,24
> -       bdnz    5b
> +       b       3b
> +
> +       /*
> +        * CRCs of exported symbols are emitted as 32-bit relocations against
> +        * the NULL .dynsym entry, with the CRC value recorded in the addend.
> +        */
> +6:     cmpdi   r0,R_PPC64_ADDR32
> +       bne     3b
> +       ld      r0,16(r9)       /* reloc->r_addend */
> +       stwx    r0,r7,r6
> +       b       3b
> +
> +7:     blr
>
> -6:     blr
>
>  .balign 8
>  p_dyn: .llong  __dynamic_start - 0b
>  p_rela:        .llong  __rela_dyn_start - 0b
> +p_rela_end:
> +       .llong  __rela_dyn_end - 0b
>  p_st:  .llong  _stext - 0b
> -
> diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
> index 7394b770ae1f..654728fc860d 100644
> --- a/arch/powerpc/kernel/vmlinux.lds.S
> +++ b/arch/powerpc/kernel/vmlinux.lds.S
> @@ -241,6 +241,7 @@ SECTIONS
>         {
>                 __rela_dyn_start = .;
>                 *(.rela*)
> +               __rela_dyn_end = .;
>         }
>  #endif
>         /* .exit.data is discarded at runtime, not link time,
> diff --git a/arch/powerpc/relocs_check.sh b/arch/powerpc/relocs_check.sh
> index ec2d5c835170..f9636b4e4548 100755
> --- a/arch/powerpc/relocs_check.sh
> +++ b/arch/powerpc/relocs_check.sh
> @@ -30,7 +30,7 @@ bad_relocs=$(
>         # On PPC64:
>         #       R_PPC64_RELATIVE, R_PPC64_NONE
>         #       R_PPC64_ADDR64 mach_<name>
> -       #       R_PPC64_ADDR64 __crc_<name>
> +       #       R_PPC64_ADDR32 __crc_<name>
>         # On PPC:
>         #       R_PPC_RELATIVE, R_PPC_ADDR16_HI,
>         #       R_PPC_ADDR16_HA,R_PPC_ADDR16_LO,
> @@ -43,7 +43,8 @@ R_PPC_ADDR16_HA
>  R_PPC_RELATIVE
>  R_PPC_NONE' |
>         grep -E -v '\<R_PPC64_ADDR64[[:space:]]+mach_' |
> -       grep -E -v '\<R_PPC64_ADDR64[[:space:]]+__crc_'
> +       grep -E -v '\<R_PPC64_ADDR32[[:space:]]+__crc_' |
> +       grep -E -v '\<R_PPC64_ADDR32[[:space:]]+\*ABS\*'
>  )
>
>  if [ -z "$bad_relocs" ]; then
> --
> 2.7.4
>

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Linus Torvalds]

On Wed, Jan 18, 2017 at 5:52 AM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
>
> It turns out that this odd treatment of absolute symbols (i.e.,
> symbols having section number SHN_ABS) is a known issue in GNU ld
>
> https://sourceware.org/ml/binutils/2012-05/msg00019.html

Ugh. I hate the GNU linker. It's slow, it does nasty crazy fixups, and
it's apparently buggy too.

I wonder what happened to gold, and why it didn't take over. I'm
assuming it had _other_ bugs.. Oh well.

                 Linus

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Linus Torvalds]

On Wed, Jan 18, 2017 at 10:27 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> I wonder what happened to gold, and why it didn't take over. I'm
> assuming it had _other_ bugs.. Oh well.

Google for gold problems, I note that it has been reported to get
"internal error"s during kernel builds - and at least some of them
have been due to ksyms.

So the core problem seems to mainly be that gcc normally itself never
generates any absolute symbols, so the whole ksyms model depends on
things that get almost zero testing in the toolchain.

Oh well.

              Linus

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Ard Biesheuvel]

On 18 January 2017 at 18:35, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Wed, Jan 18, 2017 at 10:27 AM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>>
>> I wonder what happened to gold, and why it didn't take over. I'm
>> assuming it had _other_ bugs.. Oh well.
>
> Google for gold problems, I note that it has been reported to get
> "internal error"s during kernel builds - and at least some of them
> have been due to ksyms.
>
> So the core problem seems to mainly be that gcc normally itself never
> generates any absolute symbols, so the whole ksyms model depends on
> things that get almost zero testing in the toolchain.
>
> Oh well.
>

There is one alternative that gets rid of all the hackiness, but it
does increase the CRC footprint on 32-bit platforms:

diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c
index 06121ce524a7..9f739c7224c3 100644
--- a/scripts/genksyms/genksyms.c
+++ b/scripts/genksyms/genksyms.c
@@ -693,7 +693,10 @@ void export_symbol(const char *name)
                        fputs(">\n", debugfile);

                /* Used as a linker script. */
-               printf("%s__crc_%s = 0x%08lx ;\n", mod_prefix, name, crc);
+               printf("SECTIONS { .rodata.__crc_%s : ALIGN(4) "
+                      "{ %s__crcp_%s = .; LONG(0x%08lx); } }\n"
+                      "%s__crc_%s = 0x%08lx;\n",
+                      name, mod_prefix, name, crc, mod_prefix, name, crc);
        }
 }

diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index e3508a3d6e53..5e95a552a871 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -49,8 +49,8 @@ KSYM(__kstrtab_\name):
        .section ___kcrctab\sec+\name,"a"
        .balign KCRC_ALIGN
 KSYM(__kcrctab_\name):
-       .long KSYM(__crc_\name)
-       .weak KSYM(__crc_\name)
+       .long KSYM(__crcp_\name) - .
+       .weak KSYM(__crcp_\name)
        .previous
 #endif
 #endif

(and an equivalent change in include/linux/export.h)

So the kcrctab contains the relative (signed) offset to where the CRC
is stored, which gets rid of any absolute relocations. To grab the
CRC, we only have to add the value of the kcrctab entry to its
address, and dereference the resulting pointer.

This would remove the need for patches 1 and 2, and get rid of the
overhead of the runtime relocation entries not only for arm64 but for
powerpc as well. It would also get rid of the abuse of ELF symbols
once and for all, hopefully avoiding bugs in GNU ld and gold in the
future.

For a ballpark number of 10,000 CRCs in the core kernel, this would
increase the size of the image by 40 KB for 32-bit architectures (and
if saving 40 KB is essential, chances are you won't be using
modversions in the first place). For 64-bit architectures, there would
be no change in size, of course, except for saving 24 bytes of __init
space *per CRC* for arm64 and powerpc64 with CONFIG_RELOCATABLE=y

I will send out a separate RFC so we can discuss this alternative

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Linus Torvalds]

On Wed, Jan 18, 2017 at 2:37 PM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
>
> For a ballpark number of 10,000 CRCs in the core kernel, this would
> increase the size of the image by 40 KB for 32-bit architectures (and
> if saving 40 KB is essential, chances are you won't be using
> modversions in the first place).

As you say, I don't think the space issue is much of a problem.

I'm more worried about the replacement of one crazy model that has
problems due to linker subtlety with _another_ one.

Your genksyms.c change is not exactly obvious. I looked at it, and my
brain just shut down. Why both the

  LONG(0x%08lx);

_and_ the

  "%s__crc_%s = 0x%08lx;\n"

in the linker script? I'm sure there's a good reason, but I'd like to
see a more explicit explanation fo what the generated linker script
does and what the rules are.

         Linus

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Ard Biesheuvel]

On 19 January 2017 at 00:15, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Wed, Jan 18, 2017 at 2:37 PM, Ard Biesheuvel
> <ard.biesheuvel@linaro.org> wrote:
>>
>> For a ballpark number of 10,000 CRCs in the core kernel, this would
>> increase the size of the image by 40 KB for 32-bit architectures (and
>> if saving 40 KB is essential, chances are you won't be using
>> modversions in the first place).
>
> As you say, I don't think the space issue is much of a problem.
>
> I'm more worried about the replacement of one crazy model that has
> problems due to linker subtlety with _another_ one.
>
> Your genksyms.c change is not exactly obvious. I looked at it, and my
> brain just shut down. Why both the
>
>   LONG(0x%08lx);
>
> _and_ the
>
>   "%s__crc_%s = 0x%08lx;\n"
>
> in the linker script? I'm sure there's a good reason, but I'd like to
> see a more explicit explanation fo what the generated linker script
> does and what the rules are.
>

This is simply because modpost still uses the value of the symbol
rather than the value it points to to generate the /other/ side of the
comparison (i.e., Module.symvers etc)

I will look into updating modpost to dereference the symbol as well,
and update the RFC patch accordingly.

RE: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[David Laight]

From:  Ard Biesheuvel
> Sent: 18 January 2017 13:53
..
> It turns out that this odd treatment of absolute symbols (i.e.,
> symbols having section number SHN_ABS) is a known issue in GNU ld
> 
> https://sourceware.org/ml/binutils/2012-05/msg00019.html
...

Jeepers - that is truly f*cked.

I've even used the linker script to generate absolute symbols for the
sizes of items - they really don't want relocating (ever).

If you go back to (say) RSX11/M loads of the constants (like system
call numbers and structure offsets) would be supplied by the linker
if the assembler didn't know the value.
That linker supported (more or less) arbitrary expressions in fixups.

	David

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Linus Torvalds]

On Thu, Jan 19, 2017 at 1:22 AM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
>>
>> Your genksyms.c change is not exactly obvious. I looked at it, and my
>> brain just shut down. Why both the
>>
>>   LONG(0x%08lx);
>>
>> _and_ the
>>
>>   "%s__crc_%s = 0x%08lx;\n"
>>
>> in the linker script? I'm sure there's a good reason, but I'd like to
>> see a more explicit explanation fo what the generated linker script
>> does and what the rules are.
>
> This is simply because modpost still uses the value of the symbol
> rather than the value it points to to generate the /other/ side of the
> comparison (i.e., Module.symvers etc)

Ahh, now that you explained it, it was obvious. Thanks.

But yes, I don't think we want that "both belt and suspenders"
approach, so your updated patch that does things just one way is I
think the right way.

> I will look into updating modpost to dereference the symbol as well,
> and update the RFC patch accordingly.

Yes, so your updated patch looks good to me.

I think our old "symbol with an absolute value" model was simpler
conceptually, but given the existing absolute (sic) braindamage of
linkers, I think your latest patch is probably the way to go.

If for no other reason than the fact that it doesn't depend on
something that clearly nobody else uses, and even the linker people
were confused about.

So I think the slightly more complex model of relative offsets is the
simpler one in the end if it means that we don't have to have
completely insane workarounds for linker damage.

But maybe somebody else wants to pipe up. Preferably somebody who
doesn't hate the symversions code as much as I do by now, and actually
_uses_ it ;)

                Linus

Re: [PATCH v4 3/3] modversions: treat symbol CRCs as 32 bit quantities on 64 bit archs

[Ard Biesheuvel]

On 19 January 2017 at 17:24, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Thu, Jan 19, 2017 at 1:22 AM, Ard Biesheuvel
> <ard.biesheuvel@linaro.org> wrote:
>>>
>>> Your genksyms.c change is not exactly obvious. I looked at it, and my
>>> brain just shut down. Why both the
>>>
>>>   LONG(0x%08lx);
>>>
>>> _and_ the
>>>
>>>   "%s__crc_%s = 0x%08lx;\n"
>>>
>>> in the linker script? I'm sure there's a good reason, but I'd like to
>>> see a more explicit explanation fo what the generated linker script
>>> does and what the rules are.
>>
>> This is simply because modpost still uses the value of the symbol
>> rather than the value it points to to generate the /other/ side of the
>> comparison (i.e., Module.symvers etc)
>
> Ahh, now that you explained it, it was obvious. Thanks.
>
> But yes, I don't think we want that "both belt and suspenders"
> approach, so your updated patch that does things just one way is I
> think the right way.
>
>> I will look into updating modpost to dereference the symbol as well,
>> and update the RFC patch accordingly.
>
> Yes, so your updated patch looks good to me.
>
> I think our old "symbol with an absolute value" model was simpler
> conceptually, but given the existing absolute (sic) braindamage of
> linkers, I think your latest patch is probably the way to go.
>

OK.

> If for no other reason than the fact that it doesn't depend on
> something that clearly nobody else uses, and even the linker people
> were confused about.
>
> So I think the slightly more complex model of relative offsets is the
> simpler one in the end if it means that we don't have to have
> completely insane workarounds for linker damage.
>
> But maybe somebody else wants to pipe up. Preferably somebody who
> doesn't hate the symversions code as much as I do by now, and actually
> _uses_ it ;)
>

I am not crazy about it either: I am simply trying to get rid of the
~10,000 pointless relocations in the arm64 KASLR kernel rather than
having to rely on the dodgy code that 'repairs' the CRCs at runtime.

I have noticed one slight snag though: the ARM module loader currently
has no support for the R_ARM_REL32 relocations that are emitted by the
relative references in the kcrctabs. I looked at other arches, x86,
ia64, s390, power and arm64, and those all seem to be fully equipped
in this regard, but it would be good if we could get some coverage for
this code on other architectures to find out which ones need to have
this support added as well.

Thanks,
Ard.