[kernel-hardening] [PATCH 00/30] implement KASLR for ARM

[kernel-hardening] [PATCH 00/30] implement KASLR for ARM

[Ard Biesheuvel]

This series implements randomization of the placement of the core ARM kernel
inside the lowmem region. It consists of the following parts:

- changes that allow us to build vmlinux as a PIE executable which retains
  the metadata required to fix up all absolute symbol references at runtime
- changes that eliminate absolute references from low-level code that may
  execute with the MMU off: this removes the need to perform explicit cache
  maintenance after the absolute references have been fixed up at runtime with
  the caches enabled
- changes to the core kernel startup code to take the physical offset into
  account when creating the virtual mapping (the pa-to-va mapping remains
  unchanged)
- changes to the decompressor to take the KASLR offset into account when
  placing the kernel in physical memory
- changes to the UEFI stub code to choose the KASLR offset and communicate
  it to the decompressor

Bootloader changes required for other boot environments are left as an
exercise for the reader.

To test these changes, get a UEFI image for QEMU here [0] and boot a
multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y build using:

qemu-system-aarch64 -M virt -m 2048 -bios QEMU_EFI.fd -kernel zImage \
  -device virtio-rng-pci -net none -nographic -append earlycon

and expect to see something like

[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xf0800000 - 0xff800000   ( 240 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xf0000000   ( 768 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf800000 - 0xbfe00000   (   6 MB)
[    0.000000]       .text : 0xd4208000 - 0xd4c00000   (10208 kB) <---- 
[    0.000000]       .init : 0xd5200000 - 0xd5600000   (4096 kB)  <----
[    0.000000]       .data : 0xd5600000 - 0xd5776f28   (1500 kB)  <----
[    0.000000]        .bss : 0xd57805e0 - 0xd57e60ac   ( 407 kB)  <----

[0] https://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-ARM/RELEASE_GCC49/QEMU_EFI.fd

(or update patch #26 to poke a hardcoded value into kaslr_offset directly)

Note that this series partially overlaps with my series 'ARM: add and use
convenience macros for PC relative references', for which I will not send
a follow up. I kept the necessary ones, and dropped some others that are
not actually that useful in the context of KASLR.

Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Nicolas Pitre <nico@linaro.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Kees Cook <keescook@chromium.org>
Cc: Thomas Garnier <thgarnie@google.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Cc: Dave Martin <dave.martin@arm.com>

Ard Biesheuvel (30):
  asm-generic: add .data.rel.ro sections to __ro_after_init
  ARM: assembler: introduce adr_l, ldr_l and str_l macros
  ARM: head-common.S: use PC-relative insn sequence for __proc_info
  ARM: head-common.S: use PC-relative insn sequence for idmap creation
  ARM: head.S: use PC-relative insn sequence for secondary_data
  ARM: kernel: use relative references for UP/SMP alternatives
  ARM: head: use PC-relative insn sequence for __smp_alt
  ARM: sleep.S: use PC-relative insn sequence for
    sleep_save_sp/mpidr_hash
  ARM: head.S: use PC-relative insn sequences for __fixup_pv_table
  ARM: head.S: use PC relative insn sequence to calculate PHYS_OFFSET
  ARM: kvm: replace open coded VA->PA calculations with adr_l call
  arm-soc: exynos: replace open coded VA->PA conversions
  arm-soc: mvebu: replace open coded VA->PA conversion
  arm-soc: various: replace open coded VA->PA calculation of pen_release
  ARM: kernel: switch to relative exception tables
  ARM: kernel: use relative phys-to-virt patch tables
  arm-soc: tegra: make sleep asm code runtime relocatable
  ARM: kernel: make vmlinux buildable as a PIE executable
  ARM: kernel: use PC-relative symbol references in MMU switch code
  ARM: kernel: use PC relative symbol references in suspend/resume code
  ARM: mm: export default vmalloc base address
  ARM: kernel: refer to swapper_pg_dir via its symbol
  ARM: kernel: implement randomization of the kernel load address
  ARM: decompressor: explicitly map decompressor binary cacheable
  ARM: compressed: factor out zImage header and make it extensible
  ARM: decompressor: add KASLR support
  efi/libstub: add 'max' parameter to efi_random_alloc()
  efi/libstub: check for vmalloc= command line argument
  efi/libstub: arm: reserve bootloader supplied initrd in memory map
  efi/libstub: arm: implement KASLR

 arch/arm/Kconfig                               |  19 ++
 arch/arm/Makefile                              |   5 +
 arch/arm/boot/compressed/head.S                |  46 ++---
 arch/arm/boot/compressed/vmlinux.lds.S         |   5 +-
 arch/arm/include/asm/Kbuild                    |   1 -
 arch/arm/include/asm/assembler.h               |  86 +++++++-
 arch/arm/include/asm/extable.h                 |  20 ++
 arch/arm/include/asm/futex.h                   |   2 +-
 arch/arm/include/asm/memory.h                  |   6 +-
 arch/arm/include/asm/pgtable.h                 |   1 +
 arch/arm/include/asm/processor.h               |   2 +-
 arch/arm/include/asm/uaccess.h                 |   8 +-
 arch/arm/include/asm/word-at-a-time.h          |   2 +-
 arch/arm/include/asm/zimage.h                  |  65 ++++++
 arch/arm/kernel/entry-armv.S                   |   6 +-
 arch/arm/kernel/head-common.S                  |  61 ++----
 arch/arm/kernel/head.S                         | 217 ++++++++++++--------
 arch/arm/kernel/hyp-stub.S                     |  33 ++-
 arch/arm/kernel/sleep.S                        |  27 +--
 arch/arm/kernel/swp_emulate.c                  |   4 +-
 arch/arm/kernel/vmlinux.lds.S                  |   9 +
 arch/arm/kvm/init.S                            |   8 +-
 arch/arm/lib/backtrace.S                       |   8 +-
 arch/arm/lib/getuser.S                         |  22 +-
 arch/arm/lib/putuser.S                         |  12 +-
 arch/arm/mach-exynos/headsmp.S                 |   9 +-
 arch/arm/mach-exynos/sleep.S                   |  26 +--
 arch/arm/mach-mvebu/coherency_ll.S             |   8 +-
 arch/arm/mach-prima2/headsmp.S                 |  11 +-
 arch/arm/mach-spear/headsmp.S                  |  11 +-
 arch/arm/mach-sti/headsmp.S                    |  10 +-
 arch/arm/mach-tegra/sleep-tegra20.S            |  22 +-
 arch/arm/mach-tegra/sleep-tegra30.S            |   6 +-
 arch/arm/mach-tegra/sleep.S                    |   4 +-
 arch/arm/mm/alignment.c                        |  14 +-
 arch/arm/mm/extable.c                          |   2 +-
 arch/arm/mm/mmu.c                              |   3 +-
 arch/arm/nwfpe/entry.S                         |   2 +-
 arch/arm/plat-versatile/headsmp.S              |   9 +-
 drivers/firmware/efi/libstub/arm-stub.c        |  51 +++--
 drivers/firmware/efi/libstub/arm32-stub.c      |  46 ++++-
 drivers/firmware/efi/libstub/arm64-stub.c      |   2 +-
 drivers/firmware/efi/libstub/efi-stub-helper.c |   9 +
 drivers/firmware/efi/libstub/efistub.h         |   7 +-
 drivers/firmware/efi/libstub/fdt.c             |  42 ++++
 drivers/firmware/efi/libstub/random.c          |  10 +-
 include/asm-generic/vmlinux.lds.h              |   2 +-
 include/linux/hidden.h                         |  21 ++
 scripts/sortextable.c                          |   2 +-
 49 files changed, 636 insertions(+), 368 deletions(-)
 create mode 100644 arch/arm/include/asm/extable.h
 create mode 100644 arch/arm/include/asm/zimage.h
 create mode 100644 include/linux/hidden.h

-- 
2.11.0

[kernel-hardening] [PATCH 01/30] asm-generic: add .data.rel.ro sections to __ro_after_init

[Ard Biesheuvel]

When running in PIC mode, the compiler will emit const structures
containing runtime relocatable quantities into .data.rel.ro.* sections,
so that the linker can be smart about placing them together in a segment
that is read-write initially, and is remapped read-only afterwards. This
is exactly what __ro_after_init aims to provide, so move these sections
together.

Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 include/asm-generic/vmlinux.lds.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index da0be9a8d1de..d16537b0b102 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -261,7 +261,7 @@
 #ifndef RO_AFTER_INIT_DATA
 #define RO_AFTER_INIT_DATA						\
 	VMLINUX_SYMBOL(__start_ro_after_init) = .;			\
-	*(.data..ro_after_init)						\
+	*(.data..ro_after_init .data.rel.ro.*)				\
 	VMLINUX_SYMBOL(__end_ro_after_init) = .;
 #endif
 
-- 
2.11.0

[kernel-hardening] [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Ard Biesheuvel]

Like arm64, ARM supports position independent code sequences that
produce symbol references with a greater reach than the ordinary
adr/ldr instructions.

Currently, we use open coded instruction sequences involving literals
and arithmetic operations. Instead, we can use movw/movt pairs on v7
CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
the literal into a subsection, allowing it to be emitted out of line
while retaining the ability to perform arithmetic on label offsets.

E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:

       ldr          <reg>, 222f
  111: add          <reg>, <reg>, pc
       .subsection  1
  222: .long        <sym> - (111b + 8)
       .previous

This is allowed by the assembler because, unlike ordinary sections,
subsections are combined into a single section into the object file,
and so the label references are not true cross-section references that
are visible as relocations. Note that we could even do something like

       add          <reg>, pc, #(222f - 111f) & ~0xfff
       ldr          <reg>, [<reg>, #(222f - 111f) & 0xfff]
  111: add          <reg>, <reg>, pc
       .subsection  1
  222: .long        <sym> - (111b + 8)
       .previous

if it turns out that the 4 KB range of the ldr instruction is insufficient
to reach the literal in the subsection, although this is currently not a
problem (of the 98 objects built from .S files in a multi_v7_defconfig
build, only 11 have .text sections that are over 1 KB, and the largest one
[entry-armv.o] is 3308 bytes)

Subsections have been available in binutils since 2004 at least, so
they should not cause any issues with older toolchains.

So use the above to implement the macros mov_l, adr_l, adrm_l (using ldm
to load multiple literals at once), ldr_l and str_l, all of which will
use movw/movt pairs on v7 and later CPUs, and use PC-relative literals
otherwise.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/include/asm/assembler.h | 71 ++++++++++++++++++++
 1 file changed, 71 insertions(+)

diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index ad301f107dd2..516ebaf4ff38 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -518,4 +518,75 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 #endif
 	.endm
 
+#ifdef CONFIG_THUMB2_KERNEL
+#define	ARM_PC_BIAS		4
+#else
+#define	ARM_PC_BIAS		8
+#endif
+
+	.macro		__adldst_l, op, reg, sym, tmp, c
+	.if		__LINUX_ARM_ARCH__ < 7
+	ldr\c		\tmp, 111f
+	.subsection	1
+	.align		2
+111:	.long		\sym - (222f + ARM_PC_BIAS)
+	.previous
+	.else
+	W(movw\c\())	\tmp, #:lower16:\sym - (222f + ARM_PC_BIAS)
+	W(movt\c\())	\tmp, #:upper16:\sym - (222f + ARM_PC_BIAS)
+	.endif
+222:
+	.ifc		\op, add
+	add\c		\reg, \tmp, pc
+	.elseif		CONFIG_THUMB2_KERNEL == 1
+	add		\tmp, \tmp, pc
+	\op\c		\reg, [\tmp]
+	.else
+	\op\c		\reg, [pc, \tmp]
+	.endif
+	.endm
+
+	/*
+	 * mov_l - move a constant value or [relocated] address into a register
+	 */
+	.macro		mov_l, dst:req, imm:req, cond
+	.if		__LINUX_ARM_ARCH__ < 7
+	ldr\cond	\dst, =\imm
+	.else
+	W(movw\cond\())	\dst, #:lower16:\imm
+	W(movt\cond\())	\dst, #:upper16:\imm
+	.endif
+	.endm
+
+	/*
+	 * adr_l - adr pseudo-op with unlimited range
+	 *
+	 * @dst: destination register
+	 * @sym: name of the symbol
+	 */
+	.macro		adr_l, dst:req, sym:req, cond
+	__adldst_l	add, \dst, \sym, \dst, \cond
+	.endm
+
+	/*
+	 * ldr_l - ldr <literal> pseudo-op with unlimited range
+	 *
+	 * @dst: destination register
+	 * @sym: name of the symbol
+	 */
+	.macro		ldr_l, dst:req, sym:req, cond
+	__adldst_l	ldr, \dst, \sym, \dst, \cond
+	.endm
+
+	/*
+	 * str_l - str <literal> pseudo-op with unlimited range
+	 *
+	 * @src: source register
+	 * @sym: name of the symbol
+	 * @tmp: mandatory scratch register
+	 */
+	.macro		str_l, src:req, sym:req, tmp:req, cond
+	__adldst_l	str, \src, \sym, \tmp, \cond
+	.endm
+
 #endif /* __ASM_ASSEMBLER_H__ */
-- 
2.11.0

[kernel-hardening] [PATCH 03/30] ARM: head-common.S: use PC-relative insn sequence for __proc_info

[Ard Biesheuvel]

Replace the open coded PC relative offset calculations with a pair
of adr_l invocations. This ensures these quantities are invariant
under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head-common.S | 22 ++++++--------------
 1 file changed, 6 insertions(+), 16 deletions(-)

diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
index 8733012d231f..06035488130c 100644
--- a/arch/arm/kernel/head-common.S
+++ b/arch/arm/kernel/head-common.S
@@ -150,11 +150,12 @@ ENDPROC(lookup_processor_type)
  *	r9 = cpuid (preserved)
  */
 __lookup_processor_type:
-	adr	r3, __lookup_processor_type_data
-	ldmia	r3, {r4 - r6}
-	sub	r3, r3, r4			@ get offset between virt&phys
-	add	r5, r5, r3			@ convert virt addresses to
-	add	r6, r6, r3			@ physical address space
+	/*
+	 * Look in <asm/procinfo.h> for information about the __proc_info
+	 * structure.
+	 */
+	adr_l	r5, __proc_info_begin
+	adr_l	r6, __proc_info_end
 1:	ldmia	r5, {r3, r4}			@ value, mask
 	and	r4, r4, r9			@ mask wanted bits
 	teq	r3, r4
@@ -166,17 +167,6 @@ __lookup_processor_type:
 2:	ret	lr
 ENDPROC(__lookup_processor_type)
 
-/*
- * Look in <asm/procinfo.h> for information about the __proc_info structure.
- */
-	.align	2
-	.type	__lookup_processor_type_data, %object
-__lookup_processor_type_data:
-	.long	.
-	.long	__proc_info_begin
-	.long	__proc_info_end
-	.size	__lookup_processor_type_data, . - __lookup_processor_type_data
-
 __error_lpae:
 #ifdef CONFIG_DEBUG_LL
 	adr	r0, str_lpae
-- 
2.11.0

[kernel-hardening] [PATCH 04/30] ARM: head-common.S: use PC-relative insn sequence for idmap creation

[Ard Biesheuvel]

Replace the open coded PC relative offset calculations involving
__turn_mmu_on and __turn_mmu_on_end with a pair of adr_l invocations.
This ensures these quantities are invariant under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head.S | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 04286fd9e09c..0a98aec0e39d 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -227,11 +227,8 @@ __create_page_tables:
 	 * Create identity mapping to cater for __enable_mmu.
 	 * This identity mapping will be removed by paging_init().
 	 */
-	adr	r0, __turn_mmu_on_loc
-	ldmia	r0, {r3, r5, r6}
-	sub	r0, r0, r3			@ virt->phys offset
-	add	r5, r5, r0			@ phys __turn_mmu_on
-	add	r6, r6, r0			@ phys __turn_mmu_on_end
+	adr_l	r5, __turn_mmu_on		@ _pa(__turn_mmu_on)
+	adr_l	r6, __turn_mmu_on_end		@ _pa(__turn_mmu_on_end)
 	mov	r5, r5, lsr #SECTION_SHIFT
 	mov	r6, r6, lsr #SECTION_SHIFT
 
@@ -354,11 +351,6 @@ __create_page_tables:
 	ret	lr
 ENDPROC(__create_page_tables)
 	.ltorg
-	.align
-__turn_mmu_on_loc:
-	.long	.
-	.long	__turn_mmu_on
-	.long	__turn_mmu_on_end
 
 #if defined(CONFIG_SMP)
 	.text
-- 
2.11.0

[kernel-hardening] [PATCH 05/30] ARM: head.S: use PC-relative insn sequence for secondary_data

[Ard Biesheuvel]

Replace the open coded PC relative offset calculations with adr_l
and ldr_l invocations. This ensures these quantities are invariant
under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head.S | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 0a98aec0e39d..6e9df3663a57 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -386,10 +386,8 @@ ENTRY(secondary_startup)
 	/*
 	 * Use the page tables supplied from  __cpu_up.
 	 */
-	adr	r4, __secondary_data
-	ldmia	r4, {r5, r7, r12}		@ address to jump to after
-	sub	lr, r4, r5			@ mmu has been enabled
-	add	r3, r7, lr
+	adr_l	r3, secondary_data
+	mov_l	r12, __secondary_switched
 	ldrd	r4, [r3, #0]			@ get secondary_data.pgdir
 ARM_BE8(eor	r4, r4, r5)			@ Swap r5 and r4 in BE:
 ARM_BE8(eor	r5, r4, r5)			@ it can be done in 3 steps
@@ -404,22 +402,13 @@ ARM_BE8(eor	r4, r4, r5)			@ without using a temp reg.
 ENDPROC(secondary_startup)
 ENDPROC(secondary_startup_arm)
 
-	/*
-	 * r6  = &secondary_data
-	 */
 ENTRY(__secondary_switched)
-	ldr	sp, [r7, #12]			@ get secondary_data.stack
+	ldr_l	r7, secondary_data + 12		@ get secondary_data.stack
+	mov	sp, r7
 	mov	fp, #0
 	b	secondary_start_kernel
 ENDPROC(__secondary_switched)
 
-	.align
-
-	.type	__secondary_data, %object
-__secondary_data:
-	.long	.
-	.long	secondary_data
-	.long	__secondary_switched
 #endif /* defined(CONFIG_SMP) */
 
 
-- 
2.11.0

[kernel-hardening] [PATCH 06/30] ARM: kernel: use relative references for UP/SMP alternatives

[Ard Biesheuvel]

To avoid absolute references that are subject to runtime relocation
when running a kernel built with CONFIG_RELOCATABLE=y, use relative
references in the smp.alt entries.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/include/asm/assembler.h |  4 ++--
 arch/arm/include/asm/processor.h |  2 +-
 arch/arm/kernel/head.S           | 10 +++++-----
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 516ebaf4ff38..3176bf7dcbd9 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -260,7 +260,7 @@
  */
 #define ALT_UP(instr...)					\
 	.pushsection ".alt.smp.init", "a"			;\
-	.long	9998b						;\
+	.long	9998b - .					;\
 9997:	instr							;\
 	.if . - 9997b == 2					;\
 		nop						;\
@@ -272,7 +272,7 @@
 #define ALT_UP_B(label)					\
 	.equ	up_b_offset, label - 9998b			;\
 	.pushsection ".alt.smp.init", "a"			;\
-	.long	9998b						;\
+	.long	9998b - .					;\
 	W(b)	. + up_b_offset					;\
 	.popsection
 #else
diff --git a/arch/arm/include/asm/processor.h b/arch/arm/include/asm/processor.h
index c3d5fc124a05..3667b395bb17 100644
--- a/arch/arm/include/asm/processor.h
+++ b/arch/arm/include/asm/processor.h
@@ -92,7 +92,7 @@ unsigned long get_wchan(struct task_struct *p);
 #define __ALT_SMP_ASM(smp, up)						\
 	"9998:	" smp "\n"						\
 	"	.pushsection \".alt.smp.init\", \"a\"\n"		\
-	"	.long	9998b\n"					\
+	"	.long	9998b - .\n"					\
 	"	" up "\n"						\
 	"	.popsection\n"
 #else
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 6e9df3663a57..ec22f42fd8bb 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -548,14 +548,15 @@ smp_on_up:
 __do_fixup_smp_on_up:
 	cmp	r4, r5
 	reths	lr
-	ldmia	r4!, {r0, r6}
- ARM(	str	r6, [r0, r3]	)
- THUMB(	add	r0, r0, r3	)
+	ldmia	r4, {r0, r6}
+ ARM(	str	r6, [r0, r4]	)
+ THUMB(	add	r0, r0, r4	)
+	add	r4, r4, #8
 #ifdef __ARMEB__
  THUMB(	mov	r6, r6, ror #16	)	@ Convert word order for big-endian.
 #endif
  THUMB(	strh	r6, [r0], #2	)	@ For Thumb-2, store as two halfwords
- THUMB(	mov	r6, r6, lsr #16	)	@ to be robust against misaligned r3.
+ THUMB(	mov	r6, r6, lsr #16	)	@ to be robust against misaligned r0.
  THUMB(	strh	r6, [r0]	)
 	b	__do_fixup_smp_on_up
 ENDPROC(__do_fixup_smp_on_up)
@@ -564,7 +565,6 @@ ENTRY(fixup_smp)
 	stmfd	sp!, {r4 - r6, lr}
 	mov	r4, r0
 	add	r5, r0, r1
-	mov	r3, #0
 	bl	__do_fixup_smp_on_up
 	ldmfd	sp!, {r4 - r6, pc}
 ENDPROC(fixup_smp)
-- 
2.11.0

[kernel-hardening] [PATCH 07/30] ARM: head: use PC-relative insn sequence for __smp_alt

[Ard Biesheuvel]

Replace the open coded PC relative offset calculations with a pair
of adr_l invocations. This ensures these quantities are invariant
under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head.S | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index ec22f42fd8bb..db6b823f20a4 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -523,19 +523,11 @@ ARM_BE8(rev	r0, r0)			@ byteswap if big endian
 	retne	lr
 
 __fixup_smp_on_up:
-	adr	r0, 1f
-	ldmia	r0, {r3 - r5}
-	sub	r3, r0, r3
-	add	r4, r4, r3
-	add	r5, r5, r3
+	adr_l	r4, __smpalt_begin
+	adr_l	r5, __smpalt_end
 	b	__do_fixup_smp_on_up
 ENDPROC(__fixup_smp)
 
-	.align
-1:	.word	.
-	.word	__smpalt_begin
-	.word	__smpalt_end
-
 	.pushsection .data
 	.globl	smp_on_up
 smp_on_up:
-- 
2.11.0

[kernel-hardening] [PATCH 08/30] ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash

[Ard Biesheuvel]

Replace the open coded PC relative offset calculations with adr_l and
ldr_l invocations.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/sleep.S | 16 +++-------------
 1 file changed, 3 insertions(+), 13 deletions(-)

diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
index 0f6c1000582c..3026b119d3ff 100644
--- a/arch/arm/kernel/sleep.S
+++ b/arch/arm/kernel/sleep.S
@@ -137,9 +137,8 @@ ARM_BE8(setend be)			@ ensure we are in BE mode
 	mov	r1, #0
 	ALT_SMP(mrc p15, 0, r0, c0, c0, 5)
 	ALT_UP_B(1f)
-	adr	r2, mpidr_hash_ptr
-	ldr	r3, [r2]
-	add	r2, r2, r3		@ r2 = struct mpidr_hash phys address
+	adr_l	r2, mpidr_hash		@ r2 = struct mpidr_hash phys address
+
 	/*
 	 * This ldmia relies on the memory layout of the mpidr_hash
 	 * struct mpidr_hash.
@@ -147,10 +146,7 @@ ARM_BE8(setend be)			@ ensure we are in BE mode
 	ldmia	r2, { r3-r6 }	@ r3 = mpidr mask (r4,r5,r6) = l[0,1,2] shifts
 	compute_mpidr_hash	r1, r4, r5, r6, r0, r3
 1:
-	adr	r0, _sleep_save_sp
-	ldr	r2, [r0]
-	add	r0, r0, r2
-	ldr	r0, [r0, #SLEEP_SAVE_SP_PHYS]
+	ldr_l	r0, sleep_save_sp + SLEEP_SAVE_SP_PHYS
 	ldr	r0, [r0, r1, lsl #2]
 
 	@ load phys pgd, stack, resume fn
@@ -164,12 +160,6 @@ ENDPROC(cpu_resume)
 ENDPROC(cpu_resume_arm)
 #endif
 
-	.align 2
-_sleep_save_sp:
-	.long	sleep_save_sp - .
-mpidr_hash_ptr:
-	.long	mpidr_hash - .			@ mpidr_hash struct offset
-
 	.data
 	.type	sleep_save_sp, #object
 ENTRY(sleep_save_sp)
-- 
2.11.0

[kernel-hardening] [PATCH 09/30] ARM: head.S: use PC-relative insn sequences for __fixup_pv_table

[Ard Biesheuvel]

Replace the open coded PC relative offset calculations with adr_l
and mov_l invocations. This ensures these quantities are invariant
under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head.S | 27 ++++++--------------
 1 file changed, 8 insertions(+), 19 deletions(-)

diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index db6b823f20a4..f607e290ef4b 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -578,14 +578,11 @@ ENDPROC(fixup_smp)
  */
 	__HEAD
 __fixup_pv_table:
-	adr	r0, 1f
-	ldmia	r0, {r3-r7}
+	adr_l	r6, __pv_phys_pfn_offset
+	adr_l	r7, __pv_offset			@ __pa(__pv_offset)
+	mov_l	r3, __pv_offset			@ __va(__pv_offset)
 	mvn	ip, #0
-	subs	r3, r0, r3	@ PHYS_OFFSET - PAGE_OFFSET
-	add	r4, r4, r3	@ adjust table start address
-	add	r5, r5, r3	@ adjust table end address
-	add	r6, r6, r3	@ adjust __pv_phys_pfn_offset address
-	add	r7, r7, r3	@ adjust __pv_offset address
+	subs	r3, r7, r3	@ PHYS_OFFSET - PAGE_OFFSET
 	mov	r0, r8, lsr #PAGE_SHIFT	@ convert to PFN
 	str	r0, [r6]	@ save computed PHYS_OFFSET to __pv_phys_pfn_offset
 	strcc	ip, [r7, #HIGH_OFFSET]	@ save to __pv_offset high bits
@@ -594,20 +591,15 @@ __fixup_pv_table:
 THUMB(	it	ne		@ cross section branch )
 	bne	__error
 	str	r3, [r7, #LOW_OFFSET]	@ save to __pv_offset low bits
+	adr_l	r4, __pv_table_begin
+	adr_l	r5, __pv_table_end
 	b	__fixup_a_pv_table
 ENDPROC(__fixup_pv_table)
-
-	.align
-1:	.long	.
-	.long	__pv_table_begin
-	.long	__pv_table_end
-2:	.long	__pv_phys_pfn_offset
-	.long	__pv_offset
+	.ltorg
 
 	.text
 __fixup_a_pv_table:
-	adr	r0, 3f
-	ldr	r6, [r0]
+	mov_l	r6, __pv_offset
 	add	r6, r6, r3
 	ldr	r0, [r6, #HIGH_OFFSET]	@ pv_offset high word
 	ldr	r6, [r6, #LOW_OFFSET]	@ pv_offset low word
@@ -676,9 +668,6 @@ ARM_BE8(rev16	ip, ip)
 #endif
 ENDPROC(__fixup_a_pv_table)
 
-	.align
-3:	.long __pv_offset
-
 ENTRY(fixup_pv_table)
 	stmfd	sp!, {r4 - r7, lr}
 	mov	r3, #0			@ no offset
-- 
2.11.0

[kernel-hardening] [PATCH 10/30] ARM: head.S: use PC relative insn sequence to calculate PHYS_OFFSET

[Ard Biesheuvel]

Replace the open coded arithmetic with a simple adr_l/sub pair. This
ensures these quantities are invariant under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head.S | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index f607e290ef4b..62c961849035 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -106,10 +106,8 @@ ENTRY(stext)
 #endif
 
 #ifndef CONFIG_XIP_KERNEL
-	adr	r3, 2f
-	ldmia	r3, {r4, r8}
-	sub	r4, r3, r4			@ (PHYS_OFFSET - PAGE_OFFSET)
-	add	r8, r8, r4			@ PHYS_OFFSET
+	adr_l	r8, _text			@ __pa(_text)
+	sub	r8, r8, #TEXT_OFFSET		@ PHYS_OFFSET
 #else
 	ldr	r8, =PLAT_PHYS_OFFSET		@ always constant in this case
 #endif
@@ -161,10 +159,6 @@ ENTRY(stext)
 1:	b	__enable_mmu
 ENDPROC(stext)
 	.ltorg
-#ifndef CONFIG_XIP_KERNEL
-2:	.long	.
-	.long	PAGE_OFFSET
-#endif
 
 /*
  * Setup the initial page tables.  We only setup the barest
-- 
2.11.0

[kernel-hardening] [PATCH 11/30] ARM: kvm: replace open coded VA->PA calculations with adr_l call

[Ard Biesheuvel]

Replace the open coded calculations of the actual physical address
of the KVM stub vector table with a single adr_l invocation.

Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/boot/compressed/head.S | 15 ++-------
 arch/arm/kernel/hyp-stub.S      | 33 +++++++-------------
 arch/arm/kvm/init.S             |  8 +----
 3 files changed, 15 insertions(+), 41 deletions(-)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 8a756870c238..5884e8151376 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -427,15 +427,10 @@ dtb_check_done:
 
 		/*
 		 * Compute the address of the hyp vectors after relocation.
-		 * This requires some arithmetic since we cannot directly
-		 * reference __hyp_stub_vectors in a PC-relative way.
 		 * Call __hyp_set_vectors with the new address so that we
 		 * can HVC again after the copy.
 		 */
-0:		adr	r0, 0b
-		movw	r1, #:lower16:__hyp_stub_vectors - 0b
-		movt	r1, #:upper16:__hyp_stub_vectors - 0b
-		add	r0, r0, r1
+		adr_l	r0, __hyp_stub_vectors
 		sub	r0, r0, r5
 		add	r0, r0, r10
 		bl	__hyp_set_vectors
@@ -568,17 +563,11 @@ not_relocated:	mov	r0, #0
 		cmp	r0, #HYP_MODE		@ if not booted in HYP mode...
 		bne	__enter_kernel		@ boot kernel directly
 
-		adr	r12, .L__hyp_reentry_vectors_offset
-		ldr	r0, [r12]
-		add	r0, r0, r12
-
+		adr_l	r0, __hyp_reentry_vectors
 		bl	__hyp_set_vectors
 		__HVC(0)			@ otherwise bounce to hyp mode
 
 		b	.			@ should never be reached
-
-		.align	2
-.L__hyp_reentry_vectors_offset:	.long	__hyp_reentry_vectors - .
 #else
 		b	__enter_kernel
 #endif
diff --git a/arch/arm/kernel/hyp-stub.S b/arch/arm/kernel/hyp-stub.S
index ec7e7377d423..55b5fab83861 100644
--- a/arch/arm/kernel/hyp-stub.S
+++ b/arch/arm/kernel/hyp-stub.S
@@ -36,41 +36,38 @@ ENTRY(__boot_cpu_mode)
 .text
 
 	/*
-	 * Save the primary CPU boot mode. Requires 3 scratch registers.
+	 * Save the primary CPU boot mode. Requires 2 scratch registers.
 	 */
-	.macro	store_primary_cpu_mode	reg1, reg2, reg3
+	.macro	store_primary_cpu_mode	reg1, reg2
 	mrs	\reg1, cpsr
 	and	\reg1, \reg1, #MODE_MASK
-	adr	\reg2, .L__boot_cpu_mode_offset
-	ldr	\reg3, [\reg2]
-	str	\reg1, [\reg2, \reg3]
+	str_l	\reg1, __boot_cpu_mode, \reg2
 	.endm
 
 	/*
 	 * Compare the current mode with the one saved on the primary CPU.
 	 * If they don't match, record that fact. The Z bit indicates
 	 * if there's a match or not.
-	 * Requires 3 additionnal scratch registers.
+	 * Requires 2 additionnal scratch registers.
 	 */
-	.macro	compare_cpu_mode_with_primary mode, reg1, reg2, reg3
-	adr	\reg2, .L__boot_cpu_mode_offset
-	ldr	\reg3, [\reg2]
-	ldr	\reg1, [\reg2, \reg3]
+	.macro	compare_cpu_mode_with_primary mode, reg1, reg2
+	adr_l	\reg2, __boot_cpu_mode
+	ldr	\reg1, [\reg2]
 	cmp	\mode, \reg1		@ matches primary CPU boot mode?
 	orrne	\reg1, \reg1, #BOOT_CPU_MODE_MISMATCH
-	strne	\reg1, [\reg2, \reg3]	@ record what happened and give up
+	strne	\reg1, [\reg2]		@ record what happened and give up
 	.endm
 
 #else	/* ZIMAGE */
 
-	.macro	store_primary_cpu_mode	reg1:req, reg2:req, reg3:req
+	.macro	store_primary_cpu_mode	reg1:req, reg2:req
 	.endm
 
 /*
  * The zImage loader only runs on one CPU, so we don't bother with mult-CPU
  * consistency checking:
  */
-	.macro	compare_cpu_mode_with_primary mode, reg1, reg2, reg3
+	.macro	compare_cpu_mode_with_primary mode, reg1, reg2
 	cmp	\mode, \mode
 	.endm
 
@@ -85,7 +82,7 @@ ENTRY(__boot_cpu_mode)
  */
 @ Call this from the primary CPU
 ENTRY(__hyp_stub_install)
-	store_primary_cpu_mode	r4, r5, r6
+	store_primary_cpu_mode	r4, r5
 ENDPROC(__hyp_stub_install)
 
 	@ fall through...
@@ -99,7 +96,7 @@ ENTRY(__hyp_stub_install_secondary)
 	 * If the secondary has booted with a different mode, give up
 	 * immediately.
 	 */
-	compare_cpu_mode_with_primary	r4, r5, r6, r7
+	compare_cpu_mode_with_primary	r4, r5, r6
 	retne	lr
 
 	/*
@@ -264,12 +261,6 @@ ENTRY(__hyp_reset_vectors)
 	ret	lr
 ENDPROC(__hyp_reset_vectors)
 
-#ifndef ZIMAGE
-.align 2
-.L__boot_cpu_mode_offset:
-	.long	__boot_cpu_mode - .
-#endif
-
 .align 5
 ENTRY(__hyp_stub_vectors)
 __hyp_stub_reset:	W(b)	.
diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
index 5386528665b5..d777c6fbd869 100644
--- a/arch/arm/kvm/init.S
+++ b/arch/arm/kvm/init.S
@@ -143,13 +143,7 @@ reset:
 	bic	r1, r1, r0
 	mcr	p15, 4, r1, c1, c0, 0	@ HSCTLR
 
-	/*
-	 * Install stub vectors, using ardb's VA->PA trick.
-	 */
-0:	adr	r0, 0b					@ PA(0)
-	movw	r1, #:lower16:__hyp_stub_vectors - 0b   @ VA(stub) - VA(0)
-	movt	r1, #:upper16:__hyp_stub_vectors - 0b
-	add	r1, r1, r0				@ PA(stub)
+	adr_l	r1, __hyp_stub_vectors			@ PA(stub)
 	mcr	p15, 4, r1, c12, c0, 0	@ HVBAR
 	b	exit
 
-- 
2.11.0

[kernel-hardening] [PATCH 12/30] arm-soc: exynos: replace open coded VA->PA conversions

[Ard Biesheuvel]

This replaces a couple of open coded calculations to obtain the
physical address of a far symbol with calls to the new adr_l etc
macros.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/mach-exynos/headsmp.S |  9 +------
 arch/arm/mach-exynos/sleep.S   | 26 ++++----------------
 2 files changed, 6 insertions(+), 29 deletions(-)

diff --git a/arch/arm/mach-exynos/headsmp.S b/arch/arm/mach-exynos/headsmp.S
index d3d24ab351ae..25de5aff1c41 100644
--- a/arch/arm/mach-exynos/headsmp.S
+++ b/arch/arm/mach-exynos/headsmp.S
@@ -23,10 +23,7 @@ ENTRY(exynos4_secondary_startup)
 ARM_BE8(setend	be)
 	mrc	p15, 0, r0, c0, c0, 5
 	and	r0, r0, #15
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
+	adr_l	r6, pen_release
 pen:	ldr	r7, [r6]
 	cmp	r7, r0
 	bne	pen
@@ -37,7 +34,3 @@ pen:	ldr	r7, [r6]
 	 */
 	b	secondary_startup
 ENDPROC(exynos4_secondary_startup)
-
-	.align 2
-1:	.long	.
-	.long	pen_release
diff --git a/arch/arm/mach-exynos/sleep.S b/arch/arm/mach-exynos/sleep.S
index cf950790fbdc..003465699263 100644
--- a/arch/arm/mach-exynos/sleep.S
+++ b/arch/arm/mach-exynos/sleep.S
@@ -17,6 +17,7 @@
 
 #include <linux/linkage.h>
 #include <asm/asm-offsets.h>
+#include <asm/assembler.h>
 #include <asm/hardware/cache-l2x0.h>
 #include "smc.h"
 
@@ -62,19 +63,13 @@ ENTRY(exynos_cpu_resume_ns)
 	cmp	r0, r1
 	bne	skip_cp15
 
-	adr	r0, _cp15_save_power
-	ldr	r1, [r0]
-	ldr	r1, [r0, r1]
-	adr	r0, _cp15_save_diag
-	ldr	r2, [r0]
-	ldr	r2, [r0, r2]
+	ldr_l	r1, cp15_save_power
+	ldr_l	r2, cp15_save_diag
 	mov	r0, #SMC_CMD_C15RESUME
 	dsb
 	smc	#0
 #ifdef CONFIG_CACHE_L2X0
-	adr	r0, 1f
-	ldr	r2, [r0]
-	add	r0, r2, r0
+	adr_l	r0, l2x0_saved_regs
 
 	/* Check that the address has been initialised. */
 	ldr	r1, [r0, #L2X0_R_PHY_BASE]
@@ -93,9 +88,7 @@ ENTRY(exynos_cpu_resume_ns)
 	smc	#0
 
 	/* Reload saved regs pointer because smc corrupts registers. */
-	adr	r0, 1f
-	ldr	r2, [r0]
-	add	r0, r2, r0
+	adr_l	r0, l2x0_saved_regs
 
 	ldr	r1, [r0, #L2X0_R_PWR_CTRL]
 	ldr	r2, [r0, #L2X0_R_AUX_CTRL]
@@ -114,15 +107,6 @@ skip_cp15:
 	b	cpu_resume
 ENDPROC(exynos_cpu_resume_ns)
 
-	.align
-_cp15_save_power:
-	.long	cp15_save_power - .
-_cp15_save_diag:
-	.long	cp15_save_diag - .
-#ifdef CONFIG_CACHE_L2X0
-1:	.long	l2x0_saved_regs - .
-#endif /* CONFIG_CACHE_L2X0 */
-
 	.data
 	.globl cp15_save_diag
 cp15_save_diag:
-- 
2.11.0

[kernel-hardening] [PATCH 13/30] arm-soc: mvebu: replace open coded VA->PA conversion

[Ard Biesheuvel]

This replaces an open coded calculation to obtain the physical
address of a far symbol with a call to the new ldr_l etc macro.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/mach-mvebu/coherency_ll.S | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
index 8b2fbc8b6bc6..df0ccb5cef1d 100644
--- a/arch/arm/mach-mvebu/coherency_ll.S
+++ b/arch/arm/mach-mvebu/coherency_ll.S
@@ -42,9 +42,7 @@ ENTRY(ll_get_coherency_base)
 	ldr	r1, =coherency_base
 	cmp	r1, #0
 	beq	2f
-	adr	r1, 3f
-	ldr	r3, [r1]
-	ldr	r1, [r1, r3]
+	ldr_l	r1, coherency_phys_base
 	b	2f
 1:
 	/*
@@ -160,7 +158,3 @@ ENTRY(ll_disable_coherency)
 	dsb
 	ret	lr
 ENDPROC(ll_disable_coherency)
-
-	.align 2
-3:
-	.long	coherency_phys_base - .
-- 
2.11.0

[kernel-hardening] [PATCH 14/30] arm-soc: various: replace open coded VA->PA calculation of pen_release

[Ard Biesheuvel]

This replaces a few copies of the open coded calculations of the
physical address of 'pen_release' in the secondary startup code
of a couple of platforms. This ensures these quantities are invariant
under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/mach-prima2/headsmp.S    | 11 +++--------
 arch/arm/mach-spear/headsmp.S     | 11 +++--------
 arch/arm/mach-sti/headsmp.S       | 10 +++-------
 arch/arm/plat-versatile/headsmp.S |  9 +--------
 4 files changed, 10 insertions(+), 31 deletions(-)

diff --git a/arch/arm/mach-prima2/headsmp.S b/arch/arm/mach-prima2/headsmp.S
index 209d9fc5c16c..070df700bb38 100644
--- a/arch/arm/mach-prima2/headsmp.S
+++ b/arch/arm/mach-prima2/headsmp.S
@@ -9,6 +9,8 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 
+#include <asm/assembler.h>
+
 /*
  * SIRFSOC specific entry point for secondary CPUs.  This provides
  * a "holding pen" into which all secondary cores are held until we're
@@ -17,10 +19,7 @@
 ENTRY(sirfsoc_secondary_startup)
         mrc     p15, 0, r0, c0, c0, 5
         and     r0, r0, #15
-        adr     r4, 1f
-        ldmia   r4, {r5, r6}
-        sub     r4, r4, r5
-        add     r6, r6, r4
+        adr_l   r6, pen_release
 pen:    ldr     r7, [r6]
         cmp     r7, r0
         bne     pen
@@ -31,7 +30,3 @@ pen:    ldr     r7, [r6]
          */
         b       secondary_startup
 ENDPROC(sirfsoc_secondary_startup)
-
-        .align
-1:      .long   .
-        .long   pen_release
diff --git a/arch/arm/mach-spear/headsmp.S b/arch/arm/mach-spear/headsmp.S
index c52192dc3d9f..4da01b103f33 100644
--- a/arch/arm/mach-spear/headsmp.S
+++ b/arch/arm/mach-spear/headsmp.S
@@ -13,6 +13,8 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 
+#include <asm/assembler.h>
+
 	__INIT
 
 /*
@@ -23,10 +25,7 @@
 ENTRY(spear13xx_secondary_startup)
 	mrc	p15, 0, r0, c0, c0, 5
 	and	r0, r0, #15
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
+	adr_l	r6, pen_release
 pen:	ldr	r7, [r6]
 	cmp	r7, r0
 	bne	pen
@@ -40,8 +39,4 @@ pen:	ldr	r7, [r6]
 	 * should now contain the SVC stack for this core
 	 */
 	b	secondary_startup
-
-	.align
-1:	.long	.
-	.long	pen_release
 ENDPROC(spear13xx_secondary_startup)
diff --git a/arch/arm/mach-sti/headsmp.S b/arch/arm/mach-sti/headsmp.S
index e0ad451700d5..cdf3442f397b 100644
--- a/arch/arm/mach-sti/headsmp.S
+++ b/arch/arm/mach-sti/headsmp.S
@@ -16,6 +16,8 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 
+#include <asm/assembler.h>
+
 /*
  * ST specific entry point for secondary CPUs.  This provides
  * a "holding pen" into which all secondary cores are held until we're
@@ -24,10 +26,7 @@
 ENTRY(sti_secondary_startup)
 	mrc	p15, 0, r0, c0, c0, 5
 	and	r0, r0, #15
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
+	adr_l	r6, pen_release
 pen:	ldr	r7, [r6]
 	cmp	r7, r0
 	bne	pen
@@ -38,6 +37,3 @@ pen:	ldr	r7, [r6]
 	 */
 	b	secondary_startup
 ENDPROC(sti_secondary_startup)
-
-1:	.long	.
-	.long	pen_release
diff --git a/arch/arm/plat-versatile/headsmp.S b/arch/arm/plat-versatile/headsmp.S
index 40f27e52de75..0f2a5eddac5a 100644
--- a/arch/arm/plat-versatile/headsmp.S
+++ b/arch/arm/plat-versatile/headsmp.S
@@ -21,10 +21,7 @@ ENTRY(versatile_secondary_startup)
  ARM_BE8(setend	be)
 	mrc	p15, 0, r0, c0, c0, 5
 	bic	r0, #0xff000000
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
+	adr_l	r6, pen_release
 pen:	ldr	r7, [r6]
 	cmp	r7, r0
 	bne	pen
@@ -34,8 +31,4 @@ pen:	ldr	r7, [r6]
 	 * should now contain the SVC stack for this core
 	 */
 	b	secondary_startup
-
-	.align
-1:	.long	.
-	.long	pen_release
 ENDPROC(versatile_secondary_startup)
-- 
2.11.0

[kernel-hardening] [PATCH 15/30] ARM: kernel: switch to relative exception tables

[Ard Biesheuvel]

To avoid having to relocate the contents of extable entries at
runtime when running with KASLR enabled, wire up the existing
support for emitting them as relative references. This ensures
these quantities are invariant under runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/include/asm/Kbuild           |  1 -
 arch/arm/include/asm/assembler.h      |  6 +++---
 arch/arm/include/asm/extable.h        | 20 ++++++++++++++++++
 arch/arm/include/asm/futex.h          |  2 +-
 arch/arm/include/asm/uaccess.h        |  8 +++----
 arch/arm/include/asm/word-at-a-time.h |  2 +-
 arch/arm/kernel/entry-armv.S          |  6 +++---
 arch/arm/kernel/swp_emulate.c         |  4 ++--
 arch/arm/lib/backtrace.S              |  8 +++----
 arch/arm/lib/getuser.S                | 22 ++++++++++----------
 arch/arm/lib/putuser.S                | 12 +++++------
 arch/arm/mm/alignment.c               | 14 ++++++-------
 arch/arm/mm/extable.c                 |  2 +-
 arch/arm/nwfpe/entry.S                |  2 +-
 scripts/sortextable.c                 |  2 +-
 15 files changed, 65 insertions(+), 46 deletions(-)

diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild
index 721ab5ecfb9b..f2d67fa34eb3 100644
--- a/arch/arm/include/asm/Kbuild
+++ b/arch/arm/include/asm/Kbuild
@@ -3,7 +3,6 @@ generic-y += current.h
 generic-y += early_ioremap.h
 generic-y += emergency-restart.h
 generic-y += exec.h
-generic-y += extable.h
 generic-y += irq_regs.h
 generic-y += kdebug.h
 generic-y += local.h
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 3176bf7dcbd9..09b6b28f2595 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -247,7 +247,7 @@
 9999:	x;					\
 	.pushsection __ex_table,"a";		\
 	.align	3;				\
-	.long	9999b,9001f;			\
+	.long	9999b - ., 9001f - .;		\
 	.popsection
 
 #ifdef CONFIG_SMP
@@ -383,7 +383,7 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 
 	.pushsection __ex_table,"a"
 	.align	3
-	.long	9999b, \abort
+	.long	9999b - ., \abort - .
 	.popsection
 	.endm
 
@@ -424,7 +424,7 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 
 	.pushsection __ex_table,"a"
 	.align	3
-	.long	9999b, \abort
+	.long	9999b - ., \abort - .
 	.popsection
 	.endr
 	.endm
diff --git a/arch/arm/include/asm/extable.h b/arch/arm/include/asm/extable.h
new file mode 100644
index 000000000000..3610cc116a86
--- /dev/null
+++ b/arch/arm/include/asm/extable.h
@@ -0,0 +1,20 @@
+#ifndef __ASM_EXTABLE_H
+#define __ASM_EXTABLE_H
+
+/*
+ * The exception table consists of pairs of relative offsets: the first
+ * is the relative offset to an instruction that is allowed to fault,
+ * and the second is the relative offset at which the program should
+ * continue. No registers are modified, so it is entirely up to the
+ * continuation code to figure out what to do.
+ */
+
+struct exception_table_entry
+{
+	int insn, fixup;
+};
+
+#define ARCH_HAS_RELATIVE_EXTABLE
+
+extern int fixup_exception(struct pt_regs *regs);
+#endif
diff --git a/arch/arm/include/asm/futex.h b/arch/arm/include/asm/futex.h
index 6795368ad023..31760d0ab76b 100644
--- a/arch/arm/include/asm/futex.h
+++ b/arch/arm/include/asm/futex.h
@@ -11,7 +11,7 @@
 	"3:\n"							\
 	"	.pushsection __ex_table,\"a\"\n"		\
 	"	.align	3\n"					\
-	"	.long	1b, 4f, 2b, 4f\n"			\
+	"	.long	1b - ., 4f - ., 2b - ., 4f - .\n"	\
 	"	.popsection\n"					\
 	"	.pushsection .text.fixup,\"ax\"\n"		\
 	"	.align	2\n"					\
diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h
index 0bf2347495f1..62b9bb7d991a 100644
--- a/arch/arm/include/asm/uaccess.h
+++ b/arch/arm/include/asm/uaccess.h
@@ -295,7 +295,7 @@ do {									\
 	"	.popsection\n"					\
 	"	.pushsection __ex_table,\"a\"\n"		\
 	"	.align	3\n"					\
-	"	.long	1b, 3b\n"				\
+	"	.long	1b - ., 3b - .\n"			\
 	"	.popsection"					\
 	: "+r" (err), "=&r" (x)					\
 	: "r" (addr), "i" (-EFAULT)				\
@@ -385,7 +385,7 @@ do {									\
 	"	.popsection\n"					\
 	"	.pushsection __ex_table,\"a\"\n"		\
 	"	.align	3\n"					\
-	"	.long	1b, 3b\n"				\
+	"	.long	1b - ., 3b - .\n"			\
 	"	.popsection"					\
 	: "+r" (err)						\
 	: "r" (x), "r" (__pu_addr), "i" (-EFAULT)		\
@@ -435,8 +435,8 @@ do {									\
 	"	.popsection\n"					\
 	"	.pushsection __ex_table,\"a\"\n"		\
 	"	.align	3\n"					\
-	"	.long	1b, 4b\n"				\
-	"	.long	2b, 4b\n"				\
+	"	.long	1b - ., 4b - .\n"			\
+	"	.long	2b - ., 4b - .\n"			\
 	"	.popsection"					\
 	: "+r" (err), "+r" (__pu_addr)				\
 	: "r" (x), "i" (-EFAULT)				\
diff --git a/arch/arm/include/asm/word-at-a-time.h b/arch/arm/include/asm/word-at-a-time.h
index 5831dce4b51c..d433c686d9ca 100644
--- a/arch/arm/include/asm/word-at-a-time.h
+++ b/arch/arm/include/asm/word-at-a-time.h
@@ -86,7 +86,7 @@ static inline unsigned long load_unaligned_zeropad(const void *addr)
 	"	.popsection\n"
 	"	.pushsection __ex_table,\"a\"\n"
 	"	.align	3\n"
-	"	.long	1b, 3b\n"
+	"	.long	1b - ., 3b - .\n"
 	"	.popsection"
 	: "=&r" (ret), "=&r" (offset)
 	: "r" (addr), "Qo" (*(unsigned long *)addr));
diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
index c731f0d2b2af..1c7c12123b18 100644
--- a/arch/arm/kernel/entry-armv.S
+++ b/arch/arm/kernel/entry-armv.S
@@ -570,10 +570,10 @@ ENDPROC(__und_usr)
 	ret	r9
 	.popsection
 	.pushsection __ex_table,"a"
-	.long	1b, 4b
+	.long	1b - ., 4b - .
 #if CONFIG_ARM_THUMB && __LINUX_ARM_ARCH__ >= 6 && CONFIG_CPU_V7
-	.long	2b, 4b
-	.long	3b, 4b
+	.long	2b - ., 4b - .
+	.long	3b - ., 4b - .
 #endif
 	.popsection
 
diff --git a/arch/arm/kernel/swp_emulate.c b/arch/arm/kernel/swp_emulate.c
index 3bda08bee674..a783e9c0b266 100644
--- a/arch/arm/kernel/swp_emulate.c
+++ b/arch/arm/kernel/swp_emulate.c
@@ -50,8 +50,8 @@
 	"	.previous\n"					\
 	"	.section	 __ex_table,\"a\"\n"		\
 	"	.align		3\n"				\
-	"	.long		0b, 3b\n"			\
-	"	.long		1b, 3b\n"			\
+	"	.long		0b - ., 3b - .\n"		\
+	"	.long		1b - ., 3b - .\n"		\
 	"	.previous"					\
 	: "=&r" (res), "+r" (data), "=&r" (temp)		\
 	: "r" (addr), "i" (-EAGAIN), "i" (-EFAULT)		\
diff --git a/arch/arm/lib/backtrace.S b/arch/arm/lib/backtrace.S
index 7d7952e5a3b1..84a8df7aa63c 100644
--- a/arch/arm/lib/backtrace.S
+++ b/arch/arm/lib/backtrace.S
@@ -107,10 +107,10 @@ ENDPROC(c_backtrace)
 		
 		.pushsection __ex_table,"a"
 		.align	3
-		.long	1001b, 1006b
-		.long	1002b, 1006b
-		.long	1003b, 1006b
-		.long	1004b, 1006b
+		.long	1001b - ., 1006b - .
+		.long	1002b - ., 1006b - .
+		.long	1003b - ., 1006b - .
+		.long	1004b - ., 1006b - .
 		.popsection
 
 .Lbad:		.asciz	"Backtrace aborted due to bad frame pointer <%p>\n"
diff --git a/arch/arm/lib/getuser.S b/arch/arm/lib/getuser.S
index df73914e81c8..d47b4776d308 100644
--- a/arch/arm/lib/getuser.S
+++ b/arch/arm/lib/getuser.S
@@ -133,17 +133,17 @@ ENDPROC(__get_user_bad)
 ENDPROC(__get_user_bad8)
 
 .pushsection __ex_table, "a"
-	.long	1b, __get_user_bad
-	.long	2b, __get_user_bad
-	.long	3b, __get_user_bad
-	.long	4b, __get_user_bad
-	.long	5b, __get_user_bad8
-	.long	6b, __get_user_bad8
+	.long	1b - ., __get_user_bad - .
+	.long	2b - ., __get_user_bad - .
+	.long	3b - ., __get_user_bad - .
+	.long	4b - ., __get_user_bad - .
+	.long	5b - ., __get_user_bad8 - .
+	.long	6b - ., __get_user_bad8 - .
 #ifdef __ARMEB__
-	.long   7b, __get_user_bad
-	.long	8b, __get_user_bad8
-	.long	9b, __get_user_bad8
-	.long	10b, __get_user_bad8
-	.long	11b, __get_user_bad8
+	.long   7b - ., __get_user_bad - .
+	.long	8b - ., __get_user_bad8 - .
+	.long	9b - ., __get_user_bad8 - .
+	.long	10b - ., __get_user_bad8 - .
+	.long	11b - ., __get_user_bad8 - .
 #endif
 .popsection
diff --git a/arch/arm/lib/putuser.S b/arch/arm/lib/putuser.S
index 38d660d3705f..6b854197ff48 100644
--- a/arch/arm/lib/putuser.S
+++ b/arch/arm/lib/putuser.S
@@ -89,10 +89,10 @@ __put_user_bad:
 ENDPROC(__put_user_bad)
 
 .pushsection __ex_table, "a"
-	.long	1b, __put_user_bad
-	.long	2b, __put_user_bad
-	.long	3b, __put_user_bad
-	.long	4b, __put_user_bad
-	.long	5b, __put_user_bad
-	.long	6b, __put_user_bad
+	.long	1b - ., __put_user_bad - .
+	.long	2b - ., __put_user_bad - .
+	.long	3b - ., __put_user_bad - .
+	.long	4b - ., __put_user_bad - .
+	.long	5b - ., __put_user_bad - .
+	.long	6b - ., __put_user_bad - .
 .popsection
diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
index 2c96190e018b..2c74cb826b6a 100644
--- a/arch/arm/mm/alignment.c
+++ b/arch/arm/mm/alignment.c
@@ -209,7 +209,7 @@ union offset_union {
 	"	.popsection\n"				\
 	"	.pushsection __ex_table,\"a\"\n"	\
 	"	.align	3\n"				\
-	"	.long	1b, 3b\n"			\
+	"	.long	1b - ., 3b - .\n"		\
 	"	.popsection\n"				\
 	: "=r" (err), "=&r" (val), "=r" (addr)		\
 	: "0" (err), "2" (addr))
@@ -269,8 +269,8 @@ union offset_union {
 		"	.popsection\n"				\
 		"	.pushsection __ex_table,\"a\"\n"	\
 		"	.align	3\n"				\
-		"	.long	1b, 4b\n"			\
-		"	.long	2b, 4b\n"			\
+		"	.long	1b - ., 4b - .\n"		\
+		"	.long	2b - ., 4b - .\n"		\
 		"	.popsection\n"				\
 		: "=r" (err), "=&r" (v), "=&r" (a)		\
 		: "0" (err), "1" (v), "2" (a));			\
@@ -309,10 +309,10 @@ union offset_union {
 		"	.popsection\n"				\
 		"	.pushsection __ex_table,\"a\"\n"	\
 		"	.align	3\n"				\
-		"	.long	1b, 6b\n"			\
-		"	.long	2b, 6b\n"			\
-		"	.long	3b, 6b\n"			\
-		"	.long	4b, 6b\n"			\
+		"	.long	1b - ., 6b - .\n"		\
+		"	.long	2b - ., 6b - .\n"		\
+		"	.long	3b - ., 6b - .\n"		\
+		"	.long	4b - ., 6b - .\n"		\
 		"	.popsection\n"				\
 		: "=r" (err), "=&r" (v), "=&r" (a)		\
 		: "0" (err), "1" (v), "2" (a));			\
diff --git a/arch/arm/mm/extable.c b/arch/arm/mm/extable.c
index f436f7439e46..6a8b85bf0cac 100644
--- a/arch/arm/mm/extable.c
+++ b/arch/arm/mm/extable.c
@@ -10,7 +10,7 @@ int fixup_exception(struct pt_regs *regs)
 
 	fixup = search_exception_tables(instruction_pointer(regs));
 	if (fixup) {
-		regs->ARM_pc = fixup->fixup;
+		regs->ARM_pc = (unsigned long)&fixup->fixup + fixup->fixup;
 #ifdef CONFIG_THUMB2_KERNEL
 		/* Clear the IT state to avoid nasty surprises in the fixup */
 		regs->ARM_cpsr &= ~PSR_IT_MASK;
diff --git a/arch/arm/nwfpe/entry.S b/arch/arm/nwfpe/entry.S
index 39c20afad7ed..c4fe13b31705 100644
--- a/arch/arm/nwfpe/entry.S
+++ b/arch/arm/nwfpe/entry.S
@@ -121,5 +121,5 @@ next:
 
 	.pushsection __ex_table,"a"
 	.align	3
-	.long	.Lx1, .Lfix
+	.long	.Lx1 - ., .Lfix - .
 	.popsection
diff --git a/scripts/sortextable.c b/scripts/sortextable.c
index 365a907f98b3..56a4c6714da7 100644
--- a/scripts/sortextable.c
+++ b/scripts/sortextable.c
@@ -314,6 +314,7 @@ do_file(char const *const fname)
 		break;
 
 	case EM_S390:
+	case EM_ARM:
 	case EM_AARCH64:
 	case EM_PARISC:
 	case EM_PPC:
@@ -322,7 +323,6 @@ do_file(char const *const fname)
 		break;
 	case EM_ARCOMPACT:
 	case EM_ARCV2:
-	case EM_ARM:
 	case EM_MICROBLAZE:
 	case EM_MIPS:
 	case EM_XTENSA:
-- 
2.11.0

[kernel-hardening] [PATCH 16/30] ARM: kernel: use relative phys-to-virt patch tables

[Ard Biesheuvel]

Replace the contents of the __pv_table entries with relative references
so that we don't have to relocate them at runtime when running the KASLR
kernel. This ensures these quantities are invariant under runtime
relocation, which makes any cache maintenance after runtime relocation
unnecessary.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/include/asm/memory.h |  6 +++---
 arch/arm/kernel/head.S        | 21 ++++++++++----------
 2 files changed, 13 insertions(+), 14 deletions(-)

diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 1f54e4e98c1e..47a984e3a244 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -195,7 +195,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
 	__asm__("@ __pv_stub\n"				\
 	"1:	" instr "	%0, %1, %2\n"		\
 	"	.pushsection .pv_table,\"a\"\n"		\
-	"	.long	1b\n"				\
+	"	.long	1b - .\n"			\
 	"	.popsection\n"				\
 	: "=r" (to)					\
 	: "r" (from), "I" (type))
@@ -204,7 +204,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
 	__asm__ volatile("@ __pv_stub_mov\n"		\
 	"1:	mov	%R0, %1\n"			\
 	"	.pushsection .pv_table,\"a\"\n"		\
-	"	.long	1b\n"				\
+	"	.long	1b - .\n"			\
 	"	.popsection\n"				\
 	: "=r" (t)					\
 	: "I" (__PV_BITS_7_0))
@@ -214,7 +214,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
 	"1:	adds	%Q0, %1, %2\n"			\
 	"	adc	%R0, %R0, #0\n"			\
 	"	.pushsection .pv_table,\"a\"\n"		\
-	"	.long	1b\n"				\
+	"	.long	1b - .\n"			\
 	"	.popsection\n"				\
 	: "+r" (y)					\
 	: "r" (x), "I" (__PV_BITS_31_24)		\
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 62c961849035..5d685e86148c 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -593,8 +593,7 @@ ENDPROC(__fixup_pv_table)
 
 	.text
 __fixup_a_pv_table:
-	mov_l	r6, __pv_offset
-	add	r6, r6, r3
+	adr_l	r6, __pv_offset
 	ldr	r0, [r6, #HIGH_OFFSET]	@ pv_offset high word
 	ldr	r6, [r6, #LOW_OFFSET]	@ pv_offset low word
 	mov	r6, r6, lsr #24
@@ -612,22 +611,22 @@ __fixup_a_pv_table:
 	orr	r6, r6, r7, lsl #12
 	orr	r6, #0x4000
 	b	2f
-1:	add     r7, r3
-	ldrh	ip, [r7, #2]
+1:	add	r7, r4
+	ldrh	ip, [r7, #-2]
 ARM_BE8(rev16	ip, ip)
 	tst	ip, #0x4000
 	and	ip, #0x8f00
 	orrne	ip, r6	@ mask in offset bits 31-24
 	orreq	ip, r0	@ mask in offset bits 7-0
 ARM_BE8(rev16	ip, ip)
-	strh	ip, [r7, #2]
+	strh	ip, [r7, #-2]
 	bne	2f
-	ldrh	ip, [r7]
+	ldrh	ip, [r7, #-4]
 ARM_BE8(rev16	ip, ip)
 	bic	ip, #0x20
 	orr	ip, ip, r0, lsr #16
 ARM_BE8(rev16	ip, ip)
-	strh	ip, [r7]
+	strh	ip, [r7, #-4]
 2:	cmp	r4, r5
 	ldrcc	r7, [r4], #4	@ use branch for delay slot
 	bcc	1b
@@ -639,7 +638,8 @@ ARM_BE8(rev16	ip, ip)
 	moveq	r0, #0x400000	@ set bit 22, mov to mvn instruction
 #endif
 	b	2f
-1:	ldr	ip, [r7, r3]
+1:	ldr	ip, [r7, r4]!
+	add	r4, r4, #4
 #ifdef CONFIG_CPU_ENDIAN_BE8
 	@ in BE8, we load data in BE, but instructions still in LE
 	bic	ip, ip, #0xff000000
@@ -654,9 +654,9 @@ ARM_BE8(rev16	ip, ip)
 	biceq	ip, ip, #0x400000	@ clear bit 22
 	orreq	ip, ip, r0	@ mask in offset bits 7-0
 #endif
-	str	ip, [r7, r3]
+	str	ip, [r7]
 2:	cmp	r4, r5
-	ldrcc	r7, [r4], #4	@ use branch for delay slot
+	ldrcc	r7, [r4]	@ use branch for delay slot
 	bcc	1b
 	ret	lr
 #endif
@@ -664,7 +664,6 @@ ENDPROC(__fixup_a_pv_table)
 
 ENTRY(fixup_pv_table)
 	stmfd	sp!, {r4 - r7, lr}
-	mov	r3, #0			@ no offset
 	mov	r4, r0			@ r0 = table start
 	add	r5, r0, r1		@ r1 = table size
 	bl	__fixup_a_pv_table
-- 
2.11.0

[kernel-hardening] [PATCH 17/30] arm-soc: tegra: make sleep asm code runtime relocatable

[Ard Biesheuvel]

The PIE kernel build does not allow absolute references encoded in
movw/movt instruction pairs, so use our mov_l macro instead (which
will still use such a pair unless CONFIG_RELOCATABLE is defined)

Also, avoid 32-bit absolute literals to refer to absolute symbols.
Instead, use a 16 bit reference so that PIE linker cannot get
confused whether the symbol reference is subject to relocation at
runtime.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/mach-tegra/sleep-tegra20.S | 22 ++++++++++++--------
 arch/arm/mach-tegra/sleep-tegra30.S |  6 +++---
 arch/arm/mach-tegra/sleep.S         |  4 ++--
 3 files changed, 18 insertions(+), 14 deletions(-)

diff --git a/arch/arm/mach-tegra/sleep-tegra20.S b/arch/arm/mach-tegra/sleep-tegra20.S
index 5c8e638ee51a..cab95de5c8f1 100644
--- a/arch/arm/mach-tegra/sleep-tegra20.S
+++ b/arch/arm/mach-tegra/sleep-tegra20.S
@@ -99,7 +99,7 @@ ENTRY(tegra20_cpu_shutdown)
 	cmp	r0, #0
 	reteq	lr			@ must not be called for CPU 0
 	mov32	r1, TEGRA_IRAM_RESET_BASE_VIRT
-	ldr	r2, =__tegra20_cpu1_resettable_status_offset
+	ldrh	r2, 0f
 	mov	r12, #CPU_RESETTABLE
 	strb	r12, [r1, r2]
 
@@ -121,6 +121,7 @@ ENTRY(tegra20_cpu_shutdown)
 	beq	.
 	ret	lr
 ENDPROC(tegra20_cpu_shutdown)
+0:	.short	__tegra20_cpu1_resettable_status_offset
 #endif
 
 #ifdef CONFIG_PM_SLEEP
@@ -181,6 +182,9 @@ ENTRY(tegra_pen_unlock)
 	ret     lr
 ENDPROC(tegra_pen_unlock)
 
+.L__tegra20_cpu1_resettable_status_offset:
+	.short	__tegra20_cpu1_resettable_status_offset
+
 /*
  * tegra20_cpu_clear_resettable(void)
  *
@@ -189,7 +193,7 @@ ENDPROC(tegra_pen_unlock)
  */
 ENTRY(tegra20_cpu_clear_resettable)
 	mov32	r1, TEGRA_IRAM_RESET_BASE_VIRT
-	ldr	r2, =__tegra20_cpu1_resettable_status_offset
+	ldrh	r2, .L__tegra20_cpu1_resettable_status_offset
 	mov	r12, #CPU_NOT_RESETTABLE
 	strb	r12, [r1, r2]
 	ret	lr
@@ -203,7 +207,7 @@ ENDPROC(tegra20_cpu_clear_resettable)
  */
 ENTRY(tegra20_cpu_set_resettable_soon)
 	mov32	r1, TEGRA_IRAM_RESET_BASE_VIRT
-	ldr	r2, =__tegra20_cpu1_resettable_status_offset
+	ldrh	r2, .L__tegra20_cpu1_resettable_status_offset
 	mov	r12, #CPU_RESETTABLE_SOON
 	strb	r12, [r1, r2]
 	ret	lr
@@ -217,7 +221,7 @@ ENDPROC(tegra20_cpu_set_resettable_soon)
  */
 ENTRY(tegra20_cpu_is_resettable_soon)
 	mov32	r1, TEGRA_IRAM_RESET_BASE_VIRT
-	ldr	r2, =__tegra20_cpu1_resettable_status_offset
+	ldrh	r2, .L__tegra20_cpu1_resettable_status_offset
 	ldrb	r12, [r1, r2]
 	cmp	r12, #CPU_RESETTABLE_SOON
 	moveq	r0, #1
@@ -238,11 +242,11 @@ ENTRY(tegra20_sleep_core_finish)
 	bl	tegra_disable_clean_inv_dcache
 	mov     r0, r4
 
-	mov32	r3, tegra_shut_off_mmu
+	mov_l	r3, tegra_shut_off_mmu
 	add	r3, r3, r0
 
-	mov32	r0, tegra20_tear_down_core
-	mov32	r1, tegra20_iram_start
+	mov_l	r0, tegra20_tear_down_core
+	mov_l	r1, tegra20_iram_start
 	sub	r0, r0, r1
 	mov32	r1, TEGRA_IRAM_LPx_RESUME_AREA
 	add	r0, r0, r1
@@ -265,7 +269,7 @@ ENTRY(tegra20_sleep_cpu_secondary_finish)
 	bl	tegra_disable_clean_inv_dcache
 
 	mov32	r0, TEGRA_IRAM_RESET_BASE_VIRT
-	ldr	r4, =__tegra20_cpu1_resettable_status_offset
+	ldrh	r4, .L__tegra20_cpu1_resettable_status_offset
 	mov	r3, #CPU_RESETTABLE
 	strb	r3, [r0, r4]
 
@@ -284,7 +288,7 @@ ENTRY(tegra20_sleep_cpu_secondary_finish)
 	bl	tegra_pen_lock
 
 	mov32	r0, TEGRA_IRAM_RESET_BASE_VIRT
-	ldr	r4, =__tegra20_cpu1_resettable_status_offset
+	ldrh	r4, .L__tegra20_cpu1_resettable_status_offset
 	mov	r3, #CPU_NOT_RESETTABLE
 	strb	r3, [r0, r4]
 
diff --git a/arch/arm/mach-tegra/sleep-tegra30.S b/arch/arm/mach-tegra/sleep-tegra30.S
index dd4a67dabd91..478b2ca3ef6e 100644
--- a/arch/arm/mach-tegra/sleep-tegra30.S
+++ b/arch/arm/mach-tegra/sleep-tegra30.S
@@ -261,11 +261,11 @@ ENTRY(tegra30_sleep_core_finish)
 	mov32	r6, TEGRA_FLOW_CTRL_BASE
 	mov32	r7, TEGRA_TMRUS_BASE
 
-	mov32	r3, tegra_shut_off_mmu
+	mov_l	r3, tegra_shut_off_mmu
 	add	r3, r3, r0
 
-	mov32	r0, tegra30_tear_down_core
-	mov32	r1, tegra30_iram_start
+	mov_l	r0, tegra30_tear_down_core
+	mov_l	r1, tegra30_iram_start
 	sub	r0, r0, r1
 	mov32	r1, TEGRA_IRAM_LPx_RESUME_AREA
 	add	r0, r0, r1
diff --git a/arch/arm/mach-tegra/sleep.S b/arch/arm/mach-tegra/sleep.S
index 5e3496753df1..785df3edc767 100644
--- a/arch/arm/mach-tegra/sleep.S
+++ b/arch/arm/mach-tegra/sleep.S
@@ -101,11 +101,11 @@ ENTRY(tegra_sleep_cpu_finish)
 	bl	tegra_disable_clean_inv_dcache
 
 	mov	r0, r4
-	mov32	r6, tegra_tear_down_cpu
+	mov_l	r6, tegra_tear_down_cpu
 	ldr	r1, [r6]
 	add	r1, r1, r0
 
-	mov32	r3, tegra_shut_off_mmu
+	mov_l	r3, tegra_shut_off_mmu
 	add	r3, r3, r0
 	mov	r0, r1
 
-- 
2.11.0

[kernel-hardening] [PATCH 18/30] ARM: kernel: make vmlinux buildable as a PIE executable

[Ard Biesheuvel]

Update the build flags and linker script to allow vmlinux to be build
as a PIE  binary, which retains relocation information about absolute
symbol references so that they can be fixed up at runtime. This will
be used for implementing KASLR,

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/Kconfig                 |  4 ++++
 arch/arm/Makefile                |  5 +++++
 arch/arm/include/asm/assembler.h |  7 ++++++-
 arch/arm/kernel/vmlinux.lds.S    |  9 +++++++++
 include/linux/hidden.h           | 21 ++++++++++++++++++++
 5 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 61a0cb15067e..300add3b8023 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -2085,6 +2085,10 @@ config DMI
 	  firmware need to be enabled. This would require the DMI subsystem
 	  to be enabled much earlier than we do on ARM, which is non-trivial.
 
+config RELOCATABLE
+	bool
+	select HAVE_ARCH_PREL32_RELOCATIONS
+
 endmenu
 
 menu "CPU Power Management"
diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 47d3a1ab08d2..88d9c33e24f5 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -52,6 +52,11 @@ AS		+= -EL
 LD		+= -EL
 endif
 
+ifeq ($(CONFIG_RELOCATABLE),y)
+KBUILD_CFLAGS	+= -include $(srctree)/include/linux/hidden.h -fpic
+LDFLAGS_vmlinux	+= -pie -shared -Bsymbolic
+endif
+
 #
 # The Scalar Replacement of Aggregates (SRA) optimization pass in GCC 4.9 and
 # later may result in code being generated that handles signed short and signed
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 09b6b28f2595..b2254c1dc2de 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -550,7 +550,12 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 	 * mov_l - move a constant value or [relocated] address into a register
 	 */
 	.macro		mov_l, dst:req, imm:req, cond
-	.if		__LINUX_ARM_ARCH__ < 7
+	.if		CONFIG_RELOCATABLE == 1
+	ldr_l		\dst, 333f, \cond
+	.section	".data.rel.ro.local", "aw", %progbits
+333:	.long		\imm
+	.previous
+	.elseif		__LINUX_ARM_ARCH__ < 7
 	ldr\cond	\dst, =\imm
 	.else
 	W(movw\cond\())	\dst, #:lower16:\imm
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index c83a7ba737d6..5853d4be2067 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -89,6 +89,9 @@ SECTIONS
 #endif
 		*(.discard)
 		*(.discard.*)
+		*(.ARM.exidx.discard.text)
+		*(.interp .dynamic)
+		*(.dynsym .dynstr .hash)
 	}
 
 	. = PAGE_OFFSET + TEXT_OFFSET;
@@ -209,6 +212,12 @@ SECTIONS
 		__smpalt_end = .;
 	}
 #endif
+	.rel.dyn : ALIGN(8) {
+		__rel_begin = .;
+		*(.rel .rel.* .rel.dyn)
+	}
+	__rel_end = ADDR(.rel.dyn) + SIZEOF(.rel.dyn);
+
 	.init.pv_table : {
 		__pv_table_begin = .;
 		*(.pv_table)
diff --git a/include/linux/hidden.h b/include/linux/hidden.h
new file mode 100644
index 000000000000..49b7823eed6e
--- /dev/null
+++ b/include/linux/hidden.h
@@ -0,0 +1,21 @@
+
+/*
+ * GCC assumes that we are building shared libraries or hosted binaries
+ * when the -fpic or -fpie switches are used. This results in all references
+ * to symbols with external linkage to be redirected via entries in the global
+ * offset table (GOT), which keeps .text pages clean and reduces the footprint
+ * of CoWed dirty pages to the GOT itself. It also allows symbol preemption,
+ * which is mandatory under ELF rules for shared libraries.
+ *
+ * For the kernel, we use PIC so that we can relocate the executable image at
+ * runtime. This does not involve CoW or symbol preemption, and we'd rather
+ * have relative references instead of absolute ones whenever possible.
+ * So set the default visibility to hidden: this informs the compiler that
+ * none of our symbols will ever be exported from a shared library, allowing
+ * it to use relative references where possible.
+ *
+ * Note that simply passing -fvisibility=hidden is not sufficient to achieve
+ * this: In that case, definitions will be marked hidden, but declarations
+ * will not, and we still end up with GOT entries unnecessarily.
+ */
+#pragma GCC visibility push(hidden)
-- 
2.11.0

[kernel-hardening] [PATCH 19/30] ARM: kernel: use PC-relative symbol references in MMU switch code

[Ard Biesheuvel]

To prepare for adding support for KASLR, which relocates all absolute
symbol references at runtime after the caches have been enabled,
update the MMU switch code to avoid using absolute symbol references
where possible. This ensures these quantities are invariant under
runtime relocation.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head-common.S | 39 ++++++++------------
 1 file changed, 15 insertions(+), 24 deletions(-)

diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
index 06035488130c..b74477507a12 100644
--- a/arch/arm/kernel/head-common.S
+++ b/arch/arm/kernel/head-common.S
@@ -79,9 +79,10 @@ ENDPROC(__vet_atags)
  */
 	__INIT
 __mmap_switched:
-	adr	r3, __mmap_switched_data
-
-	ldmia	r3!, {r4, r5, r6, r7}
+	adr_l	r4, __data_loc
+	adr_l	r5, _sdata
+	adr_l	r6, __bss_start
+	adr_l	r7, _end
 	cmp	r4, r5				@ Copy data segment if needed
 1:	cmpne	r5, r6
 	ldrne	fp, [r4], #4
@@ -93,9 +94,17 @@ __mmap_switched:
 	strcc	fp, [r6],#4
 	bcc	1b
 
- ARM(	ldmia	r3, {r4, r5, r6, r7, sp})
- THUMB(	ldmia	r3, {r4, r5, r6, r7}	)
- THUMB(	ldr	sp, [r3, #16]		)
+	adr_l	r3, init_thread_union + THREAD_START_SP
+	mov	sp, r3
+	adr_l	r4, processor_id
+	adr_l	r5, __machine_arch_type
+	adr_l	r6, __atags_pointer
+#ifdef CONFIG_CPU_CP15
+	adr_l	r7, cr_alignment
+#else
+	mov	r7, #0
+#endif
+
 	str	r9, [r4]			@ Save processor ID
 	str	r1, [r5]			@ Save machine type
 	str	r2, [r6]			@ Save atags pointer
@@ -104,24 +113,6 @@ __mmap_switched:
 	b	start_kernel
 ENDPROC(__mmap_switched)
 
-	.align	2
-	.type	__mmap_switched_data, %object
-__mmap_switched_data:
-	.long	__data_loc			@ r4
-	.long	_sdata				@ r5
-	.long	__bss_start			@ r6
-	.long	_end				@ r7
-	.long	processor_id			@ r4
-	.long	__machine_arch_type		@ r5
-	.long	__atags_pointer			@ r6
-#ifdef CONFIG_CPU_CP15
-	.long	cr_alignment			@ r7
-#else
-	.long	0				@ r7
-#endif
-	.long	init_thread_union + THREAD_START_SP @ sp
-	.size	__mmap_switched_data, . - __mmap_switched_data
-
 /*
  * This provides a C-API version of __lookup_processor_type
  */
-- 
2.11.0

[kernel-hardening] [PATCH 20/30] ARM: kernel: use PC relative symbol references in suspend/resume code

[Ard Biesheuvel]

Replace some unnecessary absolute references with relative ones. Also,
to prepare for runtime relocation, which occurs with the caches on,
defer taking the absolute address of cpu_resume_after_mmu() until after
the MMU is enabled.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/sleep.S | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
index 3026b119d3ff..9efd1c7d3552 100644
--- a/arch/arm/kernel/sleep.S
+++ b/arch/arm/kernel/sleep.S
@@ -60,18 +60,17 @@
 ENTRY(__cpu_suspend)
 	stmfd	sp!, {r4 - r11, lr}
 #ifdef MULTI_CPU
-	ldr	r10, =processor
-	ldr	r4, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
+	ldr_l	r4, processor + CPU_SLEEP_SIZE	@ size of CPU sleep state
 #else
-	ldr	r4, =cpu_suspend_size
+	adr_l	r4, cpu_suspend_size
 #endif
 	mov	r5, sp			@ current virtual SP
 	add	r4, r4, #12		@ Space for pgd, virt sp, phys resume fn
 	sub	sp, sp, r4		@ allocate CPU state on stack
-	ldr	r3, =sleep_save_sp
+	adr_l	r3, sleep_save_sp
 	stmfd	sp!, {r0, r1}		@ save suspend func arg and pointer
 	ldr	r3, [r3, #SLEEP_SAVE_SP_VIRT]
-	ALT_SMP(ldr r0, =mpidr_hash)
+	ALT_SMP(adr_l r0, mpidr_hash)
 	ALT_UP_B(1f)
 	/* This ldmia relies on the memory layout of the mpidr_hash struct */
 	ldmia	r0, {r1, r6-r8}	@ r1 = mpidr mask (r6,r7,r8) = l[0,1,2] shifts
@@ -100,13 +99,13 @@ ENDPROC(cpu_suspend_abort)
 	.align	5
 	.pushsection	.idmap.text,"ax"
 ENTRY(cpu_resume_mmu)
-	ldr	r3, =cpu_resume_after_mmu
 	instr_sync
 	mcr	p15, 0, r0, c1, c0, 0	@ turn on MMU, I-cache, etc
 	mrc	p15, 0, r0, c0, c0, 0	@ read id reg
 	instr_sync
 	mov	r0, r0
 	mov	r0, r0
+	ldr	r3, =cpu_resume_after_mmu
 	ret	r3			@ jump to virtual address
 ENDPROC(cpu_resume_mmu)
 	.popsection
-- 
2.11.0

[kernel-hardening] [PATCH 21/30] ARM: mm: export default vmalloc base address

[Ard Biesheuvel]

In order for the EFI stub to be able to decide over what range to
randomize the load address of the kernel, expose the definition of
the default vmalloc base address as VMALLOC_DEFAULT_BASE.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/include/asm/pgtable.h | 1 +
 arch/arm/mm/mmu.c              | 3 +--
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
index 1c462381c225..a0d4a63a6516 100644
--- a/arch/arm/include/asm/pgtable.h
+++ b/arch/arm/include/asm/pgtable.h
@@ -45,6 +45,7 @@
 #define VMALLOC_OFFSET		(8*1024*1024)
 #define VMALLOC_START		(((unsigned long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))
 #define VMALLOC_END		0xff800000UL
+#define VMALLOC_DEFAULT_BASE	(VMALLOC_END - (240 << 20) - VMALLOC_OFFSET)
 
 #define LIBRARY_TEXT_START	0x0c000000
 
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index e46a6a446cdd..566f7e657931 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1128,8 +1128,7 @@ void __init debug_ll_io_init(void)
 }
 #endif
 
-static void * __initdata vmalloc_min =
-	(void *)(VMALLOC_END - (240 << 20) - VMALLOC_OFFSET);
+static void * __initdata vmalloc_min = (void *)VMALLOC_DEFAULT_BASE;
 
 /*
  * vmalloc=size forces the vmalloc area to be exactly 'size'
-- 
2.11.0

[kernel-hardening] [PATCH 22/30] ARM: kernel: refer to swapper_pg_dir via its symbol

[Ard Biesheuvel]

The location of swapper_pg_dir is relative to the kernel, not to
PAGE_OFFSET or PHYS_OFFSET. So define the symbol relative to the
start of the kernel image, and refer to it via its name.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/kernel/head.S | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 5d685e86148c..71bc0d037bc9 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -48,14 +48,6 @@
 #define PMD_ORDER	2
 #endif
 
-	.globl	swapper_pg_dir
-	.equ	swapper_pg_dir, KERNEL_RAM_VADDR - PG_DIR_SIZE
-
-	.macro	pgtbl, rd, phys
-	add	\rd, \phys, #TEXT_OFFSET
-	sub	\rd, \rd, #PG_DIR_SIZE
-	.endm
-
 /*
  * Kernel startup entry point.
  * ---------------------------
@@ -77,6 +69,9 @@
 	.arm
 
 	__HEAD
+	.globl	swapper_pg_dir
+	.equ	swapper_pg_dir, . - PG_DIR_SIZE
+
 ENTRY(stext)
  ARM_BE8(setend	be )			@ ensure we are in BE8 mode
 
@@ -172,7 +167,7 @@ ENDPROC(stext)
  *  r4 = physical page table address
  */
 __create_page_tables:
-	pgtbl	r4, r8				@ page table address
+	adr_l	r4, swapper_pg_dir		@ page table address
 
 	/*
 	 * Clear the swapper page table
-- 
2.11.0

[kernel-hardening] [PATCH 23/30] ARM: kernel: implement randomization of the kernel load address

[Ard Biesheuvel]

This implements randomization of the placement of the kernel image
inside the lowmem region. It is intended to work together with the
decompressor to place the kernel at an offset in physical memory
that is a multiple of 2 MB, and to take the same offset into account
when creating the virtual mapping.

This uses runtime relocation of the kernel built as a PIE binary, to
fix up all absolute symbol references to refer to their runtime virtual
address. The physical-to-virtual mapping remains unchanged.

In order to allow the decompressor to hand over to the core kernel
without making assumptions that are not guaranteed to hold when
invoking the core kernel directly using bootloaders that are not
KASLR aware, the KASLR offset is expected to be placed in r3 when
entering the kernel 4 bytes past the entry point, skipping the first
instruction.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/Kconfig       |  15 +++
 arch/arm/kernel/head.S | 103 ++++++++++++++++++--
 2 files changed, 109 insertions(+), 9 deletions(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 300add3b8023..0d7b40f1061d 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1825,6 +1825,21 @@ config XEN
 	help
 	  Say Y if you want to run Linux in a Virtual Machine on Xen on ARM.
 
+config RANDOMIZE_BASE
+	bool "Randomize the address of the kernel image"
+	depends on MMU && AUTO_ZRELADDR
+	depends on !XIP_KERNEL && !ZBOOT_ROM && !HAVE_TCM
+	select RELOCATABLE
+	help
+	  Randomizes the virtual and physical address at which the kernel
+	  image is loaded, as a security feature that deters exploit attempts
+	  relying on knowledge of the location of kernel internals.
+
+	  This relies on the UEFI stub to invoke the EFI_RNG_PROTOCOL to
+	  randomize the load address of the decompressed kernel in the
+	  physical space. The same offset is applied to the virtual mapping
+	  of the kernel in the virtual space by the kernel proper.
+
 endmenu
 
 menu "Boot options"
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 71bc0d037bc9..0795da990dde 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -48,6 +48,28 @@
 #define PMD_ORDER	2
 #endif
 
+	.macro	get_kaslr_offset, reg
+#ifdef CONFIG_RANDOMIZE_BASE
+	ldr_l			\reg, __kaslr_offset
+#else
+	mov			\reg, #0
+#endif
+	.endm
+
+	.macro	add_kaslr_offset, reg, tmp
+#ifdef CONFIG_RANDOMIZE_BASE
+	get_kaslr_offset	\tmp
+	add			\reg, \reg, \tmp
+#endif
+	.endm
+
+	.macro	sub_kaslr_offset, reg, tmp
+#ifdef CONFIG_RANDOMIZE_BASE
+	get_kaslr_offset	\tmp
+	sub			\reg, \reg, \tmp
+#endif
+	.endm
+
 /*
  * Kernel startup entry point.
  * ---------------------------
@@ -73,6 +95,7 @@
 	.equ	swapper_pg_dir, . - PG_DIR_SIZE
 
 ENTRY(stext)
+	mov	r3, #0			@ normal entry point - clear r3
  ARM_BE8(setend	be )			@ ensure we are in BE8 mode
 
  THUMB(	badr	r9, 1f		)	@ Kernel is always entered in ARM.
@@ -80,6 +103,16 @@ ENTRY(stext)
  THUMB(	.thumb			)	@ switch to Thumb now.
  THUMB(1:			)
 
+#ifdef CONFIG_RANDOMIZE_BASE
+	str_l	r3, __kaslr_offset, r9	@ offset in r3 if entered via kaslr ep
+
+	.section ".bss", "aw", %nobits
+	.align	2
+__kaslr_offset:
+	.long	0			@ will be wiped before entering C code
+	.previous
+#endif
+
 #ifdef CONFIG_ARM_VIRT_EXT
 	bl	__hyp_stub_install
 #endif
@@ -103,6 +136,7 @@ ENTRY(stext)
 #ifndef CONFIG_XIP_KERNEL
 	adr_l	r8, _text			@ __pa(_text)
 	sub	r8, r8, #TEXT_OFFSET		@ PHYS_OFFSET
+	sub_kaslr_offset r8, r12
 #else
 	ldr	r8, =PLAT_PHYS_OFFSET		@ always constant in this case
 #endif
@@ -139,8 +173,8 @@ ENTRY(stext)
 	 * r0 will hold the CPU control register value, r1, r2, r4, and
 	 * r9 will be preserved.  r5 will also be preserved if LPAE.
 	 */
-	ldr	r13, =__mmap_switched		@ address to jump to after
-						@ mmu has been enabled
+	adr_l	lr, __primary_switch		@ address to jump to after
+	mov	r13, lr				@ mmu has been enabled
 	badr	lr, 1f				@ return (PIC) address
 #ifdef CONFIG_ARM_LPAE
 	mov	r5, #0				@ high TTBR0
@@ -151,7 +185,8 @@ ENTRY(stext)
 	ldr	r12, [r10, #PROCINFO_INITFUNC]
 	add	r12, r12, r10
 	ret	r12
-1:	b	__enable_mmu
+1:	get_kaslr_offset r12			@ get before turning MMU on
+	b	__enable_mmu
 ENDPROC(stext)
 	.ltorg
 
@@ -230,9 +265,14 @@ __create_page_tables:
 	/*
 	 * Map our RAM from the start to the end of the kernel .bss section.
 	 */
-	add	r0, r4, #PAGE_OFFSET >> (SECTION_SHIFT - PMD_ORDER)
-	ldr	r6, =(_end - 1)
-	orr	r3, r8, r7
+	get_kaslr_offset r3
+	add	r0, r3, #PAGE_OFFSET
+	add	r0, r4, r0, lsr #(SECTION_SHIFT - PMD_ORDER)
+	adr_l	r6, _end - 1
+	sub	r6, r6, r8
+	add	r6, r6, #PAGE_OFFSET
+	add	r3, r3, r8
+	orr	r3, r3, r7
 	add	r6, r4, r6, lsr #(SECTION_SHIFT - PMD_ORDER)
 1:	str	r3, [r0], #1 << PMD_ORDER
 	add	r3, r3, #1 << SECTION_SHIFT
@@ -376,7 +416,7 @@ ENTRY(secondary_startup)
 	 * Use the page tables supplied from  __cpu_up.
 	 */
 	adr_l	r3, secondary_data
-	mov_l	r12, __secondary_switched
+	adr_l	r12, __secondary_switch
 	ldrd	r4, [r3, #0]			@ get secondary_data.pgdir
 ARM_BE8(eor	r4, r4, r5)			@ Swap r5 and r4 in BE:
 ARM_BE8(eor	r5, r4, r5)			@ it can be done in 3 steps
@@ -414,6 +454,7 @@ ENDPROC(__secondary_switched)
  *  r4  = TTBR pointer (low word)
  *  r5  = TTBR pointer (high word if LPAE)
  *  r9  = processor ID
+ *  r12 = KASLR offset
  *  r13 = *virtual* address to jump to upon completion
  */
 __enable_mmu:
@@ -451,6 +492,7 @@ ENDPROC(__enable_mmu)
  *  r1  = machine ID
  *  r2  = atags or dtb pointer
  *  r9  = processor ID
+ *  r12 = KASLR offset
  *  r13 = *virtual* address to jump to upon completion
  *
  * other registers depend on the function called upon completion
@@ -466,10 +508,52 @@ ENTRY(__turn_mmu_on)
 	mov	r3, r3
 	mov	r3, r13
 	ret	r3
-__turn_mmu_on_end:
 ENDPROC(__turn_mmu_on)
-	.popsection
 
+__primary_switch:
+#ifdef CONFIG_RELOCATABLE
+	adr_l	r7, _text			@ r7 := __pa(_text)
+	sub	r7, r7, #TEXT_OFFSET		@ r7 := PHYS_OFFSET
+
+	adr_l	r5, __rel_begin
+	adr_l	r6, __rel_end
+	sub	r5, r5, r7
+	sub	r6, r6, r7
+
+	add	r5, r5, #PAGE_OFFSET
+	add	r6, r6, #PAGE_OFFSET
+	add	r5, r5, r12
+	add	r6, r6, r12
+
+	adr_l	r3, __stubs_start		@ __pa(__stubs_start)
+	sub	r3, r3, r7			@ offset of __stubs_start
+	add	r3, r3, #PAGE_OFFSET		@ __va(__stubs_start)
+	sub	r3, r3, #0xffff1000		@ subtract VA of stubs section
+
+0:	cmp	r5, r6
+	bge	1f
+	ldm	r5!, {r7, r8}			@ load next relocation entry
+	cmp	r8, #23				@ R_ARM_RELATIVE
+	bne	0b
+	cmp	r7, #0xff000000			@ vector page?
+	addgt	r7, r7, r3			@ fix up VA offset
+	ldr	r8, [r7, r12]
+	add	r8, r8, r12
+	str	r8, [r7, r12]
+	b	0b
+1:
+#endif
+	ldr	pc, =__mmap_switched
+ENDPROC(__primary_switch)
+
+#ifdef CONFIG_SMP
+__secondary_switch:
+	ldr	pc, =__secondary_switched
+ENDPROC(__secondary_switch)
+#endif
+	.ltorg
+__turn_mmu_on_end:
+	.popsection
 
 #ifdef CONFIG_SMP_ON_UP
 	__HEAD
@@ -570,6 +654,7 @@ __fixup_pv_table:
 	adr_l	r6, __pv_phys_pfn_offset
 	adr_l	r7, __pv_offset			@ __pa(__pv_offset)
 	mov_l	r3, __pv_offset			@ __va(__pv_offset)
+	add_kaslr_offset r3, ip
 	mvn	ip, #0
 	subs	r3, r7, r3	@ PHYS_OFFSET - PAGE_OFFSET
 	mov	r0, r8, lsr #PAGE_SHIFT	@ convert to PFN
-- 
2.11.0

[kernel-hardening] [PATCH 24/30] ARM: decompressor: explicitly map decompressor binary cacheable

[Ard Biesheuvel]

When randomizing the kernel load address, there may be a large
distance in memory between the decompressor binary and its payload
and the destination area in memory. Ensure that the decompressor
itself is mapped cacheable in this case, by tweaking the existing
routine that takes care of this for XIP decompressors.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/boot/compressed/head.S | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 5884e8151376..583cc6899d98 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -706,20 +706,24 @@ __setup_mmu:	sub	r3, r4, #16384		@ Page directory size
 		teq	r0, r2
 		bne	1b
 /*
- * If ever we are running from Flash, then we surely want the cache
- * to be enabled also for our execution instance...  We map 2MB of it
- * so there is no map overlap problem for up to 1 MB compressed kernel.
- * If the execution is in RAM then we would only be duplicating the above.
+ * Make sure our entire executable image (including payload) is mapped
+ * cacheable, in case it is located outside the region we covered above.
+ * (This may be the case if running from flash or with randomization enabled)
+ * If the regions happen to overlap, we just duplicate some of the above.
  */
 		orr	r1, r6, #0x04		@ ensure B is set for this
 		orr	r1, r1, #3 << 10
 		mov	r2, pc
+		adr_l	r9, _end
 		mov	r2, r2, lsr #20
+		mov	r9, r9, lsr #20
 		orr	r1, r1, r2, lsl #20
 		add	r0, r3, r2, lsl #2
-		str	r1, [r0], #4
+		add	r9, r3, r9, lsl #2
+0:		str	r1, [r0], #4
 		add	r1, r1, #1048576
-		str	r1, [r0]
+		cmp	r0, r9
+		bls	0b
 		mov	pc, lr
 ENDPROC(__setup_mmu)
 
-- 
2.11.0

[kernel-hardening] [PATCH 25/30] ARM: compressed: factor out zImage header and make it extensible

[Ard Biesheuvel]

To prepare for adding metadata to the zImage to put KASLR randomization
under the control of the bootloader, factor out the zImage header, and
make it extensible by adding two new fields: a magic number that cannot
be mistaken for a valid instruction, to prevent misidentification, and
an offset into the binary where an array of optional headers is placed.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/boot/compressed/head.S        |  7 +---
 arch/arm/boot/compressed/vmlinux.lds.S |  5 ++-
 arch/arm/include/asm/zimage.h          | 39 ++++++++++++++++++++
 3 files changed, 45 insertions(+), 6 deletions(-)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 583cc6899d98..e451738d8954 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -11,6 +11,7 @@
 #include <linux/linkage.h>
 #include <asm/assembler.h>
 #include <asm/v7m.h>
+#include <asm/zimage.h>
 
 #include "efi-header.S"
 
@@ -139,11 +140,7 @@ start:
 #endif
 		W(b)	1f
 
-		.word	_magic_sig	@ Magic numbers to help the loader
-		.word	_magic_start	@ absolute load/run zImage address
-		.word	_magic_end	@ zImage end address
-		.word	0x04030201	@ endianness flag
-
+		__ZIMAGE_HEADER
 		__EFI_HEADER
 1:
  ARM_BE8(	setend	be		)	@ go BE8 if compiled for BE8
diff --git a/arch/arm/boot/compressed/vmlinux.lds.S b/arch/arm/boot/compressed/vmlinux.lds.S
index 81c493156ce8..27696bc315d3 100644
--- a/arch/arm/boot/compressed/vmlinux.lds.S
+++ b/arch/arm/boot/compressed/vmlinux.lds.S
@@ -6,6 +6,8 @@
  * published by the Free Software Foundation.
  */
 
+#include <asm/zimage.h>
+
 #ifdef CONFIG_CPU_ENDIAN_BE8
 #define ZIMAGE_MAGIC(x) ( (((x) >> 24) & 0x000000ff) | \
 			  (((x) >>  8) & 0x0000ff00) | \
@@ -72,9 +74,10 @@ SECTIONS
   .pad			: { BYTE(0); . = ALIGN(8); }
   _edata = .;
 
-  _magic_sig = ZIMAGE_MAGIC(0x016f2818);
+  _magic_sig = ZIMAGE_MAGIC(ZIMAGE_HEADER_MAGIC);
   _magic_start = ZIMAGE_MAGIC(_start);
   _magic_end = ZIMAGE_MAGIC(_edata);
+  _magic_opt_sig = ZIMAGE_MAGIC(ZIMAGE_OPTIONAL_HEADER_MAGIC);
 
   . = BSS_START;
   __bss_start = .;
diff --git a/arch/arm/include/asm/zimage.h b/arch/arm/include/asm/zimage.h
new file mode 100644
index 000000000000..ff65cc3bb716
--- /dev/null
+++ b/arch/arm/include/asm/zimage.h
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2017 Linaro Ltd;  <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __ASM_ZIMAGE_H
+#define __ASM_ZIMAGE_H
+
+#define ZIMAGE_HEADER_MAGIC		0x016f2818
+#define ZIMAGE_OPTIONAL_HEADER_MAGIC	0xe7fedef0
+
+#if defined(__ASSEMBLY__) && !defined(LINKER_SCRIPT)
+
+	.macro		__ZIMAGE_HEADER
+	.word		_magic_sig	@ Magic numbers to help the loader
+	.word		_magic_start	@ absolute load/run zImage address
+	.word		_magic_end	@ zImage end address
+	.word		0x04030201	@ endianness flag
+
+	/* optional headers */
+	.word		_magic_opt_sig	@ optional header magic number
+	.word		__zimage_opt_header - .
+
+	.pushsection	".rodata", "a", %progbits
+__zimage_opt_header:
+	/*
+	 * Each header starts with a u16[2] containing id and size of the
+	 * entire header, including the u16[] itself.
+	 */
+	.long		0xffffffff	@ end of optional headers
+	.popsection
+	.endm
+
+#endif
+#endif
-- 
2.11.0

[kernel-hardening] [PATCH 26/30] ARM: decompressor: add KASLR support

[Ard Biesheuvel]

Add support to the decompressor to load the kernel at a randomized
offset, and invoke the kernel proper while passing on the information
about the offset at which the kernel was loaded.

This implementation was created with the UEFI stub in mind (which has
a rich execution environment that provides access to the platforms
random number generators), which will assign the kaslr_offset variable
directly. However, to allow other bootloaders to use this facility,
the KASLR offset is exposed via a zImage header field as well.

Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/boot/compressed/head.S |  8 ++++++
 arch/arm/include/asm/zimage.h   | 30 ++++++++++++++++++--
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index e451738d8954..7111a2cbef95 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -200,6 +200,10 @@ not_angel:
 		 */
 		mov	r4, pc
 		and	r4, r4, #0xf8000000
+#ifdef CONFIG_RANDOMIZE_BASE
+		ldr_l	r0, kaslr_offset
+		add	r4, r4, r0
+#endif
 		/* Determine final kernel image address. */
 		add	r4, r4, #TEXT_OFFSET
 #else
@@ -1353,6 +1357,10 @@ __hyp_reentry_vectors:
 
 __enter_kernel:
 		mov	r0, #0			@ must be 0
+#ifdef CONFIG_RANDOMIZE_BASE
+		ldr_l	r3, kaslr_offset
+		add	r4, r4, #4		@ skip first instruction
+#endif
  ARM(		mov	pc, r4		)	@ call kernel
  M_CLASS(	add	r4, r4, #1	)	@ enter in Thumb mode for M class
  THUMB(		bx	r4		)	@ entry point is always ARM for A/R classes
diff --git a/arch/arm/include/asm/zimage.h b/arch/arm/include/asm/zimage.h
index ff65cc3bb716..554a48ddcfd8 100644
--- a/arch/arm/include/asm/zimage.h
+++ b/arch/arm/include/asm/zimage.h
@@ -10,10 +10,15 @@
 #ifndef __ASM_ZIMAGE_H
 #define __ASM_ZIMAGE_H
 
+#include <asm/pgtable.h>
+
 #define ZIMAGE_HEADER_MAGIC		0x016f2818
 #define ZIMAGE_OPTIONAL_HEADER_MAGIC	0xe7fedef0
 
-#if defined(__ASSEMBLY__) && !defined(LINKER_SCRIPT)
+#define ZIMAGE_OPT_HDR_ID_KASLR		0x1
+
+#ifndef LINKER_SCRIPT
+#ifdef __ASSEMBLY__
 
 	.macro		__ZIMAGE_HEADER
 	.word		_magic_sig	@ Magic numbers to help the loader
@@ -31,9 +36,30 @@
 	 * Each header starts with a u16[2] containing id and size of the
 	 * entire header, including the u16[] itself.
 	 */
+
+#ifdef CONFIG_RANDOMIZE_BASE
+0:	.short		ZIMAGE_OPT_HDR_ID_KASLR
+	.short		__kaslr_hdr_size
+
+	/*
+	 * The KASLR header carries the information needed by the bootloader
+	 * to choose a randomization offset, and record it in the offset
+	 * field below.
+	 */
+ENTRY(kaslr_offset)
+	.long		0				@ kaslr offset
+	.long		CONFIG_PAGE_OFFSET		@ page offset
+	.long		VMALLOC_DEFAULT_BASE		@ start of vmalloc area
+	.long		SECTION_SIZE			@ kaslr granularity
+	.set		__kaslr_hdr_size, . - 0b
+#endif
+
 	.long		0xffffffff	@ end of optional headers
 	.popsection
 	.endm
 
-#endif
+#else /* __ASSEMBLY__ */
+extern u32 kaslr_offset;
+#endif /* __ASSEMBLY__ */
+#endif /* LINKER_SCRIPT */
 #endif
-- 
2.11.0

[kernel-hardening] [PATCH 27/30] efi/libstub: add 'max' parameter to efi_random_alloc()

[Ard Biesheuvel]

Add an upper limit to efi_random_alloc() so we can use it to randomly
allocate the ARM kernel in lowmem.

Cc: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 drivers/firmware/efi/libstub/arm64-stub.c |  2 +-
 drivers/firmware/efi/libstub/efistub.h    |  3 ++-
 drivers/firmware/efi/libstub/random.c     | 10 ++++++----
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/firmware/efi/libstub/arm64-stub.c b/drivers/firmware/efi/libstub/arm64-stub.c
index b4c2589d7c91..940766f90adb 100644
--- a/drivers/firmware/efi/libstub/arm64-stub.c
+++ b/drivers/firmware/efi/libstub/arm64-stub.c
@@ -94,7 +94,7 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table_arg,
 		*reserve_size = kernel_memsize + offset;
 		status = efi_random_alloc(sys_table_arg, *reserve_size,
 					  MIN_KIMG_ALIGN, reserve_addr,
-					  (u32)phys_seed);
+					  (u32)phys_seed, ULONG_MAX);
 
 		*image_addr = *reserve_addr + offset;
 	} else {
diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h
index 83f268c05007..3a670a5f759f 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -60,7 +60,8 @@ efi_status_t efi_get_random_bytes(efi_system_table_t *sys_table,
 
 efi_status_t efi_random_alloc(efi_system_table_t *sys_table_arg,
 			      unsigned long size, unsigned long align,
-			      unsigned long *addr, unsigned long random_seed);
+			      unsigned long *addr, unsigned long random_seed,
+			      unsigned long max);
 
 efi_status_t check_platform_features(efi_system_table_t *sys_table_arg);
 
diff --git a/drivers/firmware/efi/libstub/random.c b/drivers/firmware/efi/libstub/random.c
index 7e72954d5860..85b80a4a85b3 100644
--- a/drivers/firmware/efi/libstub/random.c
+++ b/drivers/firmware/efi/libstub/random.c
@@ -42,7 +42,8 @@ efi_status_t efi_get_random_bytes(efi_system_table_t *sys_table_arg,
  */
 static unsigned long get_entry_num_slots(efi_memory_desc_t *md,
 					 unsigned long size,
-					 unsigned long align_shift)
+					 unsigned long align_shift,
+					 unsigned long max)
 {
 	unsigned long align = 1UL << align_shift;
 	u64 first_slot, last_slot, region_end;
@@ -50,7 +51,7 @@ static unsigned long get_entry_num_slots(efi_memory_desc_t *md,
 	if (md->type != EFI_CONVENTIONAL_MEMORY)
 		return 0;
 
-	region_end = min((u64)ULONG_MAX, md->phys_addr + md->num_pages*EFI_PAGE_SIZE - 1);
+	region_end = min((u64)max, md->phys_addr + md->num_pages*EFI_PAGE_SIZE - 1);
 
 	first_slot = round_up(md->phys_addr, align);
 	last_slot = round_down(region_end - size + 1, align);
@@ -73,7 +74,8 @@ efi_status_t efi_random_alloc(efi_system_table_t *sys_table_arg,
 			      unsigned long size,
 			      unsigned long align,
 			      unsigned long *addr,
-			      unsigned long random_seed)
+			      unsigned long random_seed,
+			      unsigned long max)
 {
 	unsigned long map_size, desc_size, total_slots = 0, target_slot;
 	unsigned long buff_size;
@@ -101,7 +103,7 @@ efi_status_t efi_random_alloc(efi_system_table_t *sys_table_arg,
 		efi_memory_desc_t *md = (void *)memory_map + map_offset;
 		unsigned long slots;
 
-		slots = get_entry_num_slots(md, size, ilog2(align));
+		slots = get_entry_num_slots(md, size, ilog2(align), max);
 		MD_NUM_SLOTS(md) = slots;
 		total_slots += slots;
 	}
-- 
2.11.0

[kernel-hardening] [PATCH 28/30] efi/libstub: check for vmalloc= command line argument

[Ard Biesheuvel]

Check for and record the presence of a vmalloc= argument on the
kernel command line. We need this information on ARM systems when
implementing KASLR, given that the size of the vmalloc region will
affect the size of the lowmem region, therefore affecting the
available randomization range as well.

Cc: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 drivers/firmware/efi/libstub/efi-stub-helper.c | 9 +++++++++
 drivers/firmware/efi/libstub/efistub.h         | 1 +
 2 files changed, 10 insertions(+)

diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c b/drivers/firmware/efi/libstub/efi-stub-helper.c
index b0184360efc6..f3e9d43030ac 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -34,6 +34,7 @@ static unsigned long __chunk_size = EFI_READ_CHUNK_SIZE;
 
 static int __section(.data) __nokaslr;
 static int __section(.data) __quiet;
+static int __section(.data) __vmalloc_arg;
 
 int __pure nokaslr(void)
 {
@@ -43,6 +44,10 @@ int __pure is_quiet(void)
 {
 	return __quiet;
 }
+int __pure have_vmalloc(void)
+{
+	return __vmalloc_arg;
+}
 
 #define EFI_MMAP_NR_SLACK_SLOTS	8
 
@@ -433,6 +438,10 @@ efi_status_t efi_parse_options(char const *cmdline)
 	if (str == cmdline || (str && str > cmdline && *(str - 1) == ' '))
 		__quiet = 1;
 
+	str = strstr(cmdline, "vmalloc=");
+	if (str == cmdline || (str && str > cmdline && *(str - 1) == ' '))
+		__vmalloc_arg = 1;
+
 	/*
 	 * If no EFI parameters were specified on the cmdline we've got
 	 * nothing to do.
diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h
index 3a670a5f759f..aaf2aeb785ea 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -26,6 +26,7 @@
 
 extern int __pure nokaslr(void);
 extern int __pure is_quiet(void);
+extern int __pure have_vmalloc(void);
 
 #define pr_efi(sys_table, msg)		do {				\
 	if (!is_quiet()) efi_printk(sys_table, "EFI stub: "msg);	\
-- 
2.11.0

[kernel-hardening] [PATCH 29/30] efi/libstub: arm: reserve bootloader supplied initrd in memory map

[Ard Biesheuvel]

Under KASLR, the EFI stub may allocate the kernel anywhere in the
physical address space, which could be right on top of an initrd
if it was supplied by the bootloader (i.e., GRUB) in /chosen rather
than passed via the initrd= command line option. So allocate the
pages explicitly, this ensures that the random memory allocation
routine will disregard the region.

Note that this means that we need to defer the handle_kernel_image()
call.

Cc: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 drivers/firmware/efi/libstub/arm-stub.c | 51 ++++++++++++--------
 drivers/firmware/efi/libstub/efistub.h  |  3 ++
 drivers/firmware/efi/libstub/fdt.c      | 42 ++++++++++++++++
 3 files changed, 75 insertions(+), 21 deletions(-)

diff --git a/drivers/firmware/efi/libstub/arm-stub.c b/drivers/firmware/efi/libstub/arm-stub.c
index 8181ac179d14..f5ef5ccd5f45 100644
--- a/drivers/firmware/efi/libstub/arm-stub.c
+++ b/drivers/firmware/efi/libstub/arm-stub.c
@@ -133,6 +133,7 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
 	unsigned long reserve_size = 0;
 	enum efi_secureboot_mode secure_boot;
 	struct screen_info *si;
+	bool have_chosen_initrd;
 
 	/* Check if we were booted by the EFI firmware */
 	if (sys_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE)
@@ -183,15 +184,6 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
 
 	si = setup_graphics(sys_table);
 
-	status = handle_kernel_image(sys_table, image_addr, &image_size,
-				     &reserve_addr,
-				     &reserve_size,
-				     dram_base, image);
-	if (status != EFI_SUCCESS) {
-		pr_efi_err(sys_table, "Failed to relocate kernel\n");
-		goto fail_free_cmdline;
-	}
-
 	secure_boot = efi_get_secureboot(sys_table);
 
 	/*
@@ -209,7 +201,7 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
 
 		if (status != EFI_SUCCESS) {
 			pr_efi_err(sys_table, "Failed to load device tree!\n");
-			goto fail_free_image;
+			goto fail_free_cmdline;
 		}
 	}
 
@@ -222,16 +214,33 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
 			pr_efi(sys_table, "Using DTB from configuration table\n");
 	}
 
-	if (!fdt_addr)
+	if (!fdt_addr) {
 		pr_efi(sys_table, "Generating empty DTB\n");
+		have_chosen_initrd = false;
+	} else {
+		status = efi_reserve_dtb_initrd(sys_table, (void *)fdt_addr);
+		have_chosen_initrd = (status != EFI_NOT_FOUND);
+	}
 
-	status = handle_cmdline_files(sys_table, image, cmdline_ptr, "initrd=",
-				      efi_get_max_initrd_addr(dram_base,
-							      *image_addr),
-				      (unsigned long *)&initrd_addr,
-				      (unsigned long *)&initrd_size);
-	if (status != EFI_SUCCESS)
-		pr_efi_err(sys_table, "Failed initrd from command line!\n");
+	status = handle_kernel_image(sys_table, image_addr, &image_size,
+				     &reserve_addr, &reserve_size,
+				     dram_base, image);
+	if (status != EFI_SUCCESS) {
+		pr_efi_err(sys_table, "Failed to relocate kernel\n");
+		goto fail_free_fdt;
+	}
+
+	if (!have_chosen_initrd) {
+		status = handle_cmdline_files(sys_table, image, cmdline_ptr,
+					      "initrd=",
+					      efi_get_max_initrd_addr(dram_base,
+								      *image_addr),
+					      (unsigned long *)&initrd_addr,
+					      (unsigned long *)&initrd_size);
+		if (status != EFI_SUCCESS)
+			pr_efi_err(sys_table,
+				   "Failed initrd from command line!\n");
+	}
 
 	efi_random_get_seed(sys_table);
 
@@ -272,11 +281,11 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
 	pr_efi_err(sys_table, "Failed to update FDT and exit boot services\n");
 
 	efi_free(sys_table, initrd_size, initrd_addr);
-	efi_free(sys_table, fdt_size, fdt_addr);
-
-fail_free_image:
 	efi_free(sys_table, image_size, *image_addr);
 	efi_free(sys_table, reserve_size, reserve_addr);
+
+fail_free_fdt:
+	efi_free(sys_table, fdt_size, fdt_addr);
 fail_free_cmdline:
 	free_screen_info(sys_table, si);
 	efi_free(sys_table, cmdline_size, (unsigned long)cmdline_ptr);
diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h
index aaf2aeb785ea..35b514d7d962 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -68,4 +68,7 @@ efi_status_t check_platform_features(efi_system_table_t *sys_table_arg);
 
 efi_status_t efi_random_get_seed(efi_system_table_t *sys_table_arg);
 
+efi_status_t efi_reserve_dtb_initrd(efi_system_table_t *sys_table_arg,
+				    const void *fdt);
+
 #endif
diff --git a/drivers/firmware/efi/libstub/fdt.c b/drivers/firmware/efi/libstub/fdt.c
index 8830fa601e45..54408c95e094 100644
--- a/drivers/firmware/efi/libstub/fdt.c
+++ b/drivers/firmware/efi/libstub/fdt.c
@@ -385,3 +385,45 @@ void *get_fdt(efi_system_table_t *sys_table, unsigned long *fdt_size)
 
 	return fdt;
 }
+
+efi_status_t efi_reserve_dtb_initrd(efi_system_table_t *sys_table_arg,
+				    const void *fdt)
+{
+	int chosen, len;
+	const void *prop;
+	efi_physical_addr_t start, end;
+	unsigned long num_pages;
+	efi_status_t status;
+
+	chosen = fdt_path_offset(fdt, "/chosen");
+	if (chosen == -FDT_ERR_NOTFOUND)
+		return EFI_NOT_FOUND;
+
+	prop = fdt_getprop(fdt, chosen, "linux,initrd-start", &len);
+	if (!prop)
+		return EFI_NOT_FOUND;
+
+	start = (len == sizeof(fdt32_t)) ? fdt32_to_cpu(*(fdt32_t *)prop)
+					 : fdt64_to_cpu(*(fdt64_t *)prop);
+
+	prop = fdt_getprop(fdt, chosen, "linux,initrd-end", &len);
+	if (!prop)
+		return EFI_NOT_FOUND;
+
+	end = (len == sizeof(fdt32_t)) ? fdt32_to_cpu(*(fdt32_t *)prop)
+				       : fdt64_to_cpu(*(fdt64_t *)prop);
+
+	start = round_down(start, EFI_PAGE_SIZE);
+	num_pages = round_up(end - start, EFI_PAGE_SIZE) / EFI_PAGE_SIZE;
+
+	status = efi_call_early(allocate_pages, EFI_ALLOCATE_ADDRESS,
+				EFI_LOADER_DATA, num_pages, &start);
+
+	if (status != EFI_SUCCESS)
+		pr_efi_err(sys_table_arg,
+			   "Failed to reserve initrd area found in /chosen\n");
+	else
+		pr_efi(sys_table_arg, "Using initrd found in /chosen\n");
+
+	return status;
+}
-- 
2.11.0

[kernel-hardening] [PATCH 30/30] efi/libstub: arm: implement KASLR

[Ard Biesheuvel]

This wires up the new KASLR implementation for ARM to the random number
generator and memory allocation routines in the UEFI stub.

Given how the UEFI stub keeps track of the placement of the DTB and
potentially an initrd via its memory map, we can quite simply use
efi_random_alloc() to carve out a window for the kernel proper, and
inform the decompressor about this by setting the kaslr_offset variable
directly.

Since the presence of a vmalloc= command line option complicates the
calculations involved, let's just disable KASLR for now if a vmalloc=
command line argument was provided.

Cc: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 drivers/firmware/efi/libstub/arm32-stub.c | 46 +++++++++++++++++++-
 1 file changed, 44 insertions(+), 2 deletions(-)

diff --git a/drivers/firmware/efi/libstub/arm32-stub.c b/drivers/firmware/efi/libstub/arm32-stub.c
index becbda445913..f812cbca08ee 100644
--- a/drivers/firmware/efi/libstub/arm32-stub.c
+++ b/drivers/firmware/efi/libstub/arm32-stub.c
@@ -8,6 +8,8 @@
  */
 #include <linux/efi.h>
 #include <asm/efi.h>
+#include <asm/pgtable.h>
+#include <asm/zimage.h>
 
 #include "efistub.h"
 
@@ -200,6 +202,29 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table,
 				 efi_loaded_image_t *image)
 {
 	efi_status_t status;
+	u32 phys_seed = 0;
+
+	if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) {
+		if (have_vmalloc()) {
+			pr_efi(sys_table,
+			       "vmalloc= command line argument found, disabling KASLR\n");
+		} else if (!nokaslr()) {
+			status = efi_get_random_bytes(sys_table,
+						      sizeof(phys_seed),
+						      (u8 *)&phys_seed);
+			if (status == EFI_NOT_FOUND) {
+				pr_efi(sys_table,
+				       "EFI_RNG_PROTOCOL unavailable, no randomness supplied\n");
+			} else if (status != EFI_SUCCESS) {
+				pr_efi_err(sys_table,
+					   "efi_get_random_bytes() failed\n");
+				return status;
+			}
+		} else {
+			pr_efi(sys_table,
+			       "KASLR disabled on kernel command line\n");
+		}
+	}
 
 	/*
 	 * Verify that the DRAM base address is compatible with the ARM
@@ -210,8 +235,25 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table,
 	 */
 	dram_base = round_up(dram_base, SZ_128M);
 
-	status = reserve_kernel_base(sys_table, dram_base, reserve_addr,
-				     reserve_size);
+	if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE) || phys_seed == 0) {
+		status = reserve_kernel_base(sys_table, dram_base, reserve_addr,
+					     reserve_size);
+	} else {
+		/* the end of the lowmem region */
+		unsigned long max = dram_base + VMALLOC_DEFAULT_BASE
+				    - PAGE_OFFSET - 1;
+		/*
+		 * The DTB and initrd are covered by allocations in the UEFI
+		 * memory map, so we can create a random allocation for the
+		 * uncompressed kernel, and inform the decompressor about the
+		 * offset with respect to the base of memory.
+		 */
+		*reserve_size = MAX_UNCOMP_KERNEL_SIZE;
+		status = efi_random_alloc(sys_table, *reserve_size, SZ_2M,
+					  reserve_addr, phys_seed, max);
+		kaslr_offset = *reserve_addr - dram_base;
+	}
+
 	if (status != EFI_SUCCESS) {
 		pr_efi_err(sys_table, "Unable to allocate memory for uncompressed kernel.\n");
 		return status;
-- 
2.11.0

[kernel-hardening] Re: [PATCH 01/30] asm-generic: add .data.rel.ro sections to __ro_after_init

[Arnd Bergmann]

On Mon, Aug 14, 2017 at 2:53 PM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> When running in PIC mode, the compiler will emit const structures
> containing runtime relocatable quantities into .data.rel.ro.* sections,
> so that the linker can be smart about placing them together in a segment
> that is read-write initially, and is remapped read-only afterwards. This
> is exactly what __ro_after_init aims to provide, so move these sections
> together.
>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Acked-by: Arnd Bergmann <arnd@arndb.de>

[kernel-hardening] Re: [PATCH 17/30] arm-soc: tegra: make sleep asm code runtime relocatable

[Dave Martin]

On Mon, Aug 14, 2017 at 01:53:58PM +0100, Ard Biesheuvel wrote:
> The PIE kernel build does not allow absolute references encoded in
> movw/movt instruction pairs, so use our mov_l macro instead (which
> will still use such a pair unless CONFIG_RELOCATABLE is defined)
> 
> Also, avoid 32-bit absolute literals to refer to absolute symbols.
> Instead, use a 16 bit reference so that PIE linker cannot get
> confused whether the symbol reference is subject to relocation at
> runtime.

This sounds like we're papering over something.

If the linker is "confused", that sounds like we are either abusing
it somehow, or the linker is broken.


> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  arch/arm/mach-tegra/sleep-tegra20.S | 22 ++++++++++++--------
>  arch/arm/mach-tegra/sleep-tegra30.S |  6 +++---
>  arch/arm/mach-tegra/sleep.S         |  4 ++--
>  3 files changed, 18 insertions(+), 14 deletions(-)
> 
> diff --git a/arch/arm/mach-tegra/sleep-tegra20.S b/arch/arm/mach-tegra/sleep-tegra20.S
> index 5c8e638ee51a..cab95de5c8f1 100644
> --- a/arch/arm/mach-tegra/sleep-tegra20.S
> +++ b/arch/arm/mach-tegra/sleep-tegra20.S
> @@ -99,7 +99,7 @@ ENTRY(tegra20_cpu_shutdown)
>  	cmp	r0, #0
>  	reteq	lr			@ must not be called for CPU 0
>  	mov32	r1, TEGRA_IRAM_RESET_BASE_VIRT
> -	ldr	r2, =__tegra20_cpu1_resettable_status_offset
> +	ldrh	r2, 0f
>  	mov	r12, #CPU_RESETTABLE
>  	strb	r12, [r1, r2]
>  
> @@ -121,6 +121,7 @@ ENTRY(tegra20_cpu_shutdown)
>  	beq	.
>  	ret	lr
>  ENDPROC(tegra20_cpu_shutdown)
> +0:	.short	__tegra20_cpu1_resettable_status_offset
>  #endif
>  
>  #ifdef CONFIG_PM_SLEEP
> @@ -181,6 +182,9 @@ ENTRY(tegra_pen_unlock)
>  	ret     lr
>  ENDPROC(tegra_pen_unlock)
>  
> +.L__tegra20_cpu1_resettable_status_offset:
> +	.short	__tegra20_cpu1_resettable_status_offset
> +
>  /*
>   * tegra20_cpu_clear_resettable(void)
>   *
> @@ -189,7 +193,7 @@ ENDPROC(tegra_pen_unlock)
>   */
>  ENTRY(tegra20_cpu_clear_resettable)
>  	mov32	r1, TEGRA_IRAM_RESET_BASE_VIRT

Can we simply port mov32 to mov_l?  Or do we hit a problem with
multiplatform kernels where mov_l may involve a literal pool entry
(and does it matter)?


[...]

Cheers
---Dave

[kernel-hardening] Re: [PATCH 17/30] arm-soc: tegra: make sleep asm code runtime relocatable

[Ard Biesheuvel]

On 14 August 2017 at 15:42, Dave Martin <Dave.Martin@arm.com> wrote:
> On Mon, Aug 14, 2017 at 01:53:58PM +0100, Ard Biesheuvel wrote:
>> The PIE kernel build does not allow absolute references encoded in
>> movw/movt instruction pairs, so use our mov_l macro instead (which
>> will still use such a pair unless CONFIG_RELOCATABLE is defined)
>>
>> Also, avoid 32-bit absolute literals to refer to absolute symbols.
>> Instead, use a 16 bit reference so that PIE linker cannot get
>> confused whether the symbol reference is subject to relocation at
>> runtime.
>
> This sounds like we're papering over something.
>
> If the linker is "confused", that sounds like we are either abusing
> it somehow, or the linker is broken.
>

There is some ambiguity in how SHN_ABS symbols are treated in shared
libraries and PIE executables.

https://sourceware.org/ml/binutils/2012-05/msg00019.html

I haven't confirmed whether it actually causes problems in this
particular case, but it is safer (and not entirely inappropriate) to
use a 16-bit field for a quantity that can easily fit one.


>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>>  arch/arm/mach-tegra/sleep-tegra20.S | 22 ++++++++++++--------
>>  arch/arm/mach-tegra/sleep-tegra30.S |  6 +++---
>>  arch/arm/mach-tegra/sleep.S         |  4 ++--
>>  3 files changed, 18 insertions(+), 14 deletions(-)
>>
>> diff --git a/arch/arm/mach-tegra/sleep-tegra20.S b/arch/arm/mach-tegra/sleep-tegra20.S
>> index 5c8e638ee51a..cab95de5c8f1 100644
>> --- a/arch/arm/mach-tegra/sleep-tegra20.S
>> +++ b/arch/arm/mach-tegra/sleep-tegra20.S
>> @@ -99,7 +99,7 @@ ENTRY(tegra20_cpu_shutdown)
>>       cmp     r0, #0
>>       reteq   lr                      @ must not be called for CPU 0
>>       mov32   r1, TEGRA_IRAM_RESET_BASE_VIRT
>> -     ldr     r2, =__tegra20_cpu1_resettable_status_offset
>> +     ldrh    r2, 0f
>>       mov     r12, #CPU_RESETTABLE
>>       strb    r12, [r1, r2]
>>
>> @@ -121,6 +121,7 @@ ENTRY(tegra20_cpu_shutdown)
>>       beq     .
>>       ret     lr
>>  ENDPROC(tegra20_cpu_shutdown)
>> +0:   .short  __tegra20_cpu1_resettable_status_offset
>>  #endif
>>
>>  #ifdef CONFIG_PM_SLEEP
>> @@ -181,6 +182,9 @@ ENTRY(tegra_pen_unlock)
>>       ret     lr
>>  ENDPROC(tegra_pen_unlock)
>>
>> +.L__tegra20_cpu1_resettable_status_offset:
>> +     .short  __tegra20_cpu1_resettable_status_offset
>> +
>>  /*
>>   * tegra20_cpu_clear_resettable(void)
>>   *
>> @@ -189,7 +193,7 @@ ENDPROC(tegra_pen_unlock)
>>   */
>>  ENTRY(tegra20_cpu_clear_resettable)
>>       mov32   r1, TEGRA_IRAM_RESET_BASE_VIRT
>
> Can we simply port mov32 to mov_l?  Or do we hit a problem with
> multiplatform kernels where mov_l may involve a literal pool entry
> (and does it matter)?
>

The only place where it matters is in code that lives in idmap.text,
since the relative reference will point to the ID mapped alias of a
section that is not covered by the ID ID map. I think we should be
able to use mov_l everywhere else, and it should do the right thing
for ordinary and PIE builds

[kernel-hardening] Re: [PATCH 17/30] arm-soc: tegra: make sleep asm code runtime relocatable

[Dave Martin]

On Mon, Aug 14, 2017 at 03:49:15PM +0100, Ard Biesheuvel wrote:
> On 14 August 2017 at 15:42, Dave Martin <Dave.Martin@arm.com> wrote:
> > On Mon, Aug 14, 2017 at 01:53:58PM +0100, Ard Biesheuvel wrote:
> >> The PIE kernel build does not allow absolute references encoded in
> >> movw/movt instruction pairs, so use our mov_l macro instead (which
> >> will still use such a pair unless CONFIG_RELOCATABLE is defined)
> >>
> >> Also, avoid 32-bit absolute literals to refer to absolute symbols.
> >> Instead, use a 16 bit reference so that PIE linker cannot get
> >> confused whether the symbol reference is subject to relocation at
> >> runtime.
> >
> > This sounds like we're papering over something.
> >
> > If the linker is "confused", that sounds like we are either abusing
> > it somehow, or the linker is broken.
> >
> 
> There is some ambiguity in how SHN_ABS symbols are treated in shared
> libraries and PIE executables.
> 
> https://sourceware.org/ml/binutils/2012-05/msg00019.html
> 
> I haven't confirmed whether it actually causes problems in this
> particular case, but it is safer (and not entirely inappropriate) to
> use a 16-bit field for a quantity that can easily fit one.

OK, so, fundamental linker design flaw that cannot be fixed for
historical reasons.  Gotcha.

This "fix" feels like a bit of a hack, since it relies on the absence of
a certain relocation type that could be added later (though it seems
highly unlikely).


Cleaner options would be to expose both symbols that are subtracted,
rather than the result, or to do the subtraction at build time and
generate a header that affected files include (though that's more
invasive on the Makefile side).

May be overkill though.

> 
> >
> >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> ---
> >>  arch/arm/mach-tegra/sleep-tegra20.S | 22 ++++++++++++--------
> >>  arch/arm/mach-tegra/sleep-tegra30.S |  6 +++---
> >>  arch/arm/mach-tegra/sleep.S         |  4 ++--
> >>  3 files changed, 18 insertions(+), 14 deletions(-)
> >>
> >> diff --git a/arch/arm/mach-tegra/sleep-tegra20.S b/arch/arm/mach-tegra/sleep-tegra20.S
> >> index 5c8e638ee51a..cab95de5c8f1 100644
> >> --- a/arch/arm/mach-tegra/sleep-tegra20.S
> >> +++ b/arch/arm/mach-tegra/sleep-tegra20.S
> >> @@ -99,7 +99,7 @@ ENTRY(tegra20_cpu_shutdown)
> >>       cmp     r0, #0
> >>       reteq   lr                      @ must not be called for CPU 0
> >>       mov32   r1, TEGRA_IRAM_RESET_BASE_VIRT
> >> -     ldr     r2, =__tegra20_cpu1_resettable_status_offset
> >> +     ldrh    r2, 0f
> >>       mov     r12, #CPU_RESETTABLE
> >>       strb    r12, [r1, r2]
> >>
> >> @@ -121,6 +121,7 @@ ENTRY(tegra20_cpu_shutdown)
> >>       beq     .
> >>       ret     lr
> >>  ENDPROC(tegra20_cpu_shutdown)
> >> +0:   .short  __tegra20_cpu1_resettable_status_offset
> >>  #endif
> >>
> >>  #ifdef CONFIG_PM_SLEEP
> >> @@ -181,6 +182,9 @@ ENTRY(tegra_pen_unlock)
> >>       ret     lr
> >>  ENDPROC(tegra_pen_unlock)
> >>
> >> +.L__tegra20_cpu1_resettable_status_offset:
> >> +     .short  __tegra20_cpu1_resettable_status_offset
> >> +
> >>  /*
> >>   * tegra20_cpu_clear_resettable(void)
> >>   *
> >> @@ -189,7 +193,7 @@ ENDPROC(tegra_pen_unlock)
> >>   */
> >>  ENTRY(tegra20_cpu_clear_resettable)
> >>       mov32   r1, TEGRA_IRAM_RESET_BASE_VIRT
> >
> > Can we simply port mov32 to mov_l?  Or do we hit a problem with
> > multiplatform kernels where mov_l may involve a literal pool entry
> > (and does it matter)?
> >
> 
> The only place where it matters is in code that lives in idmap.text,
> since the relative reference will point to the ID mapped alias of a
> section that is not covered by the ID ID map. I think we should be
> able to use mov_l everywhere else, and it should do the right thing
> for ordinary and PIE builds

Fair enough.

Cheers
---Dave

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Dave Martin]

On Mon, Aug 14, 2017 at 01:53:43PM +0100, Ard Biesheuvel wrote:
> Like arm64, ARM supports position independent code sequences that
> produce symbol references with a greater reach than the ordinary
> adr/ldr instructions.
> 
> Currently, we use open coded instruction sequences involving literals
> and arithmetic operations. Instead, we can use movw/movt pairs on v7
> CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
> the literal into a subsection, allowing it to be emitted out of line
> while retaining the ability to perform arithmetic on label offsets.
> 
> E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:
> 
>        ldr          <reg>, 222f
>   111: add          <reg>, <reg>, pc
>        .subsection  1
>   222: .long        <sym> - (111b + 8)
>        .previous
> 
> This is allowed by the assembler because, unlike ordinary sections,
> subsections are combined into a single section into the object file,
> and so the label references are not true cross-section references that
> are visible as relocations. Note that we could even do something like
> 
>        add          <reg>, pc, #(222f - 111f) & ~0xfff
>        ldr          <reg>, [<reg>, #(222f - 111f) & 0xfff]
>   111: add          <reg>, <reg>, pc
>        .subsection  1
>   222: .long        <sym> - (111b + 8)
>        .previous

This is reinventing ldr=

I seem to remember ldr= barfing on things that .long happily accepts
though, was this the reason?


> if it turns out that the 4 KB range of the ldr instruction is insufficient
> to reach the literal in the subsection, although this is currently not a
> problem (of the 98 objects built from .S files in a multi_v7_defconfig
> build, only 11 have .text sections that are over 1 KB, and the largest one
> [entry-armv.o] is 3308 bytes)
> 
> Subsections have been available in binutils since 2004 at least, so
> they should not cause any issues with older toolchains.

(I also believe this to be an ancient feature, but I've not done the
digging to prove it.)

> So use the above to implement the macros mov_l, adr_l, adrm_l (using ldm
> to load multiple literals at once), ldr_l and str_l, all of which will
> use movw/movt pairs on v7 and later CPUs, and use PC-relative literals
> otherwise.
> 
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  arch/arm/include/asm/assembler.h | 71 ++++++++++++++++++++
>  1 file changed, 71 insertions(+)
> 
> diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
> index ad301f107dd2..516ebaf4ff38 100644
> --- a/arch/arm/include/asm/assembler.h
> +++ b/arch/arm/include/asm/assembler.h
> @@ -518,4 +518,75 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
>  #endif
>  	.endm
>  
> +#ifdef CONFIG_THUMB2_KERNEL
> +#define	ARM_PC_BIAS		4
> +#else
> +#define	ARM_PC_BIAS		8
> +#endif
> +
> +	.macro		__adldst_l, op, reg, sym, tmp, c
> +	.if		__LINUX_ARM_ARCH__ < 7
> +	ldr\c		\tmp, 111f
> +	.subsection	1
> +	.align		2
> +111:	.long		\sym - (222f + ARM_PC_BIAS)

See above comment about ldr=.

> +	.previous
> +	.else
> +	W(movw\c\())	\tmp, #:lower16:\sym - (222f + ARM_PC_BIAS)
> +	W(movt\c\())	\tmp, #:upper16:\sym - (222f + ARM_PC_BIAS)

Why W()?

There are no narrow forms of these instructions anyway -- if there were
then they couldn't accommodate a 16-bit immediate.

> +	.endif
> +222:
> +	.ifc		\op, add
> +	add\c		\reg, \tmp, pc
> +	.elseif		CONFIG_THUMB2_KERNEL == 1
> +	add		\tmp, \tmp, pc
> +	\op\c		\reg, [\tmp]

Shame 
	\op\c		\reg, [pc, \tmp]
doesn't work.

But it doesn't, apparently.

> +	.else
> +	\op\c		\reg, [pc, \tmp]
> +	.endif
> +	.endm
> +
> +	/*
> +	 * mov_l - move a constant value or [relocated] address into a register
> +	 */
> +	.macro		mov_l, dst:req, imm:req, cond
> +	.if		__LINUX_ARM_ARCH__ < 7
> +	ldr\cond	\dst, =\imm
> +	.else
> +	W(movw\cond\())	\dst, #:lower16:\imm
> +	W(movt\cond\())	\dst, #:upper16:\imm
> +	.endif
> +	.endm
> +
> +	/*
> +	 * adr_l - adr pseudo-op with unlimited range
> +	 *
> +	 * @dst: destination register
> +	 * @sym: name of the symbol
> +	 */
> +	.macro		adr_l, dst:req, sym:req, cond
> +	__adldst_l	add, \dst, \sym, \dst, \cond
> +	.endm
> +
> +	/*
> +	 * ldr_l - ldr <literal> pseudo-op with unlimited range
> +	 *
> +	 * @dst: destination register
> +	 * @sym: name of the symbol
> +	 */
> +	.macro		ldr_l, dst:req, sym:req, cond
> +	__adldst_l	ldr, \dst, \sym, \dst, \cond
> +	.endm
> +
> +	/*
> +	 * str_l - str <literal> pseudo-op with unlimited range
> +	 *
> +	 * @src: source register
> +	 * @sym: name of the symbol
> +	 * @tmp: mandatory scratch register
> +	 */
> +	.macro		str_l, src:req, sym:req, tmp:req, cond
> +	__adldst_l	str, \src, \sym, \tmp, \cond
> +	.endm

Cheers
---Dave

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Arnd Bergmann]

On Mon, Aug 14, 2017 at 2:53 PM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> This series implements randomization of the placement of the core ARM kernel
> inside the lowmem region. It consists of the following parts:
>
> - changes that allow us to build vmlinux as a PIE executable which retains
>   the metadata required to fix up all absolute symbol references at runtime
> - changes that eliminate absolute references from low-level code that may
>   execute with the MMU off: this removes the need to perform explicit cache
>   maintenance after the absolute references have been fixed up at runtime with
>   the caches enabled
> - changes to the core kernel startup code to take the physical offset into
>   account when creating the virtual mapping (the pa-to-va mapping remains
>   unchanged)
> - changes to the decompressor to take the KASLR offset into account when
>   placing the kernel in physical memory
> - changes to the UEFI stub code to choose the KASLR offset and communicate
>   it to the decompressor

Would it make sense to also randomize the pa-to-va mapping on top of this?
That can certainly be a later follow-up, I'm just trying to think of the options
we have, given that the kernel is now relocatable and we can support arbitrary
pa-to-va mappings already.

Can you explain how the random seed is passed from the bootloader
to the kernel when we don't use EFI? Is this implemented at all? I see
that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
the EFI boot services, but I don't see where that value gets read again
when we relocate the kernel.

      Arnd

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Dave Martin]

On Mon, Aug 14, 2017 at 01:53:43PM +0100, Ard Biesheuvel wrote:
> Like arm64, ARM supports position independent code sequences that
> produce symbol references with a greater reach than the ordinary
> adr/ldr instructions.
> 
> Currently, we use open coded instruction sequences involving literals
> and arithmetic operations. Instead, we can use movw/movt pairs on v7
> CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
> the literal into a subsection, allowing it to be emitted out of line
> while retaining the ability to perform arithmetic on label offsets.
> 
> E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:
> 
>        ldr          <reg>, 222f
>   111: add          <reg>, <reg>, pc
>        .subsection  1
>   222: .long        <sym> - (111b + 8)
>        .previous
> 
> This is allowed by the assembler because, unlike ordinary sections,
> subsections are combined into a single section into the object file,
> and so the label references are not true cross-section references that
> are visible as relocations. Note that we could even do something like
> 
>        add          <reg>, pc, #(222f - 111f) & ~0xfff
>        ldr          <reg>, [<reg>, #(222f - 111f) & 0xfff]
>   111: add          <reg>, <reg>, pc
>        .subsection  1
>   222: .long        <sym> - (111b + 8)
>        .previous
> 
> if it turns out that the 4 KB range of the ldr instruction is insufficient
> to reach the literal in the subsection, although this is currently not a
> problem (of the 98 objects built from .S files in a multi_v7_defconfig
> build, only 11 have .text sections that are over 1 KB, and the largest one
> [entry-armv.o] is 3308 bytes)
> 
> Subsections have been available in binutils since 2004 at least, so
> they should not cause any issues with older toolchains.
> 
> So use the above to implement the macros mov_l, adr_l, adrm_l (using ldm

I don't see adrm_l in this patch.

> to load multiple literals at once), ldr_l and str_l, all of which will
> use movw/movt pairs on v7 and later CPUs, and use PC-relative literals
> otherwise.

Also...

By default, I'd assume that we should port _all_ uses of :upper16:/
:lower16: to use these.  Does this series consciously do that?  Are
there any exceptions?

[...]

Cheers
---Dave

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Ard Biesheuvel]

On 14 August 2017 at 16:29, Dave Martin <Dave.Martin@arm.com> wrote:
> On Mon, Aug 14, 2017 at 01:53:43PM +0100, Ard Biesheuvel wrote:
>> Like arm64, ARM supports position independent code sequences that
>> produce symbol references with a greater reach than the ordinary
>> adr/ldr instructions.
>>
>> Currently, we use open coded instruction sequences involving literals
>> and arithmetic operations. Instead, we can use movw/movt pairs on v7
>> CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
>> the literal into a subsection, allowing it to be emitted out of line
>> while retaining the ability to perform arithmetic on label offsets.
>>
>> E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:
>>
>>        ldr          <reg>, 222f
>>   111: add          <reg>, <reg>, pc
>>        .subsection  1
>>   222: .long        <sym> - (111b + 8)
>>        .previous
>>
>> This is allowed by the assembler because, unlike ordinary sections,
>> subsections are combined into a single section into the object file,
>> and so the label references are not true cross-section references that
>> are visible as relocations. Note that we could even do something like
>>
>>        add          <reg>, pc, #(222f - 111f) & ~0xfff
>>        ldr          <reg>, [<reg>, #(222f - 111f) & 0xfff]
>>   111: add          <reg>, <reg>, pc
>>        .subsection  1
>>   222: .long        <sym> - (111b + 8)
>>        .previous
>
> This is reinventing ldr=
>
> I seem to remember ldr= barfing on things that .long happily accepts
> though, was this the reason?
>

Yes. ldr = does not accept expressions involving symbols, only plain
symbols or expressions that evaluate to constants.

So something like

ldr <reg>, =<sym> - <label>

is rejected while the equivalent

ldr <reg>, 0f
0: .long <sym> - <label>

does work.



>> if it turns out that the 4 KB range of the ldr instruction is insufficient
>> to reach the literal in the subsection, although this is currently not a
>> problem (of the 98 objects built from .S files in a multi_v7_defconfig
>> build, only 11 have .text sections that are over 1 KB, and the largest one
>> [entry-armv.o] is 3308 bytes)
>>
>> Subsections have been available in binutils since 2004 at least, so
>> they should not cause any issues with older toolchains.
>
> (I also believe this to be an ancient feature, but I've not done the
> digging to prove it.)
>

OK

>> So use the above to implement the macros mov_l, adr_l, adrm_l (using ldm
>> to load multiple literals at once), ldr_l and str_l, all of which will
>> use movw/movt pairs on v7 and later CPUs, and use PC-relative literals
>> otherwise.
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>>  arch/arm/include/asm/assembler.h | 71 ++++++++++++++++++++
>>  1 file changed, 71 insertions(+)
>>
>> diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
>> index ad301f107dd2..516ebaf4ff38 100644
>> --- a/arch/arm/include/asm/assembler.h
>> +++ b/arch/arm/include/asm/assembler.h
>> @@ -518,4 +518,75 @@ THUMB(   orr     \reg , \reg , #PSR_T_BIT        )
>>  #endif
>>       .endm
>>
>> +#ifdef CONFIG_THUMB2_KERNEL
>> +#define      ARM_PC_BIAS             4
>> +#else
>> +#define      ARM_PC_BIAS             8
>> +#endif
>> +
>> +     .macro          __adldst_l, op, reg, sym, tmp, c
>> +     .if             __LINUX_ARM_ARCH__ < 7
>> +     ldr\c           \tmp, 111f
>> +     .subsection     1
>> +     .align          2
>> +111: .long           \sym - (222f + ARM_PC_BIAS)
>
> See above comment about ldr=.
>
>> +     .previous
>> +     .else
>> +     W(movw\c\())    \tmp, #:lower16:\sym - (222f + ARM_PC_BIAS)
>> +     W(movt\c\())    \tmp, #:upper16:\sym - (222f + ARM_PC_BIAS)
>
> Why W()?
>
> There are no narrow forms of these instructions anyway -- if there were
> then they couldn't accommodate a 16-bit immediate.
>

That's a trick, actually, which I failed to add a comment for.

We use .arm sections in the thumb2 kernel, and using these macros
there would result in the wrong offset to be used. Adding the .w
suffix forces an error in the assembler which even results in a fairly
meaningful error message complaining about using .w in ARM code.

>> +     .endif
>> +222:
>> +     .ifc            \op, add
>> +     add\c           \reg, \tmp, pc
>> +     .elseif         CONFIG_THUMB2_KERNEL == 1
>> +     add             \tmp, \tmp, pc
>> +     \op\c           \reg, [\tmp]
>
> Shame
>         \op\c           \reg, [pc, \tmp]
> doesn't work.
>
> But it doesn't, apparently.
>

No, thumb2 does not allow that

>> +     .else
>> +     \op\c           \reg, [pc, \tmp]
>> +     .endif
>> +     .endm
>> +
>> +     /*
>> +      * mov_l - move a constant value or [relocated] address into a register
>> +      */
>> +     .macro          mov_l, dst:req, imm:req, cond
>> +     .if             __LINUX_ARM_ARCH__ < 7
>> +     ldr\cond        \dst, =\imm
>> +     .else
>> +     W(movw\cond\()) \dst, #:lower16:\imm
>> +     W(movt\cond\()) \dst, #:upper16:\imm
>> +     .endif
>> +     .endm
>> +
>> +     /*
>> +      * adr_l - adr pseudo-op with unlimited range
>> +      *
>> +      * @dst: destination register
>> +      * @sym: name of the symbol
>> +      */
>> +     .macro          adr_l, dst:req, sym:req, cond
>> +     __adldst_l      add, \dst, \sym, \dst, \cond
>> +     .endm
>> +
>> +     /*
>> +      * ldr_l - ldr <literal> pseudo-op with unlimited range
>> +      *
>> +      * @dst: destination register
>> +      * @sym: name of the symbol
>> +      */
>> +     .macro          ldr_l, dst:req, sym:req, cond
>> +     __adldst_l      ldr, \dst, \sym, \dst, \cond
>> +     .endm
>> +
>> +     /*
>> +      * str_l - str <literal> pseudo-op with unlimited range
>> +      *
>> +      * @src: source register
>> +      * @sym: name of the symbol
>> +      * @tmp: mandatory scratch register
>> +      */
>> +     .macro          str_l, src:req, sym:req, tmp:req, cond
>> +     __adldst_l      str, \src, \sym, \tmp, \cond
>> +     .endm
>
> Cheers
> ---Dave

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Ard Biesheuvel]

On 14 August 2017 at 16:32, Dave Martin <Dave.Martin@arm.com> wrote:
> On Mon, Aug 14, 2017 at 01:53:43PM +0100, Ard Biesheuvel wrote:
>> Like arm64, ARM supports position independent code sequences that
>> produce symbol references with a greater reach than the ordinary
>> adr/ldr instructions.
>>
>> Currently, we use open coded instruction sequences involving literals
>> and arithmetic operations. Instead, we can use movw/movt pairs on v7
>> CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
>> the literal into a subsection, allowing it to be emitted out of line
>> while retaining the ability to perform arithmetic on label offsets.
>>
>> E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:
>>
>>        ldr          <reg>, 222f
>>   111: add          <reg>, <reg>, pc
>>        .subsection  1
>>   222: .long        <sym> - (111b + 8)
>>        .previous
>>
>> This is allowed by the assembler because, unlike ordinary sections,
>> subsections are combined into a single section into the object file,
>> and so the label references are not true cross-section references that
>> are visible as relocations. Note that we could even do something like
>>
>>        add          <reg>, pc, #(222f - 111f) & ~0xfff
>>        ldr          <reg>, [<reg>, #(222f - 111f) & 0xfff]
>>   111: add          <reg>, <reg>, pc
>>        .subsection  1
>>   222: .long        <sym> - (111b + 8)
>>        .previous
>>
>> if it turns out that the 4 KB range of the ldr instruction is insufficient
>> to reach the literal in the subsection, although this is currently not a
>> problem (of the 98 objects built from .S files in a multi_v7_defconfig
>> build, only 11 have .text sections that are over 1 KB, and the largest one
>> [entry-armv.o] is 3308 bytes)
>>
>> Subsections have been available in binutils since 2004 at least, so
>> they should not cause any issues with older toolchains.
>>
>> So use the above to implement the macros mov_l, adr_l, adrm_l (using ldm
>
> I don't see adrm_l in this patch.
>

Oops (2)

Nico already mentioned that, and I failed to fix the commit log. I
added it at some point, but it wasn't really useful

>> to load multiple literals at once), ldr_l and str_l, all of which will
>> use movw/movt pairs on v7 and later CPUs, and use PC-relative literals
>> otherwise.
>
> Also...
>
> By default, I'd assume that we should port _all_ uses of :upper16:/
> :lower16: to use these.  Does this series consciously do that?  Are
> there any exceptions?
>

There aren't that many. Anything that refers to absolute symbols will
break under CONFIG_RELOCATABLE and I haven't noticed any issues (I
tested extensively with Thumb2)

I don't mind open coded movw/movt for relative references in code that
is tightly coupled to a platform that guarantees v7+ so I didn't do a
full sweep. Also, I started with 50+ patches and tried to remove the
ones that are mostly orthogonal to the KASLR stuff.

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Ard Biesheuvel]

On 14 August 2017 at 16:30, Arnd Bergmann <arnd@arndb.de> wrote:
> On Mon, Aug 14, 2017 at 2:53 PM, Ard Biesheuvel
> <ard.biesheuvel@linaro.org> wrote:
>> This series implements randomization of the placement of the core ARM kernel
>> inside the lowmem region. It consists of the following parts:
>>
>> - changes that allow us to build vmlinux as a PIE executable which retains
>>   the metadata required to fix up all absolute symbol references at runtime
>> - changes that eliminate absolute references from low-level code that may
>>   execute with the MMU off: this removes the need to perform explicit cache
>>   maintenance after the absolute references have been fixed up at runtime with
>>   the caches enabled
>> - changes to the core kernel startup code to take the physical offset into
>>   account when creating the virtual mapping (the pa-to-va mapping remains
>>   unchanged)
>> - changes to the decompressor to take the KASLR offset into account when
>>   placing the kernel in physical memory
>> - changes to the UEFI stub code to choose the KASLR offset and communicate
>>   it to the decompressor
>
> Would it make sense to also randomize the pa-to-va mapping on top of this?
> That can certainly be a later follow-up, I'm just trying to think of the options
> we have, given that the kernel is now relocatable and we can support arbitrary
> pa-to-va mappings already.
>

We could randomize PAGE_OFFSET as well. That allows you to build a
3g/1g split kernel and execute it as 2g/2g split. Pretty neat!

Randomizing the VA to PA mapping while keep PAGE_OFFSET constant will
result in either memory to be thrown away (because it is virtually
mapped below PAGE_OFFSET) or lowmem space to be wasted (because there
is a hole between PAGE_OFFSET and the VA of the lowest lowmem address)

So i think there may be opportunities, but I haven't quite figured
them out myself yet.

> Can you explain how the random seed is passed from the bootloader
> to the kernel when we don't use EFI? Is this implemented at all? I see
> that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
> the EFI boot services, but I don't see where that value gets read again
> when we relocate the kernel.

/chosen/kaslr-seed is only used on arm64, not on ARM. We could add
code to the decompressor that uses /chosen/kaslr-seed, but it is a bit
fiddly because the execution environment is so constrained, and there
is no simple access to symbols defined by the core kernel's linker
script.

On UEFI systems, the kaslr offset is calculated based on the UEFI
memory map, which describes all of memory and has reservations for the
DTB, the initrd etc. The EFI stub is linked together with the
decompressor, so passing the kaslr offset simply involves setting a
variable.

To allow other bootloaders to do the same, the kaslr metadata is
exposed via a zImage header, containing the values of PAGE_OFFSET, the
base of the vmalloc area and the randomization granularity. A
bootloader can read these values, and taking the size of DRAM and the
placement of initrd and DTB into account, it can choose a value for
kaslr offset and write it back into the zImage header.

This is a bit involved, but it is really difficult to make these
things backward compatible, i.e., passing something in a register is
not possible if that register was not mandated to be zero initially.

Similarly, the decompressor passed the kaslr offset to the startup
code in the core kernel. It does so by passing it in r3 and jumping 4
bytes past the entry point. This way, we are backward compatible with
configurations where the decompressor is not used, because in that
case, you always jump to the first instruction, which zeroes r3.

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Dave Martin]

On Mon, Aug 14, 2017 at 04:38:02PM +0100, Ard Biesheuvel wrote:
> On 14 August 2017 at 16:29, Dave Martin <Dave.Martin@arm.com> wrote:
> > On Mon, Aug 14, 2017 at 01:53:43PM +0100, Ard Biesheuvel wrote:
> >> Like arm64, ARM supports position independent code sequences that
> >> produce symbol references with a greater reach than the ordinary
> >> adr/ldr instructions.
> >>
> >> Currently, we use open coded instruction sequences involving literals
> >> and arithmetic operations. Instead, we can use movw/movt pairs on v7
> >> CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
> >> the literal into a subsection, allowing it to be emitted out of line
> >> while retaining the ability to perform arithmetic on label offsets.
> >>
> >> E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:
> >>
> >>        ldr          <reg>, 222f
> >>   111: add          <reg>, <reg>, pc
> >>        .subsection  1
> >>   222: .long        <sym> - (111b + 8)
> >>        .previous
> >>
> >> This is allowed by the assembler because, unlike ordinary sections,
> >> subsections are combined into a single section into the object file,
> >> and so the label references are not true cross-section references that
> >> are visible as relocations. Note that we could even do something like
> >>
> >>        add          <reg>, pc, #(222f - 111f) & ~0xfff
> >>        ldr          <reg>, [<reg>, #(222f - 111f) & 0xfff]
> >>   111: add          <reg>, <reg>, pc
> >>        .subsection  1
> >>   222: .long        <sym> - (111b + 8)
> >>        .previous
> >
> > This is reinventing ldr=
> >
> > I seem to remember ldr= barfing on things that .long happily accepts
> > though, was this the reason?
> >
> 
> Yes. ldr = does not accept expressions involving symbols, only plain
> symbols or expressions that evaluate to constants.
> 
> So something like
> 
> ldr <reg>, =<sym> - <label>
> 
> is rejected while the equivalent
> 
> ldr <reg>, 0f
> 0: .long <sym> - <label>
> 
> does work.

I wouldn't bother trying to rationalise gas' behaviour here.  I think
it's an accident of implementation rather than there being some
fundamental reason for it.

AFAICT gas could quite happily resolve ldr= in exactly the same way as
.long and thus not have this problem.  But we can't rewrite history.

[...]

> >> +     .macro          __adldst_l, op, reg, sym, tmp, c
> >> +     .if             __LINUX_ARM_ARCH__ < 7
> >> +     ldr\c           \tmp, 111f
> >> +     .subsection     1
> >> +     .align          2
> >> +111: .long           \sym - (222f + ARM_PC_BIAS)
> >
> > See above comment about ldr=.
> >
> >> +     .previous
> >> +     .else
> >> +     W(movw\c\())    \tmp, #:lower16:\sym - (222f + ARM_PC_BIAS)
> >> +     W(movt\c\())    \tmp, #:upper16:\sym - (222f + ARM_PC_BIAS)
> >
> > Why W()?
> >
> > There are no narrow forms of these instructions anyway -- if there were
> > then they couldn't accommodate a 16-bit immediate.
> >
> 
> That's a trick, actually, which I failed to add a comment for.
> 
> We use .arm sections in the thumb2 kernel, and using these macros
> there would result in the wrong offset to be used. Adding the .w
> suffix forces an error in the assembler which even results in a fairly
> meaningful error message complaining about using .w in ARM code.

Ewww... I think it'd be best to add a comment explaining that.

There's a fair change someone will trip over this at some point (or
worse, "fix" the assembly errors).

> 
> >> +     .endif
> >> +222:
> >> +     .ifc            \op, add
> >> +     add\c           \reg, \tmp, pc
> >> +     .elseif         CONFIG_THUMB2_KERNEL == 1
> >> +     add             \tmp, \tmp, pc
> >> +     \op\c           \reg, [\tmp]
> >
> > Shame
> >         \op\c           \reg, [pc, \tmp]
> > doesn't work.
> >
> > But it doesn't, apparently.
> >
> 
> No, thumb2 does not allow that

Meh.  Oh well.

[...]

Cheers
---Dave

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Dave Martin]

On Mon, Aug 14, 2017 at 04:40:55PM +0100, Ard Biesheuvel wrote:
> On 14 August 2017 at 16:32, Dave Martin <Dave.Martin@arm.com> wrote:
> > On Mon, Aug 14, 2017 at 01:53:43PM +0100, Ard Biesheuvel wrote:

[...]

> >> So use the above to implement the macros mov_l, adr_l, adrm_l (using ldm
> >
> > I don't see adrm_l in this patch.
> >
> 
> Oops (2)
> 
> Nico already mentioned that, and I failed to fix the commit log. I
> added it at some point, but it wasn't really useful

Thought it might be something like that.

> >> to load multiple literals at once), ldr_l and str_l, all of which will
> >> use movw/movt pairs on v7 and later CPUs, and use PC-relative literals
> >> otherwise.
> >
> > Also...
> >
> > By default, I'd assume that we should port _all_ uses of :upper16:/
> > :lower16: to use these.  Does this series consciously do that?  Are
> > there any exceptions?
> >
> 
> There aren't that many. Anything that refers to absolute symbols will
> break under CONFIG_RELOCATABLE and I haven't noticed any issues (I
> tested extensively with Thumb2)
> 
> I don't mind open coded movw/movt for relative references in code that
> is tightly coupled to a platform that guarantees v7+ so I didn't do a
> full sweep. Also, I started with 50+ patches and tried to remove the
> ones that are mostly orthogonal to the KASLR stuff.

OK, that sounds reasonable.

Cheers
---Dave

[kernel-hardening] Re: [PATCH 20/30] ARM: kernel: use PC relative symbol references in suspend/resume code

[Nicolas Pitre]

On Mon, 14 Aug 2017, Ard Biesheuvel wrote:

> Replace some unnecessary absolute references with relative ones. Also,
> to prepare for runtime relocation, which occurs with the caches on,
> defer taking the absolute address of cpu_resume_after_mmu() until after
> the MMU is enabled.
> 
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  arch/arm/kernel/sleep.S | 11 +++++------
>  1 file changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
> index 3026b119d3ff..9efd1c7d3552 100644
> --- a/arch/arm/kernel/sleep.S
> +++ b/arch/arm/kernel/sleep.S
> @@ -60,18 +60,17 @@
>  ENTRY(__cpu_suspend)
>  	stmfd	sp!, {r4 - r11, lr}
>  #ifdef MULTI_CPU
> -	ldr	r10, =processor
> -	ldr	r4, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
> +	ldr_l	r4, processor + CPU_SLEEP_SIZE	@ size of CPU sleep state
>  #else
> -	ldr	r4, =cpu_suspend_size
> +	adr_l	r4, cpu_suspend_size
>  #endif
>  	mov	r5, sp			@ current virtual SP
>  	add	r4, r4, #12		@ Space for pgd, virt sp, phys resume fn
>  	sub	sp, sp, r4		@ allocate CPU state on stack
> -	ldr	r3, =sleep_save_sp
> +	adr_l	r3, sleep_save_sp
>  	stmfd	sp!, {r0, r1}		@ save suspend func arg and pointer
>  	ldr	r3, [r3, #SLEEP_SAVE_SP_VIRT]
> -	ALT_SMP(ldr r0, =mpidr_hash)
> +	ALT_SMP(adr_l r0, mpidr_hash)
>  	ALT_UP_B(1f)

The above is dangerous. adr_l expands to more than one instruction which 
is not what ALT_SMP() was designed for. Here it might happen to work 
anyway because it is combined with ALT_UP_B() but with ALT_UP() it 
wouldn't. This is a mistake waiting to happen.


>  	/* This ldmia relies on the memory layout of the mpidr_hash struct */
>  	ldmia	r0, {r1, r6-r8}	@ r1 = mpidr mask (r6,r7,r8) = l[0,1,2] shifts
> @@ -100,13 +99,13 @@ ENDPROC(cpu_suspend_abort)
>  	.align	5
>  	.pushsection	.idmap.text,"ax"
>  ENTRY(cpu_resume_mmu)
> -	ldr	r3, =cpu_resume_after_mmu
>  	instr_sync
>  	mcr	p15, 0, r0, c1, c0, 0	@ turn on MMU, I-cache, etc
>  	mrc	p15, 0, r0, c0, c0, 0	@ read id reg
>  	instr_sync
>  	mov	r0, r0
>  	mov	r0, r0
> +	ldr	r3, =cpu_resume_after_mmu
>  	ret	r3			@ jump to virtual address
>  ENDPROC(cpu_resume_mmu)
>  	.popsection
> -- 
> 2.11.0
> 
> 

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Arnd Bergmann]

On Mon, Aug 14, 2017 at 5:49 PM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> On 14 August 2017 at 16:30, Arnd Bergmann <arnd@arndb.de> wrote:


>> Can you explain how the random seed is passed from the bootloader
>> to the kernel when we don't use EFI? Is this implemented at all? I see
>> that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
>> the EFI boot services, but I don't see where that value gets read again
>> when we relocate the kernel.

> To allow other bootloaders to do the same, the kaslr metadata is
> exposed via a zImage header, containing the values of PAGE_OFFSET, the
> base of the vmalloc area and the randomization granularity. A
> bootloader can read these values, and taking the size of DRAM and the
> placement of initrd and DTB into account, it can choose a value for
> kaslr offset and write it back into the zImage header.
>
> This is a bit involved, but it is really difficult to make these
> things backward compatible, i.e., passing something in a register is
> not possible if that register was not mandated to be zero initially.
>
> Similarly, the decompressor passed the kaslr offset to the startup
> code in the core kernel. It does so by passing it in r3 and jumping 4
> bytes past the entry point. This way, we are backward compatible with
> configurations where the decompressor is not used, because in that
> case, you always jump to the first instruction, which zeroes r3.

There are two ideas we discussed in the past (but never implemented
them obviously):

- instead of reading the "kaslr-seed" in the decompressor, it could
  simply hash all of the DT blob to get the seed. This way the bootloader
  can put the random see anywhere it likes, and as an added bonus,
  we also get a little bit more random behavior on machines that have
  no entropy source at all but that do have things like a serial number or
  mac address in DT. Obviously those would be constant across boots
  but different between machines. The OS can also store a random
  seed during shutdown in a location that the bootloader uses to
  initialize /chosen/kaslr-seed or another property that we use to seed
  the kernel PRNG at boot time.

- If we have a random number at boot but no way to pass it through
  the DT, I think we actually /can/ pass it through registers: the
  register state is undefined, so in the worst case using the XOR of
  all registers gives us the same number on each boot, but the
  if the loader is modified to store a random 32-bit number in any
  of the registers that don't pass other information, we can use that
  to calculate the kaslr-base.

         Arnd

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Nicolas Pitre]

On Mon, 14 Aug 2017, Ard Biesheuvel wrote:

> On 14 August 2017 at 16:30, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Mon, Aug 14, 2017 at 2:53 PM, Ard Biesheuvel
> > <ard.biesheuvel@linaro.org> wrote:
> >> This series implements randomization of the placement of the core ARM kernel
> >> inside the lowmem region. It consists of the following parts:
> >>
> >> - changes that allow us to build vmlinux as a PIE executable which retains
> >>   the metadata required to fix up all absolute symbol references at runtime
> >> - changes that eliminate absolute references from low-level code that may
> >>   execute with the MMU off: this removes the need to perform explicit cache
> >>   maintenance after the absolute references have been fixed up at runtime with
> >>   the caches enabled
> >> - changes to the core kernel startup code to take the physical offset into
> >>   account when creating the virtual mapping (the pa-to-va mapping remains
> >>   unchanged)
> >> - changes to the decompressor to take the KASLR offset into account when
> >>   placing the kernel in physical memory
> >> - changes to the UEFI stub code to choose the KASLR offset and communicate
> >>   it to the decompressor
> >
> > Would it make sense to also randomize the pa-to-va mapping on top of this?
> > That can certainly be a later follow-up, I'm just trying to think of the options
> > we have, given that the kernel is now relocatable and we can support arbitrary
> > pa-to-va mappings already.
> >
> 
> We could randomize PAGE_OFFSET as well. That allows you to build a
> 3g/1g split kernel and execute it as 2g/2g split. Pretty neat!

This is going to break existing user space binaries at some point. Some 
applications pretty much always assumed 3g space and are likely to fail 
otherwise. The 2g/2g config is good for people wanting to dispense with 
highmem and the ability to test their user space with it before 
deployment.

Also, this is a lot of complexity added to the kernel for a very 
negligible security gain. It is trivial from user space to determine 
what the actual PAGE_OFFSET is, so that won't be a serious deterrent.

> Randomizing the VA to PA mapping while keep PAGE_OFFSET constant will
> result in either memory to be thrown away (because it is virtually
> mapped below PAGE_OFFSET) or lowmem space to be wasted (because there
> is a hole between PAGE_OFFSET and the VA of the lowest lowmem address)
> 
> So i think there may be opportunities, but I haven't quite figured
> them out myself yet.
> 
> > Can you explain how the random seed is passed from the bootloader
> > to the kernel when we don't use EFI? Is this implemented at all? I see
> > that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
> > the EFI boot services, but I don't see where that value gets read again
> > when we relocate the kernel.
> 
> /chosen/kaslr-seed is only used on arm64, not on ARM. We could add
> code to the decompressor that uses /chosen/kaslr-seed, but it is a bit
> fiddly because the execution environment is so constrained, and there
> is no simple access to symbols defined by the core kernel's linker
> script.
> 
> On UEFI systems, the kaslr offset is calculated based on the UEFI
> memory map, which describes all of memory and has reservations for the
> DTB, the initrd etc. The EFI stub is linked together with the
> decompressor, so passing the kaslr offset simply involves setting a
> variable.
> 
> To allow other bootloaders to do the same, the kaslr metadata is
> exposed via a zImage header, containing the values of PAGE_OFFSET, the
> base of the vmalloc area and the randomization granularity. A
> bootloader can read these values, and taking the size of DRAM and the
> placement of initrd and DTB into account, it can choose a value for
> kaslr offset and write it back into the zImage header.
> 
> This is a bit involved, but it is really difficult to make these
> things backward compatible, i.e., passing something in a register is
> not possible if that register was not mandated to be zero initially.
> 
> Similarly, the decompressor passed the kaslr offset to the startup
> code in the core kernel. It does so by passing it in r3 and jumping 4
> bytes past the entry point. This way, we are backward compatible with
> configurations where the decompressor is not used, because in that
> case, you always jump to the first instruction, which zeroes r3.

Please capture all this somewhere. Either in the commit log, or in the 
Booting document, or both.


Nicolas

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Nicolas Pitre]

On Mon, 14 Aug 2017, Dave Martin wrote:

> On Mon, Aug 14, 2017 at 04:38:02PM +0100, Ard Biesheuvel wrote:
> > That's a trick, actually, which I failed to add a comment for.

Shame shame shame !

> > We use .arm sections in the thumb2 kernel, and using these macros
> > there would result in the wrong offset to be used. Adding the .w
> > suffix forces an error in the assembler which even results in a fairly
> > meaningful error message complaining about using .w in ARM code.
> 
> Ewww... I think it'd be best to add a comment explaining that.

Absolutely!


Nicolas

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Ard Biesheuvel]

On 14 August 2017 at 17:18, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Mon, 14 Aug 2017, Dave Martin wrote:
>
>> On Mon, Aug 14, 2017 at 04:38:02PM +0100, Ard Biesheuvel wrote:
>> > That's a trick, actually, which I failed to add a comment for.
>
> Shame shame shame !
>
>> > We use .arm sections in the thumb2 kernel, and using these macros
>> > there would result in the wrong offset to be used. Adding the .w
>> > suffix forces an error in the assembler which even results in a fairly
>> > meaningful error message complaining about using .w in ARM code.
>>
>> Ewww... I think it'd be best to add a comment explaining that.
>
> Absolutely!
>

Yeah, mea culpa.

But if people have better ideas how to avoid this situation, I am all ears.

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Nicolas Pitre]

On Mon, 14 Aug 2017, Arnd Bergmann wrote:

> On Mon, Aug 14, 2017 at 5:49 PM, Ard Biesheuvel
> <ard.biesheuvel@linaro.org> wrote:
> > On 14 August 2017 at 16:30, Arnd Bergmann <arnd@arndb.de> wrote:
> 
> 
> >> Can you explain how the random seed is passed from the bootloader
> >> to the kernel when we don't use EFI? Is this implemented at all? I see
> >> that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
> >> the EFI boot services, but I don't see where that value gets read again
> >> when we relocate the kernel.
> 
> > To allow other bootloaders to do the same, the kaslr metadata is
> > exposed via a zImage header, containing the values of PAGE_OFFSET, the
> > base of the vmalloc area and the randomization granularity. A
> > bootloader can read these values, and taking the size of DRAM and the
> > placement of initrd and DTB into account, it can choose a value for
> > kaslr offset and write it back into the zImage header.
> >
> > This is a bit involved, but it is really difficult to make these
> > things backward compatible, i.e., passing something in a register is
> > not possible if that register was not mandated to be zero initially.
> >
> > Similarly, the decompressor passed the kaslr offset to the startup
> > code in the core kernel. It does so by passing it in r3 and jumping 4
> > bytes past the entry point. This way, we are backward compatible with
> > configurations where the decompressor is not used, because in that
> > case, you always jump to the first instruction, which zeroes r3.
> 
> There are two ideas we discussed in the past (but never implemented
> them obviously):
> 
> - instead of reading the "kaslr-seed" in the decompressor, it could
>   simply hash all of the DT blob to get the seed. This way the bootloader
>   can put the random see anywhere it likes, and as an added bonus,
>   we also get a little bit more random behavior on machines that have
>   no entropy source at all but that do have things like a serial number or
>   mac address in DT. Obviously those would be constant across boots
>   but different between machines. The OS can also store a random
>   seed during shutdown in a location that the bootloader uses to
>   initialize /chosen/kaslr-seed or another property that we use to seed
>   the kernel PRNG at boot time.
> 
> - If we have a random number at boot but no way to pass it through
>   the DT, I think we actually /can/ pass it through registers: the
>   register state is undefined, so in the worst case using the XOR of
>   all registers gives us the same number on each boot, but the
>   if the loader is modified to store a random 32-bit number in any
>   of the registers that don't pass other information, we can use that
>   to calculate the kaslr-base.

I really like the later. This way there is no hard protocol to define 
and follow. The bootloader may exploit any source of randomness it can 
find, including the RTC in addition to the serial number for example. So 
doing both on the kernel side might actually be the best approach, 
giving the boot environment all the flexibility it wants and being 
compatible with all of them.


Nicolas

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Nicolas Pitre]

On Mon, 14 Aug 2017, Ard Biesheuvel wrote:

> On 14 August 2017 at 17:18, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Mon, 14 Aug 2017, Dave Martin wrote:
> >
> >> On Mon, Aug 14, 2017 at 04:38:02PM +0100, Ard Biesheuvel wrote:
> >> > That's a trick, actually, which I failed to add a comment for.
> >
> > Shame shame shame !
> >
> >> > We use .arm sections in the thumb2 kernel, and using these macros
> >> > there would result in the wrong offset to be used. Adding the .w
> >> > suffix forces an error in the assembler which even results in a fairly
> >> > meaningful error message complaining about using .w in ARM code.
> >>
> >> Ewww... I think it'd be best to add a comment explaining that.
> >
> > Absolutely!
> >
> 
> Yeah, mea culpa.
> 
> But if people have better ideas how to avoid this situation, I am all ears.

Just document it.  ;-)


Nicolas

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Russell King - ARM Linux]

On Mon, Aug 14, 2017 at 05:22:39PM +0100, Ard Biesheuvel wrote:
> On 14 August 2017 at 17:18, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Mon, 14 Aug 2017, Dave Martin wrote:
> >
> >> On Mon, Aug 14, 2017 at 04:38:02PM +0100, Ard Biesheuvel wrote:
> >> > That's a trick, actually, which I failed to add a comment for.
> >
> > Shame shame shame !
> >
> >> > We use .arm sections in the thumb2 kernel, and using these macros
> >> > there would result in the wrong offset to be used. Adding the .w
> >> > suffix forces an error in the assembler which even results in a fairly
> >> > meaningful error message complaining about using .w in ARM code.
> >>
> >> Ewww... I think it'd be best to add a comment explaining that.
> >
> > Absolutely!
> >
> 
> Yeah, mea culpa.
> 
> But if people have better ideas how to avoid this situation, I am all ears.

Have you tested building an ARMv7M kernel with your patches - ARMv7M is
Thumb only, so can't contain any ARM code.  If not, please try
mps2_defconfig.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up

[kernel-hardening] Re: [PATCH 02/30] ARM: assembler: introduce adr_l, ldr_l and str_l macros

[Ard Biesheuvel]

On 14 August 2017 at 17:42, Russell King - ARM Linux
<linux@armlinux.org.uk> wrote:
> On Mon, Aug 14, 2017 at 05:22:39PM +0100, Ard Biesheuvel wrote:
>> On 14 August 2017 at 17:18, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Mon, 14 Aug 2017, Dave Martin wrote:
>> >
>> >> On Mon, Aug 14, 2017 at 04:38:02PM +0100, Ard Biesheuvel wrote:
>> >> > That's a trick, actually, which I failed to add a comment for.
>> >
>> > Shame shame shame !
>> >
>> >> > We use .arm sections in the thumb2 kernel, and using these macros
>> >> > there would result in the wrong offset to be used. Adding the .w
>> >> > suffix forces an error in the assembler which even results in a fairly
>> >> > meaningful error message complaining about using .w in ARM code.
>> >>
>> >> Ewww... I think it'd be best to add a comment explaining that.
>> >
>> > Absolutely!
>> >
>>
>> Yeah, mea culpa.
>>
>> But if people have better ideas how to avoid this situation, I am all ears.
>
> Have you tested building an ARMv7M kernel with your patches - ARMv7M is
> Thumb only, so can't contain any ARM code.  If not, please try
> mps2_defconfig.
>

mps2_defconfig builds without error with these patches. I don't have
the hardware so I can't boot it, unfortunately.

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Ard Biesheuvel]

On 14 August 2017 at 17:28, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Mon, 14 Aug 2017, Arnd Bergmann wrote:
>
>> On Mon, Aug 14, 2017 at 5:49 PM, Ard Biesheuvel
>> <ard.biesheuvel@linaro.org> wrote:
>> > On 14 August 2017 at 16:30, Arnd Bergmann <arnd@arndb.de> wrote:
>>
>>
>> >> Can you explain how the random seed is passed from the bootloader
>> >> to the kernel when we don't use EFI? Is this implemented at all? I see
>> >> that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
>> >> the EFI boot services, but I don't see where that value gets read again
>> >> when we relocate the kernel.
>>
>> > To allow other bootloaders to do the same, the kaslr metadata is
>> > exposed via a zImage header, containing the values of PAGE_OFFSET, the
>> > base of the vmalloc area and the randomization granularity. A
>> > bootloader can read these values, and taking the size of DRAM and the
>> > placement of initrd and DTB into account, it can choose a value for
>> > kaslr offset and write it back into the zImage header.
>> >
>> > This is a bit involved, but it is really difficult to make these
>> > things backward compatible, i.e., passing something in a register is
>> > not possible if that register was not mandated to be zero initially.
>> >
>> > Similarly, the decompressor passed the kaslr offset to the startup
>> > code in the core kernel. It does so by passing it in r3 and jumping 4
>> > bytes past the entry point. This way, we are backward compatible with
>> > configurations where the decompressor is not used, because in that
>> > case, you always jump to the first instruction, which zeroes r3.
>>
>> There are two ideas we discussed in the past (but never implemented
>> them obviously):
>>
>> - instead of reading the "kaslr-seed" in the decompressor, it could
>>   simply hash all of the DT blob to get the seed. This way the bootloader
>>   can put the random see anywhere it likes, and as an added bonus,
>>   we also get a little bit more random behavior on machines that have
>>   no entropy source at all but that do have things like a serial number or
>>   mac address in DT. Obviously those would be constant across boots
>>   but different between machines. The OS can also store a random
>>   seed during shutdown in a location that the bootloader uses to
>>   initialize /chosen/kaslr-seed or another property that we use to seed
>>   the kernel PRNG at boot time.
>>
>> - If we have a random number at boot but no way to pass it through
>>   the DT, I think we actually /can/ pass it through registers: the
>>   register state is undefined, so in the worst case using the XOR of
>>   all registers gives us the same number on each boot, but the
>>   if the loader is modified to store a random 32-bit number in any
>>   of the registers that don't pass other information, we can use that
>>   to calculate the kaslr-base.
>
> I really like the later. This way there is no hard protocol to define
> and follow. The bootloader may exploit any source of randomness it can
> find, including the RTC in addition to the serial number for example. So
> doing both on the kernel side might actually be the best approach,
> giving the boot environment all the flexibility it wants and being
> compatible with all of them.
>

Finding a source of entropy is not trivial, but it is not the difficult part.

So when we pass some random seed to the decompressor, what will it do
with it? In order to find a suitable KASLR offset, it needs to know
the size of DRAM, the placement of the DT and potentially an initrd,
and the size of the vmalloc region in order to decide where it can put
the kernel.

In my implementation, the decompressor simply receives the offset from
the bootloader, and exposes the value of PAGE_OFFSET, the base of  the
vmalloc region and the kaslr granularity supported by the kernel. The
bootloader should already know the size of DRAM and where it loaded
the DT and initrd, so it can roll the dice in an informed manner.

Note that my UEFI stub implementation essentially does the same, which
was trivial to implement because UEFI already keeps track of all
allocations in DRAM. Note that this version simply disables KASLR if
it encounters a vmalloc= command line argument, given that it affects
the size of the lowmem region. We could enhance that to actually parse
the value, but I kept it simple for now.

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Nicolas Pitre]

On Mon, 14 Aug 2017, Ard Biesheuvel wrote:

> On 14 August 2017 at 17:28, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Mon, 14 Aug 2017, Arnd Bergmann wrote:
> >
> >> On Mon, Aug 14, 2017 at 5:49 PM, Ard Biesheuvel
> >> <ard.biesheuvel@linaro.org> wrote:
> >> > On 14 August 2017 at 16:30, Arnd Bergmann <arnd@arndb.de> wrote:
> >>
> >>
> >> >> Can you explain how the random seed is passed from the bootloader
> >> >> to the kernel when we don't use EFI? Is this implemented at all? I see
> >> >> that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
> >> >> the EFI boot services, but I don't see where that value gets read again
> >> >> when we relocate the kernel.
> >>
> >> > To allow other bootloaders to do the same, the kaslr metadata is
> >> > exposed via a zImage header, containing the values of PAGE_OFFSET, the
> >> > base of the vmalloc area and the randomization granularity. A
> >> > bootloader can read these values, and taking the size of DRAM and the
> >> > placement of initrd and DTB into account, it can choose a value for
> >> > kaslr offset and write it back into the zImage header.
> >> >
> >> > This is a bit involved, but it is really difficult to make these
> >> > things backward compatible, i.e., passing something in a register is
> >> > not possible if that register was not mandated to be zero initially.
> >> >
> >> > Similarly, the decompressor passed the kaslr offset to the startup
> >> > code in the core kernel. It does so by passing it in r3 and jumping 4
> >> > bytes past the entry point. This way, we are backward compatible with
> >> > configurations where the decompressor is not used, because in that
> >> > case, you always jump to the first instruction, which zeroes r3.
> >>
> >> There are two ideas we discussed in the past (but never implemented
> >> them obviously):
> >>
> >> - instead of reading the "kaslr-seed" in the decompressor, it could
> >>   simply hash all of the DT blob to get the seed. This way the bootloader
> >>   can put the random see anywhere it likes, and as an added bonus,
> >>   we also get a little bit more random behavior on machines that have
> >>   no entropy source at all but that do have things like a serial number or
> >>   mac address in DT. Obviously those would be constant across boots
> >>   but different between machines. The OS can also store a random
> >>   seed during shutdown in a location that the bootloader uses to
> >>   initialize /chosen/kaslr-seed or another property that we use to seed
> >>   the kernel PRNG at boot time.
> >>
> >> - If we have a random number at boot but no way to pass it through
> >>   the DT, I think we actually /can/ pass it through registers: the
> >>   register state is undefined, so in the worst case using the XOR of
> >>   all registers gives us the same number on each boot, but the
> >>   if the loader is modified to store a random 32-bit number in any
> >>   of the registers that don't pass other information, we can use that
> >>   to calculate the kaslr-base.
> >
> > I really like the later. This way there is no hard protocol to define
> > and follow. The bootloader may exploit any source of randomness it can
> > find, including the RTC in addition to the serial number for example. So
> > doing both on the kernel side might actually be the best approach,
> > giving the boot environment all the flexibility it wants and being
> > compatible with all of them.
> >
> 
> Finding a source of entropy is not trivial, but it is not the difficult part.
> 
> So when we pass some random seed to the decompressor, what will it do
> with it? In order to find a suitable KASLR offset, it needs to know
> the size of DRAM, the placement of the DT and potentially an initrd,
> and the size of the vmalloc region in order to decide where it can put
> the kernel.
> 
> In my implementation, the decompressor simply receives the offset from
> the bootloader, and exposes the value of PAGE_OFFSET, the base of  the
> vmalloc region and the kaslr granularity supported by the kernel. The
> bootloader should already know the size of DRAM and where it loaded
> the DT and initrd, so it can roll the dice in an informed manner.
> 
> Note that my UEFI stub implementation essentially does the same, which
> was trivial to implement because UEFI already keeps track of all
> allocations in DRAM. Note that this version simply disables KASLR if
> it encounters a vmalloc= command line argument, given that it affects
> the size of the lowmem region. We could enhance that to actually parse
> the value, but I kept it simple for now.

What I dislike about such an arrangement (and I've brought up this 
argument forward in a different context before) is that you create a lot 
of additional dependencies between the kernel and the boot environment. 
The kernel is no longer self-sufficient and all this boot preparation 
has to be duplicated in all boot environments from UEFI to U-Boot to 
qemu. The fact that the bootloader now has to care about very Linux 
internal concepts such as vmalloc_start makes it rather inelegant to me.

I'm even wondering if, design wise, the best solution wouldn't be for 
the actual kernel to move itself during the boot process and reboot 
itself without any external help. Not necessarily go as far as the full 
kexec danse , but boot far enough to parse the DT, initialize the 
bootmem allocator, then find a new location for itself, move it there, 
do the relocs and restart the boot process.  For this to work well, you 
would have to make a copy of the .data section so to reboot again with a 
pristine version except for one flag indicating that the move has been 
done.

Doing it this way gives you a full kernel environment to work from and 
be completely self-sufficient with zero reliance on any boot 
environment. This would even make it compatible with Image i.e. the 
compressor-less kernel. And if the DT and/or initrd is in the way then 
you could even go as far as moving them away if you wanted.

This would add some boot latency of course, but certainly in the 
sub-second range. That is negligible for those systems where KASLR is 
most relevant.


Nicolas

[kernel-hardening] Re: [PATCH 00/30] implement KASLR for ARM

[Ard Biesheuvel]

On 14 August 2017 at 19:01, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Mon, 14 Aug 2017, Ard Biesheuvel wrote:
>
>> On 14 August 2017 at 17:28, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Mon, 14 Aug 2017, Arnd Bergmann wrote:
>> >
>> >> On Mon, Aug 14, 2017 at 5:49 PM, Ard Biesheuvel
>> >> <ard.biesheuvel@linaro.org> wrote:
>> >> > On 14 August 2017 at 16:30, Arnd Bergmann <arnd@arndb.de> wrote:
>> >>
>> >>
>> >> >> Can you explain how the random seed is passed from the bootloader
>> >> >> to the kernel when we don't use EFI? Is this implemented at all? I see
>> >> >> that you add a seed to "/chosen/kaslr-seed" in the EFI stub when using
>> >> >> the EFI boot services, but I don't see where that value gets read again
>> >> >> when we relocate the kernel.
>> >>
>> >> > To allow other bootloaders to do the same, the kaslr metadata is
>> >> > exposed via a zImage header, containing the values of PAGE_OFFSET, the
>> >> > base of the vmalloc area and the randomization granularity. A
>> >> > bootloader can read these values, and taking the size of DRAM and the
>> >> > placement of initrd and DTB into account, it can choose a value for
>> >> > kaslr offset and write it back into the zImage header.
>> >> >
>> >> > This is a bit involved, but it is really difficult to make these
>> >> > things backward compatible, i.e., passing something in a register is
>> >> > not possible if that register was not mandated to be zero initially.
>> >> >
>> >> > Similarly, the decompressor passed the kaslr offset to the startup
>> >> > code in the core kernel. It does so by passing it in r3 and jumping 4
>> >> > bytes past the entry point. This way, we are backward compatible with
>> >> > configurations where the decompressor is not used, because in that
>> >> > case, you always jump to the first instruction, which zeroes r3.
>> >>
>> >> There are two ideas we discussed in the past (but never implemented
>> >> them obviously):
>> >>
>> >> - instead of reading the "kaslr-seed" in the decompressor, it could
>> >>   simply hash all of the DT blob to get the seed. This way the bootloader
>> >>   can put the random see anywhere it likes, and as an added bonus,
>> >>   we also get a little bit more random behavior on machines that have
>> >>   no entropy source at all but that do have things like a serial number or
>> >>   mac address in DT. Obviously those would be constant across boots
>> >>   but different between machines. The OS can also store a random
>> >>   seed during shutdown in a location that the bootloader uses to
>> >>   initialize /chosen/kaslr-seed or another property that we use to seed
>> >>   the kernel PRNG at boot time.
>> >>
>> >> - If we have a random number at boot but no way to pass it through
>> >>   the DT, I think we actually /can/ pass it through registers: the
>> >>   register state is undefined, so in the worst case using the XOR of
>> >>   all registers gives us the same number on each boot, but the
>> >>   if the loader is modified to store a random 32-bit number in any
>> >>   of the registers that don't pass other information, we can use that
>> >>   to calculate the kaslr-base.
>> >
>> > I really like the later. This way there is no hard protocol to define
>> > and follow. The bootloader may exploit any source of randomness it can
>> > find, including the RTC in addition to the serial number for example. So
>> > doing both on the kernel side might actually be the best approach,
>> > giving the boot environment all the flexibility it wants and being
>> > compatible with all of them.
>> >
>>
>> Finding a source of entropy is not trivial, but it is not the difficult part.
>>
>> So when we pass some random seed to the decompressor, what will it do
>> with it? In order to find a suitable KASLR offset, it needs to know
>> the size of DRAM, the placement of the DT and potentially an initrd,
>> and the size of the vmalloc region in order to decide where it can put
>> the kernel.
>>
>> In my implementation, the decompressor simply receives the offset from
>> the bootloader, and exposes the value of PAGE_OFFSET, the base of  the
>> vmalloc region and the kaslr granularity supported by the kernel. The
>> bootloader should already know the size of DRAM and where it loaded
>> the DT and initrd, so it can roll the dice in an informed manner.
>>
>> Note that my UEFI stub implementation essentially does the same, which
>> was trivial to implement because UEFI already keeps track of all
>> allocations in DRAM. Note that this version simply disables KASLR if
>> it encounters a vmalloc= command line argument, given that it affects
>> the size of the lowmem region. We could enhance that to actually parse
>> the value, but I kept it simple for now.
>
> What I dislike about such an arrangement (and I've brought up this
> argument forward in a different context before) is that you create a lot
> of additional dependencies between the kernel and the boot environment.
> The kernel is no longer self-sufficient and all this boot preparation
> has to be duplicated in all boot environments from UEFI to U-Boot to
> qemu. The fact that the bootloader now has to care about very Linux
> internal concepts such as vmalloc_start makes it rather inelegant to me.
>
> I'm even wondering if, design wise, the best solution wouldn't be for
> the actual kernel to move itself during the boot process and reboot
> itself without any external help. Not necessarily go as far as the full
> kexec danse , but boot far enough to parse the DT, initialize the
> bootmem allocator, then find a new location for itself, move it there,
> do the relocs and restart the boot process.  For this to work well, you
> would have to make a copy of the .data section so to reboot again with a
> pristine version except for one flag indicating that the move has been
> done.
>
> Doing it this way gives you a full kernel environment to work from and
> be completely self-sufficient with zero reliance on any boot
> environment. This would even make it compatible with Image i.e. the
> compressor-less kernel. And if the DT and/or initrd is in the way then
> you could even go as far as moving them away if you wanted.
>

Interestingly, this is essentially how I implemented it for arm64. It
boots to the point where it can retrieve the kaslr-seed from the DT
(and the 'nokaslr' command line argument), and either proceeds (if
there is no seed or nokaslr is passed), or it returns to the startup
code and remaps the kernel at a randomized offset.

However, the configurations are very different. The arm64 kernel
mappings are disjoint from the kernel's direct mapping, and the
physical offset is under the control of the bootloader anyway. So when
it returns to the startup code, it only updates the virtual mapping
not the physical placement.

> This would add some boot latency of course, but certainly in the
> sub-second range. That is negligible for those systems where KASLR is
> most relevant.
>

I will try to come up with something that makes it more self contained.

[kernel-hardening] Re: [PATCH 20/30] ARM: kernel: use PC relative symbol references in suspend/resume code

[Ard Biesheuvel]

On 14 August 2017 at 17:02, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Mon, 14 Aug 2017, Ard Biesheuvel wrote:
>
>> Replace some unnecessary absolute references with relative ones. Also,
>> to prepare for runtime relocation, which occurs with the caches on,
>> defer taking the absolute address of cpu_resume_after_mmu() until after
>> the MMU is enabled.
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>>  arch/arm/kernel/sleep.S | 11 +++++------
>>  1 file changed, 5 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
>> index 3026b119d3ff..9efd1c7d3552 100644
>> --- a/arch/arm/kernel/sleep.S
>> +++ b/arch/arm/kernel/sleep.S
>> @@ -60,18 +60,17 @@
>>  ENTRY(__cpu_suspend)
>>       stmfd   sp!, {r4 - r11, lr}
>>  #ifdef MULTI_CPU
>> -     ldr     r10, =processor
>> -     ldr     r4, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
>> +     ldr_l   r4, processor + CPU_SLEEP_SIZE  @ size of CPU sleep state
>>  #else
>> -     ldr     r4, =cpu_suspend_size
>> +     adr_l   r4, cpu_suspend_size
>>  #endif
>>       mov     r5, sp                  @ current virtual SP
>>       add     r4, r4, #12             @ Space for pgd, virt sp, phys resume fn
>>       sub     sp, sp, r4              @ allocate CPU state on stack
>> -     ldr     r3, =sleep_save_sp
>> +     adr_l   r3, sleep_save_sp
>>       stmfd   sp!, {r0, r1}           @ save suspend func arg and pointer
>>       ldr     r3, [r3, #SLEEP_SAVE_SP_VIRT]
>> -     ALT_SMP(ldr r0, =mpidr_hash)
>> +     ALT_SMP(adr_l r0, mpidr_hash)
>>       ALT_UP_B(1f)
>
> The above is dangerous. adr_l expands to more than one instruction which
> is not what ALT_SMP() was designed for. Here it might happen to work
> anyway because it is combined with ALT_UP_B() but with ALT_UP() it
> wouldn't. This is a mistake waiting to happen.
>

OK. I will use the opencoded sequence instead in this case. I.e.,

-       ALT_SMP(ldr r0, =mpidr_hash)
+0:     ALT_SMP(adr r0, 2f)
        ALT_UP_B(1f)
+       ldr     r1, [r0]
+       add     r0, r0, r1

and

 ENDPROC(__cpu_suspend)
+       .align  2
+2:     .long   mpidr_hash - .
        .ltorg

[kernel-hardening] Re: [PATCH 20/30] ARM: kernel: use PC relative symbol references in suspend/resume code

[Nicolas Pitre]

On Mon, 14 Aug 2017, Ard Biesheuvel wrote:

> On 14 August 2017 at 17:02, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Mon, 14 Aug 2017, Ard Biesheuvel wrote:
> >
> >> Replace some unnecessary absolute references with relative ones. Also,
> >> to prepare for runtime relocation, which occurs with the caches on,
> >> defer taking the absolute address of cpu_resume_after_mmu() until after
> >> the MMU is enabled.
> >>
> >> Cc: Russell King <linux@armlinux.org.uk>
> >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> ---
> >>  arch/arm/kernel/sleep.S | 11 +++++------
> >>  1 file changed, 5 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
> >> index 3026b119d3ff..9efd1c7d3552 100644
> >> --- a/arch/arm/kernel/sleep.S
> >> +++ b/arch/arm/kernel/sleep.S
> >> @@ -60,18 +60,17 @@
> >>  ENTRY(__cpu_suspend)
> >>       stmfd   sp!, {r4 - r11, lr}
> >>  #ifdef MULTI_CPU
> >> -     ldr     r10, =processor
> >> -     ldr     r4, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
> >> +     ldr_l   r4, processor + CPU_SLEEP_SIZE  @ size of CPU sleep state
> >>  #else
> >> -     ldr     r4, =cpu_suspend_size
> >> +     adr_l   r4, cpu_suspend_size
> >>  #endif
> >>       mov     r5, sp                  @ current virtual SP
> >>       add     r4, r4, #12             @ Space for pgd, virt sp, phys resume fn
> >>       sub     sp, sp, r4              @ allocate CPU state on stack
> >> -     ldr     r3, =sleep_save_sp
> >> +     adr_l   r3, sleep_save_sp
> >>       stmfd   sp!, {r0, r1}           @ save suspend func arg and pointer
> >>       ldr     r3, [r3, #SLEEP_SAVE_SP_VIRT]
> >> -     ALT_SMP(ldr r0, =mpidr_hash)
> >> +     ALT_SMP(adr_l r0, mpidr_hash)
> >>       ALT_UP_B(1f)
> >
> > The above is dangerous. adr_l expands to more than one instruction which
> > is not what ALT_SMP() was designed for. Here it might happen to work
> > anyway because it is combined with ALT_UP_B() but with ALT_UP() it
> > wouldn't. This is a mistake waiting to happen.
> >
> 
> OK. I will use the opencoded sequence instead in this case. I.e.,
> 
> -       ALT_SMP(ldr r0, =mpidr_hash)
> +0:     ALT_SMP(adr r0, 2f)
>         ALT_UP_B(1f)
> +       ldr     r1, [r0]
> +       add     r0, r0, r1
> 
> and
> 
>  ENDPROC(__cpu_suspend)
> +       .align  2
> +2:     .long   mpidr_hash - .
>         .ltorg
> 

Yeah... I see no way around it.

And if you make this particular case into a commit of its own, then the 
commit log may carry the above reasoning.


Nicolas

[kernel-hardening] Re: [PATCH 29/30] efi/libstub: arm: reserve bootloader supplied initrd in memory map

[Ard Biesheuvel]

On 14 August 2017 at 13:54, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> Under KASLR, the EFI stub may allocate the kernel anywhere in the
> physical address space, which could be right on top of an initrd
> if it was supplied by the bootloader (i.e., GRUB) in /chosen rather
> than passed via the initrd= command line option. So allocate the
> pages explicitly, this ensures that the random memory allocation
> routine will disregard the region.
>
> Note that this means that we need to defer the handle_kernel_image()
> call.
>
> Cc: Matt Fleming <matt@codeblueprint.co.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

If nobody objects, I will take this patch through the EFI tree
separately for v4.14, given that it fixes an issue that may affect
arm64 as well.

> ---
>  drivers/firmware/efi/libstub/arm-stub.c | 51 ++++++++++++--------
>  drivers/firmware/efi/libstub/efistub.h  |  3 ++
>  drivers/firmware/efi/libstub/fdt.c      | 42 ++++++++++++++++
>  3 files changed, 75 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/firmware/efi/libstub/arm-stub.c b/drivers/firmware/efi/libstub/arm-stub.c
> index 8181ac179d14..f5ef5ccd5f45 100644
> --- a/drivers/firmware/efi/libstub/arm-stub.c
> +++ b/drivers/firmware/efi/libstub/arm-stub.c
> @@ -133,6 +133,7 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>         unsigned long reserve_size = 0;
>         enum efi_secureboot_mode secure_boot;
>         struct screen_info *si;
> +       bool have_chosen_initrd;
>
>         /* Check if we were booted by the EFI firmware */
>         if (sys_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE)
> @@ -183,15 +184,6 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>
>         si = setup_graphics(sys_table);
>
> -       status = handle_kernel_image(sys_table, image_addr, &image_size,
> -                                    &reserve_addr,
> -                                    &reserve_size,
> -                                    dram_base, image);
> -       if (status != EFI_SUCCESS) {
> -               pr_efi_err(sys_table, "Failed to relocate kernel\n");
> -               goto fail_free_cmdline;
> -       }
> -
>         secure_boot = efi_get_secureboot(sys_table);
>
>         /*
> @@ -209,7 +201,7 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>
>                 if (status != EFI_SUCCESS) {
>                         pr_efi_err(sys_table, "Failed to load device tree!\n");
> -                       goto fail_free_image;
> +                       goto fail_free_cmdline;
>                 }
>         }
>
> @@ -222,16 +214,33 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>                         pr_efi(sys_table, "Using DTB from configuration table\n");
>         }
>
> -       if (!fdt_addr)
> +       if (!fdt_addr) {
>                 pr_efi(sys_table, "Generating empty DTB\n");
> +               have_chosen_initrd = false;
> +       } else {
> +               status = efi_reserve_dtb_initrd(sys_table, (void *)fdt_addr);
> +               have_chosen_initrd = (status != EFI_NOT_FOUND);
> +       }
>
> -       status = handle_cmdline_files(sys_table, image, cmdline_ptr, "initrd=",
> -                                     efi_get_max_initrd_addr(dram_base,
> -                                                             *image_addr),
> -                                     (unsigned long *)&initrd_addr,
> -                                     (unsigned long *)&initrd_size);
> -       if (status != EFI_SUCCESS)
> -               pr_efi_err(sys_table, "Failed initrd from command line!\n");
> +       status = handle_kernel_image(sys_table, image_addr, &image_size,
> +                                    &reserve_addr, &reserve_size,
> +                                    dram_base, image);
> +       if (status != EFI_SUCCESS) {
> +               pr_efi_err(sys_table, "Failed to relocate kernel\n");
> +               goto fail_free_fdt;
> +       }
> +
> +       if (!have_chosen_initrd) {
> +               status = handle_cmdline_files(sys_table, image, cmdline_ptr,
> +                                             "initrd=",
> +                                             efi_get_max_initrd_addr(dram_base,
> +                                                                     *image_addr),
> +                                             (unsigned long *)&initrd_addr,
> +                                             (unsigned long *)&initrd_size);
> +               if (status != EFI_SUCCESS)
> +                       pr_efi_err(sys_table,
> +                                  "Failed initrd from command line!\n");
> +       }
>
>         efi_random_get_seed(sys_table);
>
> @@ -272,11 +281,11 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>         pr_efi_err(sys_table, "Failed to update FDT and exit boot services\n");
>
>         efi_free(sys_table, initrd_size, initrd_addr);
> -       efi_free(sys_table, fdt_size, fdt_addr);
> -
> -fail_free_image:
>         efi_free(sys_table, image_size, *image_addr);
>         efi_free(sys_table, reserve_size, reserve_addr);
> +
> +fail_free_fdt:
> +       efi_free(sys_table, fdt_size, fdt_addr);
>  fail_free_cmdline:
>         free_screen_info(sys_table, si);
>         efi_free(sys_table, cmdline_size, (unsigned long)cmdline_ptr);
> diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h
> index aaf2aeb785ea..35b514d7d962 100644
> --- a/drivers/firmware/efi/libstub/efistub.h
> +++ b/drivers/firmware/efi/libstub/efistub.h
> @@ -68,4 +68,7 @@ efi_status_t check_platform_features(efi_system_table_t *sys_table_arg);
>
>  efi_status_t efi_random_get_seed(efi_system_table_t *sys_table_arg);
>
> +efi_status_t efi_reserve_dtb_initrd(efi_system_table_t *sys_table_arg,
> +                                   const void *fdt);
> +
>  #endif
> diff --git a/drivers/firmware/efi/libstub/fdt.c b/drivers/firmware/efi/libstub/fdt.c
> index 8830fa601e45..54408c95e094 100644
> --- a/drivers/firmware/efi/libstub/fdt.c
> +++ b/drivers/firmware/efi/libstub/fdt.c
> @@ -385,3 +385,45 @@ void *get_fdt(efi_system_table_t *sys_table, unsigned long *fdt_size)
>
>         return fdt;
>  }
> +
> +efi_status_t efi_reserve_dtb_initrd(efi_system_table_t *sys_table_arg,
> +                                   const void *fdt)
> +{
> +       int chosen, len;
> +       const void *prop;
> +       efi_physical_addr_t start, end;
> +       unsigned long num_pages;
> +       efi_status_t status;
> +
> +       chosen = fdt_path_offset(fdt, "/chosen");
> +       if (chosen == -FDT_ERR_NOTFOUND)
> +               return EFI_NOT_FOUND;
> +
> +       prop = fdt_getprop(fdt, chosen, "linux,initrd-start", &len);
> +       if (!prop)
> +               return EFI_NOT_FOUND;
> +
> +       start = (len == sizeof(fdt32_t)) ? fdt32_to_cpu(*(fdt32_t *)prop)
> +                                        : fdt64_to_cpu(*(fdt64_t *)prop);
> +
> +       prop = fdt_getprop(fdt, chosen, "linux,initrd-end", &len);
> +       if (!prop)
> +               return EFI_NOT_FOUND;
> +
> +       end = (len == sizeof(fdt32_t)) ? fdt32_to_cpu(*(fdt32_t *)prop)
> +                                      : fdt64_to_cpu(*(fdt64_t *)prop);
> +
> +       start = round_down(start, EFI_PAGE_SIZE);
> +       num_pages = round_up(end - start, EFI_PAGE_SIZE) / EFI_PAGE_SIZE;
> +
> +       status = efi_call_early(allocate_pages, EFI_ALLOCATE_ADDRESS,
> +                               EFI_LOADER_DATA, num_pages, &start);
> +
> +       if (status != EFI_SUCCESS)
> +               pr_efi_err(sys_table_arg,
> +                          "Failed to reserve initrd area found in /chosen\n");
> +       else
> +               pr_efi(sys_table_arg, "Using initrd found in /chosen\n");
> +
> +       return status;
> +}
> --
> 2.11.0
>

[kernel-hardening] Re: [PATCH 29/30] efi/libstub: arm: reserve bootloader supplied initrd in memory map

[Mark Rutland]

On Mon, Aug 14, 2017 at 01:54:10PM +0100, Ard Biesheuvel wrote:
> Under KASLR, the EFI stub may allocate the kernel anywhere in the
> physical address space, which could be right on top of an initrd
> if it was supplied by the bootloader (i.e., GRUB) in /chosen rather
> than passed via the initrd= command line option. So allocate the
> pages explicitly, this ensures that the random memory allocation
> routine will disregard the region.

I'm a little confused. Shouldn't the bootloader have allocated that
memory, leaving it reserved?

If it hasn't, then that region could be allcoated by anything else at
any time (e.g. by the code which loads the kernel, or some EFI timer
callback), so that sounds like a bootloader bug that we can't fix.

Thanks,
Mark.

> 
> Note that this means that we need to defer the handle_kernel_image()
> call.
> 
> Cc: Matt Fleming <matt@codeblueprint.co.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  drivers/firmware/efi/libstub/arm-stub.c | 51 ++++++++++++--------
>  drivers/firmware/efi/libstub/efistub.h  |  3 ++
>  drivers/firmware/efi/libstub/fdt.c      | 42 ++++++++++++++++
>  3 files changed, 75 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/firmware/efi/libstub/arm-stub.c b/drivers/firmware/efi/libstub/arm-stub.c
> index 8181ac179d14..f5ef5ccd5f45 100644
> --- a/drivers/firmware/efi/libstub/arm-stub.c
> +++ b/drivers/firmware/efi/libstub/arm-stub.c
> @@ -133,6 +133,7 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>  	unsigned long reserve_size = 0;
>  	enum efi_secureboot_mode secure_boot;
>  	struct screen_info *si;
> +	bool have_chosen_initrd;
>  
>  	/* Check if we were booted by the EFI firmware */
>  	if (sys_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE)
> @@ -183,15 +184,6 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>  
>  	si = setup_graphics(sys_table);
>  
> -	status = handle_kernel_image(sys_table, image_addr, &image_size,
> -				     &reserve_addr,
> -				     &reserve_size,
> -				     dram_base, image);
> -	if (status != EFI_SUCCESS) {
> -		pr_efi_err(sys_table, "Failed to relocate kernel\n");
> -		goto fail_free_cmdline;
> -	}
> -
>  	secure_boot = efi_get_secureboot(sys_table);
>  
>  	/*
> @@ -209,7 +201,7 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>  
>  		if (status != EFI_SUCCESS) {
>  			pr_efi_err(sys_table, "Failed to load device tree!\n");
> -			goto fail_free_image;
> +			goto fail_free_cmdline;
>  		}
>  	}
>  
> @@ -222,16 +214,33 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>  			pr_efi(sys_table, "Using DTB from configuration table\n");
>  	}
>  
> -	if (!fdt_addr)
> +	if (!fdt_addr) {
>  		pr_efi(sys_table, "Generating empty DTB\n");
> +		have_chosen_initrd = false;
> +	} else {
> +		status = efi_reserve_dtb_initrd(sys_table, (void *)fdt_addr);
> +		have_chosen_initrd = (status != EFI_NOT_FOUND);
> +	}
>  
> -	status = handle_cmdline_files(sys_table, image, cmdline_ptr, "initrd=",
> -				      efi_get_max_initrd_addr(dram_base,
> -							      *image_addr),
> -				      (unsigned long *)&initrd_addr,
> -				      (unsigned long *)&initrd_size);
> -	if (status != EFI_SUCCESS)
> -		pr_efi_err(sys_table, "Failed initrd from command line!\n");
> +	status = handle_kernel_image(sys_table, image_addr, &image_size,
> +				     &reserve_addr, &reserve_size,
> +				     dram_base, image);
> +	if (status != EFI_SUCCESS) {
> +		pr_efi_err(sys_table, "Failed to relocate kernel\n");
> +		goto fail_free_fdt;
> +	}
> +
> +	if (!have_chosen_initrd) {
> +		status = handle_cmdline_files(sys_table, image, cmdline_ptr,
> +					      "initrd=",
> +					      efi_get_max_initrd_addr(dram_base,
> +								      *image_addr),
> +					      (unsigned long *)&initrd_addr,
> +					      (unsigned long *)&initrd_size);
> +		if (status != EFI_SUCCESS)
> +			pr_efi_err(sys_table,
> +				   "Failed initrd from command line!\n");
> +	}
>  
>  	efi_random_get_seed(sys_table);
>  
> @@ -272,11 +281,11 @@ unsigned long efi_entry(void *handle, efi_system_table_t *sys_table,
>  	pr_efi_err(sys_table, "Failed to update FDT and exit boot services\n");
>  
>  	efi_free(sys_table, initrd_size, initrd_addr);
> -	efi_free(sys_table, fdt_size, fdt_addr);
> -
> -fail_free_image:
>  	efi_free(sys_table, image_size, *image_addr);
>  	efi_free(sys_table, reserve_size, reserve_addr);
> +
> +fail_free_fdt:
> +	efi_free(sys_table, fdt_size, fdt_addr);
>  fail_free_cmdline:
>  	free_screen_info(sys_table, si);
>  	efi_free(sys_table, cmdline_size, (unsigned long)cmdline_ptr);
> diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h
> index aaf2aeb785ea..35b514d7d962 100644
> --- a/drivers/firmware/efi/libstub/efistub.h
> +++ b/drivers/firmware/efi/libstub/efistub.h
> @@ -68,4 +68,7 @@ efi_status_t check_platform_features(efi_system_table_t *sys_table_arg);
>  
>  efi_status_t efi_random_get_seed(efi_system_table_t *sys_table_arg);
>  
> +efi_status_t efi_reserve_dtb_initrd(efi_system_table_t *sys_table_arg,
> +				    const void *fdt);
> +
>  #endif
> diff --git a/drivers/firmware/efi/libstub/fdt.c b/drivers/firmware/efi/libstub/fdt.c
> index 8830fa601e45..54408c95e094 100644
> --- a/drivers/firmware/efi/libstub/fdt.c
> +++ b/drivers/firmware/efi/libstub/fdt.c
> @@ -385,3 +385,45 @@ void *get_fdt(efi_system_table_t *sys_table, unsigned long *fdt_size)
>  
>  	return fdt;
>  }
> +
> +efi_status_t efi_reserve_dtb_initrd(efi_system_table_t *sys_table_arg,
> +				    const void *fdt)
> +{
> +	int chosen, len;
> +	const void *prop;
> +	efi_physical_addr_t start, end;
> +	unsigned long num_pages;
> +	efi_status_t status;
> +
> +	chosen = fdt_path_offset(fdt, "/chosen");
> +	if (chosen == -FDT_ERR_NOTFOUND)
> +		return EFI_NOT_FOUND;
> +
> +	prop = fdt_getprop(fdt, chosen, "linux,initrd-start", &len);
> +	if (!prop)
> +		return EFI_NOT_FOUND;
> +
> +	start = (len == sizeof(fdt32_t)) ? fdt32_to_cpu(*(fdt32_t *)prop)
> +					 : fdt64_to_cpu(*(fdt64_t *)prop);
> +
> +	prop = fdt_getprop(fdt, chosen, "linux,initrd-end", &len);
> +	if (!prop)
> +		return EFI_NOT_FOUND;
> +
> +	end = (len == sizeof(fdt32_t)) ? fdt32_to_cpu(*(fdt32_t *)prop)
> +				       : fdt64_to_cpu(*(fdt64_t *)prop);
> +
> +	start = round_down(start, EFI_PAGE_SIZE);
> +	num_pages = round_up(end - start, EFI_PAGE_SIZE) / EFI_PAGE_SIZE;
> +
> +	status = efi_call_early(allocate_pages, EFI_ALLOCATE_ADDRESS,
> +				EFI_LOADER_DATA, num_pages, &start);
> +
> +	if (status != EFI_SUCCESS)
> +		pr_efi_err(sys_table_arg,
> +			   "Failed to reserve initrd area found in /chosen\n");
> +	else
> +		pr_efi(sys_table_arg, "Using initrd found in /chosen\n");
> +
> +	return status;
> +}
> -- 
> 2.11.0
> 

[kernel-hardening] Re: [PATCH 29/30] efi/libstub: arm: reserve bootloader supplied initrd in memory map

[Ard Biesheuvel]

On 21 August 2017 at 11:37, Mark Rutland <mark.rutland@arm.com> wrote:
> On Mon, Aug 14, 2017 at 01:54:10PM +0100, Ard Biesheuvel wrote:
>> Under KASLR, the EFI stub may allocate the kernel anywhere in the
>> physical address space, which could be right on top of an initrd
>> if it was supplied by the bootloader (i.e., GRUB) in /chosen rather
>> than passed via the initrd= command line option. So allocate the
>> pages explicitly, this ensures that the random memory allocation
>> routine will disregard the region.
>
> I'm a little confused. Shouldn't the bootloader have allocated that
> memory, leaving it reserved?
>
> If it hasn't, then that region could be allcoated by anything else at
> any time (e.g. by the code which loads the kernel, or some EFI timer
> callback), so that sounds like a bootloader bug that we can't fix.
>

Yeah, thinko on my part. Thanks for spotting that.