* [PATCH v3 0/7] arm64: clean up early boot function calls
@ 2016-08-31 11:05 Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S Ard Biesheuvel
` (6 more replies)
0 siblings, 7 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
In a couple of places, the early boot code uses non-standard argument,
return value or return address registers when calling functions. This makes
the code more complicated than it needs to be, which was not a problem in the
early days, but with all the recent changes for KASLR, hibernate etc, it
makes sense to clean this up once and for all. This code removes all uses of
callee saved registers on the secondary boot and resume paths, and on the
primary boot path, it only leaves the necessary ones, and documents them
explicitly in patch #7.
I will leave it to the honourable arm64 maintainers to decide if any of
these improvements weigh up against the churn, given that this code has
already been updated numerous times over the past couple of kernel versions.
NOTE: this series applies onto today's for-next/core with fixes/core merged
on top, since it depends on Mark's commit fd363bd417ddb610 ("arm64: avoid TLB
conflict with CONFIG_RANDOMIZE_BASE")
Changes since v2:
- dropped patch that gets rid of x25/x26 as pgdir pointers, it has been merged
into for-next/core in the mean time
- fixed commit log of #1 to indicate since when the comment that it fixes had
been incorrect
- merged changes relating to the use of x27 by __enable_mmu() into a single
patch (#4)
- added Mark's R-b to all patches
Patch #1 fixes some style issues in sleep.S with no functional changes.
Patch #2 removes the use of x20 between el2_setup() and set_cpu_boot_mode_flag()
Patch #3 moves the part of the KASLR processing that resides in __enable_mmu()
into primary_switch() (which is a more suitable place, given that only the
primary boot path ever invokes it)
Patch #4 replaces the special x27 return address of __enable_mmu() with x30/lr.
Given that we can no longer dereference literals containing virtual addresses,
all callers have already been updated to return from __enable_mmu() back to the
idmap before performing a literal load + jump. Using x30 instead of x27 allows
us to merge the code that executes before __enable_mmu() with the code that
executes after it, and to change the invocation of __enable_mmu() itself into a
simple bl instruction.
Patch #5 removes the 'global' x24 register in head.S, containing __PHYS_OFFSET
Patch #6 removes the use of x28 in __primary_switched(), and replaces it with
an ordinary stack frame to preserve the return address.
Ard Biesheuvel (7):
arm64: kernel: fix style issues in sleep.S
arm64: kernel: use ordinary return/argument register for el2_setup()
arm64: head.S: move KASLR processing out of __enable_mmu()
arm64: kernel: use x30 for __enable_mmu return address
arm64: kernel: drop use of x24 from primary boot path
arm64: head.S: use ordinary stack frame for __primary_switched()
arm64: head.S: document the use of callee saved registers
arch/arm64/kernel/head.S | 147 +++++++++++---------
arch/arm64/kernel/sleep.S | 27 ++--
2 files changed, 94 insertions(+), 80 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
@ 2016-08-31 11:05 ` Ard Biesheuvel
2016-09-03 20:08 ` Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 2/7] arm64: kernel: use ordinary return/argument register for el2_setup() Ard Biesheuvel
` (5 subsequent siblings)
6 siblings, 1 reply; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
This fixes a number of style issues in sleep.S. No functional changes are
intended:
- replace absolute literal references with relative references in
__cpu_suspend_enter(), which executes from its virtual address
- replace explicit lr assignment plus branch with bl in cpu_resume(), which
aligns it with stext() and secondary_startup()
- don't export _cpu_resume()
- use adr_l for mpidr_hash reference, and fix the incorrect accompanying
comment, which has been out of date since commit cabe1c81ea5be983 ("arm64:
Change cpu_resume() to enable mmu early then access sleep_sp by va")
- replace leading spaces with tabs, and add a bit of whitespace for
readability
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/sleep.S | 21 ++++++++++----------
1 file changed, 10 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
index 1fac020761da..6adc76bf8f91 100644
--- a/arch/arm64/kernel/sleep.S
+++ b/arch/arm64/kernel/sleep.S
@@ -73,10 +73,9 @@ ENTRY(__cpu_suspend_enter)
str x2, [x0, #SLEEP_STACK_DATA_SYSTEM_REGS + CPU_CTX_SP]
/* find the mpidr_hash */
- ldr x1, =sleep_save_stash
- ldr x1, [x1]
+ ldr_l x1, sleep_save_stash
mrs x7, mpidr_el1
- ldr x9, =mpidr_hash
+ adr_l x9, mpidr_hash
ldr x10, [x9, #MPIDR_HASH_MASK]
/*
* Following code relies on the struct mpidr_hash
@@ -95,15 +94,14 @@ ENTRY(__cpu_suspend_enter)
mov x0, #1
ret
ENDPROC(__cpu_suspend_enter)
- .ltorg
.pushsection ".idmap.text", "ax"
ENTRY(cpu_resume)
bl el2_setup // if in EL2 drop to EL1 cleanly
+ bl __cpu_setup
/* enable the MMU early - so we can access sleep_save_stash by va */
- adr_l lr, __enable_mmu /* __cpu_setup will return here */
adr_l x27, _resume_switched /* __enable_mmu will branch here */
- b __cpu_setup
+ b __enable_mmu
ENDPROC(cpu_resume)
_resume_switched:
@@ -113,16 +111,17 @@ ENDPROC(_resume_switched)
.ltorg
.popsection
-ENTRY(_cpu_resume)
+_cpu_resume:
mrs x1, mpidr_el1
- adrp x8, mpidr_hash
- add x8, x8, #:lo12:mpidr_hash // x8 = struct mpidr_hash phys address
- /* retrieve mpidr_hash members to compute the hash */
+ adr_l x8, mpidr_hash // x8 = struct mpidr_hash virt address
+
+ /* retrieve mpidr_hash members to compute the hash */
ldr x2, [x8, #MPIDR_HASH_MASK]
ldp w3, w4, [x8, #MPIDR_HASH_SHIFTS]
ldp w5, w6, [x8, #(MPIDR_HASH_SHIFTS + 8)]
compute_mpidr_hash x7, x3, x4, x5, x6, x1, x2
- /* x7 contains hash index, let's use it to grab context pointer */
+
+ /* x7 contains hash index, let's use it to grab context pointer */
ldr_l x0, sleep_save_stash
ldr x0, [x0, x7, lsl #3]
add x29, x0, #SLEEP_STACK_DATA_CALLEE_REGS
--
2.7.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 2/7] arm64: kernel: use ordinary return/argument register for el2_setup()
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S Ard Biesheuvel
@ 2016-08-31 11:05 ` Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 3/7] arm64: head.S: move KASLR processing out of __enable_mmu() Ard Biesheuvel
` (4 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
The function el2_setup() passes its return value in register w20, and
in the two cases where the caller actually cares about this return value,
it is passed into set_cpu_boot_mode_flag() [almost] directly, which
expects its input in w20 as well.
So there is no reason to use a 'special' callee saved register here, but
we can simply follow the PCS for return value and first argument,
respectively.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/head.S | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 024812bfce34..647aa82f2c7b 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -210,7 +210,7 @@ efi_header_end:
ENTRY(stext)
bl preserve_boot_args
- bl el2_setup // Drop to EL1, w20=cpu_boot_mode
+ bl el2_setup // Drop to EL1, w0=cpu_boot_mode
adrp x24, __PHYS_OFFSET
and x23, x24, MIN_KIMG_ALIGN - 1 // KASLR offset, defaults to 0
bl set_cpu_boot_mode_flag
@@ -488,7 +488,7 @@ CPU_LE( bic x0, x0, #(1 << 25) ) // Clear the EE bit for EL2
CPU_BE( orr x0, x0, #(3 << 24) ) // Set the EE and E0E bits for EL1
CPU_LE( bic x0, x0, #(3 << 24) ) // Clear the EE and E0E bits for EL1
msr sctlr_el1, x0
- mov w20, #BOOT_CPU_MODE_EL1 // This cpu booted in EL1
+ mov w0, #BOOT_CPU_MODE_EL1 // This cpu booted in EL1
isb
ret
@@ -584,7 +584,7 @@ CPU_LE( movk x0, #0x30d0, lsl #16 ) // Clear EE and E0E on LE systems
cbz x2, install_el2_stub
- mov w20, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2
+ mov w0, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2
isb
ret
@@ -599,7 +599,7 @@ install_el2_stub:
PSR_MODE_EL1h)
msr spsr_el2, x0
msr elr_el2, lr
- mov w20, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2
+ mov w0, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2
eret
ENDPROC(el2_setup)
@@ -609,10 +609,10 @@ ENDPROC(el2_setup)
*/
set_cpu_boot_mode_flag:
adr_l x1, __boot_cpu_mode
- cmp w20, #BOOT_CPU_MODE_EL2
+ cmp w0, #BOOT_CPU_MODE_EL2
b.ne 1f
add x1, x1, #4
-1: str w20, [x1] // This CPU has booted in EL1
+1: str w0, [x1] // This CPU has booted in EL1
dmb sy
dc ivac, x1 // Invalidate potentially stale cache line
ret
@@ -649,7 +649,7 @@ ENTRY(__early_cpu_boot_status)
* cores are held until we're ready for them to initialise.
*/
ENTRY(secondary_holding_pen)
- bl el2_setup // Drop to EL1, w20=cpu_boot_mode
+ bl el2_setup // Drop to EL1, w0=cpu_boot_mode
bl set_cpu_boot_mode_flag
mrs x0, mpidr_el1
mov_q x1, MPIDR_HWID_BITMASK
--
2.7.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 3/7] arm64: head.S: move KASLR processing out of __enable_mmu()
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 2/7] arm64: kernel: use ordinary return/argument register for el2_setup() Ard Biesheuvel
@ 2016-08-31 11:05 ` Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 4/7] arm64: kernel: use x30 for __enable_mmu return address Ard Biesheuvel
` (3 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
The KASLR processing is only used by the primary boot path, and
complements the processing that takes place in __primary_switch().
Move the two parts together, to make the code easier to understand.
Also, fix up a minor whitespace issue.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/head.S | 72 ++++++++++++--------
1 file changed, 42 insertions(+), 30 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 647aa82f2c7b..5543068da3ae 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -222,9 +222,7 @@ ENTRY(stext)
* the TCR will have been set.
*/
bl __cpu_setup // initialise processor
- adr_l x27, __primary_switch // address to jump to after
- // MMU has been enabled
- b __enable_mmu
+ b __primary_switch
ENDPROC(stext)
/*
@@ -453,7 +451,7 @@ __primary_switched:
cbz x0, 0f // KASLR disabled? just proceed
orr x23, x23, x0 // record KASLR offset
ret x28 // we must enable KASLR, return
- // to __enable_mmu()
+ // to __primary_switch()
0:
#endif
b start_kernel
@@ -726,7 +724,6 @@ ENDPROC(__secondary_switched)
* If it isn't, park the CPU
*/
ENTRY(__enable_mmu)
- mrs x22, sctlr_el1 // preserve old SCTLR_EL1 value
mrs x1, ID_AA64MMFR0_EL1
ubfx x2, x1, #ID_AA64MMFR0_TGRAN_SHIFT, 4
cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED
@@ -747,28 +744,6 @@ ENTRY(__enable_mmu)
ic iallu
dsb nsh
isb
-#ifdef CONFIG_RANDOMIZE_BASE
- mov x19, x0 // preserve new SCTLR_EL1 value
- blr x27
-
- /*
- * If we return here, we have a KASLR displacement in x23 which we need
- * to take into account by discarding the current kernel mapping and
- * creating a new one.
- */
- msr sctlr_el1, x22 // disable the MMU
- isb
- bl __create_page_tables // recreate kernel mapping
-
- tlbi vmalle1 // Remove any stale TLB entries
- dsb nsh
-
- msr sctlr_el1, x19 // re-enable the MMU
- isb
- ic iallu // flush instructions fetched
- dsb nsh // via old mapping
- isb
-#endif
br x27
ENDPROC(__enable_mmu)
@@ -778,11 +753,11 @@ __no_granule_support:
1:
wfe
wfi
- b 1b
+ b 1b
ENDPROC(__no_granule_support)
-__primary_switch:
#ifdef CONFIG_RELOCATABLE
+__relocate_kernel:
/*
* Iterate over each entry in the relocation table, and apply the
* relocations in place.
@@ -804,8 +779,45 @@ __primary_switch:
add x13, x13, x23 // relocate
str x13, [x11, x23]
b 0b
+1: ret
+ENDPROC(__relocate_kernel)
+#endif
-1:
+__primary_switch:
+#ifdef CONFIG_RANDOMIZE_BASE
+ mov x19, x0 // preserve new SCTLR_EL1 value
+ mrs x20, sctlr_el1 // preserve old SCTLR_EL1 value
+#endif
+
+ adr x27, 0f
+ b __enable_mmu
+0:
+#ifdef CONFIG_RELOCATABLE
+ bl __relocate_kernel
+#ifdef CONFIG_RANDOMIZE_BASE
+ ldr x8, =__primary_switched
+ blr x8
+
+ /*
+ * If we return here, we have a KASLR displacement in x23 which we need
+ * to take into account by discarding the current kernel mapping and
+ * creating a new one.
+ */
+ msr sctlr_el1, x20 // disable the MMU
+ isb
+ bl __create_page_tables // recreate kernel mapping
+
+ tlbi vmalle1 // Remove any stale TLB entries
+ dsb nsh
+
+ msr sctlr_el1, x19 // re-enable the MMU
+ isb
+ ic iallu // flush instructions fetched
+ dsb nsh // via old mapping
+ isb
+
+ bl __relocate_kernel
+#endif
#endif
ldr x8, =__primary_switched
br x8
--
2.7.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 4/7] arm64: kernel: use x30 for __enable_mmu return address
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
` (2 preceding siblings ...)
2016-08-31 11:05 ` [PATCH v3 3/7] arm64: head.S: move KASLR processing out of __enable_mmu() Ard Biesheuvel
@ 2016-08-31 11:05 ` Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 5/7] arm64: kernel: drop use of x24 from primary boot path Ard Biesheuvel
` (2 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
Using x27 for passing to __enable_mmu what is essentially the return
address makes the code look more complicated than it needs to be. So
switch to x30/lr, and update the secondary and cpu_resume call sites to
simply call __enable_mmu as an ordinary function, with a bl instruction.
This requires the callers to be covered by .idmap.text.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/head.S | 21 +++++++-------------
arch/arm64/kernel/sleep.S | 8 ++------
2 files changed, 9 insertions(+), 20 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 5543068da3ae..45b865e022cc 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -675,9 +675,9 @@ secondary_startup:
* Common entry point for secondary CPUs.
*/
bl __cpu_setup // initialise processor
-
- adr_l x27, __secondary_switch // address to jump to after enabling the MMU
- b __enable_mmu
+ bl __enable_mmu
+ ldr x8, =__secondary_switched
+ br x8
ENDPROC(secondary_startup)
__secondary_switched:
@@ -716,9 +716,9 @@ ENDPROC(__secondary_switched)
* Enable the MMU.
*
* x0 = SCTLR_EL1 value for turning on the MMU.
- * x27 = *virtual* address to jump to upon completion
*
- * Other registers depend on the function called upon completion.
+ * Returns to the caller via x30/lr. This requires the caller to be covered
+ * by the .idmap.text section.
*
* Checks if the selected granule size is supported by the CPU.
* If it isn't, park the CPU
@@ -744,7 +744,7 @@ ENTRY(__enable_mmu)
ic iallu
dsb nsh
isb
- br x27
+ ret
ENDPROC(__enable_mmu)
__no_granule_support:
@@ -789,9 +789,7 @@ __primary_switch:
mrs x20, sctlr_el1 // preserve old SCTLR_EL1 value
#endif
- adr x27, 0f
- b __enable_mmu
-0:
+ bl __enable_mmu
#ifdef CONFIG_RELOCATABLE
bl __relocate_kernel
#ifdef CONFIG_RANDOMIZE_BASE
@@ -822,8 +820,3 @@ __primary_switch:
ldr x8, =__primary_switched
br x8
ENDPROC(__primary_switch)
-
-__secondary_switch:
- ldr x8, =__secondary_switched
- br x8
-ENDPROC(__secondary_switch)
diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
index 6adc76bf8f91..0f7e0b2ac64c 100644
--- a/arch/arm64/kernel/sleep.S
+++ b/arch/arm64/kernel/sleep.S
@@ -100,14 +100,10 @@ ENTRY(cpu_resume)
bl el2_setup // if in EL2 drop to EL1 cleanly
bl __cpu_setup
/* enable the MMU early - so we can access sleep_save_stash by va */
- adr_l x27, _resume_switched /* __enable_mmu will branch here */
- b __enable_mmu
-ENDPROC(cpu_resume)
-
-_resume_switched:
+ bl __enable_mmu
ldr x8, =_cpu_resume
br x8
-ENDPROC(_resume_switched)
+ENDPROC(cpu_resume)
.ltorg
.popsection
--
2.7.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 5/7] arm64: kernel: drop use of x24 from primary boot path
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
` (3 preceding siblings ...)
2016-08-31 11:05 ` [PATCH v3 4/7] arm64: kernel: use x30 for __enable_mmu return address Ard Biesheuvel
@ 2016-08-31 11:05 ` Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 6/7] arm64: head.S: use ordinary stack frame for __primary_switched() Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 7/7] arm64: head.S: document the use of callee saved registers Ard Biesheuvel
6 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
Keeping __PHYS_OFFSET in x24 is actually less clear than simply taking
the value of __PHYS_OFFSET using an adrp instruction in the three places
that we need it. So change that.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/head.S | 20 ++++++++++++--------
1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 45b865e022cc..4dee51045e79 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -211,8 +211,8 @@ efi_header_end:
ENTRY(stext)
bl preserve_boot_args
bl el2_setup // Drop to EL1, w0=cpu_boot_mode
- adrp x24, __PHYS_OFFSET
- and x23, x24, MIN_KIMG_ALIGN - 1 // KASLR offset, defaults to 0
+ adrp x23, __PHYS_OFFSET
+ and x23, x23, MIN_KIMG_ALIGN - 1 // KASLR offset, defaults to 0
bl set_cpu_boot_mode_flag
bl __create_page_tables
/*
@@ -412,6 +412,8 @@ ENDPROC(__create_page_tables)
/*
* The following fragment of code is executed with the MMU enabled.
+ *
+ * x0 = __PHYS_OFFSET
*/
.set initial_sp, init_thread_union + THREAD_START_SP
__primary_switched:
@@ -420,6 +422,12 @@ __primary_switched:
msr vbar_el1, x8 // vector table address
isb
+ str_l x21, __fdt_pointer, x5 // Save FDT pointer
+
+ ldr_l x4, kimage_vaddr // Save the offset between
+ sub x4, x4, x0 // the kernel virtual and
+ str_l x4, kimage_voffset, x5 // physical mappings
+
// Clear BSS
adr_l x0, __bss_start
mov x1, xzr
@@ -432,12 +440,6 @@ __primary_switched:
mov x4, sp
and x4, x4, #~(THREAD_SIZE - 1)
msr sp_el0, x4 // Save thread_info
- str_l x21, __fdt_pointer, x5 // Save FDT pointer
-
- ldr_l x4, kimage_vaddr // Save the offset between
- sub x4, x4, x24 // the kernel virtual and
- str_l x4, kimage_voffset, x5 // physical mappings
-
mov x29, #0
#ifdef CONFIG_KASAN
bl kasan_early_init
@@ -794,6 +796,7 @@ __primary_switch:
bl __relocate_kernel
#ifdef CONFIG_RANDOMIZE_BASE
ldr x8, =__primary_switched
+ adrp x0, __PHYS_OFFSET
blr x8
/*
@@ -818,5 +821,6 @@ __primary_switch:
#endif
#endif
ldr x8, =__primary_switched
+ adrp x0, __PHYS_OFFSET
br x8
ENDPROC(__primary_switch)
--
2.7.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 6/7] arm64: head.S: use ordinary stack frame for __primary_switched()
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
` (4 preceding siblings ...)
2016-08-31 11:05 ` [PATCH v3 5/7] arm64: kernel: drop use of x24 from primary boot path Ard Biesheuvel
@ 2016-08-31 11:05 ` Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 7/7] arm64: head.S: document the use of callee saved registers Ard Biesheuvel
6 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
Instead of stashing the value of the link register in x28 before setting
up the stack and calling into C code, create an ordinary PCS compatible
stack frame so that we can push the return address onto the stack.
Since exception handlers require a stack as well, assign the stack pointer
register before installing the vector table.
Note that this accounts for the difference between THREAD_START_SP and
THREAD_SIZE, given that the stack pointer is always decremented before
calling into any C code.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/head.S | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 4dee51045e79..29a734ee0770 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -415,13 +415,18 @@ ENDPROC(__create_page_tables)
*
* x0 = __PHYS_OFFSET
*/
- .set initial_sp, init_thread_union + THREAD_START_SP
__primary_switched:
- mov x28, lr // preserve LR
+ adrp x4, init_thread_union
+ add sp, x4, #THREAD_SIZE
+ msr sp_el0, x4 // Save thread_info
+
adr_l x8, vectors // load VBAR_EL1 with virtual
msr vbar_el1, x8 // vector table address
isb
+ stp xzr, x30, [sp, #-16]!
+ mov x29, sp
+
str_l x21, __fdt_pointer, x5 // Save FDT pointer
ldr_l x4, kimage_vaddr // Save the offset between
@@ -436,11 +441,6 @@ __primary_switched:
bl __pi_memset
dsb ishst // Make zero page visible to PTW
- adr_l sp, initial_sp, x4
- mov x4, sp
- and x4, x4, #~(THREAD_SIZE - 1)
- msr sp_el0, x4 // Save thread_info
- mov x29, #0
#ifdef CONFIG_KASAN
bl kasan_early_init
#endif
@@ -452,8 +452,8 @@ __primary_switched:
bl kaslr_early_init // parse FDT for KASLR options
cbz x0, 0f // KASLR disabled? just proceed
orr x23, x23, x0 // record KASLR offset
- ret x28 // we must enable KASLR, return
- // to __primary_switch()
+ ldp x29, x30, [sp], #16 // we must enable KASLR, return
+ ret // to __primary_switch()
0:
#endif
b start_kernel
--
2.7.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 7/7] arm64: head.S: document the use of callee saved registers
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
` (5 preceding siblings ...)
2016-08-31 11:05 ` [PATCH v3 6/7] arm64: head.S: use ordinary stack frame for __primary_switched() Ard Biesheuvel
@ 2016-08-31 11:05 ` Ard Biesheuvel
6 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-08-31 11:05 UTC (permalink / raw)
To: linux-arm-kernel
Now that the only remaining occurrences of the use of callee saved
registers are on the primary boot path, add a comment to the code
which register is used for what.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/head.S | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 29a734ee0770..427f6d3f084c 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -208,6 +208,16 @@ efi_header_end:
__INIT
+ /*
+ * The following callee saved general purpose registers are used on the
+ * primary lowlevel boot path:
+ *
+ * Register Scope Purpose
+ * x21 stext() .. start_kernel() FDT pointer passed at boot in x0
+ * x23 stext() .. start_kernel() physical misalignment/KASLR offset
+ * x28 __create_page_tables() callee preserved temp register
+ * x19/x20 __primary_switch() callee preserved temp registers
+ */
ENTRY(stext)
bl preserve_boot_args
bl el2_setup // Drop to EL1, w0=cpu_boot_mode
--
2.7.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S
2016-08-31 11:05 ` [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S Ard Biesheuvel
@ 2016-09-03 20:08 ` Ard Biesheuvel
2016-09-05 9:02 ` Will Deacon
0 siblings, 1 reply; 10+ messages in thread
From: Ard Biesheuvel @ 2016-09-03 20:08 UTC (permalink / raw)
To: linux-arm-kernel
On 31 August 2016 at 12:05, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> This fixes a number of style issues in sleep.S. No functional changes are
> intended:
> - replace absolute literal references with relative references in
> __cpu_suspend_enter(), which executes from its virtual address
> - replace explicit lr assignment plus branch with bl in cpu_resume(), which
> aligns it with stext() and secondary_startup()
> - don't export _cpu_resume()
Will,
Apologies, I sent out the wrong version of this patch in v3. Not
exporting _cpu_resume() breaks hibernate, so I dropped it from v2
(which is the version that Mark reviewed), but I reintroduced it by
accident.
Please let me know how you would like to proceed, i.e., put a fixup on
top, or resend.
Thanks,
Ard.
> - use adr_l for mpidr_hash reference, and fix the incorrect accompanying
> comment, which has been out of date since commit cabe1c81ea5be983 ("arm64:
> Change cpu_resume() to enable mmu early then access sleep_sp by va")
> - replace leading spaces with tabs, and add a bit of whitespace for
> readability
>
> Reviewed-by: Mark Rutland <mark.rutland@arm.com>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm64/kernel/sleep.S | 21 ++++++++++----------
> 1 file changed, 10 insertions(+), 11 deletions(-)
>
> diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
> index 1fac020761da..6adc76bf8f91 100644
> --- a/arch/arm64/kernel/sleep.S
> +++ b/arch/arm64/kernel/sleep.S
> @@ -73,10 +73,9 @@ ENTRY(__cpu_suspend_enter)
> str x2, [x0, #SLEEP_STACK_DATA_SYSTEM_REGS + CPU_CTX_SP]
>
> /* find the mpidr_hash */
> - ldr x1, =sleep_save_stash
> - ldr x1, [x1]
> + ldr_l x1, sleep_save_stash
> mrs x7, mpidr_el1
> - ldr x9, =mpidr_hash
> + adr_l x9, mpidr_hash
> ldr x10, [x9, #MPIDR_HASH_MASK]
> /*
> * Following code relies on the struct mpidr_hash
> @@ -95,15 +94,14 @@ ENTRY(__cpu_suspend_enter)
> mov x0, #1
> ret
> ENDPROC(__cpu_suspend_enter)
> - .ltorg
>
> .pushsection ".idmap.text", "ax"
> ENTRY(cpu_resume)
> bl el2_setup // if in EL2 drop to EL1 cleanly
> + bl __cpu_setup
> /* enable the MMU early - so we can access sleep_save_stash by va */
> - adr_l lr, __enable_mmu /* __cpu_setup will return here */
> adr_l x27, _resume_switched /* __enable_mmu will branch here */
> - b __cpu_setup
> + b __enable_mmu
> ENDPROC(cpu_resume)
>
> _resume_switched:
> @@ -113,16 +111,17 @@ ENDPROC(_resume_switched)
> .ltorg
> .popsection
>
> -ENTRY(_cpu_resume)
> +_cpu_resume:
> mrs x1, mpidr_el1
> - adrp x8, mpidr_hash
> - add x8, x8, #:lo12:mpidr_hash // x8 = struct mpidr_hash phys address
> - /* retrieve mpidr_hash members to compute the hash */
> + adr_l x8, mpidr_hash // x8 = struct mpidr_hash virt address
> +
> + /* retrieve mpidr_hash members to compute the hash */
> ldr x2, [x8, #MPIDR_HASH_MASK]
> ldp w3, w4, [x8, #MPIDR_HASH_SHIFTS]
> ldp w5, w6, [x8, #(MPIDR_HASH_SHIFTS + 8)]
> compute_mpidr_hash x7, x3, x4, x5, x6, x1, x2
> - /* x7 contains hash index, let's use it to grab context pointer */
> +
> + /* x7 contains hash index, let's use it to grab context pointer */
> ldr_l x0, sleep_save_stash
> ldr x0, [x0, x7, lsl #3]
> add x29, x0, #SLEEP_STACK_DATA_CALLEE_REGS
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S
2016-09-03 20:08 ` Ard Biesheuvel
@ 2016-09-05 9:02 ` Will Deacon
0 siblings, 0 replies; 10+ messages in thread
From: Will Deacon @ 2016-09-05 9:02 UTC (permalink / raw)
To: linux-arm-kernel
Hi Ard,
On Sat, Sep 03, 2016 at 09:08:13PM +0100, Ard Biesheuvel wrote:
> On 31 August 2016 at 12:05, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > This fixes a number of style issues in sleep.S. No functional changes are
> > intended:
> > - replace absolute literal references with relative references in
> > __cpu_suspend_enter(), which executes from its virtual address
> > - replace explicit lr assignment plus branch with bl in cpu_resume(), which
> > aligns it with stext() and secondary_startup()
> > - don't export _cpu_resume()
>
> Apologies, I sent out the wrong version of this patch in v3. Not
> exporting _cpu_resume() breaks hibernate, so I dropped it from v2
> (which is the version that Mark reviewed), but I reintroduced it by
> accident.
No problem, these things happen.
> Please let me know how you would like to proceed, i.e., put a fixup on
> top, or resend.
I already pushed this out on for-next/core, so please send a fixup on top.
I could've sworn I did an allmodconfig build test before pushing out, but
apparently not.
Will
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2016-09-05 9:02 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-31 11:05 [PATCH v3 0/7] arm64: clean up early boot function calls Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 1/7] arm64: kernel: fix style issues in sleep.S Ard Biesheuvel
2016-09-03 20:08 ` Ard Biesheuvel
2016-09-05 9:02 ` Will Deacon
2016-08-31 11:05 ` [PATCH v3 2/7] arm64: kernel: use ordinary return/argument register for el2_setup() Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 3/7] arm64: head.S: move KASLR processing out of __enable_mmu() Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 4/7] arm64: kernel: use x30 for __enable_mmu return address Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 5/7] arm64: kernel: drop use of x24 from primary boot path Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 6/7] arm64: head.S: use ordinary stack frame for __primary_switched() Ard Biesheuvel
2016-08-31 11:05 ` [PATCH v3 7/7] arm64: head.S: document the use of callee saved registers Ard Biesheuvel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.