* [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch
@ 2015-10-28 0:50 Anton Blanchard
2015-10-28 0:50 ` [PATCH 02/19] powerpc: Don't disable MSR bits in do_load_up_transact_*() functions Anton Blanchard
` (17 more replies)
0 siblings, 18 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Writing the MSR is slow, so we want to avoid it whenever possible.
A subsequent patch will add a debug option that strictly manages the
FP/VMX/VSX unavailable bits. For now just remove it, matching what
we do in other areas of the kernel (eg enable_kernel_altivec()).
A context switch microbenchmark using yield():
http://ozlabs.org/~anton/junkcode/context_switch2.c
./context_switch2 --test=yield --fp 0 0
shows an improvement of almost 3% on POWER8.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/kernel/entry_64.S | 15 +--------------
1 file changed, 1 insertion(+), 14 deletions(-)
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index a94f155..93bb284 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -453,26 +453,13 @@ _GLOBAL(_switch)
SAVE_8GPRS(14, r1)
SAVE_10GPRS(22, r1)
mflr r20 /* Return to switch caller */
- mfmsr r22
- li r0, MSR_FP
-#ifdef CONFIG_VSX
-BEGIN_FTR_SECTION
- oris r0,r0,MSR_VSX@h /* Disable VSX */
-END_FTR_SECTION_IFSET(CPU_FTR_VSX)
-#endif /* CONFIG_VSX */
#ifdef CONFIG_ALTIVEC
BEGIN_FTR_SECTION
- oris r0,r0,MSR_VEC@h /* Disable altivec */
mfspr r24,SPRN_VRSAVE /* save vrsave register value */
std r24,THREAD_VRSAVE(r3)
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
#endif /* CONFIG_ALTIVEC */
- and. r0,r0,r22
- beq+ 1f
- andc r22,r22,r0
- MTMSRD(r22)
- isync
-1: std r20,_NIP(r1)
+ std r20,_NIP(r1)
mfcr r23
std r23,_CCR(r1)
std r1,KSP(r3) /* Set old stack pointer */
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 02/19] powerpc: Don't disable MSR bits in do_load_up_transact_*() functions
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 03/19] powerpc: Create context switch helpers save_sprs() and restore_sprs() Anton Blanchard
` (16 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Similar to the non TM load_up_*() functions, don't disable the MSR
bits on the way out.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/kernel/fpu.S | 4 ----
arch/powerpc/kernel/vector.S | 4 ----
2 files changed, 8 deletions(-)
diff --git a/arch/powerpc/kernel/fpu.S b/arch/powerpc/kernel/fpu.S
index 9ad236e..38eb79b 100644
--- a/arch/powerpc/kernel/fpu.S
+++ b/arch/powerpc/kernel/fpu.S
@@ -73,10 +73,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
MTFSF_L(fr0)
REST_32FPVSRS(0, R4, R7)
- /* FP/VSX off again */
- MTMSRD(r6)
- SYNC
-
blr
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
diff --git a/arch/powerpc/kernel/vector.S b/arch/powerpc/kernel/vector.S
index f5c80d5..1c54259 100644
--- a/arch/powerpc/kernel/vector.S
+++ b/arch/powerpc/kernel/vector.S
@@ -29,10 +29,6 @@ _GLOBAL(do_load_up_transact_altivec)
addi r10,r3,THREAD_TRANSACT_VRSTATE
REST_32VRS(0,r4,r10)
- /* Disable VEC again. */
- MTMSRD(r6)
- isync
-
blr
#endif
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 03/19] powerpc: Create context switch helpers save_sprs() and restore_sprs()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
2015-10-28 0:50 ` [PATCH 02/19] powerpc: Don't disable MSR bits in do_load_up_transact_*() functions Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 04/19] powerpc: Remove redundant mflr in _switch Anton Blanchard
` (15 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Move all our context switch SPR save and restore code into two
helpers. We do a few optimisations:
- Group all mfsprs and all mtsprs. In many cases an mtspr sets a
scoreboarding bit that an mfspr waits on, so the current practise of
mfspr A; mtspr A; mfpsr B; mtspr B is the worst scheduling we can
do.
- SPR writes are slow, so check that the value is changing before
writing it.
A context switch microbenchmark using yield():
http://ozlabs.org/~anton/junkcode/context_switch2.c
./context_switch2 --test=yield 0 0
shows an improvement of almost 10% on POWER8.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/processor.h | 1 +
arch/powerpc/include/asm/switch_to.h | 11 -----
arch/powerpc/kernel/entry_64.S | 60 +----------------------
arch/powerpc/kernel/process.c | 92 +++++++++++++++++++++++++++++++-----
4 files changed, 82 insertions(+), 82 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
index 5afea36..c273f3e 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -294,6 +294,7 @@ struct thread_struct {
#endif
#ifdef CONFIG_PPC64
unsigned long dscr;
+ unsigned long fscr;
/*
* This member element dscr_inherit indicates that the process
* has explicitly attempted and changed the DSCR register value
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 15cca17..33a071d 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -15,17 +15,6 @@ extern struct task_struct *__switch_to(struct task_struct *,
struct thread_struct;
extern struct task_struct *_switch(struct thread_struct *prev,
struct thread_struct *next);
-#ifdef CONFIG_PPC_BOOK3S_64
-static inline void save_early_sprs(struct thread_struct *prev)
-{
- if (cpu_has_feature(CPU_FTR_ARCH_207S))
- prev->tar = mfspr(SPRN_TAR);
- if (cpu_has_feature(CPU_FTR_DSCR))
- prev->dscr = mfspr(SPRN_DSCR);
-}
-#else
-static inline void save_early_sprs(struct thread_struct *prev) {}
-#endif
extern void enable_kernel_fp(void);
extern void enable_kernel_altivec(void);
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 93bb284..e84e5bc 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -453,29 +453,12 @@ _GLOBAL(_switch)
SAVE_8GPRS(14, r1)
SAVE_10GPRS(22, r1)
mflr r20 /* Return to switch caller */
-#ifdef CONFIG_ALTIVEC
-BEGIN_FTR_SECTION
- mfspr r24,SPRN_VRSAVE /* save vrsave register value */
- std r24,THREAD_VRSAVE(r3)
-END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
-#endif /* CONFIG_ALTIVEC */
+
std r20,_NIP(r1)
mfcr r23
std r23,_CCR(r1)
std r1,KSP(r3) /* Set old stack pointer */
-#ifdef CONFIG_PPC_BOOK3S_64
-BEGIN_FTR_SECTION
- /* Event based branch registers */
- mfspr r0, SPRN_BESCR
- std r0, THREAD_BESCR(r3)
- mfspr r0, SPRN_EBBHR
- std r0, THREAD_EBBHR(r3)
- mfspr r0, SPRN_EBBRR
- std r0, THREAD_EBBRR(r3)
-END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
-#endif
-
#ifdef CONFIG_SMP
/* We need a sync somewhere here to make sure that if the
* previous task gets rescheduled on another CPU, it sees all
@@ -563,47 +546,6 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_1T_SEGMENT)
mr r1,r8 /* start using new stack pointer */
std r7,PACAKSAVE(r13)
-#ifdef CONFIG_PPC_BOOK3S_64
-BEGIN_FTR_SECTION
- /* Event based branch registers */
- ld r0, THREAD_BESCR(r4)
- mtspr SPRN_BESCR, r0
- ld r0, THREAD_EBBHR(r4)
- mtspr SPRN_EBBHR, r0
- ld r0, THREAD_EBBRR(r4)
- mtspr SPRN_EBBRR, r0
-
- ld r0,THREAD_TAR(r4)
- mtspr SPRN_TAR,r0
-END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
-#endif
-
-#ifdef CONFIG_ALTIVEC
-BEGIN_FTR_SECTION
- ld r0,THREAD_VRSAVE(r4)
- mtspr SPRN_VRSAVE,r0 /* if G4, restore VRSAVE reg */
-END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
-#endif /* CONFIG_ALTIVEC */
-#ifdef CONFIG_PPC64
-BEGIN_FTR_SECTION
- lwz r6,THREAD_DSCR_INHERIT(r4)
- ld r0,THREAD_DSCR(r4)
- cmpwi r6,0
- bne 1f
- ld r0,PACA_DSCR_DEFAULT(r13)
-1:
-BEGIN_FTR_SECTION_NESTED(70)
- mfspr r8, SPRN_FSCR
- rldimi r8, r6, FSCR_DSCR_LG, (63 - FSCR_DSCR_LG)
- mtspr SPRN_FSCR, r8
-END_FTR_SECTION_NESTED(CPU_FTR_ARCH_207S, CPU_FTR_ARCH_207S, 70)
- cmpd r0,r25
- beq 2f
- mtspr SPRN_DSCR,r0
-2:
-END_FTR_SECTION_IFSET(CPU_FTR_DSCR)
-#endif
-
ld r6,_CCR(r1)
mtcrf 0xFF,r6
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 75b6676..3aabed4 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -742,6 +742,73 @@ void restore_tm_state(struct pt_regs *regs)
#define __switch_to_tm(prev)
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+static inline void save_sprs(struct thread_struct *t)
+{
+#ifdef CONFIG_ALTIVEC
+ if (cpu_has_feature(cpu_has_feature(CPU_FTR_ALTIVEC)))
+ t->vrsave = mfspr(SPRN_VRSAVE);
+#endif
+#ifdef CONFIG_PPC_BOOK3S_64
+ if (cpu_has_feature(CPU_FTR_DSCR))
+ t->dscr = mfspr(SPRN_DSCR);
+
+ if (cpu_has_feature(CPU_FTR_ARCH_207S)) {
+ t->bescr = mfspr(SPRN_BESCR);
+ t->ebbhr = mfspr(SPRN_EBBHR);
+ t->ebbrr = mfspr(SPRN_EBBRR);
+
+ t->fscr = mfspr(SPRN_FSCR);
+
+ /*
+ * Note that the TAR is not available for use in the kernel.
+ * (To provide this, the TAR should be backed up/restored on
+ * exception entry/exit instead, and be in pt_regs. FIXME,
+ * this should be in pt_regs anyway (for debug).)
+ */
+ t->tar = mfspr(SPRN_TAR);
+ }
+#endif
+}
+
+static inline void restore_sprs(struct thread_struct *old_thread,
+ struct thread_struct *new_thread)
+{
+#ifdef CONFIG_ALTIVEC
+ if (cpu_has_feature(CPU_FTR_ALTIVEC) &&
+ old_thread->vrsave != new_thread->vrsave)
+ mtspr(SPRN_VRSAVE, new_thread->vrsave);
+#endif
+#ifdef CONFIG_PPC_BOOK3S_64
+ if (cpu_has_feature(CPU_FTR_DSCR)) {
+ u64 dscr = get_paca()->dscr_default;
+ u64 fscr = old_thread->fscr & ~FSCR_DSCR;
+
+ if (new_thread->dscr_inherit) {
+ dscr = new_thread->dscr;
+ fscr |= FSCR_DSCR;
+ }
+
+ if (old_thread->dscr != dscr)
+ mtspr(SPRN_DSCR, dscr);
+
+ if (old_thread->fscr != fscr)
+ mtspr(SPRN_FSCR, fscr);
+ }
+
+ if (cpu_has_feature(CPU_FTR_ARCH_207S)) {
+ if (old_thread->bescr != new_thread->bescr)
+ mtspr(SPRN_BESCR, new_thread->bescr);
+ if (old_thread->ebbhr != new_thread->ebbhr)
+ mtspr(SPRN_EBBHR, new_thread->ebbhr);
+ if (old_thread->ebbrr != new_thread->ebbrr)
+ mtspr(SPRN_EBBRR, new_thread->ebbrr);
+
+ if (old_thread->tar != new_thread->tar)
+ mtspr(SPRN_TAR, new_thread->tar);
+ }
+#endif
+}
+
struct task_struct *__switch_to(struct task_struct *prev,
struct task_struct *new)
{
@@ -751,17 +818,16 @@ struct task_struct *__switch_to(struct task_struct *prev,
struct ppc64_tlb_batch *batch;
#endif
+ new_thread = &new->thread;
+ old_thread = ¤t->thread;
+
WARN_ON(!irqs_disabled());
- /* Back up the TAR and DSCR across context switches.
- * Note that the TAR is not available for use in the kernel. (To
- * provide this, the TAR should be backed up/restored on exception
- * entry/exit instead, and be in pt_regs. FIXME, this should be in
- * pt_regs anyway (for debug).)
- * Save the TAR and DSCR here before we do treclaim/trecheckpoint as
- * these will change them.
+ /*
+ * We need to save SPRs before treclaim/trecheckpoint as these will
+ * change a number of them.
*/
- save_early_sprs(&prev->thread);
+ save_sprs(&prev->thread);
__switch_to_tm(prev);
@@ -844,10 +910,6 @@ struct task_struct *__switch_to(struct task_struct *prev,
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
#endif
-
- new_thread = &new->thread;
- old_thread = ¤t->thread;
-
#ifdef CONFIG_PPC64
/*
* Collect processor utilization data per process
@@ -883,6 +945,10 @@ struct task_struct *__switch_to(struct task_struct *prev,
last = _switch(old_thread, new_thread);
+ /* Need to recalculate these after calling _switch() */
+ old_thread = &last->thread;
+ new_thread = ¤t->thread;
+
#ifdef CONFIG_PPC_BOOK3S_64
if (current_thread_info()->local_flags & _TLF_LAZY_MMU) {
current_thread_info()->local_flags &= ~_TLF_LAZY_MMU;
@@ -891,6 +957,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
}
#endif /* CONFIG_PPC_BOOK3S_64 */
+ restore_sprs(old_thread, new_thread);
+
return last;
}
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 04/19] powerpc: Remove redundant mflr in _switch
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
2015-10-28 0:50 ` [PATCH 02/19] powerpc: Don't disable MSR bits in do_load_up_transact_*() functions Anton Blanchard
2015-10-28 0:50 ` [PATCH 03/19] powerpc: Create context switch helpers save_sprs() and restore_sprs() Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 05/19] powerpc: Remove UP only lazy floating point and vector optimisations Anton Blanchard
` (14 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
No need to execute mflr twice.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/kernel/entry_64.S | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index e84e5bc..c8b4225 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -452,9 +452,7 @@ _GLOBAL(_switch)
/* r3-r13 are caller saved -- Cort */
SAVE_8GPRS(14, r1)
SAVE_10GPRS(22, r1)
- mflr r20 /* Return to switch caller */
-
- std r20,_NIP(r1)
+ std r0,_NIP(r1) /* Return to switch caller */
mfcr r23
std r23,_CCR(r1)
std r1,KSP(r3) /* Set old stack pointer */
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 05/19] powerpc: Remove UP only lazy floating point and vector optimisations
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (2 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 04/19] powerpc: Remove redundant mflr in _switch Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 06/19] powerpc: Simplify TM restore checks Anton Blanchard
` (13 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
The UP only lazy floating point and vector optimisations were written
back when SMP was not common, and neither glibc nor gcc used vector
instructions. Now SMP is very common, glibc aggressively uses vector
instructions and gcc autovectorises.
We want to add new optimisations that apply to both UP and SMP, but
in preparation for that remove these UP only optimisations.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/processor.h | 6 --
arch/powerpc/include/asm/switch_to.h | 8 ---
arch/powerpc/kernel/fpu.S | 35 -----------
arch/powerpc/kernel/head_fsl_booke.S | 32 ----------
arch/powerpc/kernel/idle_power7.S | 7 ---
arch/powerpc/kernel/process.c | 113 +----------------------------------
arch/powerpc/kernel/signal_32.c | 18 ------
arch/powerpc/kernel/signal_64.c | 18 ------
arch/powerpc/kernel/vector.S | 68 ---------------------
9 files changed, 1 insertion(+), 304 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
index c273f3e..a2e8918 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -88,12 +88,6 @@ struct task_struct;
void start_thread(struct pt_regs *regs, unsigned long fdptr, unsigned long sp);
void release_thread(struct task_struct *);
-/* Lazy FPU handling on uni-processor */
-extern struct task_struct *last_task_used_math;
-extern struct task_struct *last_task_used_altivec;
-extern struct task_struct *last_task_used_vsx;
-extern struct task_struct *last_task_used_spe;
-
#ifdef CONFIG_PPC32
#if CONFIG_TASK_SIZE > CONFIG_KERNEL_START
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 33a071d..bd1d933 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -27,14 +27,6 @@ extern void giveup_spe(struct task_struct *);
extern void load_up_spe(struct task_struct *);
extern void switch_booke_debug_regs(struct debug_reg *new_debug);
-#ifndef CONFIG_SMP
-extern void discard_lazy_cpu_state(void);
-#else
-static inline void discard_lazy_cpu_state(void)
-{
-}
-#endif
-
#ifdef CONFIG_PPC_FPU
extern void flush_fp_to_thread(struct task_struct *);
extern void giveup_fpu(struct task_struct *);
diff --git a/arch/powerpc/kernel/fpu.S b/arch/powerpc/kernel/fpu.S
index 38eb79b..50d2352 100644
--- a/arch/powerpc/kernel/fpu.S
+++ b/arch/powerpc/kernel/fpu.S
@@ -132,31 +132,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
SYNC
MTMSRD(r5) /* enable use of fpu now */
isync
-/*
- * For SMP, we don't do lazy FPU switching because it just gets too
- * horrendously complex, especially when a task switches from one CPU
- * to another. Instead we call giveup_fpu in switch_to.
- */
-#ifndef CONFIG_SMP
- LOAD_REG_ADDRBASE(r3, last_task_used_math)
- toreal(r3)
- PPC_LL r4,ADDROFF(last_task_used_math)(r3)
- PPC_LCMPI 0,r4,0
- beq 1f
- toreal(r4)
- addi r4,r4,THREAD /* want last_task_used_math->thread */
- addi r10,r4,THREAD_FPSTATE
- SAVE_32FPVSRS(0, R5, R10)
- mffs fr0
- stfd fr0,FPSTATE_FPSCR(r10)
- PPC_LL r5,PT_REGS(r4)
- toreal(r5)
- PPC_LL r4,_MSR-STACK_FRAME_OVERHEAD(r5)
- li r10,MSR_FP|MSR_FE0|MSR_FE1
- andc r4,r4,r10 /* disable FP for previous task */
- PPC_STL r4,_MSR-STACK_FRAME_OVERHEAD(r5)
-1:
-#endif /* CONFIG_SMP */
/* enable use of FP after return */
#ifdef CONFIG_PPC32
mfspr r5,SPRN_SPRG_THREAD /* current task's THREAD (phys) */
@@ -175,11 +150,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
lfd fr0,FPSTATE_FPSCR(r10)
MTFSF_L(fr0)
REST_32FPVSRS(0, R4, R10)
-#ifndef CONFIG_SMP
- subi r4,r5,THREAD
- fromreal(r4)
- PPC_STL r4,ADDROFF(last_task_used_math)(r3)
-#endif /* CONFIG_SMP */
/* restore registers and return */
/* we haven't used ctr or xer or lr */
blr
@@ -226,11 +196,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
andc r4,r4,r3 /* disable FP for previous task */
PPC_STL r4,_MSR-STACK_FRAME_OVERHEAD(r5)
1:
-#ifndef CONFIG_SMP
- li r5,0
- LOAD_REG_ADDRBASE(r4,last_task_used_math)
- PPC_STL r5,ADDROFF(last_task_used_math)(r4)
-#endif /* CONFIG_SMP */
blr
/*
diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S
index fffd1f9..ec936ab 100644
--- a/arch/powerpc/kernel/head_fsl_booke.S
+++ b/arch/powerpc/kernel/head_fsl_booke.S
@@ -857,29 +857,6 @@ _GLOBAL(load_up_spe)
oris r5,r5,MSR_SPE@h
mtmsr r5 /* enable use of SPE now */
isync
-/*
- * For SMP, we don't do lazy SPE switching because it just gets too
- * horrendously complex, especially when a task switches from one CPU
- * to another. Instead we call giveup_spe in switch_to.
- */
-#ifndef CONFIG_SMP
- lis r3,last_task_used_spe@ha
- lwz r4,last_task_used_spe@l(r3)
- cmpi 0,r4,0
- beq 1f
- addi r4,r4,THREAD /* want THREAD of last_task_used_spe */
- SAVE_32EVRS(0,r10,r4,THREAD_EVR0)
- evxor evr10, evr10, evr10 /* clear out evr10 */
- evmwumiaa evr10, evr10, evr10 /* evr10 <- ACC = 0 * 0 + ACC */
- li r5,THREAD_ACC
- evstddx evr10, r4, r5 /* save off accumulator */
- lwz r5,PT_REGS(r4)
- lwz r4,_MSR-STACK_FRAME_OVERHEAD(r5)
- lis r10,MSR_SPE@h
- andc r4,r4,r10 /* disable SPE for previous task */
- stw r4,_MSR-STACK_FRAME_OVERHEAD(r5)
-1:
-#endif /* !CONFIG_SMP */
/* enable use of SPE after return */
oris r9,r9,MSR_SPE@h
mfspr r5,SPRN_SPRG_THREAD /* current task's THREAD (phys) */
@@ -889,10 +866,6 @@ _GLOBAL(load_up_spe)
evlddx evr4,r10,r5
evmra evr4,evr4
REST_32EVRS(0,r10,r5,THREAD_EVR0)
-#ifndef CONFIG_SMP
- subi r4,r5,THREAD
- stw r4,last_task_used_spe@l(r3)
-#endif /* !CONFIG_SMP */
blr
/*
@@ -1035,11 +1008,6 @@ _GLOBAL(giveup_spe)
andc r4,r4,r3 /* disable SPE for previous task */
stw r4,_MSR-STACK_FRAME_OVERHEAD(r5)
1:
-#ifndef CONFIG_SMP
- li r5,0
- lis r4,last_task_used_spe@ha
- stw r5,last_task_used_spe@l(r4)
-#endif /* !CONFIG_SMP */
blr
#endif /* CONFIG_SPE */
diff --git a/arch/powerpc/kernel/idle_power7.S b/arch/powerpc/kernel/idle_power7.S
index 112ccf4..cf4fb54 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -89,13 +89,6 @@ _GLOBAL(power7_powersave_common)
std r0,_LINK(r1)
std r0,_NIP(r1)
-#ifndef CONFIG_SMP
- /* Make sure FPU, VSX etc... are flushed as we may lose
- * state when going to nap mode
- */
- bl discard_lazy_cpu_state
-#endif /* CONFIG_SMP */
-
/* Hard disable interrupts */
mfmsr r9
rldicl r9,r9,48,1
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 3aabed4..e098f43 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -67,13 +67,6 @@
extern unsigned long _get_SP(void);
-#ifndef CONFIG_SMP
-struct task_struct *last_task_used_math = NULL;
-struct task_struct *last_task_used_altivec = NULL;
-struct task_struct *last_task_used_vsx = NULL;
-struct task_struct *last_task_used_spe = NULL;
-#endif
-
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
void giveup_fpu_maybe_transactional(struct task_struct *tsk)
{
@@ -134,16 +127,14 @@ void flush_fp_to_thread(struct task_struct *tsk)
*/
preempt_disable();
if (tsk->thread.regs->msr & MSR_FP) {
-#ifdef CONFIG_SMP
/*
* This should only ever be called for current or
* for a stopped child process. Since we save away
- * the FP register state on context switch on SMP,
+ * the FP register state on context switch,
* there is something wrong if a stopped child appears
* to still have its FP state in the CPU registers.
*/
BUG_ON(tsk != current);
-#endif
giveup_fpu_maybe_transactional(tsk);
}
preempt_enable();
@@ -156,14 +147,10 @@ void enable_kernel_fp(void)
{
WARN_ON(preemptible());
-#ifdef CONFIG_SMP
if (current->thread.regs && (current->thread.regs->msr & MSR_FP))
giveup_fpu_maybe_transactional(current);
else
giveup_fpu(NULL); /* just enables FP for kernel */
-#else
- giveup_fpu_maybe_transactional(last_task_used_math);
-#endif /* CONFIG_SMP */
}
EXPORT_SYMBOL(enable_kernel_fp);
@@ -172,14 +159,10 @@ void enable_kernel_altivec(void)
{
WARN_ON(preemptible());
-#ifdef CONFIG_SMP
if (current->thread.regs && (current->thread.regs->msr & MSR_VEC))
giveup_altivec_maybe_transactional(current);
else
giveup_altivec_notask();
-#else
- giveup_altivec_maybe_transactional(last_task_used_altivec);
-#endif /* CONFIG_SMP */
}
EXPORT_SYMBOL(enable_kernel_altivec);
@@ -192,9 +175,7 @@ void flush_altivec_to_thread(struct task_struct *tsk)
if (tsk->thread.regs) {
preempt_disable();
if (tsk->thread.regs->msr & MSR_VEC) {
-#ifdef CONFIG_SMP
BUG_ON(tsk != current);
-#endif
giveup_altivec_maybe_transactional(tsk);
}
preempt_enable();
@@ -208,14 +189,10 @@ void enable_kernel_vsx(void)
{
WARN_ON(preemptible());
-#ifdef CONFIG_SMP
if (current->thread.regs && (current->thread.regs->msr & MSR_VSX))
giveup_vsx(current);
else
giveup_vsx(NULL); /* just enable vsx for kernel - force */
-#else
- giveup_vsx(last_task_used_vsx);
-#endif /* CONFIG_SMP */
}
EXPORT_SYMBOL(enable_kernel_vsx);
@@ -232,9 +209,7 @@ void flush_vsx_to_thread(struct task_struct *tsk)
if (tsk->thread.regs) {
preempt_disable();
if (tsk->thread.regs->msr & MSR_VSX) {
-#ifdef CONFIG_SMP
BUG_ON(tsk != current);
-#endif
giveup_vsx(tsk);
}
preempt_enable();
@@ -249,14 +224,10 @@ void enable_kernel_spe(void)
{
WARN_ON(preemptible());
-#ifdef CONFIG_SMP
if (current->thread.regs && (current->thread.regs->msr & MSR_SPE))
giveup_spe(current);
else
giveup_spe(NULL); /* just enable SPE for kernel - force */
-#else
- giveup_spe(last_task_used_spe);
-#endif /* __SMP __ */
}
EXPORT_SYMBOL(enable_kernel_spe);
@@ -265,9 +236,7 @@ void flush_spe_to_thread(struct task_struct *tsk)
if (tsk->thread.regs) {
preempt_disable();
if (tsk->thread.regs->msr & MSR_SPE) {
-#ifdef CONFIG_SMP
BUG_ON(tsk != current);
-#endif
tsk->thread.spefscr = mfspr(SPRN_SPEFSCR);
giveup_spe(tsk);
}
@@ -276,32 +245,6 @@ void flush_spe_to_thread(struct task_struct *tsk)
}
#endif /* CONFIG_SPE */
-#ifndef CONFIG_SMP
-/*
- * If we are doing lazy switching of CPU state (FP, altivec or SPE),
- * and the current task has some state, discard it.
- */
-void discard_lazy_cpu_state(void)
-{
- preempt_disable();
- if (last_task_used_math == current)
- last_task_used_math = NULL;
-#ifdef CONFIG_ALTIVEC
- if (last_task_used_altivec == current)
- last_task_used_altivec = NULL;
-#endif /* CONFIG_ALTIVEC */
-#ifdef CONFIG_VSX
- if (last_task_used_vsx == current)
- last_task_used_vsx = NULL;
-#endif /* CONFIG_VSX */
-#ifdef CONFIG_SPE
- if (last_task_used_spe == current)
- last_task_used_spe = NULL;
-#endif
- preempt_enable();
-}
-#endif /* CONFIG_SMP */
-
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
void do_send_trap(struct pt_regs *regs, unsigned long address,
unsigned long error_code, int signal_code, int breakpt)
@@ -831,30 +774,9 @@ struct task_struct *__switch_to(struct task_struct *prev,
__switch_to_tm(prev);
-#ifdef CONFIG_SMP
- /* avoid complexity of lazy save/restore of fpu
- * by just saving it every time we switch out if
- * this task used the fpu during the last quantum.
- *
- * If it tries to use the fpu again, it'll trap and
- * reload its fp regs. So we don't have to do a restore
- * every switch, just a save.
- * -- Cort
- */
if (prev->thread.regs && (prev->thread.regs->msr & MSR_FP))
giveup_fpu(prev);
#ifdef CONFIG_ALTIVEC
- /*
- * If the previous thread used altivec in the last quantum
- * (thus changing altivec regs) then save them.
- * We used to check the VRSAVE register but not all apps
- * set it, so we don't rely on it now (and in fact we need
- * to save & restore VSCR even if VRSAVE == 0). -- paulus
- *
- * On SMP we always save/restore altivec regs just to avoid the
- * complexity of changing processors.
- * -- Cort
- */
if (prev->thread.regs && (prev->thread.regs->msr & MSR_VEC))
giveup_altivec(prev);
#endif /* CONFIG_ALTIVEC */
@@ -864,39 +786,10 @@ struct task_struct *__switch_to(struct task_struct *prev,
__giveup_vsx(prev);
#endif /* CONFIG_VSX */
#ifdef CONFIG_SPE
- /*
- * If the previous thread used spe in the last quantum
- * (thus changing spe regs) then save them.
- *
- * On SMP we always save/restore spe regs just to avoid the
- * complexity of changing processors.
- */
if ((prev->thread.regs && (prev->thread.regs->msr & MSR_SPE)))
giveup_spe(prev);
#endif /* CONFIG_SPE */
-#else /* CONFIG_SMP */
-#ifdef CONFIG_ALTIVEC
- /* Avoid the trap. On smp this this never happens since
- * we don't set last_task_used_altivec -- Cort
- */
- if (new->thread.regs && last_task_used_altivec == new)
- new->thread.regs->msr |= MSR_VEC;
-#endif /* CONFIG_ALTIVEC */
-#ifdef CONFIG_VSX
- if (new->thread.regs && last_task_used_vsx == new)
- new->thread.regs->msr |= MSR_VSX;
-#endif /* CONFIG_VSX */
-#ifdef CONFIG_SPE
- /* Avoid the trap. On smp this this never happens since
- * we don't set last_task_used_spe
- */
- if (new->thread.regs && last_task_used_spe == new)
- new->thread.regs->msr |= MSR_SPE;
-#endif /* CONFIG_SPE */
-
-#endif /* CONFIG_SMP */
-
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
switch_booke_debug_regs(&new->thread.debug);
#else
@@ -1111,13 +1004,10 @@ void show_regs(struct pt_regs * regs)
void exit_thread(void)
{
- discard_lazy_cpu_state();
}
void flush_thread(void)
{
- discard_lazy_cpu_state();
-
#ifdef CONFIG_HAVE_HW_BREAKPOINT
flush_ptrace_hw_breakpoint(current);
#else /* CONFIG_HAVE_HW_BREAKPOINT */
@@ -1355,7 +1245,6 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
regs->msr = MSR_USER32;
}
#endif
- discard_lazy_cpu_state();
#ifdef CONFIG_VSX
current->thread.used_vsr = 0;
#endif
diff --git a/arch/powerpc/kernel/signal_32.c b/arch/powerpc/kernel/signal_32.c
index 0dbee46..3cd7a32 100644
--- a/arch/powerpc/kernel/signal_32.c
+++ b/arch/powerpc/kernel/signal_32.c
@@ -687,15 +687,6 @@ static long restore_user_regs(struct pt_regs *regs,
if (sig)
regs->msr = (regs->msr & ~MSR_LE) | (msr & MSR_LE);
- /*
- * Do this before updating the thread state in
- * current->thread.fpr/vr/evr. That way, if we get preempted
- * and another task grabs the FPU/Altivec/SPE, it won't be
- * tempted to save the current CPU state into the thread_struct
- * and corrupt what we are writing there.
- */
- discard_lazy_cpu_state();
-
#ifdef CONFIG_ALTIVEC
/*
* Force the process to reload the altivec registers from
@@ -798,15 +789,6 @@ static long restore_tm_user_regs(struct pt_regs *regs,
/* Restore the previous little-endian mode */
regs->msr = (regs->msr & ~MSR_LE) | (msr & MSR_LE);
- /*
- * Do this before updating the thread state in
- * current->thread.fpr/vr/evr. That way, if we get preempted
- * and another task grabs the FPU/Altivec/SPE, it won't be
- * tempted to save the current CPU state into the thread_struct
- * and corrupt what we are writing there.
- */
- discard_lazy_cpu_state();
-
#ifdef CONFIG_ALTIVEC
regs->msr &= ~MSR_VEC;
if (msr & MSR_VEC) {
diff --git a/arch/powerpc/kernel/signal_64.c b/arch/powerpc/kernel/signal_64.c
index 20756df..6f2b555 100644
--- a/arch/powerpc/kernel/signal_64.c
+++ b/arch/powerpc/kernel/signal_64.c
@@ -350,15 +350,6 @@ static long restore_sigcontext(struct pt_regs *regs, sigset_t *set, int sig,
err |= __get_user(set->sig[0], &sc->oldmask);
/*
- * Do this before updating the thread state in
- * current->thread.fpr/vr. That way, if we get preempted
- * and another task grabs the FPU/Altivec, it won't be
- * tempted to save the current CPU state into the thread_struct
- * and corrupt what we are writing there.
- */
- discard_lazy_cpu_state();
-
- /*
* Force reload of FP/VEC.
* This has to be done before copying stuff into current->thread.fpr/vr
* for the reasons explained in the previous comment.
@@ -465,15 +456,6 @@ static long restore_tm_sigcontexts(struct pt_regs *regs,
err |= __get_user(regs->result, &sc->gp_regs[PT_RESULT]);
/*
- * Do this before updating the thread state in
- * current->thread.fpr/vr. That way, if we get preempted
- * and another task grabs the FPU/Altivec, it won't be
- * tempted to save the current CPU state into the thread_struct
- * and corrupt what we are writing there.
- */
- discard_lazy_cpu_state();
-
- /*
* Force reload of FP/VEC.
* This has to be done before copying stuff into current->thread.fpr/vr
* for the reasons explained in the previous comment.
diff --git a/arch/powerpc/kernel/vector.S b/arch/powerpc/kernel/vector.S
index 1c54259..1757c0c 100644
--- a/arch/powerpc/kernel/vector.S
+++ b/arch/powerpc/kernel/vector.S
@@ -80,39 +80,6 @@ _GLOBAL(load_up_altivec)
MTMSRD(r5) /* enable use of AltiVec now */
isync
-/*
- * For SMP, we don't do lazy VMX switching because it just gets too
- * horrendously complex, especially when a task switches from one CPU
- * to another. Instead we call giveup_altvec in switch_to.
- * VRSAVE isn't dealt with here, that is done in the normal context
- * switch code. Note that we could rely on vrsave value to eventually
- * avoid saving all of the VREGs here...
- */
-#ifndef CONFIG_SMP
- LOAD_REG_ADDRBASE(r3, last_task_used_altivec)
- toreal(r3)
- PPC_LL r4,ADDROFF(last_task_used_altivec)(r3)
- PPC_LCMPI 0,r4,0
- beq 1f
-
- /* Save VMX state to last_task_used_altivec's THREAD struct */
- toreal(r4)
- addi r4,r4,THREAD
- addi r6,r4,THREAD_VRSTATE
- SAVE_32VRS(0,r5,r6)
- mfvscr v0
- li r10,VRSTATE_VSCR
- stvx v0,r10,r6
- /* Disable VMX for last_task_used_altivec */
- PPC_LL r5,PT_REGS(r4)
- toreal(r5)
- PPC_LL r4,_MSR-STACK_FRAME_OVERHEAD(r5)
- lis r10,MSR_VEC@h
- andc r4,r4,r10
- PPC_STL r4,_MSR-STACK_FRAME_OVERHEAD(r5)
-1:
-#endif /* CONFIG_SMP */
-
/* Hack: if we get an altivec unavailable trap with VRSAVE
* set to all zeros, we assume this is a broken application
* that fails to set it properly, and thus we switch it to
@@ -141,12 +108,6 @@ _GLOBAL(load_up_altivec)
lvx v0,r10,r6
mtvscr v0
REST_32VRS(0,r4,r6)
-#ifndef CONFIG_SMP
- /* Update last_task_used_altivec to 'current' */
- subi r4,r5,THREAD /* Back to 'current' */
- fromreal(r4)
- PPC_STL r4,ADDROFF(last_task_used_altivec)(r3)
-#endif /* CONFIG_SMP */
/* restore registers and return */
blr
@@ -199,11 +160,6 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_VSX)
andc r4,r4,r3 /* disable FP for previous task */
PPC_STL r4,_MSR-STACK_FRAME_OVERHEAD(r5)
1:
-#ifndef CONFIG_SMP
- li r5,0
- LOAD_REG_ADDRBASE(r4,last_task_used_altivec)
- PPC_STL r5,ADDROFF(last_task_used_altivec)(r4)
-#endif /* CONFIG_SMP */
blr
#ifdef CONFIG_VSX
@@ -226,20 +182,6 @@ _GLOBAL(load_up_vsx)
andis. r5,r12,MSR_VEC@h
beql+ load_up_altivec /* skip if already loaded */
-#ifndef CONFIG_SMP
- ld r3,last_task_used_vsx@got(r2)
- ld r4,0(r3)
- cmpdi 0,r4,0
- beq 1f
- /* Disable VSX for last_task_used_vsx */
- addi r4,r4,THREAD
- ld r5,PT_REGS(r4)
- ld r4,_MSR-STACK_FRAME_OVERHEAD(r5)
- lis r6,MSR_VSX@h
- andc r6,r4,r6
- std r6,_MSR-STACK_FRAME_OVERHEAD(r5)
-1:
-#endif /* CONFIG_SMP */
ld r4,PACACURRENT(r13)
addi r4,r4,THREAD /* Get THREAD */
li r6,1
@@ -247,11 +189,6 @@ _GLOBAL(load_up_vsx)
/* enable use of VSX after return */
oris r12,r12,MSR_VSX@h
std r12,_MSR(r1)
-#ifndef CONFIG_SMP
- /* Update last_task_used_vsx to 'current' */
- ld r4,PACACURRENT(r13)
- std r4,0(r3)
-#endif /* CONFIG_SMP */
b fast_exception_return
/*
@@ -277,11 +214,6 @@ _GLOBAL(__giveup_vsx)
andc r4,r4,r3 /* disable VSX for previous task */
std r4,_MSR-STACK_FRAME_OVERHEAD(r5)
1:
-#ifndef CONFIG_SMP
- li r5,0
- ld r4,last_task_used_vsx@got(r2)
- std r5,0(r4)
-#endif /* CONFIG_SMP */
blr
#endif /* CONFIG_VSX */
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 06/19] powerpc: Simplify TM restore checks
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (3 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 05/19] powerpc: Remove UP only lazy floating point and vector optimisations Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 07/19] powerpc: Create mtmsrd_isync() Anton Blanchard
` (12 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Instead of having multiple giveup_*_maybe_transactional() functions,
separate out the TM check into a new function called
check_if_tm_restore_required().
This will make it easier to optimise the giveup_*() functions in a
subsequent patch.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/kernel/process.c | 53 ++++++++++++++++---------------------------
1 file changed, 19 insertions(+), 34 deletions(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index e098f43..ef64219 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -68,7 +68,7 @@
extern unsigned long _get_SP(void);
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
-void giveup_fpu_maybe_transactional(struct task_struct *tsk)
+static void check_if_tm_restore_required(struct task_struct *tsk)
{
/*
* If we are saving the current thread's registers, and the
@@ -82,31 +82,9 @@ void giveup_fpu_maybe_transactional(struct task_struct *tsk)
tsk->thread.ckpt_regs.msr = tsk->thread.regs->msr;
set_thread_flag(TIF_RESTORE_TM);
}
-
- giveup_fpu(tsk);
-}
-
-void giveup_altivec_maybe_transactional(struct task_struct *tsk)
-{
- /*
- * If we are saving the current thread's registers, and the
- * thread is in a transactional state, set the TIF_RESTORE_TM
- * bit so that we know to restore the registers before
- * returning to userspace.
- */
- if (tsk == current && tsk->thread.regs &&
- MSR_TM_ACTIVE(tsk->thread.regs->msr) &&
- !test_thread_flag(TIF_RESTORE_TM)) {
- tsk->thread.ckpt_regs.msr = tsk->thread.regs->msr;
- set_thread_flag(TIF_RESTORE_TM);
- }
-
- giveup_altivec(tsk);
}
-
#else
-#define giveup_fpu_maybe_transactional(tsk) giveup_fpu(tsk)
-#define giveup_altivec_maybe_transactional(tsk) giveup_altivec(tsk)
+static inline void check_if_tm_restore_required(struct task_struct *tsk) { }
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
#ifdef CONFIG_PPC_FPU
@@ -135,7 +113,8 @@ void flush_fp_to_thread(struct task_struct *tsk)
* to still have its FP state in the CPU registers.
*/
BUG_ON(tsk != current);
- giveup_fpu_maybe_transactional(tsk);
+ check_if_tm_restore_required(tsk);
+ giveup_fpu(tsk);
}
preempt_enable();
}
@@ -147,10 +126,12 @@ void enable_kernel_fp(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_FP))
- giveup_fpu_maybe_transactional(current);
- else
+ if (current->thread.regs && (current->thread.regs->msr & MSR_FP)) {
+ check_if_tm_restore_required(current);
+ giveup_fpu(current);
+ } else {
giveup_fpu(NULL); /* just enables FP for kernel */
+ }
}
EXPORT_SYMBOL(enable_kernel_fp);
@@ -159,10 +140,12 @@ void enable_kernel_altivec(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_VEC))
- giveup_altivec_maybe_transactional(current);
- else
+ if (current->thread.regs && (current->thread.regs->msr & MSR_VEC)) {
+ check_if_tm_restore_required(current);
+ giveup_altivec(current);
+ } else {
giveup_altivec_notask();
+ }
}
EXPORT_SYMBOL(enable_kernel_altivec);
@@ -176,7 +159,8 @@ void flush_altivec_to_thread(struct task_struct *tsk)
preempt_disable();
if (tsk->thread.regs->msr & MSR_VEC) {
BUG_ON(tsk != current);
- giveup_altivec_maybe_transactional(tsk);
+ check_if_tm_restore_required(tsk);
+ giveup_altivec(tsk);
}
preempt_enable();
}
@@ -198,8 +182,9 @@ EXPORT_SYMBOL(enable_kernel_vsx);
void giveup_vsx(struct task_struct *tsk)
{
- giveup_fpu_maybe_transactional(tsk);
- giveup_altivec_maybe_transactional(tsk);
+ check_if_tm_restore_required(tsk);
+ giveup_fpu(tsk);
+ giveup_altivec(tsk);
__giveup_vsx(tsk);
}
EXPORT_SYMBOL(giveup_vsx);
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 07/19] powerpc: Create mtmsrd_isync()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (4 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 06/19] powerpc: Simplify TM restore checks Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 08/19] powerpc: Remove NULL task struct pointer checks in FP and vector code Anton Blanchard
` (11 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
mtmsrd_isync() will do an mtmsrd followed by an isync on older
processors. On newer processors we avoid the isync via a feature fixup.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/reg.h | 8 ++++++++
arch/powerpc/kernel/process.c | 30 ++++++++++++++++++++++--------
2 files changed, 30 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index a908ada..987dac0 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -1193,12 +1193,20 @@
#define __mtmsrd(v, l) asm volatile("mtmsrd %0," __stringify(l) \
: : "r" (v) : "memory")
#define mtmsr(v) __mtmsrd((v), 0)
+#define __MTMSR "mtmsrd"
#else
#define mtmsr(v) asm volatile("mtmsr %0" : \
: "r" ((unsigned long)(v)) \
: "memory")
+#define __MTMSR "mtmsr"
#endif
+static inline void mtmsr_isync(unsigned long val)
+{
+ asm volatile(__MTMSR " %0; " ASM_FTR_IFCLR("isync", "nop", %1) : :
+ "r" (val), "i" (CPU_FTR_ARCH_206) : "memory");
+}
+
#define mfspr(rn) ({unsigned long rval; \
asm volatile("mfspr %0," __stringify(rn) \
: "=r" (rval)); rval;})
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index ef64219..5bf8ec2 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -130,7 +130,10 @@ void enable_kernel_fp(void)
check_if_tm_restore_required(current);
giveup_fpu(current);
} else {
- giveup_fpu(NULL); /* just enables FP for kernel */
+ u64 oldmsr = mfmsr();
+
+ if (!(oldmsr & MSR_FP))
+ mtmsr_isync(oldmsr | MSR_FP);
}
}
EXPORT_SYMBOL(enable_kernel_fp);
@@ -144,7 +147,10 @@ void enable_kernel_altivec(void)
check_if_tm_restore_required(current);
giveup_altivec(current);
} else {
- giveup_altivec_notask();
+ u64 oldmsr = mfmsr();
+
+ if (!(oldmsr & MSR_VEC))
+ mtmsr_isync(oldmsr | MSR_VEC);
}
}
EXPORT_SYMBOL(enable_kernel_altivec);
@@ -173,10 +179,14 @@ void enable_kernel_vsx(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_VSX))
+ if (current->thread.regs && (current->thread.regs->msr & MSR_VSX)) {
giveup_vsx(current);
- else
- giveup_vsx(NULL); /* just enable vsx for kernel - force */
+ } else {
+ u64 oldmsr = mfmsr();
+
+ if (!(oldmsr & MSR_VSX))
+ mtmsr_isync(oldmsr | MSR_VSX);
+ }
}
EXPORT_SYMBOL(enable_kernel_vsx);
@@ -209,10 +219,14 @@ void enable_kernel_spe(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_SPE))
+ if (current->thread.regs && (current->thread.regs->msr & MSR_SPE)) {
giveup_spe(current);
- else
- giveup_spe(NULL); /* just enable SPE for kernel - force */
+ } else {
+ u64 oldmsr = mfmsr();
+
+ if (!(oldmsr & MSR_SPE))
+ mtmsr_isync(oldmsr | MSR_SPE);
+ }
}
EXPORT_SYMBOL(enable_kernel_spe);
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 08/19] powerpc: Remove NULL task struct pointer checks in FP and vector code
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (5 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 07/19] powerpc: Create mtmsrd_isync() Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 09/19] powerpc: Move part of giveup_fpu,altivec,spe into c Anton Blanchard
` (10 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
We used to allow giveup_*() to be called with a NULL task struct
pointer. Now those cases are handled in the caller we can remove
the checks. We can also remove giveup_altivec_notask() which is also
unused.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/switch_to.h | 1 -
arch/powerpc/kernel/fpu.S | 2 --
arch/powerpc/kernel/head_fsl_booke.S | 2 --
arch/powerpc/kernel/vector.S | 14 --------------
4 files changed, 19 deletions(-)
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index bd1d933..042aaf0 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -38,7 +38,6 @@ static inline void giveup_fpu(struct task_struct *t) { }
#ifdef CONFIG_ALTIVEC
extern void flush_altivec_to_thread(struct task_struct *);
extern void giveup_altivec(struct task_struct *);
-extern void giveup_altivec_notask(void);
#else
static inline void flush_altivec_to_thread(struct task_struct *t)
{
diff --git a/arch/powerpc/kernel/fpu.S b/arch/powerpc/kernel/fpu.S
index 50d2352..71bdce2 100644
--- a/arch/powerpc/kernel/fpu.S
+++ b/arch/powerpc/kernel/fpu.S
@@ -173,8 +173,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
MTMSRD(r5) /* enable use of fpu now */
SYNC_601
isync
- PPC_LCMPI 0,r3,0
- beqlr- /* if no previous owner, done */
addi r3,r3,THREAD /* want THREAD of task */
PPC_LL r6,THREAD_FPSAVEAREA(r3)
PPC_LL r5,PT_REGS(r3)
diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S
index ec936ab..d6980bb 100644
--- a/arch/powerpc/kernel/head_fsl_booke.S
+++ b/arch/powerpc/kernel/head_fsl_booke.S
@@ -992,8 +992,6 @@ _GLOBAL(giveup_spe)
oris r5,r5,MSR_SPE@h
mtmsr r5 /* enable use of SPE now */
isync
- cmpi 0,r3,0
- beqlr- /* if no previous owner, done */
addi r3,r3,THREAD /* want THREAD of task */
lwz r5,PT_REGS(r3)
cmpi 0,r5,0
diff --git a/arch/powerpc/kernel/vector.S b/arch/powerpc/kernel/vector.S
index 1757c0c..b31528c 100644
--- a/arch/powerpc/kernel/vector.S
+++ b/arch/powerpc/kernel/vector.S
@@ -111,16 +111,6 @@ _GLOBAL(load_up_altivec)
/* restore registers and return */
blr
-_GLOBAL(giveup_altivec_notask)
- mfmsr r3
- andis. r4,r3,MSR_VEC@h
- bnelr /* Already enabled? */
- oris r3,r3,MSR_VEC@h
- SYNC
- MTMSRD(r3) /* enable use of VMX now */
- isync
- blr
-
/*
* giveup_altivec(tsk)
* Disable VMX for the task given as the argument,
@@ -133,8 +123,6 @@ _GLOBAL(giveup_altivec)
SYNC
MTMSRD(r5) /* enable use of VMX now */
isync
- PPC_LCMPI 0,r3,0
- beqlr /* if no previous owner, done */
addi r3,r3,THREAD /* want THREAD of task */
PPC_LL r7,THREAD_VRSAVEAREA(r3)
PPC_LL r5,PT_REGS(r3)
@@ -203,8 +191,6 @@ _GLOBAL(__giveup_vsx)
mtmsrd r5 /* enable use of VSX now */
isync
- cmpdi 0,r3,0
- beqlr- /* if no previous owner, done */
addi r3,r3,THREAD /* want THREAD of task */
ld r5,PT_REGS(r3)
cmpdi 0,r5,0
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 09/19] powerpc: Move part of giveup_fpu,altivec,spe into c
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (6 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 08/19] powerpc: Remove NULL task struct pointer checks in FP and vector code Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 3:17 ` [PATCH 09/19] powerpc: Move part of giveup_fpu, altivec, spe " kbuild test robot
2015-10-28 0:50 ` [PATCH 10/19] powerpc: Move part of giveup_vsx " Anton Blanchard
` (9 subsequent siblings)
17 siblings, 1 reply; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Move the MSR modification into new c functions. Removing it from
the low level functions will allow us to avoid costly MSR writes
by batching them up.
Move the check_if_tm_restore_required() check into these new functions.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/switch_to.h | 12 ++++-----
arch/powerpc/kernel/fpu.S | 16 ++---------
arch/powerpc/kernel/head_fsl_booke.S | 8 ++----
arch/powerpc/kernel/ppc_ksyms.c | 6 -----
arch/powerpc/kernel/process.c | 52 +++++++++++++++++++++++++++++++++---
arch/powerpc/kernel/vector.S | 10 ++-----
6 files changed, 60 insertions(+), 44 deletions(-)
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 042aaf0..6bcfb89 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -30,21 +30,21 @@ extern void switch_booke_debug_regs(struct debug_reg *new_debug);
#ifdef CONFIG_PPC_FPU
extern void flush_fp_to_thread(struct task_struct *);
extern void giveup_fpu(struct task_struct *);
+extern void __giveup_fpu(struct task_struct *);
#else
static inline void flush_fp_to_thread(struct task_struct *t) { }
static inline void giveup_fpu(struct task_struct *t) { }
+static inline void __giveup_fpu(struct task_struct *t) { }
#endif
#ifdef CONFIG_ALTIVEC
extern void flush_altivec_to_thread(struct task_struct *);
extern void giveup_altivec(struct task_struct *);
+extern void __giveup_altivec(struct task_struct *);
#else
-static inline void flush_altivec_to_thread(struct task_struct *t)
-{
-}
-static inline void giveup_altivec(struct task_struct *t)
-{
-}
+static inline void flush_altivec_to_thread(struct task_struct *t) { }
+static inline void giveup_altivec(struct task_struct *t) { }
+static inline void __giveup_altivec(struct task_struct *t) { }
#endif
#ifdef CONFIG_VSX
diff --git a/arch/powerpc/kernel/fpu.S b/arch/powerpc/kernel/fpu.S
index 71bdce2..431ab57 100644
--- a/arch/powerpc/kernel/fpu.S
+++ b/arch/powerpc/kernel/fpu.S
@@ -155,24 +155,12 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
blr
/*
- * giveup_fpu(tsk)
+ * __giveup_fpu(tsk)
* Disable FP for the task given as the argument,
* and save the floating-point registers in its thread_struct.
* Enables the FPU for use in the kernel on return.
*/
-_GLOBAL(giveup_fpu)
- mfmsr r5
- ori r5,r5,MSR_FP
-#ifdef CONFIG_VSX
-BEGIN_FTR_SECTION
- oris r5,r5,MSR_VSX@h
-END_FTR_SECTION_IFSET(CPU_FTR_VSX)
-#endif
- SYNC_601
- ISYNC_601
- MTMSRD(r5) /* enable use of fpu now */
- SYNC_601
- isync
+_GLOBAL(__giveup_fpu)
addi r3,r3,THREAD /* want THREAD of task */
PPC_LL r6,THREAD_FPSAVEAREA(r3)
PPC_LL r5,PT_REGS(r3)
diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S
index d6980bb..f705171 100644
--- a/arch/powerpc/kernel/head_fsl_booke.S
+++ b/arch/powerpc/kernel/head_fsl_booke.S
@@ -984,14 +984,10 @@ _GLOBAL(__setup_ehv_ivors)
#ifdef CONFIG_SPE
/*
- * extern void giveup_spe(struct task_struct *prev)
+ * extern void __giveup_spe(struct task_struct *prev)
*
*/
-_GLOBAL(giveup_spe)
- mfmsr r5
- oris r5,r5,MSR_SPE@h
- mtmsr r5 /* enable use of SPE now */
- isync
+_GLOBAL(__giveup_spe)
addi r3,r3,THREAD /* want THREAD of task */
lwz r5,PT_REGS(r3)
cmpi 0,r5,0
diff --git a/arch/powerpc/kernel/ppc_ksyms.c b/arch/powerpc/kernel/ppc_ksyms.c
index 202963e..41e1607 100644
--- a/arch/powerpc/kernel/ppc_ksyms.c
+++ b/arch/powerpc/kernel/ppc_ksyms.c
@@ -19,13 +19,11 @@ EXPORT_SYMBOL(_mcount);
#endif
#ifdef CONFIG_PPC_FPU
-EXPORT_SYMBOL(giveup_fpu);
EXPORT_SYMBOL(load_fp_state);
EXPORT_SYMBOL(store_fp_state);
#endif
#ifdef CONFIG_ALTIVEC
-EXPORT_SYMBOL(giveup_altivec);
EXPORT_SYMBOL(load_vr_state);
EXPORT_SYMBOL(store_vr_state);
#endif
@@ -34,10 +32,6 @@ EXPORT_SYMBOL(store_vr_state);
EXPORT_SYMBOL_GPL(__giveup_vsx);
#endif
-#ifdef CONFIG_SPE
-EXPORT_SYMBOL(giveup_spe);
-#endif
-
#ifdef CONFIG_EPAPR_PARAVIRT
EXPORT_SYMBOL(epapr_hypercall_start);
#endif
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 5bf8ec2..ca8b36f 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -88,6 +88,25 @@ static inline void check_if_tm_restore_required(struct task_struct *tsk) { }
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
#ifdef CONFIG_PPC_FPU
+void giveup_fpu(struct task_struct *tsk)
+{
+ u64 oldmsr = mfmsr();
+ u64 newmsr;
+
+ check_if_tm_restore_required(tsk);
+
+ newmsr = oldmsr | MSR_FP;
+#ifdef CONFIG_VSX
+ if (cpu_has_feature(CPU_FTR_VSX))
+ newmsr |= MSR_VSX;
+#endif
+ if (oldmsr != newmsr)
+ mtmsr_isync(newmsr);
+
+ __giveup_fpu(tsk);
+}
+EXPORT_SYMBOL(giveup_fpu);
+
/*
* Make sure the floating-point register state in the
* the thread_struct is up to date for task tsk.
@@ -113,7 +132,6 @@ void flush_fp_to_thread(struct task_struct *tsk)
* to still have its FP state in the CPU registers.
*/
BUG_ON(tsk != current);
- check_if_tm_restore_required(tsk);
giveup_fpu(tsk);
}
preempt_enable();
@@ -127,7 +145,6 @@ void enable_kernel_fp(void)
WARN_ON(preemptible());
if (current->thread.regs && (current->thread.regs->msr & MSR_FP)) {
- check_if_tm_restore_required(current);
giveup_fpu(current);
} else {
u64 oldmsr = mfmsr();
@@ -139,12 +156,26 @@ void enable_kernel_fp(void)
EXPORT_SYMBOL(enable_kernel_fp);
#ifdef CONFIG_ALTIVEC
+void giveup_altivec(struct task_struct *tsk)
+{
+ u64 oldmsr = mfmsr();
+ u64 newmsr;
+
+ check_if_tm_restore_required(tsk);
+
+ newmsr = oldmsr | MSR_VEC;
+ if (oldmsr != newmsr)
+ mtmsr_isync(newmsr);
+
+ __giveup_altivec(tsk);
+}
+EXPORT_SYMBOL(giveup_altivec);
+
void enable_kernel_altivec(void)
{
WARN_ON(preemptible());
if (current->thread.regs && (current->thread.regs->msr & MSR_VEC)) {
- check_if_tm_restore_required(current);
giveup_altivec(current);
} else {
u64 oldmsr = mfmsr();
@@ -165,7 +196,6 @@ void flush_altivec_to_thread(struct task_struct *tsk)
preempt_disable();
if (tsk->thread.regs->msr & MSR_VEC) {
BUG_ON(tsk != current);
- check_if_tm_restore_required(tsk);
giveup_altivec(tsk);
}
preempt_enable();
@@ -214,6 +244,20 @@ EXPORT_SYMBOL_GPL(flush_vsx_to_thread);
#endif /* CONFIG_VSX */
#ifdef CONFIG_SPE
+void giveup_spe(struct task_struct *)
+{
+ u64 oldmsr = mfmsr();
+ u64 newmsr;
+
+ check_if_tm_restore_required(tsk);
+
+ newmsr = oldmsr | MSR_SPE;
+ if (oldmsr != newmsr)
+ mtmsr_isync(newmsr);
+
+ __giveup_spe(tsk);
+}
+EXPORT_SYMBOL(giveup_spe);
void enable_kernel_spe(void)
{
diff --git a/arch/powerpc/kernel/vector.S b/arch/powerpc/kernel/vector.S
index b31528c..6e925b4 100644
--- a/arch/powerpc/kernel/vector.S
+++ b/arch/powerpc/kernel/vector.S
@@ -112,17 +112,11 @@ _GLOBAL(load_up_altivec)
blr
/*
- * giveup_altivec(tsk)
+ * __giveup_altivec(tsk)
* Disable VMX for the task given as the argument,
* and save the vector registers in its thread_struct.
- * Enables the VMX for use in the kernel on return.
*/
-_GLOBAL(giveup_altivec)
- mfmsr r5
- oris r5,r5,MSR_VEC@h
- SYNC
- MTMSRD(r5) /* enable use of VMX now */
- isync
+_GLOBAL(__giveup_altivec)
addi r3,r3,THREAD /* want THREAD of task */
PPC_LL r7,THREAD_VRSAVEAREA(r3)
PPC_LL r5,PT_REGS(r3)
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 10/19] powerpc: Move part of giveup_vsx into c
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (7 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 09/19] powerpc: Move part of giveup_fpu,altivec,spe into c Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:50 ` [PATCH 11/19] crypto: vmx: Only call enable_kernel_vsx() Anton Blanchard
` (8 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Move the MSR modification into c. Removing it from the assembly
function will allow us to avoid costly MSR writes by batching them
up.
Check the FP and VMX bits before calling the relevant giveup_*()
function. This makes giveup_vsx() and flush_vsx_to_thread() perform
more like their sister functions, and allows us to use
flush_vsx_to_thread() in the signal code.
Move the check_if_tm_restore_required() check in.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/kernel/process.c | 28 +++++++++++++++++++---------
arch/powerpc/kernel/signal_32.c | 4 ++--
arch/powerpc/kernel/signal_64.c | 4 ++--
arch/powerpc/kernel/vector.S | 6 ------
4 files changed, 23 insertions(+), 19 deletions(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index ca8b36f..c39aa5a 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -205,6 +205,25 @@ EXPORT_SYMBOL_GPL(flush_altivec_to_thread);
#endif /* CONFIG_ALTIVEC */
#ifdef CONFIG_VSX
+void giveup_vsx(struct task_struct *tsk)
+{
+ u64 oldmsr = mfmsr();
+ u64 newmsr;
+
+ check_if_tm_restore_required(tsk);
+
+ newmsr = oldmsr | (MSR_FP|MSR_VEC|MSR_VSX);
+ if (oldmsr != newmsr)
+ mtmsr_isync(newmsr);
+
+ if (tsk->thread.regs->msr & MSR_FP)
+ __giveup_fpu(tsk);
+ if (tsk->thread.regs->msr & MSR_VEC)
+ __giveup_altivec(tsk);
+ __giveup_vsx(tsk);
+}
+EXPORT_SYMBOL(giveup_vsx);
+
void enable_kernel_vsx(void)
{
WARN_ON(preemptible());
@@ -220,15 +239,6 @@ void enable_kernel_vsx(void)
}
EXPORT_SYMBOL(enable_kernel_vsx);
-void giveup_vsx(struct task_struct *tsk)
-{
- check_if_tm_restore_required(tsk);
- giveup_fpu(tsk);
- giveup_altivec(tsk);
- __giveup_vsx(tsk);
-}
-EXPORT_SYMBOL(giveup_vsx);
-
void flush_vsx_to_thread(struct task_struct *tsk)
{
if (tsk->thread.regs) {
diff --git a/arch/powerpc/kernel/signal_32.c b/arch/powerpc/kernel/signal_32.c
index 3cd7a32..4022cbb 100644
--- a/arch/powerpc/kernel/signal_32.c
+++ b/arch/powerpc/kernel/signal_32.c
@@ -458,7 +458,7 @@ static int save_user_regs(struct pt_regs *regs, struct mcontext __user *frame,
* contains valid data
*/
if (current->thread.used_vsr && ctx_has_vsx_region) {
- __giveup_vsx(current);
+ flush_vsx_to_thread(current);
if (copy_vsx_to_user(&frame->mc_vsregs, current))
return 1;
msr |= MSR_VSX;
@@ -606,7 +606,7 @@ static int save_tm_user_regs(struct pt_regs *regs,
* contains valid data
*/
if (current->thread.used_vsr) {
- __giveup_vsx(current);
+ flush_vsx_to_thread(current);
if (copy_vsx_to_user(&frame->mc_vsregs, current))
return 1;
if (msr & MSR_VSX) {
diff --git a/arch/powerpc/kernel/signal_64.c b/arch/powerpc/kernel/signal_64.c
index 6f2b555..3b23399 100644
--- a/arch/powerpc/kernel/signal_64.c
+++ b/arch/powerpc/kernel/signal_64.c
@@ -147,7 +147,7 @@ static long setup_sigcontext(struct sigcontext __user *sc, struct pt_regs *regs,
* VMX data.
*/
if (current->thread.used_vsr && ctx_has_vsx_region) {
- __giveup_vsx(current);
+ flush_vsx_to_thread(current);
v_regs += ELF_NVRREG;
err |= copy_vsx_to_user(v_regs, current);
/* set MSR_VSX in the MSR value in the frame to
@@ -270,7 +270,7 @@ static long setup_tm_sigcontexts(struct sigcontext __user *sc,
* VMX data.
*/
if (current->thread.used_vsr) {
- __giveup_vsx(current);
+ flush_vsx_to_thread(current);
v_regs += ELF_NVRREG;
tm_v_regs += ELF_NVRREG;
diff --git a/arch/powerpc/kernel/vector.S b/arch/powerpc/kernel/vector.S
index 6e925b4..98675b0 100644
--- a/arch/powerpc/kernel/vector.S
+++ b/arch/powerpc/kernel/vector.S
@@ -177,14 +177,8 @@ _GLOBAL(load_up_vsx)
* __giveup_vsx(tsk)
* Disable VSX for the task given as the argument.
* Does NOT save vsx registers.
- * Enables the VSX for use in the kernel on return.
*/
_GLOBAL(__giveup_vsx)
- mfmsr r5
- oris r5,r5,MSR_VSX@h
- mtmsrd r5 /* enable use of VSX now */
- isync
-
addi r3,r3,THREAD /* want THREAD of task */
ld r5,PT_REGS(r3)
cmpdi 0,r5,0
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 11/19] crypto: vmx: Only call enable_kernel_vsx()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (8 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 10/19] powerpc: Move part of giveup_vsx " Anton Blanchard
@ 2015-10-28 0:50 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 12/19] powerpc: Create msr_check_and_{set,clear}() Anton Blanchard
` (7 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:50 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
With the recent change to enable_kernel_vsx(), we no longer need
to call enable_kernel_fp() and enable_kernel_altivec().
Signed-off-by: Anton Blanchard <anton@samba.org>
---
drivers/crypto/vmx/aes.c | 3 ---
drivers/crypto/vmx/aes_cbc.c | 3 ---
drivers/crypto/vmx/aes_ctr.c | 3 ---
drivers/crypto/vmx/ghash.c | 8 --------
4 files changed, 17 deletions(-)
diff --git a/drivers/crypto/vmx/aes.c b/drivers/crypto/vmx/aes.c
index 263af70..20539fb 100644
--- a/drivers/crypto/vmx/aes.c
+++ b/drivers/crypto/vmx/aes.c
@@ -83,7 +83,6 @@ static int p8_aes_setkey(struct crypto_tfm *tfm, const u8 *key,
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
ret = aes_p8_set_encrypt_key(key, keylen * 8, &ctx->enc_key);
ret += aes_p8_set_decrypt_key(key, keylen * 8, &ctx->dec_key);
@@ -103,7 +102,6 @@ static void p8_aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src)
} else {
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
aes_p8_encrypt(src, dst, &ctx->enc_key);
pagefault_enable();
@@ -120,7 +118,6 @@ static void p8_aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src)
} else {
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
aes_p8_decrypt(src, dst, &ctx->dec_key);
pagefault_enable();
diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cbc.c
index 0b8fe2e..8847b92 100644
--- a/drivers/crypto/vmx/aes_cbc.c
+++ b/drivers/crypto/vmx/aes_cbc.c
@@ -84,7 +84,6 @@ static int p8_aes_cbc_setkey(struct crypto_tfm *tfm, const u8 *key,
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
ret = aes_p8_set_encrypt_key(key, keylen * 8, &ctx->enc_key);
ret += aes_p8_set_decrypt_key(key, keylen * 8, &ctx->dec_key);
@@ -115,7 +114,6 @@ static int p8_aes_cbc_encrypt(struct blkcipher_desc *desc,
} else {
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
blkcipher_walk_init(&walk, dst, src, nbytes);
@@ -156,7 +154,6 @@ static int p8_aes_cbc_decrypt(struct blkcipher_desc *desc,
} else {
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
blkcipher_walk_init(&walk, dst, src, nbytes);
diff --git a/drivers/crypto/vmx/aes_ctr.c b/drivers/crypto/vmx/aes_ctr.c
index ee1306c..8095866 100644
--- a/drivers/crypto/vmx/aes_ctr.c
+++ b/drivers/crypto/vmx/aes_ctr.c
@@ -81,7 +81,6 @@ static int p8_aes_ctr_setkey(struct crypto_tfm *tfm, const u8 *key,
struct p8_aes_ctr_ctx *ctx = crypto_tfm_ctx(tfm);
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
ret = aes_p8_set_encrypt_key(key, keylen * 8, &ctx->enc_key);
pagefault_enable();
@@ -100,7 +99,6 @@ static void p8_aes_ctr_final(struct p8_aes_ctr_ctx *ctx,
unsigned int nbytes = walk->nbytes;
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
aes_p8_encrypt(ctrblk, keystream, &ctx->enc_key);
pagefault_enable();
@@ -133,7 +131,6 @@ static int p8_aes_ctr_crypt(struct blkcipher_desc *desc,
ret = blkcipher_walk_virt_block(desc, &walk, AES_BLOCK_SIZE);
while ((nbytes = walk.nbytes) >= AES_BLOCK_SIZE) {
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
aes_p8_ctr32_encrypt_blocks(walk.src.virt.addr,
walk.dst.virt.addr,
diff --git a/drivers/crypto/vmx/ghash.c b/drivers/crypto/vmx/ghash.c
index 2183a2e..1f4586c 100644
--- a/drivers/crypto/vmx/ghash.c
+++ b/drivers/crypto/vmx/ghash.c
@@ -118,9 +118,7 @@ static int p8_ghash_setkey(struct crypto_shash *tfm, const u8 *key,
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
- enable_kernel_fp();
gcm_init_p8(ctx->htable, (const u64 *) key);
pagefault_enable();
preempt_enable();
@@ -149,9 +147,7 @@ static int p8_ghash_update(struct shash_desc *desc,
GHASH_DIGEST_SIZE - dctx->bytes);
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
- enable_kernel_fp();
gcm_ghash_p8(dctx->shash, ctx->htable,
dctx->buffer, GHASH_DIGEST_SIZE);
pagefault_enable();
@@ -164,9 +160,7 @@ static int p8_ghash_update(struct shash_desc *desc,
if (len) {
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
- enable_kernel_fp();
gcm_ghash_p8(dctx->shash, ctx->htable, src, len);
pagefault_enable();
preempt_enable();
@@ -195,9 +189,7 @@ static int p8_ghash_final(struct shash_desc *desc, u8 *out)
dctx->buffer[i] = 0;
preempt_disable();
pagefault_disable();
- enable_kernel_altivec();
enable_kernel_vsx();
- enable_kernel_fp();
gcm_ghash_p8(dctx->shash, ctx->htable,
dctx->buffer, GHASH_DIGEST_SIZE);
pagefault_enable();
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 12/19] powerpc: Create msr_check_and_{set,clear}()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (9 preceding siblings ...)
2015-10-28 0:50 ` [PATCH 11/19] crypto: vmx: Only call enable_kernel_vsx() Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 13/19] powerpc: Create disable_kernel_{fp,altivec,vsx,spe}() Anton Blanchard
` (6 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Create helper functions to set and clear MSR bits after first
checking if they are already set. Grouping them will make it
easy to avoid the MSR writes in a subsequent optimisation.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/kernel/process.c | 107 ++++++++++++++++++++----------------------
1 file changed, 52 insertions(+), 55 deletions(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index c39aa5a..5f244d0 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -87,23 +87,46 @@ static void check_if_tm_restore_required(struct task_struct *tsk)
static inline void check_if_tm_restore_required(struct task_struct *tsk) { }
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
-#ifdef CONFIG_PPC_FPU
-void giveup_fpu(struct task_struct *tsk)
+static void msr_check_and_set(unsigned long bits)
{
- u64 oldmsr = mfmsr();
- u64 newmsr;
+ unsigned long oldmsr = mfmsr();
+ unsigned long newmsr;
- check_if_tm_restore_required(tsk);
+ newmsr = oldmsr | bits;
- newmsr = oldmsr | MSR_FP;
#ifdef CONFIG_VSX
- if (cpu_has_feature(CPU_FTR_VSX))
+ if (cpu_has_feature(CPU_FTR_VSX) && (bits & MSR_FP))
newmsr |= MSR_VSX;
#endif
+
if (oldmsr != newmsr)
mtmsr_isync(newmsr);
+}
+
+static void msr_check_and_clear(unsigned long bits)
+{
+ unsigned long oldmsr = mfmsr();
+ unsigned long newmsr;
+
+ newmsr = oldmsr & ~bits;
+
+#ifdef CONFIG_VSX
+ if (cpu_has_feature(CPU_FTR_VSX) && (bits & MSR_FP))
+ newmsr &= ~MSR_VSX;
+#endif
+ if (oldmsr != newmsr)
+ mtmsr_isync(newmsr);
+}
+
+#ifdef CONFIG_PPC_FPU
+void giveup_fpu(struct task_struct *tsk)
+{
+ check_if_tm_restore_required(tsk);
+
+ msr_check_and_set(MSR_FP);
__giveup_fpu(tsk);
+ msr_check_and_clear(MSR_FP);
}
EXPORT_SYMBOL(giveup_fpu);
@@ -144,30 +167,21 @@ void enable_kernel_fp(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_FP)) {
- giveup_fpu(current);
- } else {
- u64 oldmsr = mfmsr();
+ msr_check_and_set(MSR_FP);
- if (!(oldmsr & MSR_FP))
- mtmsr_isync(oldmsr | MSR_FP);
- }
+ if (current->thread.regs && (current->thread.regs->msr & MSR_FP))
+ __giveup_fpu(current);
}
EXPORT_SYMBOL(enable_kernel_fp);
#ifdef CONFIG_ALTIVEC
void giveup_altivec(struct task_struct *tsk)
{
- u64 oldmsr = mfmsr();
- u64 newmsr;
-
check_if_tm_restore_required(tsk);
- newmsr = oldmsr | MSR_VEC;
- if (oldmsr != newmsr)
- mtmsr_isync(newmsr);
-
+ msr_check_and_set(MSR_VEC);
__giveup_altivec(tsk);
+ msr_check_and_clear(MSR_VEC);
}
EXPORT_SYMBOL(giveup_altivec);
@@ -175,14 +189,10 @@ void enable_kernel_altivec(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_VEC)) {
- giveup_altivec(current);
- } else {
- u64 oldmsr = mfmsr();
+ msr_check_and_set(MSR_VEC);
- if (!(oldmsr & MSR_VEC))
- mtmsr_isync(oldmsr | MSR_VEC);
- }
+ if (current->thread.regs && (current->thread.regs->msr & MSR_VEC))
+ __giveup_altivec(current);
}
EXPORT_SYMBOL(enable_kernel_altivec);
@@ -207,20 +217,15 @@ EXPORT_SYMBOL_GPL(flush_altivec_to_thread);
#ifdef CONFIG_VSX
void giveup_vsx(struct task_struct *tsk)
{
- u64 oldmsr = mfmsr();
- u64 newmsr;
-
check_if_tm_restore_required(tsk);
- newmsr = oldmsr | (MSR_FP|MSR_VEC|MSR_VSX);
- if (oldmsr != newmsr)
- mtmsr_isync(newmsr);
-
+ msr_check_and_set(MSR_FP|MSR_VEC|MSR_VSX);
if (tsk->thread.regs->msr & MSR_FP)
__giveup_fpu(tsk);
if (tsk->thread.regs->msr & MSR_VEC)
__giveup_altivec(tsk);
__giveup_vsx(tsk);
+ msr_check_and_clear(MSR_FP|MSR_VEC|MSR_VSX);
}
EXPORT_SYMBOL(giveup_vsx);
@@ -228,13 +233,14 @@ void enable_kernel_vsx(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_VSX)) {
- giveup_vsx(current);
- } else {
- u64 oldmsr = mfmsr();
+ msr_check_and_set(MSR_FP|MSR_VEC|MSR_VSX);
- if (!(oldmsr & MSR_VSX))
- mtmsr_isync(oldmsr | MSR_VSX);
+ if (current->thread.regs && (current->thread.regs->msr & MSR_VSX)) {
+ if (current->thread.regs->msr & MSR_FP)
+ __giveup_fpu(current);
+ if (current->thread.regs->msr & MSR_VEC)
+ __giveup_altivec(current);
+ __giveup_vsx(current);
}
}
EXPORT_SYMBOL(enable_kernel_vsx);
@@ -256,16 +262,11 @@ EXPORT_SYMBOL_GPL(flush_vsx_to_thread);
#ifdef CONFIG_SPE
void giveup_spe(struct task_struct *)
{
- u64 oldmsr = mfmsr();
- u64 newmsr;
-
check_if_tm_restore_required(tsk);
- newmsr = oldmsr | MSR_SPE;
- if (oldmsr != newmsr)
- mtmsr_isync(newmsr);
-
+ msr_check_and_set(MSR_SPE);
__giveup_spe(tsk);
+ msr_check_and_clear(MSR_SPE);
}
EXPORT_SYMBOL(giveup_spe);
@@ -273,14 +274,10 @@ void enable_kernel_spe(void)
{
WARN_ON(preemptible());
- if (current->thread.regs && (current->thread.regs->msr & MSR_SPE)) {
- giveup_spe(current);
- } else {
- u64 oldmsr = mfmsr();
+ msr_check_and_set(MSR_SPE);
- if (!(oldmsr & MSR_SPE))
- mtmsr_isync(oldmsr | MSR_SPE);
- }
+ if (current->thread.regs && (current->thread.regs->msr & MSR_SPE))
+ __giveup_spe(current);
}
EXPORT_SYMBOL(enable_kernel_spe);
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 13/19] powerpc: Create disable_kernel_{fp,altivec,vsx,spe}()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (10 preceding siblings ...)
2015-10-28 0:51 ` [PATCH 12/19] powerpc: Create msr_check_and_{set,clear}() Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 14/19] powerpc: Add ppc_strict_facility_enable boot option Anton Blanchard
` (5 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
The enable_kernel_*() functions leave the relevant MSR bits enabled
until we exit the kernel sometime later. Create disable versions
that wrap the kernel use of FP, Altivec VSX or SPE.
While we don't want to disable it normally for performance reasons
(MSR writes are slow), it will be used for a debug boot option that
does this and catches bad uses in other areas of the kernel.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/crypto/aes-spe-glue.c | 1 +
arch/powerpc/crypto/sha1-spe-glue.c | 1 +
arch/powerpc/crypto/sha256-spe-glue.c | 1 +
arch/powerpc/include/asm/switch_to.h | 5 +++++
arch/powerpc/kernel/align.c | 2 ++
arch/powerpc/kvm/book3s_paired_singles.c | 1 +
arch/powerpc/kvm/book3s_pr.c | 4 ++++
arch/powerpc/kvm/booke.c | 4 ++++
arch/powerpc/lib/vmx-helper.c | 2 ++
arch/powerpc/lib/xor_vmx.c | 4 ++++
drivers/crypto/vmx/aes.c | 3 +++
drivers/crypto/vmx/aes_cbc.c | 3 +++
drivers/crypto/vmx/aes_ctr.c | 3 +++
drivers/crypto/vmx/ghash.c | 4 ++++
lib/raid6/altivec.uc | 1 +
15 files changed, 39 insertions(+)
diff --git a/arch/powerpc/crypto/aes-spe-glue.c b/arch/powerpc/crypto/aes-spe-glue.c
index bd5e63f..93ee046 100644
--- a/arch/powerpc/crypto/aes-spe-glue.c
+++ b/arch/powerpc/crypto/aes-spe-glue.c
@@ -85,6 +85,7 @@ static void spe_begin(void)
static void spe_end(void)
{
+ disable_kernel_spe();
/* reenable preemption */
preempt_enable();
}
diff --git a/arch/powerpc/crypto/sha1-spe-glue.c b/arch/powerpc/crypto/sha1-spe-glue.c
index 3e1d222..f9ebc38 100644
--- a/arch/powerpc/crypto/sha1-spe-glue.c
+++ b/arch/powerpc/crypto/sha1-spe-glue.c
@@ -46,6 +46,7 @@ static void spe_begin(void)
static void spe_end(void)
{
+ disable_kernel_spe();
/* reenable preemption */
preempt_enable();
}
diff --git a/arch/powerpc/crypto/sha256-spe-glue.c b/arch/powerpc/crypto/sha256-spe-glue.c
index f4a616f..718a079 100644
--- a/arch/powerpc/crypto/sha256-spe-glue.c
+++ b/arch/powerpc/crypto/sha256-spe-glue.c
@@ -47,6 +47,7 @@ static void spe_begin(void)
static void spe_end(void)
{
+ disable_kernel_spe();
/* reenable preemption */
preempt_enable();
}
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 6bcfb89..dbc4caa 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -27,6 +27,11 @@ extern void giveup_spe(struct task_struct *);
extern void load_up_spe(struct task_struct *);
extern void switch_booke_debug_regs(struct debug_reg *new_debug);
+static inline void disable_kernel_fp(void) { }
+static inline void disable_kernel_altivec(void) { }
+static inline void disable_kernel_spe(void) { }
+static inline void disable_kernel_vsx(void) { }
+
#ifdef CONFIG_PPC_FPU
extern void flush_fp_to_thread(struct task_struct *);
extern void giveup_fpu(struct task_struct *);
diff --git a/arch/powerpc/kernel/align.c b/arch/powerpc/kernel/align.c
index 86150fb..8e7cb8e 100644
--- a/arch/powerpc/kernel/align.c
+++ b/arch/powerpc/kernel/align.c
@@ -960,6 +960,7 @@ int fix_alignment(struct pt_regs *regs)
preempt_disable();
enable_kernel_fp();
cvt_df(&data.dd, (float *)&data.x32.low32);
+ disable_kernel_fp();
preempt_enable();
#else
return 0;
@@ -1000,6 +1001,7 @@ int fix_alignment(struct pt_regs *regs)
preempt_disable();
enable_kernel_fp();
cvt_fd((float *)&data.x32.low32, &data.dd);
+ disable_kernel_fp();
preempt_enable();
#else
return 0;
diff --git a/arch/powerpc/kvm/book3s_paired_singles.c b/arch/powerpc/kvm/book3s_paired_singles.c
index a759d9a..eab96cf 100644
--- a/arch/powerpc/kvm/book3s_paired_singles.c
+++ b/arch/powerpc/kvm/book3s_paired_singles.c
@@ -1265,6 +1265,7 @@ int kvmppc_emulate_paired_single(struct kvm_run *run, struct kvm_vcpu *vcpu)
if (rcomp)
kvmppc_set_cr(vcpu, cr);
+ disable_kernel_fp();
preempt_enable();
return emulated;
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 64891b0..49f5dad 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -751,6 +751,7 @@ static int kvmppc_handle_ext(struct kvm_vcpu *vcpu, unsigned int exit_nr,
preempt_disable();
enable_kernel_fp();
load_fp_state(&vcpu->arch.fp);
+ disable_kernel_fp();
t->fp_save_area = &vcpu->arch.fp;
preempt_enable();
}
@@ -760,6 +761,7 @@ static int kvmppc_handle_ext(struct kvm_vcpu *vcpu, unsigned int exit_nr,
preempt_disable();
enable_kernel_altivec();
load_vr_state(&vcpu->arch.vr);
+ disable_kernel_altivec();
t->vr_save_area = &vcpu->arch.vr;
preempt_enable();
#endif
@@ -788,6 +790,7 @@ static void kvmppc_handle_lost_ext(struct kvm_vcpu *vcpu)
preempt_disable();
enable_kernel_fp();
load_fp_state(&vcpu->arch.fp);
+ disable_kernel_fp();
preempt_enable();
}
#ifdef CONFIG_ALTIVEC
@@ -795,6 +798,7 @@ static void kvmppc_handle_lost_ext(struct kvm_vcpu *vcpu)
preempt_disable();
enable_kernel_altivec();
load_vr_state(&vcpu->arch.vr);
+ disable_kernel_altivec();
preempt_enable();
}
#endif
diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index fd58751..778ef86 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -98,6 +98,7 @@ void kvmppc_vcpu_disable_spe(struct kvm_vcpu *vcpu)
preempt_disable();
enable_kernel_spe();
kvmppc_save_guest_spe(vcpu);
+ disable_kernel_spe();
vcpu->arch.shadow_msr &= ~MSR_SPE;
preempt_enable();
}
@@ -107,6 +108,7 @@ static void kvmppc_vcpu_enable_spe(struct kvm_vcpu *vcpu)
preempt_disable();
enable_kernel_spe();
kvmppc_load_guest_spe(vcpu);
+ disable_kernel_spe();
vcpu->arch.shadow_msr |= MSR_SPE;
preempt_enable();
}
@@ -141,6 +143,7 @@ static inline void kvmppc_load_guest_fp(struct kvm_vcpu *vcpu)
if (!(current->thread.regs->msr & MSR_FP)) {
enable_kernel_fp();
load_fp_state(&vcpu->arch.fp);
+ disable_kernel_fp();
current->thread.fp_save_area = &vcpu->arch.fp;
current->thread.regs->msr |= MSR_FP;
}
@@ -182,6 +185,7 @@ static inline void kvmppc_load_guest_altivec(struct kvm_vcpu *vcpu)
if (!(current->thread.regs->msr & MSR_VEC)) {
enable_kernel_altivec();
load_vr_state(&vcpu->arch.vr);
+ disable_kernel_altivec();
current->thread.vr_save_area = &vcpu->arch.vr;
current->thread.regs->msr |= MSR_VEC;
}
diff --git a/arch/powerpc/lib/vmx-helper.c b/arch/powerpc/lib/vmx-helper.c
index ac93a3b..b27e030 100644
--- a/arch/powerpc/lib/vmx-helper.c
+++ b/arch/powerpc/lib/vmx-helper.c
@@ -46,6 +46,7 @@ int enter_vmx_usercopy(void)
*/
int exit_vmx_usercopy(void)
{
+ disable_kernel_altivec();
pagefault_enable();
preempt_enable();
return 0;
@@ -70,6 +71,7 @@ int enter_vmx_copy(void)
*/
void *exit_vmx_copy(void *dest)
{
+ disable_kernel_altivec();
preempt_enable();
return dest;
}
diff --git a/arch/powerpc/lib/xor_vmx.c b/arch/powerpc/lib/xor_vmx.c
index e905f7c..07f49f1 100644
--- a/arch/powerpc/lib/xor_vmx.c
+++ b/arch/powerpc/lib/xor_vmx.c
@@ -74,6 +74,7 @@ void xor_altivec_2(unsigned long bytes, unsigned long *v1_in,
v2 += 4;
} while (--lines > 0);
+ disable_kernel_altivec();
preempt_enable();
}
EXPORT_SYMBOL(xor_altivec_2);
@@ -102,6 +103,7 @@ void xor_altivec_3(unsigned long bytes, unsigned long *v1_in,
v3 += 4;
} while (--lines > 0);
+ disable_kernel_altivec();
preempt_enable();
}
EXPORT_SYMBOL(xor_altivec_3);
@@ -135,6 +137,7 @@ void xor_altivec_4(unsigned long bytes, unsigned long *v1_in,
v4 += 4;
} while (--lines > 0);
+ disable_kernel_altivec();
preempt_enable();
}
EXPORT_SYMBOL(xor_altivec_4);
@@ -172,6 +175,7 @@ void xor_altivec_5(unsigned long bytes, unsigned long *v1_in,
v5 += 4;
} while (--lines > 0);
+ disable_kernel_altivec();
preempt_enable();
}
EXPORT_SYMBOL(xor_altivec_5);
diff --git a/drivers/crypto/vmx/aes.c b/drivers/crypto/vmx/aes.c
index 20539fb..022c7ab 100644
--- a/drivers/crypto/vmx/aes.c
+++ b/drivers/crypto/vmx/aes.c
@@ -86,6 +86,7 @@ static int p8_aes_setkey(struct crypto_tfm *tfm, const u8 *key,
enable_kernel_vsx();
ret = aes_p8_set_encrypt_key(key, keylen * 8, &ctx->enc_key);
ret += aes_p8_set_decrypt_key(key, keylen * 8, &ctx->dec_key);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
@@ -104,6 +105,7 @@ static void p8_aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src)
pagefault_disable();
enable_kernel_vsx();
aes_p8_encrypt(src, dst, &ctx->enc_key);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
}
@@ -120,6 +122,7 @@ static void p8_aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src)
pagefault_disable();
enable_kernel_vsx();
aes_p8_decrypt(src, dst, &ctx->dec_key);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
}
diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cbc.c
index 8847b92..1881b3f 100644
--- a/drivers/crypto/vmx/aes_cbc.c
+++ b/drivers/crypto/vmx/aes_cbc.c
@@ -87,6 +87,7 @@ static int p8_aes_cbc_setkey(struct crypto_tfm *tfm, const u8 *key,
enable_kernel_vsx();
ret = aes_p8_set_encrypt_key(key, keylen * 8, &ctx->enc_key);
ret += aes_p8_set_decrypt_key(key, keylen * 8, &ctx->dec_key);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
@@ -127,6 +128,7 @@ static int p8_aes_cbc_encrypt(struct blkcipher_desc *desc,
ret = blkcipher_walk_done(desc, &walk, nbytes);
}
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
}
@@ -167,6 +169,7 @@ static int p8_aes_cbc_decrypt(struct blkcipher_desc *desc,
ret = blkcipher_walk_done(desc, &walk, nbytes);
}
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
}
diff --git a/drivers/crypto/vmx/aes_ctr.c b/drivers/crypto/vmx/aes_ctr.c
index 8095866..2d58b18 100644
--- a/drivers/crypto/vmx/aes_ctr.c
+++ b/drivers/crypto/vmx/aes_ctr.c
@@ -83,6 +83,7 @@ static int p8_aes_ctr_setkey(struct crypto_tfm *tfm, const u8 *key,
pagefault_disable();
enable_kernel_vsx();
ret = aes_p8_set_encrypt_key(key, keylen * 8, &ctx->enc_key);
+ disable_kernel_vsx();
pagefault_enable();
ret += crypto_blkcipher_setkey(ctx->fallback, key, keylen);
@@ -101,6 +102,7 @@ static void p8_aes_ctr_final(struct p8_aes_ctr_ctx *ctx,
pagefault_disable();
enable_kernel_vsx();
aes_p8_encrypt(ctrblk, keystream, &ctx->enc_key);
+ disable_kernel_vsx();
pagefault_enable();
crypto_xor(keystream, src, nbytes);
@@ -139,6 +141,7 @@ static int p8_aes_ctr_crypt(struct blkcipher_desc *desc,
AES_BLOCK_SIZE,
&ctx->enc_key,
walk.iv);
+ disable_kernel_vsx();
pagefault_enable();
/* We need to update IV mostly for last bytes/round */
diff --git a/drivers/crypto/vmx/ghash.c b/drivers/crypto/vmx/ghash.c
index 1f4586c..6c999cb0 100644
--- a/drivers/crypto/vmx/ghash.c
+++ b/drivers/crypto/vmx/ghash.c
@@ -120,6 +120,7 @@ static int p8_ghash_setkey(struct crypto_shash *tfm, const u8 *key,
pagefault_disable();
enable_kernel_vsx();
gcm_init_p8(ctx->htable, (const u64 *) key);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
return crypto_shash_setkey(ctx->fallback, key, keylen);
@@ -150,6 +151,7 @@ static int p8_ghash_update(struct shash_desc *desc,
enable_kernel_vsx();
gcm_ghash_p8(dctx->shash, ctx->htable,
dctx->buffer, GHASH_DIGEST_SIZE);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
src += GHASH_DIGEST_SIZE - dctx->bytes;
@@ -162,6 +164,7 @@ static int p8_ghash_update(struct shash_desc *desc,
pagefault_disable();
enable_kernel_vsx();
gcm_ghash_p8(dctx->shash, ctx->htable, src, len);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
src += len;
@@ -192,6 +195,7 @@ static int p8_ghash_final(struct shash_desc *desc, u8 *out)
enable_kernel_vsx();
gcm_ghash_p8(dctx->shash, ctx->htable,
dctx->buffer, GHASH_DIGEST_SIZE);
+ disable_kernel_vsx();
pagefault_enable();
preempt_enable();
dctx->bytes = 0;
diff --git a/lib/raid6/altivec.uc b/lib/raid6/altivec.uc
index bec27fc..682aae8 100644
--- a/lib/raid6/altivec.uc
+++ b/lib/raid6/altivec.uc
@@ -101,6 +101,7 @@ static void raid6_altivec$#_gen_syndrome(int disks, size_t bytes, void **ptrs)
raid6_altivec$#_gen_syndrome_real(disks, bytes, ptrs);
+ disable_kernel_altivec();
preempt_enable();
}
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 14/19] powerpc: Add ppc_strict_facility_enable boot option
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (11 preceding siblings ...)
2015-10-28 0:51 ` [PATCH 13/19] powerpc: Create disable_kernel_{fp,altivec,vsx,spe}() Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 15/19] powerpc: Remove fp_enable() and vec_enable(), use msr_check_and_{set, clear}() Anton Blanchard
` (4 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Add a boot option that strictly manages the MSR unavailable bits.
This catches kernel uses of FP/Altivec/SPE that would otherwise
corrupt user state.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
Documentation/kernel-parameters.txt | 6 ++++++
arch/powerpc/include/asm/reg.h | 9 +++++++++
arch/powerpc/include/asm/switch_to.h | 24 +++++++++++++++++++-----
arch/powerpc/kernel/process.c | 17 +++++++++++++++--
4 files changed, 49 insertions(+), 7 deletions(-)
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 22a4b68..f535a02 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -2939,6 +2939,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
may be specified.
Format: <port>,<port>....
+ ppc_strict_facility_enable
+ [PPC] This option catches any kernel floating point,
+ Altivec, VSX and SPE outside of regions specifically
+ allowed (eg kernel_enable_fpu()/kernel_disable_fpu()).
+ There is some performance impact when enabling this.
+
print-fatal-signals=
[KNL] debug: print fatal signals
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index 987dac0..eb2986e 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -1214,6 +1214,15 @@ static inline void mtmsr_isync(unsigned long val)
: "r" ((unsigned long)(v)) \
: "memory")
+extern void msr_check_and_set(unsigned long bits);
+extern bool strict_msr_control;
+extern void __msr_check_and_clear(unsigned long bits);
+static inline void msr_check_and_clear(unsigned long bits)
+{
+ if (strict_msr_control)
+ __msr_check_and_clear(bits);
+}
+
static inline unsigned long mfvtb (void)
{
#ifdef CONFIG_PPC_BOOK3S_64
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index dbc4caa..b6a4951 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -4,6 +4,8 @@
#ifndef _ASM_POWERPC_SWITCH_TO_H
#define _ASM_POWERPC_SWITCH_TO_H
+#include <asm/reg.h>
+
struct thread_struct;
struct task_struct;
struct pt_regs;
@@ -27,15 +29,15 @@ extern void giveup_spe(struct task_struct *);
extern void load_up_spe(struct task_struct *);
extern void switch_booke_debug_regs(struct debug_reg *new_debug);
-static inline void disable_kernel_fp(void) { }
-static inline void disable_kernel_altivec(void) { }
-static inline void disable_kernel_spe(void) { }
-static inline void disable_kernel_vsx(void) { }
-
#ifdef CONFIG_PPC_FPU
extern void flush_fp_to_thread(struct task_struct *);
extern void giveup_fpu(struct task_struct *);
extern void __giveup_fpu(struct task_struct *);
+static inline void disable_kernel_fp(void)
+{
+ msr_check_and_clear(MSR_FP);
+}
+
#else
static inline void flush_fp_to_thread(struct task_struct *t) { }
static inline void giveup_fpu(struct task_struct *t) { }
@@ -46,6 +48,10 @@ static inline void __giveup_fpu(struct task_struct *t) { }
extern void flush_altivec_to_thread(struct task_struct *);
extern void giveup_altivec(struct task_struct *);
extern void __giveup_altivec(struct task_struct *);
+static inline void disable_kernel_altivec(void)
+{
+ msr_check_and_clear(MSR_VEC);
+}
#else
static inline void flush_altivec_to_thread(struct task_struct *t) { }
static inline void giveup_altivec(struct task_struct *t) { }
@@ -54,6 +60,10 @@ static inline void __giveup_altivec(struct task_struct *t) { }
#ifdef CONFIG_VSX
extern void flush_vsx_to_thread(struct task_struct *);
+static inline void disable_kernel_vsx(void)
+{
+ msr_check_and_clear(MSR_FP|MSR_VEC|MSR_VSX);
+}
#else
static inline void flush_vsx_to_thread(struct task_struct *t)
{
@@ -62,6 +72,10 @@ static inline void flush_vsx_to_thread(struct task_struct *t)
#ifdef CONFIG_SPE
extern void flush_spe_to_thread(struct task_struct *);
+static inline void disable_kernel_spe(void)
+{
+ msr_check_and_clear(MSR_SPE);
+}
#else
static inline void flush_spe_to_thread(struct task_struct *t)
{
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 5f244d0..878ea17 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -87,7 +87,19 @@ static void check_if_tm_restore_required(struct task_struct *tsk)
static inline void check_if_tm_restore_required(struct task_struct *tsk) { }
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
-static void msr_check_and_set(unsigned long bits)
+bool strict_msr_control;
+EXPORT_SYMBOL(strict_msr_control);
+
+static int __init enable_strict_msr_control(char *str)
+{
+ strict_msr_control = true;
+ pr_info("Enabling strict facility control\n");
+
+ return 0;
+}
+early_param("ppc_strict_facility_enable", enable_strict_msr_control);
+
+void msr_check_and_set(unsigned long bits)
{
unsigned long oldmsr = mfmsr();
unsigned long newmsr;
@@ -103,7 +115,7 @@ static void msr_check_and_set(unsigned long bits)
mtmsr_isync(newmsr);
}
-static void msr_check_and_clear(unsigned long bits)
+void __msr_check_and_clear(unsigned long bits)
{
unsigned long oldmsr = mfmsr();
unsigned long newmsr;
@@ -118,6 +130,7 @@ static void msr_check_and_clear(unsigned long bits)
if (oldmsr != newmsr)
mtmsr_isync(newmsr);
}
+EXPORT_SYMBOL(__msr_check_and_clear);
#ifdef CONFIG_PPC_FPU
void giveup_fpu(struct task_struct *tsk)
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 15/19] powerpc: Remove fp_enable() and vec_enable(), use msr_check_and_{set, clear}()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (12 preceding siblings ...)
2015-10-28 0:51 ` [PATCH 14/19] powerpc: Add ppc_strict_facility_enable boot option Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 16/19] powerpc: create giveup_all() Anton Blanchard
` (3 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
More consolidation of our MSR available bit handling.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/processor.h | 2 --
arch/powerpc/kernel/fpu.S | 16 ----------------
arch/powerpc/kernel/process.c | 6 ++++--
arch/powerpc/kernel/vector.S | 10 ----------
4 files changed, 4 insertions(+), 30 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
index a2e8918..ac23308 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -380,8 +380,6 @@ extern int set_endian(struct task_struct *tsk, unsigned int val);
extern int get_unalign_ctl(struct task_struct *tsk, unsigned long adr);
extern int set_unalign_ctl(struct task_struct *tsk, unsigned int val);
-extern void fp_enable(void);
-extern void vec_enable(void);
extern void load_fp_state(struct thread_fp_state *fp);
extern void store_fp_state(struct thread_fp_state *fp);
extern void load_vr_state(struct thread_vr_state *vr);
diff --git a/arch/powerpc/kernel/fpu.S b/arch/powerpc/kernel/fpu.S
index 431ab57..2117eac 100644
--- a/arch/powerpc/kernel/fpu.S
+++ b/arch/powerpc/kernel/fpu.S
@@ -77,22 +77,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
/*
- * Enable use of the FPU, and VSX if possible, for the caller.
- */
-_GLOBAL(fp_enable)
- mfmsr r3
- ori r3,r3,MSR_FP
-#ifdef CONFIG_VSX
-BEGIN_FTR_SECTION
- oris r3,r3,MSR_VSX@h
-END_FTR_SECTION_IFSET(CPU_FTR_VSX)
-#endif
- SYNC
- MTMSRD(r3)
- isync /* (not necessary for arch 2.02 and later) */
- blr
-
-/*
* Load state from memory into FP registers including FPSCR.
* Assumes the caller has enabled FP in the MSR.
*/
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 878ea17..cf2d811 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -732,13 +732,15 @@ void restore_tm_state(struct pt_regs *regs)
msr_diff = current->thread.ckpt_regs.msr & ~regs->msr;
msr_diff &= MSR_FP | MSR_VEC | MSR_VSX;
if (msr_diff & MSR_FP) {
- fp_enable();
+ msr_check_and_set(MSR_FP);
load_fp_state(¤t->thread.fp_state);
+ msr_check_and_clear(MSR_FP);
regs->msr |= current->thread.fpexc_mode;
}
if (msr_diff & MSR_VEC) {
- vec_enable();
+ msr_check_and_set(MSR_VEC);
load_vr_state(¤t->thread.vr_state);
+ msr_check_and_clear(MSR_VEC);
}
regs->msr |= msr_diff;
}
diff --git a/arch/powerpc/kernel/vector.S b/arch/powerpc/kernel/vector.S
index 98675b0..162d0f7 100644
--- a/arch/powerpc/kernel/vector.S
+++ b/arch/powerpc/kernel/vector.S
@@ -33,16 +33,6 @@ _GLOBAL(do_load_up_transact_altivec)
#endif
/*
- * Enable use of VMX/Altivec for the caller.
- */
-_GLOBAL(vec_enable)
- mfmsr r3
- oris r3,r3,MSR_VEC@h
- MTMSRD(r3)
- isync
- blr
-
-/*
* Load state from memory into VMX registers including VSCR.
* Assumes the caller has enabled VMX in the MSR.
*/
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 16/19] powerpc: create giveup_all()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (13 preceding siblings ...)
2015-10-28 0:51 ` [PATCH 15/19] powerpc: Remove fp_enable() and vec_enable(), use msr_check_and_{set, clear}() Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 17/19] powerpc: create flush_all_to_thread() Anton Blanchard
` (2 subsequent siblings)
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Create a single function that gives everything up (FP, VMX, VSX, SPE).
Doing this all at once means we only do one MSR write.
A context switch microbenchmark using yield():
http://ozlabs.org/~anton/junkcode/context_switch2.c
./context_switch2 --test=yield --fp --altivec --vector 0 0
shows an improvement of 3% on POWER8.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/switch_to.h | 1 +
arch/powerpc/kernel/process.c | 75 ++++++++++++++++++++++++++++--------
arch/powerpc/kvm/book3s_pr.c | 17 +-------
3 files changed, 63 insertions(+), 30 deletions(-)
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index b6a4951..0b4a339 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -27,6 +27,7 @@ extern void giveup_vsx(struct task_struct *);
extern void enable_kernel_spe(void);
extern void giveup_spe(struct task_struct *);
extern void load_up_spe(struct task_struct *);
+extern void giveup_all(struct task_struct *);
extern void switch_booke_debug_regs(struct debug_reg *new_debug);
#ifdef CONFIG_PPC_FPU
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index cf2d811..5ca416c 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -308,6 +308,64 @@ void flush_spe_to_thread(struct task_struct *tsk)
}
#endif /* CONFIG_SPE */
+static unsigned long msr_all_available;
+
+static int __init init_msr_all_available(void)
+{
+#ifdef CONFIG_PPC_FPU
+ msr_all_available |= MSR_FP;
+#endif
+#ifdef CONFIG_ALTIVEC
+ if (cpu_has_feature(CPU_FTR_ALTIVEC))
+ msr_all_available |= MSR_VEC;
+#endif
+#ifdef CONFIG_VSX
+ if (cpu_has_feature(CPU_FTR_VSX))
+ msr_all_available |= MSR_VSX;
+#endif
+#ifdef CONFIG_SPE
+ if (cpu_has_feature(CPU_FTR_SPE))
+ msr_all_available |= MSR_SPE;
+#endif
+
+ return 0;
+}
+early_initcall(init_msr_all_available);
+
+void giveup_all(struct task_struct *tsk)
+{
+ unsigned long usermsr;
+
+ if (!tsk->thread.regs)
+ return;
+
+ usermsr = tsk->thread.regs->msr;
+
+ if ((usermsr & msr_all_available) == 0)
+ return;
+
+ msr_check_and_set(msr_all_available);
+
+#ifdef CONFIG_PPC_FPU
+ if (usermsr & MSR_FP)
+ __giveup_fpu(tsk);
+#endif
+#ifdef CONFIG_ALTIVEC
+ if (usermsr & MSR_VEC)
+ __giveup_altivec(tsk);
+#endif
+#ifdef CONFIG_VSX
+ if (usermsr & MSR_VSX)
+ __giveup_vsx(tsk);
+#endif
+#ifdef CONFIG_SPE
+ if (usermsr & MSR_SPE)
+ __giveup_spe(tsk);
+#endif
+
+ msr_check_and_clear(msr_all_available);
+}
+
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
void do_send_trap(struct pt_regs *regs, unsigned long address,
unsigned long error_code, int signal_code, int breakpt)
@@ -839,21 +897,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
__switch_to_tm(prev);
- if (prev->thread.regs && (prev->thread.regs->msr & MSR_FP))
- giveup_fpu(prev);
-#ifdef CONFIG_ALTIVEC
- if (prev->thread.regs && (prev->thread.regs->msr & MSR_VEC))
- giveup_altivec(prev);
-#endif /* CONFIG_ALTIVEC */
-#ifdef CONFIG_VSX
- if (prev->thread.regs && (prev->thread.regs->msr & MSR_VSX))
- /* VMX and FPU registers are already save here */
- __giveup_vsx(prev);
-#endif /* CONFIG_VSX */
-#ifdef CONFIG_SPE
- if ((prev->thread.regs && (prev->thread.regs->msr & MSR_SPE)))
- giveup_spe(prev);
-#endif /* CONFIG_SPE */
+ /* Save FPU, Altivec, VSX and SPE state */
+ giveup_all(prev);
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
switch_booke_debug_regs(&new->thread.debug);
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 49f5dad..a78e0e6 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -1490,21 +1490,8 @@ static int kvmppc_vcpu_run_pr(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
goto out;
/* interrupts now hard-disabled */
- /* Save FPU state in thread_struct */
- if (current->thread.regs->msr & MSR_FP)
- giveup_fpu(current);
-
-#ifdef CONFIG_ALTIVEC
- /* Save Altivec state in thread_struct */
- if (current->thread.regs->msr & MSR_VEC)
- giveup_altivec(current);
-#endif
-
-#ifdef CONFIG_VSX
- /* Save VSX state in thread_struct */
- if (current->thread.regs->msr & MSR_VSX)
- __giveup_vsx(current);
-#endif
+ /* Save FPU, Altivec and VSX state */
+ giveup_all(current);
/* Preload FPU if it's enabled */
if (kvmppc_get_msr(vcpu) & MSR_FP)
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 17/19] powerpc: create flush_all_to_thread()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (14 preceding siblings ...)
2015-10-28 0:51 ` [PATCH 16/19] powerpc: create giveup_all() Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 18/19] powerpc: Rearrange __switch_to() Anton Blanchard
2015-10-28 0:51 ` [PATCH 19/19] powerpc: clean up asm/switch_to.h Anton Blanchard
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Create a single function that flushes everything (FP, VMX, VSX, SPE).
Doing this all at once means we only do one MSR write.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/switch_to.h | 1 +
arch/powerpc/kernel/process.c | 22 ++++++++++++++++++----
arch/powerpc/kvm/book3s_hv.c | 5 ++---
3 files changed, 21 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 0b4a339..08e2d65 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -28,6 +28,7 @@ extern void enable_kernel_spe(void);
extern void giveup_spe(struct task_struct *);
extern void load_up_spe(struct task_struct *);
extern void giveup_all(struct task_struct *);
+extern void flush_all_to_thread(struct task_struct *);
extern void switch_booke_debug_regs(struct debug_reg *new_debug);
#ifdef CONFIG_PPC_FPU
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 5ca416c..875ab50 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -366,6 +366,23 @@ void giveup_all(struct task_struct *tsk)
msr_check_and_clear(msr_all_available);
}
+void flush_all_to_thread(struct task_struct *tsk)
+{
+ if (tsk->thread.regs) {
+ preempt_disable();
+ BUG_ON(tsk != current);
+ giveup_all(tsk);
+
+#ifdef CONFIG_SPE
+ if (tsk->thread.regs->msr & MSR_SPE)
+ tsk->thread.spefscr = mfspr(SPRN_SPEFSCR);
+#endif
+
+ preempt_enable();
+ }
+}
+EXPORT_SYMBOL(flush_all_to_thread);
+
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
void do_send_trap(struct pt_regs *regs, unsigned long address,
unsigned long error_code, int signal_code, int breakpt)
@@ -1136,10 +1153,7 @@ release_thread(struct task_struct *t)
*/
int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
{
- flush_fp_to_thread(src);
- flush_altivec_to_thread(src);
- flush_vsx_to_thread(src);
- flush_spe_to_thread(src);
+ flush_all_to_thread(src);
/*
* Flush TM state out so we can copy it. __switch_to_tm() does this
* flush but it removes the checkpointed state from the current CPU and
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 9c26c5a..c165f4a 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2700,9 +2700,8 @@ static int kvmppc_vcpu_run_hv(struct kvm_run *run, struct kvm_vcpu *vcpu)
goto out;
}
- flush_fp_to_thread(current);
- flush_altivec_to_thread(current);
- flush_vsx_to_thread(current);
+ flush_all_to_thread(current);
+
vcpu->arch.wqp = &vcpu->arch.vcore->wq;
vcpu->arch.pgdir = current->mm->pgd;
vcpu->arch.state = KVMPPC_VCPU_BUSY_IN_HOST;
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 18/19] powerpc: Rearrange __switch_to()
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (15 preceding siblings ...)
2015-10-28 0:51 ` [PATCH 17/19] powerpc: create flush_all_to_thread() Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 0:51 ` [PATCH 19/19] powerpc: clean up asm/switch_to.h Anton Blanchard
17 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Most of __switch_to() is housekeeping, TLB batching, timekeeping etc.
Move these away from the more complex and critical context switching
code.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/kernel/process.c | 52 +++++++++++++++++++++----------------------
1 file changed, 26 insertions(+), 26 deletions(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 875ab50..398f7bf 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -906,30 +906,6 @@ struct task_struct *__switch_to(struct task_struct *prev,
WARN_ON(!irqs_disabled());
- /*
- * We need to save SPRs before treclaim/trecheckpoint as these will
- * change a number of them.
- */
- save_sprs(&prev->thread);
-
- __switch_to_tm(prev);
-
- /* Save FPU, Altivec, VSX and SPE state */
- giveup_all(prev);
-
-#ifdef CONFIG_PPC_ADV_DEBUG_REGS
- switch_booke_debug_regs(&new->thread.debug);
-#else
-/*
- * For PPC_BOOK3S_64, we use the hw-breakpoint interfaces that would
- * schedule DABR
- */
-#ifndef CONFIG_HAVE_HW_BREAKPOINT
- if (unlikely(!hw_brk_match(this_cpu_ptr(¤t_brk), &new->thread.hw_brk)))
- __set_breakpoint(&new->thread.hw_brk);
-#endif /* CONFIG_HAVE_HW_BREAKPOINT */
-#endif
-
#ifdef CONFIG_PPC64
/*
* Collect processor utilization data per process
@@ -954,6 +930,30 @@ struct task_struct *__switch_to(struct task_struct *prev,
}
#endif /* CONFIG_PPC_BOOK3S_64 */
+#ifdef CONFIG_PPC_ADV_DEBUG_REGS
+ switch_booke_debug_regs(&new->thread.debug);
+#else
+/*
+ * For PPC_BOOK3S_64, we use the hw-breakpoint interfaces that would
+ * schedule DABR
+ */
+#ifndef CONFIG_HAVE_HW_BREAKPOINT
+ if (unlikely(!hw_brk_match(this_cpu_ptr(¤t_brk), &new->thread.hw_brk)))
+ __set_breakpoint(&new->thread.hw_brk);
+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
+#endif
+
+ /*
+ * We need to save SPRs before treclaim/trecheckpoint as these will
+ * change a number of them.
+ */
+ save_sprs(&prev->thread);
+
+ __switch_to_tm(prev);
+
+ /* Save FPU, Altivec, VSX and SPE state */
+ giveup_all(prev);
+
/*
* We can't take a PMU exception inside _switch() since there is a
* window where the kernel stack SLB and the kernel stack are out
@@ -969,6 +969,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
old_thread = &last->thread;
new_thread = ¤t->thread;
+ restore_sprs(old_thread, new_thread);
+
#ifdef CONFIG_PPC_BOOK3S_64
if (current_thread_info()->local_flags & _TLF_LAZY_MMU) {
current_thread_info()->local_flags &= ~_TLF_LAZY_MMU;
@@ -977,8 +979,6 @@ struct task_struct *__switch_to(struct task_struct *prev,
}
#endif /* CONFIG_PPC_BOOK3S_64 */
- restore_sprs(old_thread, new_thread);
-
return last;
}
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 19/19] powerpc: clean up asm/switch_to.h
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
` (16 preceding siblings ...)
2015-10-28 0:51 ` [PATCH 18/19] powerpc: Rearrange __switch_to() Anton Blanchard
@ 2015-10-28 0:51 ` Anton Blanchard
2015-10-28 3:19 ` kbuild test robot
2015-10-28 4:24 ` kbuild test robot
17 siblings, 2 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-28 0:51 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur; +Cc: linuxppc-dev
Remove a bunch of unnecessary fallback functions and group
things in a more logical way.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/switch_to.h | 39 ++++++++++--------------------------
1 file changed, 11 insertions(+), 28 deletions(-)
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 08e2d65..171ce13 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -14,24 +14,18 @@ extern struct task_struct *__switch_to(struct task_struct *,
struct task_struct *);
#define switch_to(prev, next, last) ((last) = __switch_to((prev), (next)))
-struct thread_struct;
extern struct task_struct *_switch(struct thread_struct *prev,
struct thread_struct *next);
-extern void enable_kernel_fp(void);
-extern void enable_kernel_altivec(void);
-extern void enable_kernel_vsx(void);
+extern void switch_booke_debug_regs(struct debug_reg *new_debug);
+
extern int emulate_altivec(struct pt_regs *);
-extern void __giveup_vsx(struct task_struct *);
-extern void giveup_vsx(struct task_struct *);
-extern void enable_kernel_spe(void);
-extern void giveup_spe(struct task_struct *);
-extern void load_up_spe(struct task_struct *);
-extern void giveup_all(struct task_struct *);
+
extern void flush_all_to_thread(struct task_struct *);
-extern void switch_booke_debug_regs(struct debug_reg *new_debug);
+extern void giveup_all(struct task_struct *);
#ifdef CONFIG_PPC_FPU
+extern void enable_kernel_fp(void);
extern void flush_fp_to_thread(struct task_struct *);
extern void giveup_fpu(struct task_struct *);
extern void __giveup_fpu(struct task_struct *);
@@ -39,14 +33,10 @@ static inline void disable_kernel_fp(void)
{
msr_check_and_clear(MSR_FP);
}
-
-#else
-static inline void flush_fp_to_thread(struct task_struct *t) { }
-static inline void giveup_fpu(struct task_struct *t) { }
-static inline void __giveup_fpu(struct task_struct *t) { }
#endif
#ifdef CONFIG_ALTIVEC
+extern void enable_kernel_altivec(void);
extern void flush_altivec_to_thread(struct task_struct *);
extern void giveup_altivec(struct task_struct *);
extern void __giveup_altivec(struct task_struct *);
@@ -54,34 +44,27 @@ static inline void disable_kernel_altivec(void)
{
msr_check_and_clear(MSR_VEC);
}
-#else
-static inline void flush_altivec_to_thread(struct task_struct *t) { }
-static inline void giveup_altivec(struct task_struct *t) { }
-static inline void __giveup_altivec(struct task_struct *t) { }
#endif
#ifdef CONFIG_VSX
+extern void enable_kernel_vsx(void);
extern void flush_vsx_to_thread(struct task_struct *);
+extern void giveup_vsx(struct task_struct *);
+extern void __giveup_vsx(struct task_struct *);
static inline void disable_kernel_vsx(void)
{
msr_check_and_clear(MSR_FP|MSR_VEC|MSR_VSX);
}
-#else
-static inline void flush_vsx_to_thread(struct task_struct *t)
-{
-}
#endif
#ifdef CONFIG_SPE
+extern void enable_kernel_spe(void);
extern void flush_spe_to_thread(struct task_struct *);
+extern void giveup_spe(struct task_struct *);
static inline void disable_kernel_spe(void)
{
msr_check_and_clear(MSR_SPE);
}
-#else
-static inline void flush_spe_to_thread(struct task_struct *t)
-{
-}
#endif
static inline void clear_task_ebb(struct task_struct *t)
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH 09/19] powerpc: Move part of giveup_fpu, altivec, spe into c
2015-10-28 0:50 ` [PATCH 09/19] powerpc: Move part of giveup_fpu,altivec,spe into c Anton Blanchard
@ 2015-10-28 3:17 ` kbuild test robot
0 siblings, 0 replies; 23+ messages in thread
From: kbuild test robot @ 2015-10-28 3:17 UTC (permalink / raw)
To: Anton Blanchard
Cc: kbuild-all, benh, paulus, mpe, mikey, cyrilbur, linuxppc-dev
[-- Attachment #1: Type: text/plain, Size: 2028 bytes --]
Hi Anton,
[auto build test ERROR on powerpc/next -- if it's inappropriate base, please suggest rules for selecting the more suitable base]
url: https://github.com/0day-ci/linux/commits/Anton-Blanchard/powerpc-Don-t-disable-kernel-FP-VMX-VSX-MSR-bits-on-context-switch/20151028-091736
config: powerpc-mpc85xx_cds_defconfig (attached as .config)
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=powerpc
All errors (new ones prefixed by >>):
arch/powerpc/kernel/process.c: In function 'giveup_spe':
>> arch/powerpc/kernel/process.c:247:24: error: parameter name omitted
void giveup_spe(struct task_struct *)
^
>> arch/powerpc/kernel/process.c:252:31: error: 'tsk' undeclared (first use in this function)
check_if_tm_restore_required(tsk);
^
arch/powerpc/kernel/process.c:252:31: note: each undeclared identifier is reported only once for each function it appears in
>> arch/powerpc/kernel/process.c:258:2: error: implicit declaration of function '__giveup_spe' [-Werror=implicit-function-declaration]
__giveup_spe(tsk);
^
cc1: all warnings being treated as errors
vim +247 arch/powerpc/kernel/process.c
241 }
242 }
243 EXPORT_SYMBOL_GPL(flush_vsx_to_thread);
244 #endif /* CONFIG_VSX */
245
246 #ifdef CONFIG_SPE
> 247 void giveup_spe(struct task_struct *)
248 {
249 u64 oldmsr = mfmsr();
250 u64 newmsr;
251
> 252 check_if_tm_restore_required(tsk);
253
254 newmsr = oldmsr | MSR_SPE;
255 if (oldmsr != newmsr)
256 mtmsr_isync(newmsr);
257
> 258 __giveup_spe(tsk);
259 }
260 EXPORT_SYMBOL(giveup_spe);
261
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 11378 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 19/19] powerpc: clean up asm/switch_to.h
2015-10-28 0:51 ` [PATCH 19/19] powerpc: clean up asm/switch_to.h Anton Blanchard
@ 2015-10-28 3:19 ` kbuild test robot
2015-10-28 4:24 ` kbuild test robot
1 sibling, 0 replies; 23+ messages in thread
From: kbuild test robot @ 2015-10-28 3:19 UTC (permalink / raw)
To: Anton Blanchard
Cc: kbuild-all, benh, paulus, mpe, mikey, cyrilbur, linuxppc-dev
[-- Attachment #1: Type: text/plain, Size: 3148 bytes --]
Hi Anton,
[auto build test ERROR on powerpc/next -- if it's inappropriate base, please suggest rules for selecting the more suitable base]
url: https://github.com/0day-ci/linux/commits/Anton-Blanchard/powerpc-Don-t-disable-kernel-FP-VMX-VSX-MSR-bits-on-context-switch/20151028-091736
config: powerpc-virtex_defconfig (attached as .config)
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=powerpc
All errors (new ones prefixed by >>):
arch/powerpc/kernel/ptrace.c: In function 'fpr_get':
>> arch/powerpc/kernel/ptrace.c:368:2: error: implicit declaration of function 'flush_fp_to_thread' [-Werror=implicit-function-declaration]
flush_fp_to_thread(target);
^
cc1: all warnings being treated as errors
--
arch/powerpc/kernel/align.c: In function 'fix_alignment':
>> arch/powerpc/kernel/align.c:908:3: error: implicit declaration of function 'flush_fp_to_thread' [-Werror=implicit-function-declaration]
flush_fp_to_thread(current);
^
cc1: all warnings being treated as errors
--
arch/powerpc/kernel/signal_32.c: In function 'save_user_regs':
>> arch/powerpc/kernel/signal_32.c:415:2: error: implicit declaration of function 'flush_fp_to_thread' [-Werror=implicit-function-declaration]
flush_fp_to_thread(current);
^
cc1: all warnings being treated as errors
--
arch/powerpc/kernel/process.c: In function 'enable_kernel_fp':
>> arch/powerpc/kernel/process.c:186:3: error: implicit declaration of function '__giveup_fpu' [-Werror=implicit-function-declaration]
__giveup_fpu(current);
^
cc1: all warnings being treated as errors
--
arch/powerpc/kernel/traps.c: In function 'parse_fpe':
>> arch/powerpc/kernel/traps.c:860:2: error: implicit declaration of function 'flush_fp_to_thread' [-Werror=implicit-function-declaration]
flush_fp_to_thread(current);
^
cc1: all warnings being treated as errors
vim +/flush_fp_to_thread +368 arch/powerpc/kernel/ptrace.c
f65255e8 Roland McGrath 2007-12-20 362 void *kbuf, void __user *ubuf)
f65255e8 Roland McGrath 2007-12-20 363 {
c6e6771b Michael Neuling 2008-06-25 364 #ifdef CONFIG_VSX
de79f7b9 Paul Mackerras 2013-09-10 365 u64 buf[33];
c6e6771b Michael Neuling 2008-06-25 366 int i;
c6e6771b Michael Neuling 2008-06-25 367 #endif
f65255e8 Roland McGrath 2007-12-20 @368 flush_fp_to_thread(target);
f65255e8 Roland McGrath 2007-12-20 369
c6e6771b Michael Neuling 2008-06-25 370 #ifdef CONFIG_VSX
c6e6771b Michael Neuling 2008-06-25 371 /* copy to local buffer then write that out */
:::::: The code at line 368 was first introduced by commit
:::::: f65255e8d51ecbc6c9eef20d39e0377d19b658ca [POWERPC] Use user_regset accessors for FP regs
:::::: TO: Roland McGrath <roland@redhat.com>
:::::: CC: Paul Mackerras <paulus@samba.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 12779 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 19/19] powerpc: clean up asm/switch_to.h
2015-10-28 0:51 ` [PATCH 19/19] powerpc: clean up asm/switch_to.h Anton Blanchard
2015-10-28 3:19 ` kbuild test robot
@ 2015-10-28 4:24 ` kbuild test robot
1 sibling, 0 replies; 23+ messages in thread
From: kbuild test robot @ 2015-10-28 4:24 UTC (permalink / raw)
To: Anton Blanchard
Cc: kbuild-all, benh, paulus, mpe, mikey, cyrilbur, linuxppc-dev
[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]
Hi Anton,
[auto build test ERROR on powerpc/next -- if it's inappropriate base, please suggest rules for selecting the more suitable base]
url: https://github.com/0day-ci/linux/commits/Anton-Blanchard/powerpc-Don-t-disable-kernel-FP-VMX-VSX-MSR-bits-on-context-switch/20151028-091736
config: powerpc-ppc6xx_defconfig (attached as .config)
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=powerpc
All errors (new ones prefixed by >>):
arch/powerpc/kernel/swsusp.c: In function 'save_processor_state':
>> arch/powerpc/kernel/swsusp.c:25:2: error: implicit declaration of function 'flush_spe_to_thread' [-Werror=implicit-function-declaration]
flush_spe_to_thread(current);
^
cc1: all warnings being treated as errors
vim +/flush_spe_to_thread +25 arch/powerpc/kernel/swsusp.c
be9c94dd Johannes Berg 2007-04-30 19 /*
be9c94dd Johannes Berg 2007-04-30 20 * flush out all the special registers so we don't need
be9c94dd Johannes Berg 2007-04-30 21 * to save them in the snapshot
be9c94dd Johannes Berg 2007-04-30 22 */
be9c94dd Johannes Berg 2007-04-30 23 flush_fp_to_thread(current);
be9c94dd Johannes Berg 2007-04-30 24 flush_altivec_to_thread(current);
be9c94dd Johannes Berg 2007-04-30 @25 flush_spe_to_thread(current);
543b9fd3 Johannes Berg 2007-05-03 26
543b9fd3 Johannes Berg 2007-05-03 27 #ifdef CONFIG_PPC64
543b9fd3 Johannes Berg 2007-05-03 28 hard_irq_disable();
:::::: The code at line 25 was first introduced by commit
:::::: be9c94dd7776467813419f49fabe8017bc2d4c81 [POWERPC] Fix suspend states again
:::::: TO: Johannes Berg <johannes@sipsolutions.net>
:::::: CC: Paul Mackerras <paulus@samba.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 27223 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 17/19] powerpc: create flush_all_to_thread()
2015-10-29 0:43 [PATCH 00/19] Context switch improvements Anton Blanchard
@ 2015-10-29 0:44 ` Anton Blanchard
0 siblings, 0 replies; 23+ messages in thread
From: Anton Blanchard @ 2015-10-29 0:44 UTC (permalink / raw)
To: benh, paulus, mpe, mikey, cyrilbur, scottwood; +Cc: linuxppc-dev
Create a single function that flushes everything (FP, VMX, VSX, SPE).
Doing this all at once means we only do one MSR write.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
arch/powerpc/include/asm/switch_to.h | 1 +
arch/powerpc/kernel/process.c | 22 ++++++++++++++++++----
arch/powerpc/kernel/swsusp.c | 4 +---
arch/powerpc/kvm/book3s_hv.c | 5 ++---
4 files changed, 22 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 8f85678..81d46a4 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -27,6 +27,7 @@ extern void giveup_vsx(struct task_struct *);
extern void enable_kernel_spe(void);
extern void load_up_spe(struct task_struct *);
extern void giveup_all(struct task_struct *);
+extern void flush_all_to_thread(struct task_struct *);
extern void switch_booke_debug_regs(struct debug_reg *new_debug);
#ifdef CONFIG_PPC_FPU
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index a1540ee..e66d625 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -366,6 +366,23 @@ void giveup_all(struct task_struct *tsk)
msr_check_and_clear(msr_all_available);
}
+void flush_all_to_thread(struct task_struct *tsk)
+{
+ if (tsk->thread.regs) {
+ preempt_disable();
+ BUG_ON(tsk != current);
+ giveup_all(tsk);
+
+#ifdef CONFIG_SPE
+ if (tsk->thread.regs->msr & MSR_SPE)
+ tsk->thread.spefscr = mfspr(SPRN_SPEFSCR);
+#endif
+
+ preempt_enable();
+ }
+}
+EXPORT_SYMBOL(flush_all_to_thread);
+
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
void do_send_trap(struct pt_regs *regs, unsigned long address,
unsigned long error_code, int signal_code, int breakpt)
@@ -1136,10 +1153,7 @@ release_thread(struct task_struct *t)
*/
int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
{
- flush_fp_to_thread(src);
- flush_altivec_to_thread(src);
- flush_vsx_to_thread(src);
- flush_spe_to_thread(src);
+ flush_all_to_thread(src);
/*
* Flush TM state out so we can copy it. __switch_to_tm() does this
* flush but it removes the checkpointed state from the current CPU and
diff --git a/arch/powerpc/kernel/swsusp.c b/arch/powerpc/kernel/swsusp.c
index eae33e1..6669b17 100644
--- a/arch/powerpc/kernel/swsusp.c
+++ b/arch/powerpc/kernel/swsusp.c
@@ -20,9 +20,7 @@ void save_processor_state(void)
* flush out all the special registers so we don't need
* to save them in the snapshot
*/
- flush_fp_to_thread(current);
- flush_altivec_to_thread(current);
- flush_spe_to_thread(current);
+ flush_all_to_thread(current);
#ifdef CONFIG_PPC64
hard_irq_disable();
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 9c26c5a..c165f4a 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2700,9 +2700,8 @@ static int kvmppc_vcpu_run_hv(struct kvm_run *run, struct kvm_vcpu *vcpu)
goto out;
}
- flush_fp_to_thread(current);
- flush_altivec_to_thread(current);
- flush_vsx_to_thread(current);
+ flush_all_to_thread(current);
+
vcpu->arch.wqp = &vcpu->arch.vcore->wq;
vcpu->arch.pgdir = current->mm->pgd;
vcpu->arch.state = KVMPPC_VCPU_BUSY_IN_HOST;
--
2.5.0
^ permalink raw reply related [flat|nested] 23+ messages in thread
end of thread, other threads:[~2015-10-29 1:06 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-10-28 0:50 [PATCH 01/19] powerpc: Don't disable kernel FP/VMX/VSX MSR bits on context switch Anton Blanchard
2015-10-28 0:50 ` [PATCH 02/19] powerpc: Don't disable MSR bits in do_load_up_transact_*() functions Anton Blanchard
2015-10-28 0:50 ` [PATCH 03/19] powerpc: Create context switch helpers save_sprs() and restore_sprs() Anton Blanchard
2015-10-28 0:50 ` [PATCH 04/19] powerpc: Remove redundant mflr in _switch Anton Blanchard
2015-10-28 0:50 ` [PATCH 05/19] powerpc: Remove UP only lazy floating point and vector optimisations Anton Blanchard
2015-10-28 0:50 ` [PATCH 06/19] powerpc: Simplify TM restore checks Anton Blanchard
2015-10-28 0:50 ` [PATCH 07/19] powerpc: Create mtmsrd_isync() Anton Blanchard
2015-10-28 0:50 ` [PATCH 08/19] powerpc: Remove NULL task struct pointer checks in FP and vector code Anton Blanchard
2015-10-28 0:50 ` [PATCH 09/19] powerpc: Move part of giveup_fpu,altivec,spe into c Anton Blanchard
2015-10-28 3:17 ` [PATCH 09/19] powerpc: Move part of giveup_fpu, altivec, spe " kbuild test robot
2015-10-28 0:50 ` [PATCH 10/19] powerpc: Move part of giveup_vsx " Anton Blanchard
2015-10-28 0:50 ` [PATCH 11/19] crypto: vmx: Only call enable_kernel_vsx() Anton Blanchard
2015-10-28 0:51 ` [PATCH 12/19] powerpc: Create msr_check_and_{set,clear}() Anton Blanchard
2015-10-28 0:51 ` [PATCH 13/19] powerpc: Create disable_kernel_{fp,altivec,vsx,spe}() Anton Blanchard
2015-10-28 0:51 ` [PATCH 14/19] powerpc: Add ppc_strict_facility_enable boot option Anton Blanchard
2015-10-28 0:51 ` [PATCH 15/19] powerpc: Remove fp_enable() and vec_enable(), use msr_check_and_{set, clear}() Anton Blanchard
2015-10-28 0:51 ` [PATCH 16/19] powerpc: create giveup_all() Anton Blanchard
2015-10-28 0:51 ` [PATCH 17/19] powerpc: create flush_all_to_thread() Anton Blanchard
2015-10-28 0:51 ` [PATCH 18/19] powerpc: Rearrange __switch_to() Anton Blanchard
2015-10-28 0:51 ` [PATCH 19/19] powerpc: clean up asm/switch_to.h Anton Blanchard
2015-10-28 3:19 ` kbuild test robot
2015-10-28 4:24 ` kbuild test robot
2015-10-29 0:43 [PATCH 00/19] Context switch improvements Anton Blanchard
2015-10-29 0:44 ` [PATCH 17/19] powerpc: create flush_all_to_thread() Anton Blanchard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).