[PATCH 0/3] CR4 fixes (some are urgent)

[PATCH 0/3] CR4 fixes (some are urgent)

[Andy Lutomirski]

Hi Ingo, etc:

Patches 1 and 2 fix boot regressions.  Patch 1 needs to be
backported.  Patch 2 doesn't because the regression is new in 4.8
(although it's probably unobservable because the regression fixed in
patch 1 also prevented bootup on affected CPUs).

Patch 3 cleans the whole mess up.  It's probably not 4.8 material,
but I'm sending it as part of this series for ease of review.  If
you like, you could apply patches 1 and 2 to x86/urgent, merge into
x86/asm, and apply patch 3 on top.

Andy Lutomirski (3):
  x86/init: Fix cr4_init_shadow() on CR4-less machines
  x86/boot: Fix another __read_cr4() case on 486
  x86/asm: Get rid of __read_cr4_safe()

 arch/x86/include/asm/paravirt.h       |  4 ----
 arch/x86/include/asm/paravirt_types.h |  1 -
 arch/x86/include/asm/special_insns.h  | 22 +++++++---------------
 arch/x86/kernel/paravirt.c            |  1 -
 arch/x86/kernel/process_32.c          |  2 +-
 arch/x86/kernel/setup.c               |  9 +++++++--
 arch/x86/power/cpu.c                  |  2 +-
 arch/x86/xen/enlighten.c              |  1 -
 8 files changed, 16 insertions(+), 26 deletions(-)

-- 
2.7.4

[PATCH 1/3] x86/init: Fix cr4_init_shadow() on CR4-less machines

[Andy Lutomirski]

cr4_init_shadow() will panic on 486-like machines without CR4.  Fix
it using __read_cr4_safe().

Cc: stable@vger.kernel.org
Reported-by: david@saggiorato.net
Fixes: 1e02ce4cccdc ("x86: Store a per-cpu shadow copy of CR4")
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/include/asm/tlbflush.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 6fa85944af83..dee8a70382ba 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
 /* Initialize cr4 shadow for this CPU. */
 static inline void cr4_init_shadow(void)
 {
-	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
+	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
 }
 
 /* Set in this cpu's CR4. */
-- 
2.7.4

[PATCH 2/3] x86/boot: Fix another __read_cr4() case on 486

[Andy Lutomirski]

The condition for reading CR4 was wrong: there are some CPUs with
CPUID but not CR4.  Rather than trying to make the condition exact,
using __read_cr4_safe().

Reported-by: david@saggiorato.net
Fixes: 18bc7bd523e0 ("x86/boot: Synchronize trampoline_cr4_features and mmu_cr4_features directly")
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/kernel/setup.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 0fa60f5f5a16..5930a4d191b4 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1137,9 +1137,13 @@ void __init setup_arch(char **cmdline_p)
 	 * auditing all the early-boot CR4 manipulation would be needed to
 	 * rule it out.
 	 */
-	if (boot_cpu_data.cpuid_level >= 0)
-		/* A CPU has %cr4 if and only if it has CPUID. */
-		mmu_cr4_features = __read_cr4();
+	if (boot_cpu_data.cpuid_level >= 0) {
+		/*
+		 * CPUs without CPUID don't have CR4.  CPUs with CPUID
+		 * usually have CR4.
+		 */
+		mmu_cr4_features = __read_cr4_safe();
+	}
 
 	memblock_set_current_limit(get_max_mapped());
 
-- 
2.7.4

[PATCH 3/3] x86/asm: Get rid of __read_cr4_safe()

[Andy Lutomirski]

We use __read_cr4() vs __read_cr4_safe() inconsistently.  On
CR4-less CPUs, all CR4 bits are effectively clear, so we can make
the code simpler and more robust by making __read_cr4() always fix
up faults on 32-bit kernels.

This may fix some bugs on old 486-like CPUs, but I don't have any
easy way to test that.

Cc: david@saggiorato.net
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/include/asm/paravirt.h       |  4 ----
 arch/x86/include/asm/paravirt_types.h |  1 -
 arch/x86/include/asm/special_insns.h  | 22 +++++++---------------
 arch/x86/include/asm/tlbflush.h       |  2 +-
 arch/x86/kernel/paravirt.c            |  1 -
 arch/x86/kernel/process_32.c          |  2 +-
 arch/x86/kernel/setup.c               |  5 +++--
 arch/x86/power/cpu.c                  |  2 +-
 arch/x86/xen/enlighten.c              |  1 -
 9 files changed, 13 insertions(+), 27 deletions(-)

diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 2970d22d7766..91b6f4eed3fd 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -80,10 +80,6 @@ static inline unsigned long __read_cr4(void)
 {
 	return PVOP_CALL0(unsigned long, pv_cpu_ops.read_cr4);
 }
-static inline unsigned long __read_cr4_safe(void)
-{
-	return PVOP_CALL0(unsigned long, pv_cpu_ops.read_cr4_safe);
-}
 
 static inline void __write_cr4(unsigned long x)
 {
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 7fa9e7740ba3..fcf243f077ac 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -108,7 +108,6 @@ struct pv_cpu_ops {
 	unsigned long (*read_cr0)(void);
 	void (*write_cr0)(unsigned long);
 
-	unsigned long (*read_cr4_safe)(void);
 	unsigned long (*read_cr4)(void);
 	void (*write_cr4)(unsigned long);
 
diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h
index 587d7914ea4b..19a2224f9e16 100644
--- a/arch/x86/include/asm/special_insns.h
+++ b/arch/x86/include/asm/special_insns.h
@@ -59,22 +59,19 @@ static inline void native_write_cr3(unsigned long val)
 static inline unsigned long native_read_cr4(void)
 {
 	unsigned long val;
-	asm volatile("mov %%cr4,%0\n\t" : "=r" (val), "=m" (__force_order));
-	return val;
-}
-
-static inline unsigned long native_read_cr4_safe(void)
-{
-	unsigned long val;
-	/* This could fault if %cr4 does not exist. In x86_64, a cr4 always
-	 * exists, so it will never fail. */
 #ifdef CONFIG_X86_32
+	/*
+	 * This could fault if CR4 does not exist.  Non-existent CR4
+	 * is functionally equivalent to CR4 == 0.  Keep it simple and pretend
+	 * that CR4 == 0 on CPUs that don't have CR4.
+	 */
 	asm volatile("1: mov %%cr4, %0\n"
 		     "2:\n"
 		     _ASM_EXTABLE(1b, 2b)
 		     : "=r" (val), "=m" (__force_order) : "0" (0));
 #else
-	val = native_read_cr4();
+	/* CR4 always exists on x86_64. */
+	asm volatile("mov %%cr4,%0\n\t" : "=r" (val), "=m" (__force_order));
 #endif
 	return val;
 }
@@ -182,11 +179,6 @@ static inline unsigned long __read_cr4(void)
 	return native_read_cr4();
 }
 
-static inline unsigned long __read_cr4_safe(void)
-{
-	return native_read_cr4_safe();
-}
-
 static inline void __write_cr4(unsigned long x)
 {
 	native_write_cr4(x);
diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index dee8a70382ba..6fa85944af83 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
 /* Initialize cr4 shadow for this CPU. */
 static inline void cr4_init_shadow(void)
 {
-	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
+	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
 }
 
 /* Set in this cpu's CR4. */
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 1acfd76e3e26..ed03b1c5b1b9 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -332,7 +332,6 @@ __visible struct pv_cpu_ops pv_cpu_ops = {
 	.read_cr0 = native_read_cr0,
 	.write_cr0 = native_write_cr0,
 	.read_cr4 = native_read_cr4,
-	.read_cr4_safe = native_read_cr4_safe,
 	.write_cr4 = native_write_cr4,
 #ifdef CONFIG_X86_64
 	.read_cr8 = native_read_cr8,
diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c
index d86be29c38c7..e5da9ba220d1 100644
--- a/arch/x86/kernel/process_32.c
+++ b/arch/x86/kernel/process_32.c
@@ -101,7 +101,7 @@ void __show_regs(struct pt_regs *regs, int all)
 	cr0 = read_cr0();
 	cr2 = read_cr2();
 	cr3 = read_cr3();
-	cr4 = __read_cr4_safe();
+	cr4 = __read_cr4();
 	printk(KERN_DEFAULT "CR0: %08lx CR2: %08lx CR3: %08lx CR4: %08lx\n",
 			cr0, cr2, cr3, cr4);
 
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 5930a4d191b4..09d245e8e2da 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1140,9 +1140,10 @@ void __init setup_arch(char **cmdline_p)
 	if (boot_cpu_data.cpuid_level >= 0) {
 		/*
 		 * CPUs without CPUID don't have CR4.  CPUs with CPUID
-		 * usually have CR4.
+		 * usually have CR4.  On CPUID-less CPUs, this avoids
+		 * an unnecessary fixup, mainly for historical reasons.
 		 */
-		mmu_cr4_features = __read_cr4_safe();
+		mmu_cr4_features = __read_cr4();
 	}
 
 	memblock_set_current_limit(get_max_mapped());
diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c
index b12c26e2e309..53cace2ec0e2 100644
--- a/arch/x86/power/cpu.c
+++ b/arch/x86/power/cpu.c
@@ -130,7 +130,7 @@ static void __save_processor_state(struct saved_context *ctxt)
 	ctxt->cr0 = read_cr0();
 	ctxt->cr2 = read_cr2();
 	ctxt->cr3 = read_cr3();
-	ctxt->cr4 = __read_cr4_safe();
+	ctxt->cr4 = __read_cr4();
 #ifdef CONFIG_X86_64
 	ctxt->cr8 = read_cr8();
 #endif
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index b86ebb1a9a7f..e2cf8fcea6bb 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1237,7 +1237,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
 	.write_cr0 = xen_write_cr0,
 
 	.read_cr4 = native_read_cr4,
-	.read_cr4_safe = native_read_cr4_safe,
 	.write_cr4 = xen_write_cr4,
 
 #ifdef CONFIG_X86_64
-- 
2.7.4

Re: [PATCH 1/3] x86/init: Fix cr4_init_shadow() on CR4-less machines

[Borislav Petkov]

On Wed, Sep 28, 2016 at 12:34:14PM -0700, Andy Lutomirski wrote:
> cr4_init_shadow() will panic on 486-like machines without CR4.  Fix
> it using __read_cr4_safe().
> 
> Cc: stable@vger.kernel.org
> Reported-by: david@saggiorato.net
> Fixes: 1e02ce4cccdc ("x86: Store a per-cpu shadow copy of CR4")
> Signed-off-by: Andy Lutomirski <luto@kernel.org>
> ---
>  arch/x86/include/asm/tlbflush.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
> index 6fa85944af83..dee8a70382ba 100644
> --- a/arch/x86/include/asm/tlbflush.h
> +++ b/arch/x86/include/asm/tlbflush.h
> @@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
>  /* Initialize cr4 shadow for this CPU. */
>  static inline void cr4_init_shadow(void)
>  {
> -	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
> +	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
>  }
>  
>  /* Set in this cpu's CR4. */
> -- 

Reviewed-by: Borislav Petkov <bp@suse.de>

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

Re: [PATCH 2/3] x86/boot: Fix another __read_cr4() case on 486

[Borislav Petkov]

On Wed, Sep 28, 2016 at 12:34:15PM -0700, Andy Lutomirski wrote:
> The condition for reading CR4 was wrong: there are some CPUs with
> CPUID but not CR4.  Rather than trying to make the condition exact,
> using __read_cr4_safe().
> 
> Reported-by: david@saggiorato.net
> Fixes: 18bc7bd523e0 ("x86/boot: Synchronize trampoline_cr4_features and mmu_cr4_features directly")
> Signed-off-by: Andy Lutomirski <luto@kernel.org>
> ---
>  arch/x86/kernel/setup.c | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 0fa60f5f5a16..5930a4d191b4 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -1137,9 +1137,13 @@ void __init setup_arch(char **cmdline_p)
>  	 * auditing all the early-boot CR4 manipulation would be needed to
>  	 * rule it out.
>  	 */
> -	if (boot_cpu_data.cpuid_level >= 0)
> -		/* A CPU has %cr4 if and only if it has CPUID. */
> -		mmu_cr4_features = __read_cr4();
> +	if (boot_cpu_data.cpuid_level >= 0) {
> +		/*
> +		 * CPUs without CPUID don't have CR4.  CPUs with CPUID
> +		 * usually have CR4.
> +		 */
> +		mmu_cr4_features = __read_cr4_safe();
> +	}

Why are we even doing the CPUID check instead of unconditionally doing
__read_cr4_safe()?

The safe variant will give 0 on !CR4 machines.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[tip:x86/urgent] x86/init: Fix cr4_init_shadow() on CR4-less machines

[tip-bot for Andy Lutomirski]

Commit-ID:  e1bfc11c5a6f40222a698a818dc269113245820e
Gitweb:     http://git.kernel.org/tip/e1bfc11c5a6f40222a698a818dc269113245820e
Author:     Andy Lutomirski <luto@kernel.org>
AuthorDate: Wed, 28 Sep 2016 12:34:14 -0700
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 29 Sep 2016 19:08:30 +0200

x86/init: Fix cr4_init_shadow() on CR4-less machines

cr4_init_shadow() will panic on 486-like machines without CR4.  Fix
it using __read_cr4_safe().

Reported-by: david@saggiorato.net
Signed-off-by: Andy Lutomirski <luto@kernel.org>
Reviewed-by: Borislav Petkov <bp@suse.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Fixes: 1e02ce4cccdc ("x86: Store a per-cpu shadow copy of CR4")
Link: http://lkml.kernel.org/r/43a20f81fb504013bf613913dc25574b45336a61.1475091074.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/include/asm/tlbflush.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 6fa8594..dee8a70 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
 /* Initialize cr4 shadow for this CPU. */
 static inline void cr4_init_shadow(void)
 {
-	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
+	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
 }
 
 /* Set in this cpu's CR4. */

Re: [PATCH 2/3] x86/boot: Fix another __read_cr4() case on 486

[Andy Lutomirski]

On Thu, Sep 29, 2016 at 4:48 AM, Borislav Petkov <bp@alien8.de> wrote:
> On Wed, Sep 28, 2016 at 12:34:15PM -0700, Andy Lutomirski wrote:
>> The condition for reading CR4 was wrong: there are some CPUs with
>> CPUID but not CR4.  Rather than trying to make the condition exact,
>> using __read_cr4_safe().
>>
>> Reported-by: david@saggiorato.net
>> Fixes: 18bc7bd523e0 ("x86/boot: Synchronize trampoline_cr4_features and mmu_cr4_features directly")
>> Signed-off-by: Andy Lutomirski <luto@kernel.org>
>> ---
>>  arch/x86/kernel/setup.c | 10 +++++++---
>>  1 file changed, 7 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 0fa60f5f5a16..5930a4d191b4 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -1137,9 +1137,13 @@ void __init setup_arch(char **cmdline_p)
>>        * auditing all the early-boot CR4 manipulation would be needed to
>>        * rule it out.
>>        */
>> -     if (boot_cpu_data.cpuid_level >= 0)
>> -             /* A CPU has %cr4 if and only if it has CPUID. */
>> -             mmu_cr4_features = __read_cr4();
>> +     if (boot_cpu_data.cpuid_level >= 0) {
>> +             /*
>> +              * CPUs without CPUID don't have CR4.  CPUs with CPUID
>> +              * usually have CR4.
>> +              */
>> +             mmu_cr4_features = __read_cr4_safe();
>> +     }
>
> Why are we even doing the CPUID check instead of unconditionally doing
> __read_cr4_safe()?
>
> The safe variant will give 0 on !CR4 machines.

Will do.