* [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup
@ 2020-06-05 20:07 Sean Christopherson
2020-06-08 10:12 ` Liam Merwick
2020-06-08 14:24 ` mlevitsk
0 siblings, 2 replies; 5+ messages in thread
From: Sean Christopherson @ 2020-06-05 20:07 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86,
Rafael J. Wysocki, Pavel Machek
Cc: H. Peter Anvin, linux-kernel, linux-pm, Brad Campbell,
Maxim Levitsky, Paolo Bonzini, kvm, Sean Christopherson
Reinitialize IA32_FEAT_CTL on the BSP during wakeup to handle the case
where firmware doesn't initialize or save/restore across S3. This fixes
a bug where IA32_FEAT_CTL is left uninitialized and results in VMXON
taking a #GP due to VMX not being fully enabled, i.e. breaks KVM.
Use init_ia32_feat_ctl() to "restore" IA32_FEAT_CTL as it already deals
with the case where the MSR is locked, and because APs already redo
init_ia32_feat_ctl() during suspend by virtue of the SMP boot flow being
used to reinitialize APs upon wakeup. Do the call in the early wakeup
flow to avoid dependencies in the syscore_ops chain, e.g. simply adding
a resume hook is not guaranteed to work, as KVM does VMXON in its own
resume hook, kvm_resume(), when KVM has active guests.
Reported-by: Brad Campbell <lists2009@fnarfbargle.com>
Cc: Maxim Levitsky <mlevitsk@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: kvm@vger.kernel.org
Fixes: 21bd3467a58e ("KVM: VMX: Drop initialization of IA32_FEAT_CTL MSR")
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
arch/x86/include/asm/cpu.h | 5 +++++
arch/x86/kernel/cpu/cpu.h | 4 ----
arch/x86/power/cpu.c | 6 ++++++
3 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/arch/x86/include/asm/cpu.h b/arch/x86/include/asm/cpu.h
index dd17c2da1af5..da78ccbd493b 100644
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -58,4 +58,9 @@ static inline bool handle_guest_split_lock(unsigned long ip)
return false;
}
#endif
+#ifdef CONFIG_IA32_FEAT_CTL
+void init_ia32_feat_ctl(struct cpuinfo_x86 *c);
+#else
+static inline void init_ia32_feat_ctl(struct cpuinfo_x86 *c) {}
+#endif
#endif /* _ASM_X86_CPU_H */
diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h
index 37fdefd14f28..38ab6e115eac 100644
--- a/arch/x86/kernel/cpu/cpu.h
+++ b/arch/x86/kernel/cpu/cpu.h
@@ -80,8 +80,4 @@ extern void x86_spec_ctrl_setup_ap(void);
extern u64 x86_read_arch_cap_msr(void);
-#ifdef CONFIG_IA32_FEAT_CTL
-void init_ia32_feat_ctl(struct cpuinfo_x86 *c);
-#endif
-
#endif /* ARCH_X86_CPU_H */
diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c
index aaff9ed7ff45..b0d3c5ca6d80 100644
--- a/arch/x86/power/cpu.c
+++ b/arch/x86/power/cpu.c
@@ -193,6 +193,8 @@ static void fix_processor_context(void)
*/
static void notrace __restore_processor_state(struct saved_context *ctxt)
{
+ struct cpuinfo_x86 *c;
+
if (ctxt->misc_enable_saved)
wrmsrl(MSR_IA32_MISC_ENABLE, ctxt->misc_enable);
/*
@@ -263,6 +265,10 @@ static void notrace __restore_processor_state(struct saved_context *ctxt)
mtrr_bp_restore();
perf_restore_debug_store();
msr_restore_context(ctxt);
+
+ c = &cpu_data(smp_processor_id());
+ if (cpu_has(c, X86_FEATURE_MSR_IA32_FEAT_CTL))
+ init_ia32_feat_ctl(c);
}
/* Needed by apm.c */
--
2.26.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup
2020-06-05 20:07 [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup Sean Christopherson
@ 2020-06-08 10:12 ` Liam Merwick
2020-06-08 17:29 ` Sean Christopherson
2020-06-08 14:24 ` mlevitsk
1 sibling, 1 reply; 5+ messages in thread
From: Liam Merwick @ 2020-06-08 10:12 UTC (permalink / raw)
To: Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, x86, Rafael J. Wysocki, Pavel Machek
Cc: H. Peter Anvin, linux-kernel, linux-pm, Brad Campbell,
Maxim Levitsky, Paolo Bonzini, kvm, liam.merwick
On 05/06/2020 21:07, Sean Christopherson wrote:
> Reinitialize IA32_FEAT_CTL on the BSP during wakeup to handle the case
> where firmware doesn't initialize or save/restore across S3. This fixes
> a bug where IA32_FEAT_CTL is left uninitialized and results in VMXON
> taking a #GP due to VMX not being fully enabled, i.e. breaks KVM.
>
> Use init_ia32_feat_ctl() to "restore" IA32_FEAT_CTL as it already deals
> with the case where the MSR is locked, and because APs already redo
> init_ia32_feat_ctl() during suspend by virtue of the SMP boot flow being
> used to reinitialize APs upon wakeup. Do the call in the early wakeup
> flow to avoid dependencies in the syscore_ops chain, e.g. simply adding
> a resume hook is not guaranteed to work, as KVM does VMXON in its own
> resume hook, kvm_resume(), when KVM has active guests.
>
> Reported-by: Brad Campbell <lists2009@fnarfbargle.com>
> Cc: Maxim Levitsky <mlevitsk@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: kvm@vger.kernel.org
Should it have the following tag since it fixes a commit introduced in 5.6?
Cc: stable@vger.kernel.org # v5.6
> Fixes: 21bd3467a58e ("KVM: VMX: Drop initialization of IA32_FEAT_CTL MSR")
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
> ---
> arch/x86/include/asm/cpu.h | 5 +++++
> arch/x86/kernel/cpu/cpu.h | 4 ----
> arch/x86/power/cpu.c | 6 ++++++
> 3 files changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/include/asm/cpu.h b/arch/x86/include/asm/cpu.h
> index dd17c2da1af5..da78ccbd493b 100644
> --- a/arch/x86/include/asm/cpu.h
> +++ b/arch/x86/include/asm/cpu.h
> @@ -58,4 +58,9 @@ static inline bool handle_guest_split_lock(unsigned long ip)
> return false;
> }
> #endif
> +#ifdef CONFIG_IA32_FEAT_CTL
> +void init_ia32_feat_ctl(struct cpuinfo_x86 *c);
> +#else
> +static inline void init_ia32_feat_ctl(struct cpuinfo_x86 *c) {}
> +#endif
> #endif /* _ASM_X86_CPU_H */
> diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h
> index 37fdefd14f28..38ab6e115eac 100644
> --- a/arch/x86/kernel/cpu/cpu.h
> +++ b/arch/x86/kernel/cpu/cpu.h
> @@ -80,8 +80,4 @@ extern void x86_spec_ctrl_setup_ap(void);
>
> extern u64 x86_read_arch_cap_msr(void);
>
> -#ifdef CONFIG_IA32_FEAT_CTL
> -void init_ia32_feat_ctl(struct cpuinfo_x86 *c);
> -#endif
> -
> #endif /* ARCH_X86_CPU_H */
> diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c
> index aaff9ed7ff45..b0d3c5ca6d80 100644
> --- a/arch/x86/power/cpu.c
> +++ b/arch/x86/power/cpu.c
> @@ -193,6 +193,8 @@ static void fix_processor_context(void)
> */
> static void notrace __restore_processor_state(struct saved_context *ctxt)
> {
> + struct cpuinfo_x86 *c;
> +
> if (ctxt->misc_enable_saved)
> wrmsrl(MSR_IA32_MISC_ENABLE, ctxt->misc_enable);
> /*
> @@ -263,6 +265,10 @@ static void notrace __restore_processor_state(struct saved_context *ctxt)
> mtrr_bp_restore();
> perf_restore_debug_store();
> msr_restore_context(ctxt);
> +
> + c = &cpu_data(smp_processor_id());
> + if (cpu_has(c, X86_FEATURE_MSR_IA32_FEAT_CTL))
> + init_ia32_feat_ctl(c);
> }
>
> /* Needed by apm.c */
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup
2020-06-05 20:07 [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup Sean Christopherson
2020-06-08 10:12 ` Liam Merwick
@ 2020-06-08 14:24 ` mlevitsk
1 sibling, 0 replies; 5+ messages in thread
From: mlevitsk @ 2020-06-08 14:24 UTC (permalink / raw)
To: Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, x86, Rafael J. Wysocki, Pavel Machek
Cc: H. Peter Anvin, linux-kernel, linux-pm, Brad Campbell,
Paolo Bonzini, kvm
On Fri, 2020-06-05 at 13:07 -0700, Sean Christopherson wrote:
> Reinitialize IA32_FEAT_CTL on the BSP during wakeup to handle the
> case
> where firmware doesn't initialize or save/restore across S3. This
> fixes
> a bug where IA32_FEAT_CTL is left uninitialized and results in VMXON
> taking a #GP due to VMX not being fully enabled, i.e. breaks KVM.
>
> Use init_ia32_feat_ctl() to "restore" IA32_FEAT_CTL as it already
> deals
> with the case where the MSR is locked, and because APs already redo
> init_ia32_feat_ctl() during suspend by virtue of the SMP boot flow
> being
> used to reinitialize APs upon wakeup. Do the call in the early
> wakeup
> flow to avoid dependencies in the syscore_ops chain, e.g. simply
> adding
> a resume hook is not guaranteed to work, as KVM does VMXON in its own
> resume hook, kvm_resume(), when KVM has active guests.
>
> Reported-by: Brad Campbell <lists2009@fnarfbargle.com>
> Cc: Maxim Levitsky <mlevitsk@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: kvm@vger.kernel.org
> Fixes: 21bd3467a58e ("KVM: VMX: Drop initialization of IA32_FEAT_CTL
> MSR")
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> ---
> arch/x86/include/asm/cpu.h | 5 +++++
> arch/x86/kernel/cpu/cpu.h | 4 ----
> arch/x86/power/cpu.c | 6 ++++++
> 3 files changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/include/asm/cpu.h b/arch/x86/include/asm/cpu.h
> index dd17c2da1af5..da78ccbd493b 100644
> --- a/arch/x86/include/asm/cpu.h
> +++ b/arch/x86/include/asm/cpu.h
> @@ -58,4 +58,9 @@ static inline bool handle_guest_split_lock(unsigned
> long ip)
> return false;
> }
> #endif
> +#ifdef CONFIG_IA32_FEAT_CTL
> +void init_ia32_feat_ctl(struct cpuinfo_x86 *c);
> +#else
> +static inline void init_ia32_feat_ctl(struct cpuinfo_x86 *c) {}
> +#endif
> #endif /* _ASM_X86_CPU_H */
> diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h
> index 37fdefd14f28..38ab6e115eac 100644
> --- a/arch/x86/kernel/cpu/cpu.h
> +++ b/arch/x86/kernel/cpu/cpu.h
> @@ -80,8 +80,4 @@ extern void x86_spec_ctrl_setup_ap(void);
>
> extern u64 x86_read_arch_cap_msr(void);
>
> -#ifdef CONFIG_IA32_FEAT_CTL
> -void init_ia32_feat_ctl(struct cpuinfo_x86 *c);
> -#endif
> -
> #endif /* ARCH_X86_CPU_H */
> diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c
> index aaff9ed7ff45..b0d3c5ca6d80 100644
> --- a/arch/x86/power/cpu.c
> +++ b/arch/x86/power/cpu.c
> @@ -193,6 +193,8 @@ static void fix_processor_context(void)
> */
> static void notrace __restore_processor_state(struct saved_context
> *ctxt)
> {
> + struct cpuinfo_x86 *c;
> +
> if (ctxt->misc_enable_saved)
> wrmsrl(MSR_IA32_MISC_ENABLE, ctxt->misc_enable);
> /*
> @@ -263,6 +265,10 @@ static void notrace
> __restore_processor_state(struct saved_context *ctxt)
> mtrr_bp_restore();
> perf_restore_debug_store();
> msr_restore_context(ctxt);
> +
> + c = &cpu_data(smp_processor_id());
> + if (cpu_has(c, X86_FEATURE_MSR_IA32_FEAT_CTL))
> + init_ia32_feat_ctl(c);
> }
>
> /* Needed by apm.c */
I don't have currently an active VMX system to test this on,
but from the code and from my knowelege of this area this looks all
right.
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Best regards,
Maxim Levitsky
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup
2020-06-08 10:12 ` Liam Merwick
@ 2020-06-08 17:29 ` Sean Christopherson
2020-06-15 12:15 ` Borislav Petkov
0 siblings, 1 reply; 5+ messages in thread
From: Sean Christopherson @ 2020-06-08 17:29 UTC (permalink / raw)
To: Liam Merwick
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86,
Rafael J. Wysocki, Pavel Machek, H. Peter Anvin, linux-kernel,
linux-pm, Brad Campbell, Maxim Levitsky, Paolo Bonzini, kvm
On Mon, Jun 08, 2020 at 11:12:35AM +0100, Liam Merwick wrote:
> On 05/06/2020 21:07, Sean Christopherson wrote:
> >Reinitialize IA32_FEAT_CTL on the BSP during wakeup to handle the case
> >where firmware doesn't initialize or save/restore across S3. This fixes
> >a bug where IA32_FEAT_CTL is left uninitialized and results in VMXON
> >taking a #GP due to VMX not being fully enabled, i.e. breaks KVM.
> >
> >Use init_ia32_feat_ctl() to "restore" IA32_FEAT_CTL as it already deals
> >with the case where the MSR is locked, and because APs already redo
> >init_ia32_feat_ctl() during suspend by virtue of the SMP boot flow being
> >used to reinitialize APs upon wakeup. Do the call in the early wakeup
> >flow to avoid dependencies in the syscore_ops chain, e.g. simply adding
> >a resume hook is not guaranteed to work, as KVM does VMXON in its own
> >resume hook, kvm_resume(), when KVM has active guests.
> >
> >Reported-by: Brad Campbell <lists2009@fnarfbargle.com>
> >Cc: Maxim Levitsky <mlevitsk@redhat.com>
> >Cc: Paolo Bonzini <pbonzini@redhat.com>
> >Cc: kvm@vger.kernel.org
>
> Should it have the following tag since it fixes a commit introduced in 5.6?
> Cc: stable@vger.kernel.org # v5.6
It definitely warrants a backport to v5.6. I didn't include a Cc to stable
because I swear I had seen an email fly by that stated an explicit Cc is
unnecessary/unwanted for tip-tree patches, but per a recent statement from
Boris it looks like I'm simply confused[*]. I'll add the Cc in v2.
[*] https://lkml.kernel.org/r/20200417164752.GF7322@zn.tnic
> >Fixes: 21bd3467a58e ("KVM: VMX: Drop initialization of IA32_FEAT_CTL MSR")
> >Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
>
> Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup
2020-06-08 17:29 ` Sean Christopherson
@ 2020-06-15 12:15 ` Borislav Petkov
0 siblings, 0 replies; 5+ messages in thread
From: Borislav Petkov @ 2020-06-15 12:15 UTC (permalink / raw)
To: Sean Christopherson, Greg KH, Sasha Levin
Cc: Liam Merwick, Thomas Gleixner, Ingo Molnar, x86,
Rafael J. Wysocki, Pavel Machek, H. Peter Anvin, linux-kernel,
linux-pm, Brad Campbell, Maxim Levitsky, Paolo Bonzini, kvm
On Mon, Jun 08, 2020 at 10:29:21AM -0700, Sean Christopherson wrote:
> On Mon, Jun 08, 2020 at 11:12:35AM +0100, Liam Merwick wrote:
> > On 05/06/2020 21:07, Sean Christopherson wrote:
> > >Reinitialize IA32_FEAT_CTL on the BSP during wakeup to handle the case
> > >where firmware doesn't initialize or save/restore across S3. This fixes
> > >a bug where IA32_FEAT_CTL is left uninitialized and results in VMXON
> > >taking a #GP due to VMX not being fully enabled, i.e. breaks KVM.
> > >
> > >Use init_ia32_feat_ctl() to "restore" IA32_FEAT_CTL as it already deals
> > >with the case where the MSR is locked, and because APs already redo
> > >init_ia32_feat_ctl() during suspend by virtue of the SMP boot flow being
> > >used to reinitialize APs upon wakeup. Do the call in the early wakeup
> > >flow to avoid dependencies in the syscore_ops chain, e.g. simply adding
> > >a resume hook is not guaranteed to work, as KVM does VMXON in its own
> > >resume hook, kvm_resume(), when KVM has active guests.
> > >
> > >Reported-by: Brad Campbell <lists2009@fnarfbargle.com>
> > >Cc: Maxim Levitsky <mlevitsk@redhat.com>
> > >Cc: Paolo Bonzini <pbonzini@redhat.com>
> > >Cc: kvm@vger.kernel.org
> >
> > Should it have the following tag since it fixes a commit introduced in 5.6?
> > Cc: stable@vger.kernel.org # v5.6
>
> It definitely warrants a backport to v5.6. I didn't include a Cc to stable
> because I swear I had seen an email fly by that stated an explicit Cc is
> unnecessary/unwanted for tip-tree patches, but per a recent statement from
> Boris it looks like I'm simply confused[*]. I'll add the Cc in v2.
>
> [*] https://lkml.kernel.org/r/20200417164752.GF7322@zn.tnic
Yeah, I was simply parroting what Greg has told me. Maybe he should
finally do that script. :-P
Also, I believe Sasha's Skynet machine already does that...
CCed both.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-06-15 12:15 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-05 20:07 [PATCH] x86/cpu: Reinitialize IA32_FEAT_CTL MSR on BSP during wakeup Sean Christopherson
2020-06-08 10:12 ` Liam Merwick
2020-06-08 17:29 ` Sean Christopherson
2020-06-15 12:15 ` Borislav Petkov
2020-06-08 14:24 ` mlevitsk
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).