* [PATCH 3.10 024/319] KVM: MIPS: Drop other CPU ASIDs on guest MMU changes
2017-02-05 19:21 [PATCH 3.10 023/319] KVM: MIPS: Precalculate MMIO load resume PC Willy Tarreau
@ 2017-02-05 19:21 ` Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 025/319] KVM: nVMX: postpone VMCS changes on MSR_IA32_APICBASE write Willy Tarreau
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Willy Tarreau @ 2017-02-05 19:21 UTC (permalink / raw)
To: linux-kernel, stable, linux
Cc: James Hogan, Paolo Bonzini, Radim Krčmář,
Ralf Baechle, linux-mips, kvm, Jiri Slaby, Willy Tarreau
From: James Hogan <james.hogan@imgtec.com>
commit 91e4f1b6073dd680d86cdb7e42d7cccca9db39d8 upstream.
When a guest TLB entry is replaced by TLBWI or TLBWR, we only invalidate
TLB entries on the local CPU. This doesn't work correctly on an SMP host
when the guest is migrated to a different physical CPU, as it could pick
up stale TLB mappings from the last time the vCPU ran on that physical
CPU.
Therefore invalidate both user and kernel host ASIDs on other CPUs,
which will cause new ASIDs to be generated when it next runs on those
CPUs.
We're careful only to do this if the TLB entry was already valid, and
only for the kernel ASID where the virtual address it mapped is outside
of the guest user address range.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Radim KrÄmář" <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
Cc: <stable@vger.kernel.org>
Cc: Jiri Slaby <jslaby@suse.cz>
[james.hogan@imgtec.com: Backport to 3.10..3.16]
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
arch/mips/kvm/kvm_mips_emul.c | 61 +++++++++++++++++++++++++++++++++++++------
1 file changed, 53 insertions(+), 8 deletions(-)
diff --git a/arch/mips/kvm/kvm_mips_emul.c b/arch/mips/kvm/kvm_mips_emul.c
index e5977f2..4cfb5bd 100644
--- a/arch/mips/kvm/kvm_mips_emul.c
+++ b/arch/mips/kvm/kvm_mips_emul.c
@@ -310,6 +310,47 @@ enum emulation_result kvm_mips_emul_tlbr(struct kvm_vcpu *vcpu)
return er;
}
+/**
+ * kvm_mips_invalidate_guest_tlb() - Indicates a change in guest MMU map.
+ * @vcpu: VCPU with changed mappings.
+ * @tlb: TLB entry being removed.
+ *
+ * This is called to indicate a single change in guest MMU mappings, so that we
+ * can arrange TLB flushes on this and other CPUs.
+ */
+static void kvm_mips_invalidate_guest_tlb(struct kvm_vcpu *vcpu,
+ struct kvm_mips_tlb *tlb)
+{
+ int cpu, i;
+ bool user;
+
+ /* No need to flush for entries which are already invalid */
+ if (!((tlb->tlb_lo0 | tlb->tlb_lo1) & MIPS3_PG_V))
+ return;
+ /* User address space doesn't need flushing for KSeg2/3 changes */
+ user = tlb->tlb_hi < KVM_GUEST_KSEG0;
+
+ preempt_disable();
+
+ /*
+ * Probe the shadow host TLB for the entry being overwritten, if one
+ * matches, invalidate it
+ */
+ kvm_mips_host_tlb_inv(vcpu, tlb->tlb_hi);
+
+ /* Invalidate the whole ASID on other CPUs */
+ cpu = smp_processor_id();
+ for_each_possible_cpu(i) {
+ if (i == cpu)
+ continue;
+ if (user)
+ vcpu->arch.guest_user_asid[i] = 0;
+ vcpu->arch.guest_kernel_asid[i] = 0;
+ }
+
+ preempt_enable();
+}
+
/* Write Guest TLB Entry @ Index */
enum emulation_result kvm_mips_emul_tlbwi(struct kvm_vcpu *vcpu)
{
@@ -331,10 +372,8 @@ enum emulation_result kvm_mips_emul_tlbwi(struct kvm_vcpu *vcpu)
}
tlb = &vcpu->arch.guest_tlb[index];
-#if 1
- /* Probe the shadow host TLB for the entry being overwritten, if one matches, invalidate it */
- kvm_mips_host_tlb_inv(vcpu, tlb->tlb_hi);
-#endif
+
+ kvm_mips_invalidate_guest_tlb(vcpu, tlb);
tlb->tlb_mask = kvm_read_c0_guest_pagemask(cop0);
tlb->tlb_hi = kvm_read_c0_guest_entryhi(cop0);
@@ -373,10 +412,7 @@ enum emulation_result kvm_mips_emul_tlbwr(struct kvm_vcpu *vcpu)
tlb = &vcpu->arch.guest_tlb[index];
-#if 1
- /* Probe the shadow host TLB for the entry being overwritten, if one matches, invalidate it */
- kvm_mips_host_tlb_inv(vcpu, tlb->tlb_hi);
-#endif
+ kvm_mips_invalidate_guest_tlb(vcpu, tlb);
tlb->tlb_mask = kvm_read_c0_guest_pagemask(cop0);
tlb->tlb_hi = kvm_read_c0_guest_entryhi(cop0);
@@ -419,6 +455,7 @@ kvm_mips_emulate_CP0(uint32_t inst, uint32_t *opc, uint32_t cause,
int32_t rt, rd, copz, sel, co_bit, op;
uint32_t pc = vcpu->arch.pc;
unsigned long curr_pc;
+ int cpu, i;
/*
* Update PC and hold onto current PC in case there is
@@ -538,8 +575,16 @@ kvm_mips_emulate_CP0(uint32_t inst, uint32_t *opc, uint32_t cause,
ASID_MASK,
vcpu->arch.gprs[rt] & ASID_MASK);
+ preempt_disable();
/* Blow away the shadow host TLBs */
kvm_mips_flush_host_tlb(1);
+ cpu = smp_processor_id();
+ for_each_possible_cpu(i)
+ if (i != cpu) {
+ vcpu->arch.guest_user_asid[i] = 0;
+ vcpu->arch.guest_kernel_asid[i] = 0;
+ }
+ preempt_enable();
}
kvm_write_c0_guest_entryhi(cop0,
vcpu->arch.gprs[rt]);
--
2.8.0.rc2.1.gbe9624a
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3.10 025/319] KVM: nVMX: postpone VMCS changes on MSR_IA32_APICBASE write
2017-02-05 19:21 [PATCH 3.10 023/319] KVM: MIPS: Precalculate MMIO load resume PC Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 024/319] KVM: MIPS: Drop other CPU ASIDs on guest MMU changes Willy Tarreau
@ 2017-02-05 19:21 ` Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 026/319] KVM: MIPS: Make ERET handle ERL before EXL Willy Tarreau
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Willy Tarreau @ 2017-02-05 19:21 UTC (permalink / raw)
To: linux-kernel, stable, linux
Cc: Radim KrÄmář, Willy Tarreau
From: Radim KrÄmář <rkrcmar@redhat.com>
commit dccbfcf52cebb8963246eba5b177b77f26b34da0 upstream.
If vmcs12 does not intercept APIC_BASE writes, then KVM will handle the
write with vmcs02 as the current VMCS.
This will incorrectly apply modifications intended for vmcs01 to vmcs02
and L2 can use it to gain access to L0's x2APIC registers by disabling
virtualized x2APIC while using msr bitmap that assumes enabled.
Postpone execution of vmx_set_virtual_x2apic_mode until vmcs01 is the
current VMCS. An alternative solution would temporarily make vmcs01 the
current VMCS, but it requires more care.
Fixes: 8d14695f9542 ("x86, apicv: add virtual x2apic support")
Reported-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Wanpeng Li <wanpeng.li@hotmail.com>
Signed-off-by: Radim KrÄmář <rkrcmar@redhat.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
arch/x86/kvm/vmx.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 335fe70..7e9ca58 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -366,6 +366,7 @@ struct nested_vmx {
struct list_head vmcs02_pool;
int vmcs02_num;
u64 vmcs01_tsc_offset;
+ bool change_vmcs01_virtual_x2apic_mode;
/* L2 must run next, and mustn't decide to exit to L1. */
bool nested_run_pending;
/*
@@ -6702,6 +6703,12 @@ static void vmx_set_virtual_x2apic_mode(struct kvm_vcpu *vcpu, bool set)
{
u32 sec_exec_control;
+ /* Postpone execution until vmcs01 is the current VMCS. */
+ if (is_guest_mode(vcpu)) {
+ to_vmx(vcpu)->nested.change_vmcs01_virtual_x2apic_mode = true;
+ return;
+ }
+
/*
* There is not point to enable virtualize x2apic without enable
* apicv
@@ -8085,6 +8092,12 @@ static void nested_vmx_vmexit(struct kvm_vcpu *vcpu)
/* Update TSC_OFFSET if TSC was changed while L2 ran */
vmcs_write64(TSC_OFFSET, vmx->nested.vmcs01_tsc_offset);
+ if (vmx->nested.change_vmcs01_virtual_x2apic_mode) {
+ vmx->nested.change_vmcs01_virtual_x2apic_mode = false;
+ vmx_set_virtual_x2apic_mode(vcpu,
+ vcpu->arch.apic_base & X2APIC_ENABLE);
+ }
+
/* This is needed for same reason as it was needed in prepare_vmcs02 */
vmx->host_rsp = 0;
--
2.8.0.rc2.1.gbe9624a
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3.10 026/319] KVM: MIPS: Make ERET handle ERL before EXL
2017-02-05 19:21 [PATCH 3.10 023/319] KVM: MIPS: Precalculate MMIO load resume PC Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 024/319] KVM: MIPS: Drop other CPU ASIDs on guest MMU changes Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 025/319] KVM: nVMX: postpone VMCS changes on MSR_IA32_APICBASE write Willy Tarreau
@ 2017-02-05 19:21 ` Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 027/319] KVM: x86: fix wbinvd_dirty_mask use-after-free Willy Tarreau
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Willy Tarreau @ 2017-02-05 19:21 UTC (permalink / raw)
To: linux-kernel, stable, linux
Cc: James Hogan, Paolo Bonzini, Radim Krčmář,
Ralf Baechle, linux-mips, kvm, Willy Tarreau
From: James Hogan <james.hogan@imgtec.com>
commit ede5f3e7b54a4347be4d8525269eae50902bd7cd upstream.
The ERET instruction to return from exception is used for returning from
exception level (Status.EXL) and error level (Status.ERL). If both bits
are set however we should be returning from ERL first, as ERL can
interrupt EXL, for example when an NMI is taken. KVM however checks EXL
first.
Fix the order of the checks to match the pseudocode in the instruction
set manual.
Fixes: e685c689f3a8 ("KVM/MIPS32: Privileged instruction/target branch emulation.")
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Radim KrÄmář" <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
arch/mips/kvm/kvm_mips_emul.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/mips/kvm/kvm_mips_emul.c b/arch/mips/kvm/kvm_mips_emul.c
index 4cfb5bd..7162854 100644
--- a/arch/mips/kvm/kvm_mips_emul.c
+++ b/arch/mips/kvm/kvm_mips_emul.c
@@ -254,15 +254,15 @@ enum emulation_result kvm_mips_emul_eret(struct kvm_vcpu *vcpu)
struct mips_coproc *cop0 = vcpu->arch.cop0;
enum emulation_result er = EMULATE_DONE;
- if (kvm_read_c0_guest_status(cop0) & ST0_EXL) {
+ if (kvm_read_c0_guest_status(cop0) & ST0_ERL) {
+ kvm_clear_c0_guest_status(cop0, ST0_ERL);
+ vcpu->arch.pc = kvm_read_c0_guest_errorepc(cop0);
+ } else if (kvm_read_c0_guest_status(cop0) & ST0_EXL) {
kvm_debug("[%#lx] ERET to %#lx\n", vcpu->arch.pc,
kvm_read_c0_guest_epc(cop0));
kvm_clear_c0_guest_status(cop0, ST0_EXL);
vcpu->arch.pc = kvm_read_c0_guest_epc(cop0);
- } else if (kvm_read_c0_guest_status(cop0) & ST0_ERL) {
- kvm_clear_c0_guest_status(cop0, ST0_ERL);
- vcpu->arch.pc = kvm_read_c0_guest_errorepc(cop0);
} else {
printk("[%#lx] ERET when MIPS_SR_EXL|MIPS_SR_ERL == 0\n",
vcpu->arch.pc);
--
2.8.0.rc2.1.gbe9624a
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3.10 027/319] KVM: x86: fix wbinvd_dirty_mask use-after-free
2017-02-05 19:21 [PATCH 3.10 023/319] KVM: MIPS: Precalculate MMIO load resume PC Willy Tarreau
` (2 preceding siblings ...)
2017-02-05 19:21 ` [PATCH 3.10 026/319] KVM: MIPS: Make ERET handle ERL before EXL Willy Tarreau
@ 2017-02-05 19:21 ` Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 028/319] KVM: x86: fix missed SRCU usage in kvm_lapic_set_vapic_addr Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 029/319] KVM: Disable irq while unregistering user notifier Willy Tarreau
5 siblings, 0 replies; 7+ messages in thread
From: Willy Tarreau @ 2017-02-05 19:21 UTC (permalink / raw)
To: linux-kernel, stable, linux
Cc: Ido Yariv, Radim Krčmář, Willy Tarreau
From: Ido Yariv <ido@wizery.com>
commit bd768e146624cbec7122ed15dead8daa137d909d upstream.
vcpu->arch.wbinvd_dirty_mask may still be used after freeing it,
corrupting memory. For example, the following call trace may set a bit
in an already freed cpu mask:
kvm_arch_vcpu_load
vcpu_load
vmx_free_vcpu_nested
vmx_free_vcpu
kvm_arch_vcpu_free
Fix this by deferring freeing of wbinvd_dirty_mask.
Signed-off-by: Ido Yariv <ido@wizery.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Radim KrÄmář <rkrcmar@redhat.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
arch/x86/kvm/x86.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 8e57771..fc68806 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6509,11 +6509,13 @@ void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
void kvm_arch_vcpu_free(struct kvm_vcpu *vcpu)
{
+ void *wbinvd_dirty_mask = vcpu->arch.wbinvd_dirty_mask;
+
kvmclock_reset(vcpu);
- free_cpumask_var(vcpu->arch.wbinvd_dirty_mask);
fx_free(vcpu);
kvm_x86_ops->vcpu_free(vcpu);
+ free_cpumask_var(wbinvd_dirty_mask);
}
struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm,
--
2.8.0.rc2.1.gbe9624a
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3.10 028/319] KVM: x86: fix missed SRCU usage in kvm_lapic_set_vapic_addr
2017-02-05 19:21 [PATCH 3.10 023/319] KVM: MIPS: Precalculate MMIO load resume PC Willy Tarreau
` (3 preceding siblings ...)
2017-02-05 19:21 ` [PATCH 3.10 027/319] KVM: x86: fix wbinvd_dirty_mask use-after-free Willy Tarreau
@ 2017-02-05 19:21 ` Willy Tarreau
2017-02-05 19:21 ` [PATCH 3.10 029/319] KVM: Disable irq while unregistering user notifier Willy Tarreau
5 siblings, 0 replies; 7+ messages in thread
From: Willy Tarreau @ 2017-02-05 19:21 UTC (permalink / raw)
To: linux-kernel, stable, linux
Cc: Paolo Bonzini, Andrew Honig, Radim Krčmář, Willy Tarreau
From: Paolo Bonzini <pbonzini@redhat.com>
commit 7301d6abaea926d685832f7e1f0c37dd206b01f4 upstream.
Reported by syzkaller:
[ INFO: suspicious RCU usage. ]
4.9.0-rc4+ #47 Not tainted
-------------------------------
./include/linux/kvm_host.h:536 suspicious rcu_dereference_check() usage!
stack backtrace:
CPU: 1 PID: 6679 Comm: syz-executor Not tainted 4.9.0-rc4+ #47
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
ffff880039e2f6d0 ffffffff81c2e46b ffff88003e3a5b40 0000000000000000
0000000000000001 ffffffff83215600 ffff880039e2f700 ffffffff81334ea9
ffffc9000730b000 0000000000000004 ffff88003c4f8420 ffff88003d3f8000
Call Trace:
[< inline >] __dump_stack lib/dump_stack.c:15
[<ffffffff81c2e46b>] dump_stack+0xb3/0x118 lib/dump_stack.c:51
[<ffffffff81334ea9>] lockdep_rcu_suspicious+0x139/0x180 kernel/locking/lockdep.c:4445
[< inline >] __kvm_memslots include/linux/kvm_host.h:534
[< inline >] kvm_memslots include/linux/kvm_host.h:541
[<ffffffff8105d6ae>] kvm_gfn_to_hva_cache_init+0xa1e/0xce0 virt/kvm/kvm_main.c:1941
[<ffffffff8112685d>] kvm_lapic_set_vapic_addr+0xed/0x140 arch/x86/kvm/lapic.c:2217
Reported-by: Dmitry Vyukov <dvyukov@google.com>
Fixes: fda4e2e85589191b123d31cdc21fd33ee70f50fd
Cc: Andrew Honig <ahonig@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Radim KrÄmář <rkrcmar@redhat.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
arch/x86/kvm/x86.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index fc68806..1072083 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3182,6 +3182,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
};
case KVM_SET_VAPIC_ADDR: {
struct kvm_vapic_addr va;
+ int idx;
r = -EINVAL;
if (!irqchip_in_kernel(vcpu->kvm))
@@ -3189,7 +3190,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
r = -EFAULT;
if (copy_from_user(&va, argp, sizeof va))
goto out;
+ idx = srcu_read_lock(&vcpu->kvm->srcu);
r = kvm_lapic_set_vapic_addr(vcpu, va.vapic_addr);
+ srcu_read_unlock(&vcpu->kvm->srcu, idx);
break;
}
case KVM_X86_SETUP_MCE: {
--
2.8.0.rc2.1.gbe9624a
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3.10 029/319] KVM: Disable irq while unregistering user notifier
2017-02-05 19:21 [PATCH 3.10 023/319] KVM: MIPS: Precalculate MMIO load resume PC Willy Tarreau
` (4 preceding siblings ...)
2017-02-05 19:21 ` [PATCH 3.10 028/319] KVM: x86: fix missed SRCU usage in kvm_lapic_set_vapic_addr Willy Tarreau
@ 2017-02-05 19:21 ` Willy Tarreau
5 siblings, 0 replies; 7+ messages in thread
From: Willy Tarreau @ 2017-02-05 19:21 UTC (permalink / raw)
To: linux-kernel, stable, linux
Cc: Ignacio Alvarado, Radim Krčmář, Willy Tarreau
From: Ignacio Alvarado <ikalvarado@google.com>
commit 1650b4ebc99da4c137bfbfc531be4a2405f951dd upstream.
Function user_notifier_unregister should be called only once for each
registered user notifier.
Function kvm_arch_hardware_disable can be executed from an IPI context
which could cause a race condition with a VCPU returning to user mode
and attempting to unregister the notifier.
Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>
Fixes: 18863bdd60f8 ("KVM: x86 shared msr infrastructure")
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Radim KrÄmář <rkrcmar@redhat.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
arch/x86/kvm/x86.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1072083..b70b67b 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -178,7 +178,18 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
struct kvm_shared_msrs *locals
= container_of(urn, struct kvm_shared_msrs, urn);
struct kvm_shared_msr_values *values;
+ unsigned long flags;
+ /*
+ * Disabling irqs at this point since the following code could be
+ * interrupted and executed through kvm_arch_hardware_disable()
+ */
+ local_irq_save(flags);
+ if (locals->registered) {
+ locals->registered = false;
+ user_return_notifier_unregister(urn);
+ }
+ local_irq_restore(flags);
for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
values = &locals->values[slot];
if (values->host != values->curr) {
@@ -186,8 +197,6 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
values->curr = values->host;
}
}
- locals->registered = false;
- user_return_notifier_unregister(urn);
}
static void shared_msr_update(unsigned slot, u32 msr)
--
2.8.0.rc2.1.gbe9624a
^ permalink raw reply related [flat|nested] 7+ messages in thread