From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753441AbdKILBq (ORCPT ); Thu, 9 Nov 2017 06:01:46 -0500 Received: from mail-oi0-f67.google.com ([209.85.218.67]:43083 "EHLO mail-oi0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753016AbdKILBo (ORCPT ); Thu, 9 Nov 2017 06:01:44 -0500 X-Google-Smtp-Source: AGs4zMbudzPFBNUCSBQLDN2L7uGLtHeQwaVfNyDpXMR/oiorM1419iM4movzyjawW8fRQvytJqjcs+/BdBmgOmOQUlk= MIME-Version: 1.0 In-Reply-To: References: <1510192934-5369-1-git-send-email-wanpeng.li@hotmail.com> <1510192934-5369-3-git-send-email-wanpeng.li@hotmail.com> From: Wanpeng Li Date: Thu, 9 Nov 2017 19:01:43 +0800 Message-ID: Subject: Re: [PATCH RESEND 2/3] KVM: Add paravirt remote TLB flush To: Paolo Bonzini Cc: "linux-kernel@vger.kernel.org" , kvm , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Wanpeng Li , Eduardo Valentin Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by nfs id vA9B1o8P028595 2017-11-09 18:48 GMT+08:00 Paolo Bonzini : > On 09/11/2017 03:02, Wanpeng Li wrote: >> From: Wanpeng Li >> >> Remote flushing api's does a busy wait which is fine in bare-metal >> scenario. But with-in the guest, the vcpus might have been pre-empted >> or blocked. In this scenario, the initator vcpu would end up >> busy-waiting for a long amount of time. >> >> This patch set implements para-virt flush tlbs making sure that it >> does not wait for vcpus that are sleeping. And all the sleeping vcpus >> flush the tlb on guest enter. >> >> The best result is achieved when we're overcommiting the host by running >> multiple vCPUs on each pCPU. In this case PV tlb flush avoids touching >> vCPUs which are not scheduled and avoid the wait on the main CPU. >> >> Test on a Haswell i7 desktop 4 cores (2HT), so 8 pCPUs, running ebizzy in >> one linux guest. >> >> ebizzy -M >> vanilla optimized boost >> 8 vCPUs 10152 10083 -0.68% >> 16 vCPUs 1224 4866 297.5% >> 24 vCPUs 1109 3871 249% >> 32 vCPUs 1025 3375 229.3% >> >> Cc: Paolo Bonzini >> Cc: Radim Krčmář >> Signed-off-by: Wanpeng Li >> --- >> arch/x86/include/uapi/asm/kvm_para.h | 1 + >> arch/x86/kernel/kvm.c | 29 +++++++++++++++++++++++++++++ >> 2 files changed, 30 insertions(+) >> >> diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h >> index ff23ce9..189e354 100644 >> --- a/arch/x86/include/uapi/asm/kvm_para.h >> +++ b/arch/x86/include/uapi/asm/kvm_para.h >> @@ -52,6 +52,7 @@ struct kvm_steal_time { >> >> #define KVM_VCPU_NOT_PREEMPTED (0 << 0) >> #define KVM_VCPU_PREEMPTED (1 << 0) >> +#define KVM_VCPU_SHOULD_FLUSH (1 << 1) >> >> #define KVM_CLOCK_PAIRING_WALLCLOCK 0 >> struct kvm_clock_pairing { >> diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c >> index 1b1b641..2e2f3ae 100644 >> --- a/arch/x86/kernel/kvm.c >> +++ b/arch/x86/kernel/kvm.c >> @@ -465,6 +465,33 @@ static void __init kvm_apf_trap_init(void) >> update_intr_gate(X86_TRAP_PF, async_page_fault); >> } >> >> +static void kvm_flush_tlb_others(const struct cpumask *cpumask, >> + const struct flush_tlb_info *info) >> +{ >> + u8 state; >> + int cpu; >> + struct kvm_steal_time *src; >> + cpumask_t flushmask; >> + >> + >> + cpumask_copy(&flushmask, cpumask); >> + /* >> + * We have to call flush only on online vCPUs. And >> + * queue flush_on_enter for pre-empted vCPUs >> + */ >> + for_each_cpu(cpu, cpumask) { >> + src = &per_cpu(steal_time, cpu); >> + state = src->preempted; >> + if ((state & KVM_VCPU_PREEMPTED)) { >> + if (cmpxchg(&src->preempted, state, state | 1 << >> + KVM_VCPU_SHOULD_FLUSH)) >> + cpumask_clear_cpu(cpu, &flushmask); >> + } >> + } >> + >> + native_flush_tlb_others(&flushmask, info); >> +} >> + >> void __init kvm_guest_init(void) >> { >> int i; >> @@ -484,6 +511,8 @@ void __init kvm_guest_init(void) >> pv_time_ops.steal_clock = kvm_steal_clock; >> } >> >> + pv_mmu_ops.flush_tlb_others = kvm_flush_tlb_others; > > This needs to be keyed on a new CPUID feature bit. Eduardo is also Will do. > adding a new "PV_DEDICATED" hint and you might disable PV TLB flush when > PV_DEDICATED is set. Why disable PV TLB flush for PV_DEDICATED(qspinlock)? Regards, Wanpeng Li