Re: [Xen-devel] [PATCH 2/2] x86/tlb: use Xen L0 assisted TLB flush when available

[Wei Liu <wl@xen.org>]

On Tue, Dec 24, 2019 at 02:26:16PM +0100, Roger Pau Monne wrote:
> Use Xen's L0 HVMOP_flush_tlbs hypercall when available in order to
> perform flushes. This greatly increases the performance of tlb flushes
> when running with a high amount of vCPUs as a Xen guest, and is
> specially important when running in shim mode.
> 
> The following figures are from a PV guest running `make -j342 xen` in
> shim mode with 32 vCPUs.
> 
> Using x2APIC and ALLBUT shorthand:
> real	4m35.973s
> user	4m35.110s
> sys	36m24.117s
> 
> Using L0 assisted flush:
> real	1m17.391s
> user	4m42.413s
> sys	6m20.773s
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
>  xen/arch/x86/guest/xen/xen.c    | 11 +++++++++++
>  xen/arch/x86/smp.c              |  6 ++++++
>  xen/include/asm-x86/guest/xen.h |  7 +++++++
>  3 files changed, 24 insertions(+)
> 
> diff --git a/xen/arch/x86/guest/xen/xen.c b/xen/arch/x86/guest/xen/xen.c
> index 6dbc5f953f..e6493caecf 100644
> --- a/xen/arch/x86/guest/xen/xen.c
> +++ b/xen/arch/x86/guest/xen/xen.c
> @@ -281,6 +281,17 @@ int xg_free_unused_page(mfn_t mfn)
>      return rangeset_remove_range(mem, mfn_x(mfn), mfn_x(mfn));
>  }
>  
> +int xg_flush_tlbs(void)
> +{
> +    int rc;
> +
> +    do {
> +        rc = xen_hypercall_hvm_op(HVMOP_flush_tlbs, NULL);
> +    } while ( rc == -ERESTART );
> +
> +    return rc;
> +}
> +

Is it possible to make this a hook in the hypervisor_op?

I can foresee there will be something similar for Hyper-V.

>  static void ap_resume(void *unused)
>  {
>      map_vcpuinfo();
> diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
> index 427c33db9d..a892db28c1 100644
> --- a/xen/arch/x86/smp.c
> +++ b/xen/arch/x86/smp.c
> @@ -15,6 +15,7 @@
>  #include <xen/perfc.h>
>  #include <xen/spinlock.h>
>  #include <asm/current.h>
> +#include <asm/guest.h>
>  #include <asm/smp.h>
>  #include <asm/mc146818rtc.h>
>  #include <asm/flushtlb.h>
> @@ -235,6 +236,11 @@ void flush_area_mask(const cpumask_t *mask, const void *va, unsigned int flags)
>      {
>          bool cpus_locked = false;
>  
> +        if ( xen_guest &&

Also it would be better to not expose xen_guest here. It is x86 generic
code after all.

I would probably introduce a function to tell if Xen is running
virtualised or not.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel