[PATCH] x86/hvm: avoid p2m lookups for vlapic accesses.

[PATCH] x86/hvm: avoid p2m lookups for vlapic accesses.

[Tim Deegan]

It seems like this patch helps the XP performance problem, and it
doesn't hurt in other cases, so I think it should probably go in even if
we find a general solution.

commit ae8868ff4fc5a56e729399b031b27157d94febbf
Author: Tim Deegan <tim@xen.org>
Date:   Thu May 9 13:06:53 2013 +0100

    x86/hvm: avoid p2m lookups for vlapic accesses.
    
    The LAPIC base address is a known GFN, so we can skip looking up the
    p2m: we know it should be handled as emulated MMIO.  That helps
    performance in older Windows OSes, which make a _lot_ of TPR accesses.
    
    This will change the behaviour of any OS that maps other
    memory/devices at its LAPIC address; the new behaviour (the LAPIC
    mapping always wins) is closer to actual hardware behaviour.
    
    Signed-off-by: Tim Deegan <tim@xen.org>

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 7c3cb15..05ce054 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1361,6 +1361,17 @@ int hvm_hap_nested_page_fault(paddr_t gpa,
         }
     }
 
+    /* For the benefit of 32-bit WinXP (& older Windows) on AMD CPUs,
+     * a fast path for LAPIC accesses, skipping the p2m lookup. */
+    if ( !nestedhvm_vcpu_in_guestmode(v)
+         && gfn == vlapic_base_address(vcpu_vlapic(v)) >> PAGE_SHIFT )
+    {
+        if ( !handle_mmio() )
+            hvm_inject_hw_exception(TRAP_gp_fault, 0);
+        rc = 1;
+        goto out;
+    }
+
     p2m = p2m_get_hostp2m(v->domain);
     mfn = get_gfn_type_access(p2m, gfn, &p2mt, &p2ma, 
                               P2M_ALLOC | (access_w ? P2M_UNSHARE : 0), NULL);
@@ -2471,6 +2482,12 @@ static enum hvm_copy_result __hvm_copy(
             gfn = addr >> PAGE_SHIFT;
         }
 
+        /* For the benefit of 32-bit WinXP (& older Windows) on AMD CPUs,
+         * a fast path for LAPIC accesses, skipping the p2m lookup. */
+        if ( !nestedhvm_vcpu_in_guestmode(curr)
+             && gfn == vlapic_base_address(vcpu_vlapic(curr)) >> PAGE_SHIFT )
+            return HVMCOPY_bad_gfn_to_mfn;
+
         page = get_page_from_gfn(curr->domain, gfn, &p2mt, P2M_UNSHARE);
 
         if ( p2m_is_paging(p2mt) )

Re: [PATCH] x86/hvm: avoid p2m lookups for vlapic accesses.

[George Dunlap]

On Thu, May 9, 2013 at 1:12 PM, Tim Deegan <tim@xen.org> wrote:
> It seems like this patch helps the XP performance problem, and it
> doesn't hurt in other cases, so I think it should probably go in even if
> we find a general solution.
>
> commit ae8868ff4fc5a56e729399b031b27157d94febbf
> Author: Tim Deegan <tim@xen.org>
> Date:   Thu May 9 13:06:53 2013 +0100
>
>     x86/hvm: avoid p2m lookups for vlapic accesses.
>
>     The LAPIC base address is a known GFN, so we can skip looking up the
>     p2m: we know it should be handled as emulated MMIO.  That helps
>     performance in older Windows OSes, which make a _lot_ of TPR accesses.
>
>     This will change the behaviour of any OS that maps other
>     memory/devices at its LAPIC address; the new behaviour (the LAPIC
>     mapping always wins) is closer to actual hardware behaviour.
>
>     Signed-off-by: Tim Deegan <tim@xen.org>

Acked-by: George Dunlap <george.dunlap@eu.citrix.com>

>
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 7c3cb15..05ce054 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -1361,6 +1361,17 @@ int hvm_hap_nested_page_fault(paddr_t gpa,
>          }
>      }
>
> +    /* For the benefit of 32-bit WinXP (& older Windows) on AMD CPUs,
> +     * a fast path for LAPIC accesses, skipping the p2m lookup. */
> +    if ( !nestedhvm_vcpu_in_guestmode(v)
> +         && gfn == vlapic_base_address(vcpu_vlapic(v)) >> PAGE_SHIFT )
> +    {
> +        if ( !handle_mmio() )
> +            hvm_inject_hw_exception(TRAP_gp_fault, 0);
> +        rc = 1;
> +        goto out;
> +    }
> +
>      p2m = p2m_get_hostp2m(v->domain);
>      mfn = get_gfn_type_access(p2m, gfn, &p2mt, &p2ma,
>                                P2M_ALLOC | (access_w ? P2M_UNSHARE : 0), NULL);
> @@ -2471,6 +2482,12 @@ static enum hvm_copy_result __hvm_copy(
>              gfn = addr >> PAGE_SHIFT;
>          }
>
> +        /* For the benefit of 32-bit WinXP (& older Windows) on AMD CPUs,
> +         * a fast path for LAPIC accesses, skipping the p2m lookup. */
> +        if ( !nestedhvm_vcpu_in_guestmode(curr)
> +             && gfn == vlapic_base_address(vcpu_vlapic(curr)) >> PAGE_SHIFT )
> +            return HVMCOPY_bad_gfn_to_mfn;
> +
>          page = get_page_from_gfn(curr->domain, gfn, &p2mt, P2M_UNSHARE);
>
>          if ( p2m_is_paging(p2mt) )
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH] x86/hvm: avoid p2m lookups for vlapic accesses.

[Jan Beulich]

>>> On 09.05.13 at 14:12, Tim Deegan <tim@xen.org> wrote:
> It seems like this patch helps the XP performance problem, and it
> doesn't hurt in other cases, so I think it should probably go in even if
> we find a general solution.
> 
> commit ae8868ff4fc5a56e729399b031b27157d94febbf
> Author: Tim Deegan <tim@xen.org>
> Date:   Thu May 9 13:06:53 2013 +0100
> 
>     x86/hvm: avoid p2m lookups for vlapic accesses.
>     
>     The LAPIC base address is a known GFN, so we can skip looking up the
>     p2m: we know it should be handled as emulated MMIO.  That helps
>     performance in older Windows OSes, which make a _lot_ of TPR accesses.
>     
>     This will change the behaviour of any OS that maps other
>     memory/devices at its LAPIC address; the new behaviour (the LAPIC
>     mapping always wins) is closer to actual hardware behaviour.
>     
>     Signed-off-by: Tim Deegan <tim@xen.org>

I'd prefer PFN_DOWN() to be used instead of the explicit shifts,
but irrespective of that

Acked-by: Jan Beulich <jbeulich@suse.com>

along with the question whether this should be backported later to
4.2/4.1.

Jan

> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 7c3cb15..05ce054 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -1361,6 +1361,17 @@ int hvm_hap_nested_page_fault(paddr_t gpa,
>          }
>      }
>  
> +    /* For the benefit of 32-bit WinXP (& older Windows) on AMD CPUs,
> +     * a fast path for LAPIC accesses, skipping the p2m lookup. */
> +    if ( !nestedhvm_vcpu_in_guestmode(v)
> +         && gfn == vlapic_base_address(vcpu_vlapic(v)) >> PAGE_SHIFT )
> +    {
> +        if ( !handle_mmio() )
> +            hvm_inject_hw_exception(TRAP_gp_fault, 0);
> +        rc = 1;
> +        goto out;
> +    }
> +
>      p2m = p2m_get_hostp2m(v->domain);
>      mfn = get_gfn_type_access(p2m, gfn, &p2mt, &p2ma, 
>                                P2M_ALLOC | (access_w ? P2M_UNSHARE : 0), NULL);
> @@ -2471,6 +2482,12 @@ static enum hvm_copy_result __hvm_copy(
>              gfn = addr >> PAGE_SHIFT;
>          }
>  
> +        /* For the benefit of 32-bit WinXP (& older Windows) on AMD CPUs,
> +         * a fast path for LAPIC accesses, skipping the p2m lookup. */
> +        if ( !nestedhvm_vcpu_in_guestmode(curr)
> +             && gfn == vlapic_base_address(vcpu_vlapic(curr)) >> PAGE_SHIFT )
> +            return HVMCOPY_bad_gfn_to_mfn;
> +
>          page = get_page_from_gfn(curr->domain, gfn, &p2mt, P2M_UNSHARE);
>  
>          if ( p2m_is_paging(p2mt) )

Re: [PATCH] x86/hvm: avoid p2m lookups for vlapic accesses.

[Tim Deegan]

At 09:49 +0100 on 10 May (1368179395), Jan Beulich wrote:
> >>> On 09.05.13 at 14:12, Tim Deegan <tim@xen.org> wrote:
> > commit ae8868ff4fc5a56e729399b031b27157d94febbf
> > Author: Tim Deegan <tim@xen.org>
> > Date:   Thu May 9 13:06:53 2013 +0100
> > 
> >     x86/hvm: avoid p2m lookups for vlapic accesses.
> >     
> >     The LAPIC base address is a known GFN, so we can skip looking up the
> >     p2m: we know it should be handled as emulated MMIO.  That helps
> >     performance in older Windows OSes, which make a _lot_ of TPR accesses.
> >     
> >     This will change the behaviour of any OS that maps other
> >     memory/devices at its LAPIC address; the new behaviour (the LAPIC
> >     mapping always wins) is closer to actual hardware behaviour.
> >     
> >     Signed-off-by: Tim Deegan <tim@xen.org>
> 
> I'd prefer PFN_DOWN() to be used instead of the explicit shifts,

Good point; done.

> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> along with the question whether this should be backported later to
> 4.2/4.1.

I think it should be considered for 4.2; no need for it in 4.1 as the
p2m changes that caused the problem only appear in 4.2

Cheers,

Tim