[PATCH] KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register

[PATCH] KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register

[Like Xu]

From: Like Xu <likexu@tencent.com>

If we run the following perf command in an AMD Milan guest:

  perf stat \
  -e cpu/event=0x1d0/ \
  -e cpu/event=0x1c7/ \
  -e cpu/umask=0x1f,event=0x18e/ \
  -e cpu/umask=0x7,event=0x18e/ \
  -e cpu/umask=0x18,event=0x18e/ \
  ./workload

dmesg will report a #GP warning from an unchecked MSR access
error on MSR_F15H_PERF_CTLx.

This is because according to APM (Revision: 4.03) Figure 13-7,
the bits [35:32] of AMD PerfEvtSeln register is a part of the
event select encoding, which extends the EVENT_SELECT field
from 8 bits to 12 bits.

Opportunistically update pmu->reserved_bits for reserved bit 19.

Reported-by: Jim Mattson <jmattson@google.com>
Fixes: ca724305a2b0 ("KVM: x86/vPMU: Implement AMD vPMU code for KVM")
Signed-off-by: Like Xu <likexu@tencent.com>
---
 arch/x86/kvm/svm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
index 871c426ec389..b4095dfeeee6 100644
--- a/arch/x86/kvm/svm/pmu.c
+++ b/arch/x86/kvm/svm/pmu.c
@@ -281,7 +281,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 		pmu->nr_arch_gp_counters = AMD64_NUM_COUNTERS;
 
 	pmu->counter_bitmask[KVM_PMC_GP] = ((u64)1 << 48) - 1;
-	pmu->reserved_bits = 0xffffffff00200000ull;
+	pmu->reserved_bits = 0xfffffff000280000ull;
 	pmu->version = 1;
 	/* not applicable to AMD; but clean them to prevent any fall out */
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
-- 
2.33.1

Re: [PATCH] KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register

[Paolo Bonzini]

On 11/18/21 14:03, Like Xu wrote:
> 
> This is because according to APM (Revision: 4.03) Figure 13-7,
> the bits [35:32] of AMD PerfEvtSeln register is a part of the
> event select encoding, which extends the EVENT_SELECT field
> from 8 bits to 12 bits.

Queued, thanks.

Paolo

Re: [PATCH] KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register

[Paolo Bonzini]

On 11/18/21 14:03, Like Xu wrote:
> From: Like Xu <likexu@tencent.com>
> 
> If we run the following perf command in an AMD Milan guest:
> 
>    perf stat \
>    -e cpu/event=0x1d0/ \
>    -e cpu/event=0x1c7/ \
>    -e cpu/umask=0x1f,event=0x18e/ \
>    -e cpu/umask=0x7,event=0x18e/ \
>    -e cpu/umask=0x18,event=0x18e/ \
>    ./workload
> 
> dmesg will report a #GP warning from an unchecked MSR access
> error on MSR_F15H_PERF_CTLx.
> 
> This is because according to APM (Revision: 4.03) Figure 13-7,
> the bits [35:32] of AMD PerfEvtSeln register is a part of the
> event select encoding, which extends the EVENT_SELECT field
> from 8 bits to 12 bits.
> 
> Opportunistically update pmu->reserved_bits for reserved bit 19.
> 
> Reported-by: Jim Mattson <jmattson@google.com>
> Fixes: ca724305a2b0 ("KVM: x86/vPMU: Implement AMD vPMU code for KVM")
> Signed-off-by: Like Xu <likexu@tencent.com>
> ---
>   arch/x86/kvm/svm/pmu.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
> index 871c426ec389..b4095dfeeee6 100644
> --- a/arch/x86/kvm/svm/pmu.c
> +++ b/arch/x86/kvm/svm/pmu.c
> @@ -281,7 +281,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
>   		pmu->nr_arch_gp_counters = AMD64_NUM_COUNTERS;
>   
>   	pmu->counter_bitmask[KVM_PMC_GP] = ((u64)1 << 48) - 1;
> -	pmu->reserved_bits = 0xffffffff00200000ull;
> +	pmu->reserved_bits = 0xfffffff000280000ull;
>   	pmu->version = 1;
>   	/* not applicable to AMD; but clean them to prevent any fall out */
>   	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
> 

Queued, thanks.

Re: [PATCH] KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register

[Jim Mattson]

On Thu, Nov 18, 2021 at 5:03 AM Like Xu <like.xu.linux@gmail.com> wrote:
>
> From: Like Xu <likexu@tencent.com>
>
> If we run the following perf command in an AMD Milan guest:
>
>   perf stat \
>   -e cpu/event=0x1d0/ \
>   -e cpu/event=0x1c7/ \
>   -e cpu/umask=0x1f,event=0x18e/ \
>   -e cpu/umask=0x7,event=0x18e/ \
>   -e cpu/umask=0x18,event=0x18e/ \
>   ./workload
>
> dmesg will report a #GP warning from an unchecked MSR access
> error on MSR_F15H_PERF_CTLx.
>
> This is because according to APM (Revision: 4.03) Figure 13-7,
> the bits [35:32] of AMD PerfEvtSeln register is a part of the
> event select encoding, which extends the EVENT_SELECT field
> from 8 bits to 12 bits.
>
> Opportunistically update pmu->reserved_bits for reserved bit 19.
>
> Reported-by: Jim Mattson <jmattson@google.com>
> Fixes: ca724305a2b0 ("KVM: x86/vPMU: Implement AMD vPMU code for KVM")
> Signed-off-by: Like Xu <likexu@tencent.com>
> ---
>  arch/x86/kvm/svm/pmu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
> index 871c426ec389..b4095dfeeee6 100644
> --- a/arch/x86/kvm/svm/pmu.c
> +++ b/arch/x86/kvm/svm/pmu.c
> @@ -281,7 +281,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
>                 pmu->nr_arch_gp_counters = AMD64_NUM_COUNTERS;
>
>         pmu->counter_bitmask[KVM_PMC_GP] = ((u64)1 << 48) - 1;
> -       pmu->reserved_bits = 0xffffffff00200000ull;
> +       pmu->reserved_bits = 0xfffffff000280000ull;

Bits 40 and 41 are guest mode and host mode. They cannot be reserved
if the guest supports nested SVM.

>         pmu->version = 1;
>         /* not applicable to AMD; but clean them to prevent any fall out */
>         pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
> --
> 2.33.1
>

Re: [PATCH] KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register

[Like Xu]

On 12/2/2022 4:39 pm, Jim Mattson wrote:
>> -       pmu->reserved_bits = 0xffffffff00200000ull;
>> +       pmu->reserved_bits = 0xfffffff000280000ull;
> Bits 40 and 41 are guest mode and host mode. They cannot be reserved
> if the guest supports nested SVM.
> 

Indeed, we need (some hands) to do more pmu tests on nested SVM.

Re: [PATCH] KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register

[Jim Mattson]

On Tue, Feb 15, 2022 at 11:47 PM Like Xu <like.xu.linux@gmail.com> wrote:
>
> On 12/2/2022 4:39 pm, Jim Mattson wrote:
> >> -       pmu->reserved_bits = 0xffffffff00200000ull;
> >> +       pmu->reserved_bits = 0xfffffff000280000ull;
> > Bits 40 and 41 are guest mode and host mode. They cannot be reserved
> > if the guest supports nested SVM.
> >
>
> Indeed, we need (some hands) to do more pmu tests on nested SVM.

Actually, it's not just nested SVM.

When we enable vPMU for an Ubuntu guest that is incapable of nested
SVM, we see errors like the following:

root@Ubuntu1804:~# perf stat -e r26 -a sleep 1

 Performance counter stats for 'system wide':

                 0      r26


       1.001070977 seconds time elapsed

Feb 23 03:59:58 Ubuntu1804 kernel: [  405.379957] unchecked MSR access
error: WRMSR to 0xc0010200 (tried to write 0x0000020000130026) at rIP:
0xffffffff9b276a28 (native_write_msr+0x8/0x30)
Feb 23 03:59:58 Ubuntu1804 kernel: [  405.379958] Call Trace:
Feb 23 03:59:58 Ubuntu1804 kernel: [  405.379963]
amd_pmu_disable_event+0x27/0x90

If the standard Linux perf tool sets "exclude_guest" by default, even
when EFER.SVME is clear, then amd_core_hw_config() in the guest kernel
will set bit 41 (again, without checking EFER.SVME). This WRMSR should
not raise #GP.

Current AMD hardware doesn't raise #GP for any value written to a
PerfEvtSeln MSR. I don't think KVM should ever synthesize a #GP
either. Perhaps we should just mask off the bits that you have
indicated as reserved, above.