Re: [PATCH v3 0/4] KVM: x86: hyper-v: Conditionally allow SynIC with APICv/AVIC

From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Maxim Levitsky <mlevitsk@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>,
	linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [PATCH v3 0/4] KVM: x86: hyper-v: Conditionally allow SynIC with APICv/AVIC
Date: Mon, 14 Jun 2021 09:40:18 +0200	[thread overview]
Message-ID: <87zgvsx5b1.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <f294faba4e5d25aba8773f36170d1309236edd3b.camel@redhat.com>

Maxim Levitsky <mlevitsk@redhat.com> writes:

> On Wed, 2021-06-09 at 17:09 +0200, Vitaly Kuznetsov wrote:
>> Changes since v2:
>> - First two patches got merged, rebase.
>> - Use 'enable_apicv = avic = ...' in PATCH1 [Paolo]
>> - Collect R-b tags for PATCH2 [Sean, Max]
>> - Use hv_apicv_update_work() to get out of SRCU lock [Max]
>> - "KVM: x86: Check for pending interrupts when APICv is getting disabled"
>>   added.
>> 
>> Original description:
>> 
>> APICV_INHIBIT_REASON_HYPERV is currently unconditionally forced upon
>> SynIC activation as SynIC's AutoEOI is incompatible with APICv/AVIC. It is,
>> however, possible to track whether the feature was actually used by the
>> guest and only inhibit APICv/AVIC when needed.
>> 
>> The series can be tested with the followin hack:
>> 
>> diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
>> index 9a48f138832d..65a9974f80d9 100644
>> --- a/arch/x86/kvm/cpuid.c
>> +++ b/arch/x86/kvm/cpuid.c
>> @@ -147,6 +147,13 @@ void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu)
>>                                            vcpu->arch.ia32_misc_enable_msr &
>>                                            MSR_IA32_MISC_ENABLE_MWAIT);
>>         }
>> +
>> +       /* Dirty hack: force HV_DEPRECATING_AEOI_RECOMMENDED. Not to be merged! */
>> +       best = kvm_find_cpuid_entry(vcpu, HYPERV_CPUID_ENLIGHTMENT_INFO, 0);
>> +       if (best) {
>> +               best->eax &= ~HV_X64_APIC_ACCESS_RECOMMENDED;
>> +               best->eax |= HV_DEPRECATING_AEOI_RECOMMENDED;
>> +       }
>>  }
>>  EXPORT_SYMBOL_GPL(kvm_update_cpuid_runtime);
>>  
>> Vitaly Kuznetsov (4):
>>   KVM: x86: Use common 'enable_apicv' variable for both APICv and AVIC
>>   KVM: x86: Drop vendor specific functions for APICv/AVIC enablement
>>   KVM: x86: Check for pending interrupts when APICv is getting disabled
>>   KVM: x86: hyper-v: Deactivate APICv only when AutoEOI feature is in
>>     use
>> 
>>  arch/x86/include/asm/kvm_host.h |  9 +++++-
>>  arch/x86/kvm/hyperv.c           | 51 +++++++++++++++++++++++++++++----
>>  arch/x86/kvm/svm/avic.c         | 14 ++++-----
>>  arch/x86/kvm/svm/svm.c          | 22 ++++++++------
>>  arch/x86/kvm/svm/svm.h          |  2 --
>>  arch/x86/kvm/vmx/capabilities.h |  1 -
>>  arch/x86/kvm/vmx/vmx.c          |  2 --
>>  arch/x86/kvm/x86.c              | 18 ++++++++++--
>>  8 files changed, 86 insertions(+), 33 deletions(-)
>> 
>
> Hi!
>
> I hate to say it, but at least one of my VMs doesn't boot amymore
> with avic=1, after the recent updates. I'll bisect this soon,
> but this is likely related to this series.
>
> I will also review this series very soon.
>
> When the VM fails, it hangs on the OVMF screen and I see this
> in qemu logs:
>
> KVM: injection failed, MSI lost (Operation not permitted)
> KVM: injection failed, MSI lost (Operation not permitted)
> KVM: injection failed, MSI lost (Operation not permitted)
> KVM: injection failed, MSI lost (Operation not permitted)
> KVM: injection failed, MSI lost (Operation not permitted)
> KVM: injection failed, MSI lost (Operation not permitted)
>

-EPERM?? Interesting... strace(1) may come handy...

$ git grep EPERM kvm/queue arch/x86/kvm/ 
kvm/queue:arch/x86/kvm/x86.c:           ret = -KVM_EPERM;
(just this one)

kvm_emulate_hypercall():
...
b3646477d458f arch/x86/kvm/x86.c (Jason Baron                2021-01-14 22:27:56 -0500  8433)   if (static_call(kvm_x86_get_cpl)(vcpu) != 0) {
07708c4af1346 arch/x86/kvm/x86.c (Jan Kiszka                 2009-08-03 18:43:28 +0200  8434)           ret = -KVM_EPERM;
696ca779a928d arch/x86/kvm/x86.c (Radim Krčmář               2018-05-24 17:50:56 +0200  8435)           goto out;
07708c4af1346 arch/x86/kvm/x86.c (Jan Kiszka                 2009-08-03 18:43:28 +0200  8436)   }
...

Doesn't seem we have any updates here, curious what your bisection will
point us to.

-- 
Vitaly