Re: [PATCH v2 12/15] kvm: i8254: Check LAPIC EOI pending when injecting irq on SVM AVIC

From: "Suthikulpanit, Suravee" <Suravee.Suthikulpanit@amd.com>
To: Alexander Graf <graf@amazon.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>
Cc: "pbonzini@redhat.com" <pbonzini@redhat.com>,
	"rkrcmar@redhat.com" <rkrcmar@redhat.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"jschoenh@amazon.de" <jschoenh@amazon.de>,
	"karahmed@amazon.de" <karahmed@amazon.de>,
	"rimasluk@amazon.com" <rimasluk@amazon.com>,
	"Grimm, Jon" <Jon.Grimm@amd.com>
Subject: Re: [PATCH v2 12/15] kvm: i8254: Check LAPIC EOI pending when injecting irq on SVM AVIC
Date: Mon, 26 Aug 2019 20:46:20 +0000	[thread overview]
Message-ID: <8320fecb-de61-1a01-b90d-a45f224de287@amd.com> (raw)
In-Reply-To: <ac9fa8d4-2c25-52a5-3938-3ce373b3c3e0@amazon.com>

Alex,

On 8/19/2019 5:42 AM, Alexander Graf wrote:
> 
> 
> On 15.08.19 18:25, Suthikulpanit, Suravee wrote:
>> ACK notifiers don't work with AMD SVM w/ AVIC when the PIT interrupt
>> is delivered as edge-triggered fixed interrupt since AMD processors
>> cannot exit on EOI for these interrupts.
>>
>> Add code to check LAPIC pending EOI before injecting any pending PIT
>> interrupt on AMD SVM when AVIC is activated.
>>
>> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
>> ---
>>   arch/x86/kvm/i8254.c | 31 +++++++++++++++++++++++++------
>>   1 file changed, 25 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c
>> index 4a6dc54..31c4a9b 100644
>> --- a/arch/x86/kvm/i8254.c
>> +++ b/arch/x86/kvm/i8254.c
>> @@ -34,10 +34,12 @@
>>   #include <linux/kvm_host.h>
>>   #include <linux/slab.h>
>> +#include <asm/virtext.h>
>>   #include "ioapic.h"
>>   #include "irq.h"
>>   #include "i8254.h"
>> +#include "lapic.h"
>>   #include "x86.h"
>>   #ifndef CONFIG_X86_64
>> @@ -236,6 +238,12 @@ static void destroy_pit_timer(struct kvm_pit *pit)
>>       kthread_flush_work(&pit->expired);
>>   }
>> +static inline void kvm_pit_reset_reinject(struct kvm_pit *pit)
>> +{
>> +    atomic_set(&pit->pit_state.pending, 0);
>> +    atomic_set(&pit->pit_state.irq_ack, 1);
>> +}
>> +
>>   static void pit_do_work(struct kthread_work *work)
>>   {
>>       struct kvm_pit *pit = container_of(work, struct kvm_pit, expired);
>> @@ -244,6 +252,23 @@ static void pit_do_work(struct kthread_work *work)
>>       int i;
>>       struct kvm_kpit_state *ps = &pit->pit_state;
>> +    /*
>> +     * Since, AMD SVM AVIC accelerates write access to APIC EOI
>> +     * register for edge-trigger interrupts. PIT will not be able
>> +     * to receive the IRQ ACK notifier and will always be zero.
>> +     * Therefore, we check if any LAPIC EOI pending for vector 0
>> +     * and reset irq_ack if no pending.
>> +     */
>> +    if (cpu_has_svm(NULL) && kvm->arch.apicv_state == APICV_ACTIVATED) {
>> +        int eoi = 0;
>> +
>> +        kvm_for_each_vcpu(i, vcpu, kvm)
>> +            if (kvm_apic_pending_eoi(vcpu, 0))
>> +                eoi++;
>> +        if (!eoi)
>> +            kvm_pit_reset_reinject(pit);
> 
> In which case would eoi be != 0 when APIC-V is active?

That would be the case when guest has not processed and/or still processing the interrupt.
Once the guest writes to APIC EOI register for edge-triggered interrupt for vector 0,
and the AVIC hardware accelerated the access by clearing the highest priority ISR bit,
then the eoi should be zero.

Suravee