From mboxrd@z Thu Jan  1 00:00:00 1970
From: Marcelo Tosatti <mtosatti@redhat.com>
Subject: Re: [PATCH 2/2] x86, apicv: Add Posted Interrupt supporting
Date: Thu, 31 Jan 2013 11:44:43 -0200
Message-ID: <20130131134443.GA4419@amt.cnet>
References: <1355383780-1367-1-git-send-email-yang.z.zhang@intel.com>
 <1355383780-1367-3-git-send-email-yang.z.zhang@intel.com>
 <20130124234339.GA18719@amt.cnet>
 <A9667DDFB95DB7438FA9D7D576C3D87E31AB93@SHSMSX101.ccr.corp.intel.com>
 <20130130230311.GA19081@amt.cnet>
 <20130131094348.GN15004@redhat.com>
 <20130131133245.GB3179@amt.cnet>
 <20130131133837.GB23213@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: "Zhang, Yang Z" <yang.z.zhang@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"Shan, Haitao" <haitao.shan@intel.com>,
	"Zhang, Xiantao" <xiantao.zhang@intel.com>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	"Anvin, H Peter" <h.peter.anvin@intel.com>
To: Gleb Natapov <gleb@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:17576 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1751731Ab3AaNo5 (ORCPT <rfc822;kvm@vger.kernel.org>);
	Thu, 31 Jan 2013 08:44:57 -0500
Content-Disposition: inline
In-Reply-To: <20130131133837.GB23213@redhat.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Thu, Jan 31, 2013 at 03:38:37PM +0200, Gleb Natapov wrote:
> On Thu, Jan 31, 2013 at 11:32:45AM -0200, Marcelo Tosatti wrote:
> > On Thu, Jan 31, 2013 at 11:43:48AM +0200, Gleb Natapov wrote:
> > > On Wed, Jan 30, 2013 at 09:03:11PM -0200, Marcelo Tosatti wrote:
> > > > Posted interrupt patch:
> > > > 2) Must move IN_GUEST_MODE assignment after local_irq_disable, in
> > > > vcpu_enter_guest function. Otherwise:
> > > > 
> > > > cpu0			vcpu1<->cpu1
> > > > 
> > > > 			vcpu->mode = IN_GUEST_MODE
> > > > 
> > > > if IN_GUEST_MODE == true
> > > > 	send IPI
> > > > 			local_irq_disable
> > > > 
> > > > PIR not transferred to VIRR, misses interrupt.
> > > > 
> > > cpu0 will set KVM_REQ_EVENT, so vmentry will be aborted after
> > > local_irq_disable() by ->requests check.
> > 
> > Yes, but you don't want KVM_REQ_EVENT+kick. It defeats the purpose 
> > of posted interrupts. You want
> > 
> > if vcpu in guest mode
> > 	send posted interrupt IPI
> > else
> > 	KVM_REQ_EVENT+kick
> > 
> I am thinking:
> 
>  set KVM_REQ_EVENT
>  if pi is enabled
>      send posted interrupt IPI
>  else
>      kick

KVM_REQ_EVENT must be after sending posted interrupt IPI. Otherwise on
the vcpu entry side

test_and_clear(KVM_REQ_EVENT) {
	No bits set in PIR
   }

What about item 4 below?

> > > > 3) Must check outstanding PIR notification bit unconditionally on
> > > > every VM-entry, because:
> > > > 
> > > > 	1. local_irq_disable
> > > > 	2. vcpu->mode = IN_GUEST_MODE
> > > > 	3. vmenter
> > > > 	4. vmexit
> > > > 	5. vcpu->mode = OUTSIDE_GUEST_MODE
> > > > 
> > > > If PIR-IPI-interrupt is sent between an event which triggers VM-exit
> > > > (for example, an external interrupt due to a device), and step 5
> > > > (assignment of vcpu->mode), the PIR->VIRR transfer before vmentry must
> > > > be made.
> > > Not sure I understand, but I think KVM_REQ_EVENT will cover that too.
> > 
> > See above.
> > 
> > > > 
> > > > 4) Today, an interrupt notification is cached on IRR until its delivered - further
> > > > interrupt injection is not generating further interrupt notification
> > > > bits. With PIR, behaviour changes: Its possible to have one bit in PIR and another 
> > > > on IRR APIC page (if timing is right). Is this harmless? Why?
> > > > 
> > > > 
> > > 
> > > --
> > > 			Gleb.
> 
> --
> 			Gleb.