RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set

["Wu, Feng" <feng.wu@intel.com>]

> -----Original Message-----
> From: Zhang, Yang Z
> Sent: Friday, December 19, 2014 1:26 PM
> To: Wu, Feng; Paolo Bonzini; kvm@vger.kernel.org
> Cc: iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org;
> kvm@vger.kernel.org
> Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
> 
> Wu, Feng wrote on 2014-12-19:
> >
> >
> > Zhang, Yang Z wrote on 2014-12-19:
> >> Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is
> >> set
> >>
> >> Wu, Feng wrote on 2014-12-19:
> >>>
> >>>
> >>> Zhang, Yang Z wrote on 2014-12-19:
> >>>> Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN'
> >>>> is set
> >>>>
> >>>> Wu, Feng wrote on 2014-12-19:
> >>>>>
> >>>>>
> >>>>> iommu-bounces@lists.linux-foundation.org wrote on
> >>>> mailto:iommu-bounces@lists.linux-foundation.org] On Behalf Of:
> >>>>>> Cc: iommu@lists.linux-foundation.org;
> >>>>>> linux-kernel@vger.kernel.org; kvm@vger.kernel.org
> >>>>>> Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN'
> >>>>>> is set
> >>>>>>
> >>>>>> Paolo Bonzini wrote on 2014-12-18:
> >>>>>>>
> >>>>>>>
> >>>>>>> On 18/12/2014 04:14, Wu, Feng wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> linux-kernel-owner@vger.kernel.org wrote on
> >>>>>> mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of Paolo:
> >>>>>>>>> x86@kernel.org; Gleb Natapov; Paolo Bonzini;
> >>>>>>>>> dwmw2@infradead.org;
> >>>>>>>>> joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org; Alex
> Williamson;
> >>>>>>>>> joro-zLv9SwRftAIdnm+Jiang Liu Cc:
> >>>>>>>>> iommu@lists.linux-foundation.org;
> >>>>>>>>> linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; KVM
> list;
> >>>>>>>>> Eric Auger Subject: Re: [v3 25/26] KVM: Suppress
> >>>>>>>>> posted-interrupt when 'SN' is set
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On 12/12/2014 16:14, Feng Wu wrote:
> >>>>>>>>>> Currently, we don't support urgent interrupt, all
> >>>>>>>>>> interrupts are recognized as non-urgent interrupt, so we
> >>>>>>>>>> cannot send posted-interrupt when 'SN' is set.
> >>>>>>>>>
> >>>>>>>>> Can this happen?  If the vcpu is in guest mode, it cannot
> >>>>>>>>> have been scheduled out, and that's the only case when SN is set.
> >>>>>>>>>
> >>>>>>>>> Paolo
> >>>>>>>>
> >>>>>>>> Currently, the only place where SN is set is vCPU is
> >>>>>>>> preempted and
> >>>>>>
> >>>>>> If the vCPU is preempted, shouldn't the subsequent be ignored?
> >>>>>> What happens if a PI is occurs when vCPU is preempted?
> >>>>>
> >>>>> If a vCPU is preempted, the 'SN' bit is set, the subsequent
> >>>>> interrupts are suppressed for posting.
> >>>>
> >>>> I mean what happens if we don't set SN bit. From my point, if
> >>>> preempter already disabled the interrupt, it is ok to leave SN
> >>>> bit as zero. But if preempter enabled the interrupt, doesn't this
> >>>> mean he allow interrupt to happen? BTW, since there already has
> >>>> ON bit, so this means there only have one interrupt arrived at
> >>>> most and it doesn't hurt performance. Do we really need to set SN bit?
> >>>
> >>>
> >>> See this scenario:
> >>> vCPU0 is running on pCPU0
> >>> --> vCPU0 is preempted by vCPU1
> >>> --> Then vCPU1 is running on pCPU0 and vCPU0 is waiting for
> >>> --> schedule in runqueue
> >>>
> >>> If the we don't set SN for vCPU0, then all subsequent interrupts
> >>> for
> >>> vCPU0 is posted to vCPU1, this will consume hardware and software
> >>
> >> The PI vector for vCPU1 is notification vector, but the PI vector
> >> for
> >> vCPU0 should be wakeup vector. Why vCPU1 will consume this PI event?
> >
> > Wakeup vector is only used for blocking case, when vCPU is preempted
> > and waiting in the runqueue, the NV is the notification vector.
> 
> I see your point. But from performance point, if we can schedule the vCPU to
> another PCPU to handle the interrupt, it would helpful. But I remember current
> KVM will not schedule the vCPU in run queue (even though it got preempted) to
> another pCPU to run(Am I right?). So it may hard to do it.
> 

KVM is using the Linux scheduler, when the preempted vCPU (in runqueue) is
scheduled again depends on the scheduling algorithm itself, I think it is a little
hard for us to get involved.

I think what you mentioned is a little like the urgent interrupt in VT-d PI Spec.
For this kind of interrupts, if an interrupt is coming for an preempted vCPU
(waiting in the run queue), we need to schedule the vCPU immediately. This
is some real time things. And we don't support urgent interrupt so far.

Thanks,
Feng

> >
> > Thanks,
> > Feng
> >
> >>
> >>> efforts and in fact it is not needed at all. If SN is set for
> >>> vCPU0, VT-d hardware will not issue Notification Event for vCPU0
> >>> when an interrupt is for it, but just setting the related PIR bit.
> >>>
> >>> Thanks,
> >>> Feng
> >>>
> >>>>
> >>>>>
> >>>>> Thanks,
> >>>>> Feng
> >>>>>
> >>>>>>
> >>>>>>>> waiting for the next scheduling in the runqueue. But I am not
> >>>>>>>> sure whether we need to set SN for other purpose in future.
> >>>>>>>> Adding SN checking here is just to follow the Spec.
> >>>>>>>> non-urgent interrupts are suppressed
> >>>>>>> when SN is set.
> >>>>>>>
> >>>>>>> I would change that to a WARN_ON_ONCE then.
> >>>>>>
> >>>>>>
> >>>>>> Best regards,
> >>>>>> Yang
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> iommu mailing list
> >>>>>> iommu@lists.linux-foundation.org
> >>>>>> https://lists.linuxfoundation.org/mailman/listinfo/iommu
> >>>>
> >>>>
> >>>> Best regards,
> >>>> Yang
> >>>>
> >>
> >>
> >> Best regards,
> >> Yang
> >>
> 
> 
> Best regards,
> Yang
>