Re: [PATCH v2 3/4] VMX: Assign the right value to 'NDST' field in a concern case

From: "Wu, Feng" <feng.wu@intel.com>
To: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>, Keir Fraser <keir@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Dario Faggioli <dario.faggioli@citrix.com>,
	"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
	Jan Beulich <jbeulich@suse.com>, "Wu, Feng" <feng.wu@intel.com>
Subject: Re: [PATCH v2 3/4] VMX: Assign the right value to 'NDST' field in a concern case
Date: Fri, 24 Jun 2016 09:08:45 +0000	[thread overview]
Message-ID: <E959C4978C3B6342920538CF579893F01972E6E1@SHSMSX103.ccr.corp.intel.com> (raw)
In-Reply-To: <CAFLBxZbk_JyrGsmXmZ10-HmE9xy+188mKN=6qZgiEL3L+_xe8A@mail.gmail.com>

> -----Original Message-----
> From: dunlapg@gmail.com [mailto:dunlapg@gmail.com] On Behalf Of George
> Dunlap
> Sent: Thursday, June 23, 2016 2:01 AM
> To: Wu, Feng <feng.wu@intel.com>
> Cc: xen-devel@lists.xen.org; Tian, Kevin <kevin.tian@intel.com>; Keir Fraser
> <keir@xen.org>; Andrew Cooper <andrew.cooper3@citrix.com>; Dario Faggioli
> <dario.faggioli@citrix.com>; Jan Beulich <jbeulich@suse.com>
> Subject: Re: [Xen-devel] [PATCH v2 3/4] VMX: Assign the right value to 'NDST'
> field in a concern case
> 
> On Thu, May 26, 2016 at 2:39 PM, Feng Wu <feng.wu@intel.com> wrote:
> > Normally, in vmx_cpu_block() 'NDST' filed should have the same
> > value with 'dest' or 'MASK_INSR(dest, PI_xAPIC_NDST_MASK)' depending
> > on whether x2apic is enabled. However, in the following scenario,
> > 'NDST' has different value:
> >
> > 'vcpu_block' hook gets assigned in vmx_pi_hooks_assign(), but all
> > other three PI hooks have not been assigned or not been excuted yet.
> > And during this interval, we are running in vmx_vcpu_block(), then
> > 'NDST' may have different value.
> >
> > This patch fix this concern case.
> >
> > Signed-off-by: Feng Wu <feng.wu@intel.com>
> 
> I agree with Jan that a cleaner solution here would be making sure
> that all the appropriate state is actually set up for all vcpus before
> leaving vmx_pi_hooks_assign().  With the patch you propose, the
> following sequence of events is possible:
> 
> * vcpu 0 starts running on a pcpu
> * a device is assigned, causing the hooks to be set
> * an interrupt from the device is routed to vcpu 0, but it is not
> actually delivered properly, since ndst is not pointing to the right
> processor.
> 
> One option would be to pause all vcpus before setting the hooks and
> then un-pause them; this would force all the vcpus to go through
> vmx_pi_switch_to() before vmx_vcpu_block().  Another would be to grab
> the scheduler lock for each pcpu and write the vcpu's ndst with the
> appropriate value before setting the hooks.

That sounds a great idea. Besides that, maybe we can also pause/unpause
the domain before/after unsetting the hooks, then we don't need to
care about the race condition when vmx_pi_hooks_deassign() and
vmx_vcpu_block() get called at the same time. After unpause the domain,
we can safely remove the vCPUs from the per-cpu blocking list if needed.

Thanks,
Feng

> 
>  -George
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel