Re: vmx: VT-d posted-interrupt core logic handling

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: "Wu, Feng" <feng.wu@intel.com>
Cc: Lars Kurth <lars.kurth@citrix.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	George Dunlap <George.Dunlap@eu.citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Dario Faggioli <dario.faggioli@citrix.com>,
	Ian Jackson <Ian.Jackson@eu.citrix.com>,
	GeorgeDunlap <george.dunlap@citrix.com>,
	"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
	David Vrabel <david.vrabel@citrix.com>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: vmx: VT-d posted-interrupt core logic handling
Date: Tue, 17 May 2016 09:27:13 -0400	[thread overview]
Message-ID: <20160517132712.GA28261@char.us.oracle.com> (raw)
In-Reply-To: <E959C4978C3B6342920538CF579893F00C36BAC0@SHSMSX104.ccr.corp.intel.com>

On Thu, Mar 10, 2016 at 01:36:34PM +0000, Wu, Feng wrote:
> 
> 
> > -----Original Message-----
> > From: Tian, Kevin
> > Sent: Thursday, March 10, 2016 6:06 PM
> > To: Jan Beulich <JBeulich@suse.com>
> > Cc: Andrew Cooper <andrew.cooper3@citrix.com>; Dario Faggioli
> > <dario.faggioli@citrix.com>; David Vrabel <david.vrabel@citrix.com>;
> > GeorgeDunlap <george.dunlap@citrix.com>; Lars Kurth <lars.kurth@citrix.com>;
> > George Dunlap <George.Dunlap@eu.citrix.com>; Ian Jackson
> > <Ian.Jackson@eu.citrix.com>; Wu, Feng <feng.wu@intel.com>; xen-
> > devel@lists.xen.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > Subject: RE: vmx: VT-d posted-interrupt core logic handling
> > 
> > 
> > Hi, Jan,
> > 
> > I'm thinking your earlier idea about evenly distributed list:
> > 
> > --
> > Ah, right, I think that limitation was named before, yet I've
> > forgotten about it again. But that only slightly alters the
> > suggestion: To distribute vCPU-s evenly would then require to
> > change their placement on the pCPU in the course of entering
> > blocked state.
> > --
> > 
> > Actually after more thinking, there is no hard requirement that
> > the vcpu must block on the pcpu which is configured in 'NDST'
> > of that vcpu's PI descriptor. What really matters, is that the
> > vcpu is added to the linked list of the very pcpu, then when PI
> > notification comes we can always find out the vcpu struct from
> > that pcpu's linked list. Of course one drawback of such placement
> > is additional IPI incurred in wake up path.
> > 
> > Then one possible optimized policy within vmx_vcpu_block could
> > be:
> > 
> > (Say PCPU1 which VCPU1 is currently blocked on)
> > - As long as the #vcpus in the linked list on PCPU1 is below a
> > threshold (say 16), add VCPU1 to the list. NDST set to PCPU1;
> > Upon PI notification on PCPU1, local linked list is searched to
> > find VCPU1 and then VCPU1 will be unblocked on PCPU1;
> > 
> > - Otherwise, add VCPU1 to PCPU2 based on a simple distribution
> > algorithm (based on vcpu_id/vm_id). VCPU1 still blocks on PCPU1
> > but NDST set to PCPU2. Upon notification on PCPU2, local linked
> > list is searched to find VCPU1 and then an IPI is sent to PCPU1 to
> > unblock VCPU1;
> > 
> > Feng, do you see any overlook here? :-)
> 
> Kevin, thanks for the suggestion, it sounds a good idea, I will think
> it a bit more and do some trials based on it.

Hey!

I am curious how the trials went? This feature is pretty awesome and
I am wondering what the next step is?

Thanks!

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel