Re: [PATCH v2] x86/hvm: re-work viridian APIC assist code

From: Paul Durrant <Paul.Durrant@citrix.com>
To: 'David Woodhouse' <dwmw2@infradead.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Eslam Elnikety <elnikety@amazon.de>,
	Andrew Cooper <Andrew.Cooper3@citrix.com>,
	Shan Haitao <haitao.shan@intel.com>,
	Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v2] x86/hvm: re-work viridian APIC assist code
Date: Wed, 5 Sep 2018 09:36:22 +0000	[thread overview]
Message-ID: <88abe6b0dee947ad93af543653dde12a@AMSPEX02CL03.citrite.net> (raw)
In-Reply-To: <1536093061.31549.47.camel@infradead.org>

> -----Original Message-----
> From: David Woodhouse [mailto:dwmw2@infradead.org]
> Sent: 04 September 2018 21:31
> To: Paul Durrant <Paul.Durrant@citrix.com>; xen-devel@lists.xenproject.org
> Cc: Andrew Cooper <Andrew.Cooper3@citrix.com>; Jan Beulich
> <jbeulich@suse.com>; Eslam Elnikety <elnikety@amazon.de>; Shan Haitao
> <haitao.shan@intel.com>
> Subject: Re: [Xen-devel] [PATCH v2] x86/hvm: re-work viridian APIC assist
> code
> 
> On Mon, 2018-09-03 at 10:12 +0000, Paul Durrant wrote:
> >
> > I believe APIC assist is intended for fully synthetic interrupts.
> 
> Hm, if by 'fully synthetic interrupts' you mean
> vlapic_virtual_intr_delivery_enabled(), then no I think APIC assist
> doesn't get used in that case at all.

No, what I meant was that it was my belief that Windows would avoid APIC assist for what it sees as devices on a hardware bus but - now I think about it - that would not be wonderfully useful behaviour and would certainly squash the reason I put the support in anyway, which is for the edge triggered per-cpu event channel upcalls from Xen (since the driver registering the interrupts is bound to the Xen platform PCI device).

> 
> > Is it definitely this patch that causes the problem? It was only
> > intended to fix previous incorrectness but, if this is the culprit,
> > then it's clearly caused collateral damage in a logically unrelated
> > area.
> 
> Not entirely. The performance gain we observed with APIC assist in the
> first place was basically stolen. It wasn't just bypassing the vmexit
> for that EOI; it was *so* much faster because it actually didn't ever
> do the EOI properly at all.
> 
> You fixed that omission and unsurprisingly it got slower again; most of
> the apparent benefit of APIC assist is lost. But that's because it was
> never really doing the right thing in the first place.
> 
> That EOI handling for unmaskable MSI is really painfully slow, so my
> hack bypasses it in the common case where it isn't really necessary.
> FWIW I've done it in my tree with a single per-domain flag rather than
> a per-vector bitmap now, which makes it slightly simpler.

I see. Given that Windows has used APIC assist to circumvent its EOI then I wonder whether we can get away with essentially doing the same. I.e. for a completed APIC assist found in vlapic_has_pending_irq() we simply clear the APIC assist and highest vector in the ISR, rather than calling through to vlapic_EOI_set() and suffering the overhead. I'll spin up a patch and give it a whirl.

  Paul
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel