From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Wu, Feng" Subject: Re: [PATCH v9 01/17] VT-d Posted-intterrupt (PI) design Date: Tue, 24 Nov 2015 06:49:32 +0000 Message-ID: References: <1446540207-4806-1-git-send-email-feng.wu@intel.com> <1446540207-4806-2-git-send-email-feng.wu@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Tian, Kevin" , "xen-devel@lists.xen.org" Cc: "Wu, Feng" , George Dunlap , Andrew Cooper , Jan Beulich , "Zhang, Yang Z" , Keir Fraser List-Id: xen-devel@lists.xenproject.org > -----Original Message----- > From: Tian, Kevin > Sent: Tuesday, November 24, 2015 2:35 PM > To: Wu, Feng ; xen-devel@lists.xen.org > Cc: Zhang, Yang Z ; Jan Beulich > ; Keir Fraser ; Andrew Cooper > ; George Dunlap > > Subject: RE: [PATCH v9 01/17] VT-d Posted-intterrupt (PI) design > > > From: Wu, Feng > > Sent: Tuesday, November 03, 2015 4:43 PM > > > > > > diff --git a/docs/misc/vtd-pi.txt b/docs/misc/vtd-pi.txt > > new file mode 100644 > > index 0000000..f9b4637 > > --- /dev/null > > +++ b/docs/misc/vtd-pi.txt > > @@ -0,0 +1,329 @@ > > +Authors: Feng Wu > > + > > +VT-d Posted-interrupt (PI) design for XEN > > + > > +Background > > +========== > > +With the development of virtualization, there are more and more device > > +assignment requirements. However, today when a VM is running with > > +assigned devices (such as, NIC), external interrupt handling for the > assigned > > +devices always needs VMM intervention. > > + > > +VT-d Posted-interrupt is a more enhanced method to handle interrupts > > +in the virtualization environment. Interrupt posting is the process by > > +which an interrupt request is recorded in a memory-resident > > +posted-interrupt-descriptor structure by the root-complex, followed by > > +an optional notification event issued to the CPU complex. > > Some clarification required here. Is "Interrupt Posting" only used to > represent the process of VT-d Posted-interrupt, or also for CPU-side > posting through IPI? From above context, and later explanation about > "Processor Support" and "Root-complex Support", looks the former > is true. Then how do we call CPU-side posting? Thanks for the comments. "Interrupt posting" in fact contains vt-d side and cpu side. So I will reword this to make it clear. Thanks, Feng > > I think a one-sentence definition about those terms in the start would > be helpful. > > > + > > +With VT-d Posted-interrupt we can get the following advantages: > > +- Direct delivery of external interrupts to running vCPUs without VMM > > +intervention > > +- Decrease the interrupt migration complexity. On vCPU migration, > software > > +can atomically co-migrate all interrupts targeting the migrating vCPU. For > > +virtual machines with assigned devices, migrating a vCPU across pCPUs > > +either incurs the overhead of forwarding interrupts in software (e.g. via > VMM > > +generated IPIs), or complexity to independently migrate each interrupt > targeting > > +the vCPU to the new pCPU. However, after enabling VT-d PI, the > destination vCPU > > +of an external interrupt from assigned devices is stored in the IRTE (i.e. > > +Posted-interrupt Descriptor Address), when vCPU is migrated to another > pCPU, > > +we will set this new pCPU in the 'NDST' filed of Posted-interrupt > descriptor, this > > +make the interrupt migration automatic. > > + > > +Here is what Xen currently does for external interrupts from assigned > devices: > > + > > +When a VM is running and an external interrupt from an assigned device > occurs > > +for it. VM-EXIT happens, then: > > + > > +vmx_do_extint() --> do_IRQ() --> __do_IRQ_guest() --> hvm_do_IRQ_dpci() > --> > > +raise_softirq_for(pirq_dpci) --> raise_softirq(HVM_DPCI_SOFTIRQ) > > + > > +softirq HVM_DPCI_SOFTIRQ is bound to dpci_softirq() > > + > > +dpci_softirq() --> hvm_dirq_assist() --> vmsi_deliver_pirq() --> > vmsi_deliver() --> > > +vmsi_inj_irq() --> vlapic_set_irq() > > + > > +vlapic_set_irq() does the following things: > > +1. If CPU-side posted-interrupt is supported, call > vmx_deliver_posted_intr() to deliver > > +the virtual interrupt via posted-interrupt infrastructure. > > +2. Else if CPU-side posted-interrupt is not supported, set the related vIRR > in vLAPIC > > +page and call vcpu_kick() to kick the related vCPU. Before VM-Entry, > vmx_intr_assist() > > +will help to inject the interrupt to guests. > > + > > +However, after VT-d PI is supported, when a guest is running in non-root > and an > > +external interrupt from an assigned device occurs for it. No VM-Exit is > needed, > > ". No VM-Exit ..." -> ", no VM-Exit..." > > > +the guest can handle this totally in non-root mode, thus avoiding all the > above > > +code flow. > > + > > +Posted-interrupt Introduction > > +======================== > > +There are two components to the Posted-interrupt architecture: > > "to the" -> "in the" > > > Thanks > Kevin