From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Tian, Kevin" Subject: Re: [PATCH v9 01/17] VT-d Posted-intterrupt (PI) design Date: Tue, 24 Nov 2015 06:34:58 +0000 Message-ID: References: <1446540207-4806-1-git-send-email-feng.wu@intel.com> <1446540207-4806-2-git-send-email-feng.wu@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1446540207-4806-2-git-send-email-feng.wu@intel.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Wu, Feng" , "xen-devel@lists.xen.org" Cc: "Zhang, Yang Z" , Andrew Cooper , Keir Fraser , George Dunlap , Jan Beulich List-Id: xen-devel@lists.xenproject.org > From: Wu, Feng > Sent: Tuesday, November 03, 2015 4:43 PM > > > diff --git a/docs/misc/vtd-pi.txt b/docs/misc/vtd-pi.txt > new file mode 100644 > index 0000000..f9b4637 > --- /dev/null > +++ b/docs/misc/vtd-pi.txt > @@ -0,0 +1,329 @@ > +Authors: Feng Wu > + > +VT-d Posted-interrupt (PI) design for XEN > + > +Background > +========== > +With the development of virtualization, there are more and more device > +assignment requirements. However, today when a VM is running with > +assigned devices (such as, NIC), external interrupt handling for the assigned > +devices always needs VMM intervention. > + > +VT-d Posted-interrupt is a more enhanced method to handle interrupts > +in the virtualization environment. Interrupt posting is the process by > +which an interrupt request is recorded in a memory-resident > +posted-interrupt-descriptor structure by the root-complex, followed by > +an optional notification event issued to the CPU complex. Some clarification required here. Is "Interrupt Posting" only used to represent the process of VT-d Posted-interrupt, or also for CPU-side posting through IPI? From above context, and later explanation about "Processor Support" and "Root-complex Support", looks the former is true. Then how do we call CPU-side posting? I think a one-sentence definition about those terms in the start would be helpful. > + > +With VT-d Posted-interrupt we can get the following advantages: > +- Direct delivery of external interrupts to running vCPUs without VMM > +intervention > +- Decrease the interrupt migration complexity. On vCPU migration, software > +can atomically co-migrate all interrupts targeting the migrating vCPU. For > +virtual machines with assigned devices, migrating a vCPU across pCPUs > +either incurs the overhead of forwarding interrupts in software (e.g. via VMM > +generated IPIs), or complexity to independently migrate each interrupt targeting > +the vCPU to the new pCPU. However, after enabling VT-d PI, the destination vCPU > +of an external interrupt from assigned devices is stored in the IRTE (i.e. > +Posted-interrupt Descriptor Address), when vCPU is migrated to another pCPU, > +we will set this new pCPU in the 'NDST' filed of Posted-interrupt descriptor, this > +make the interrupt migration automatic. > + > +Here is what Xen currently does for external interrupts from assigned devices: > + > +When a VM is running and an external interrupt from an assigned device occurs > +for it. VM-EXIT happens, then: > + > +vmx_do_extint() --> do_IRQ() --> __do_IRQ_guest() --> hvm_do_IRQ_dpci() --> > +raise_softirq_for(pirq_dpci) --> raise_softirq(HVM_DPCI_SOFTIRQ) > + > +softirq HVM_DPCI_SOFTIRQ is bound to dpci_softirq() > + > +dpci_softirq() --> hvm_dirq_assist() --> vmsi_deliver_pirq() --> vmsi_deliver() --> > +vmsi_inj_irq() --> vlapic_set_irq() > + > +vlapic_set_irq() does the following things: > +1. If CPU-side posted-interrupt is supported, call vmx_deliver_posted_intr() to deliver > +the virtual interrupt via posted-interrupt infrastructure. > +2. Else if CPU-side posted-interrupt is not supported, set the related vIRR in vLAPIC > +page and call vcpu_kick() to kick the related vCPU. Before VM-Entry, vmx_intr_assist() > +will help to inject the interrupt to guests. > + > +However, after VT-d PI is supported, when a guest is running in non-root and an > +external interrupt from an assigned device occurs for it. No VM-Exit is needed, ". No VM-Exit ..." -> ", no VM-Exit..." > +the guest can handle this totally in non-root mode, thus avoiding all the above > +code flow. > + > +Posted-interrupt Introduction > +======================== > +There are two components to the Posted-interrupt architecture: "to the" -> "in the" Thanks Kevin