From: Marc Zyngier <marc.zyngier@arm.com> To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Cc: Andre Przywara <Andre.Przywara@arm.com> Subject: [PATCH v4 26/26] KVM: arm/arm64: GICv4: Theory of operations Date: Fri, 6 Oct 2017 16:34:01 +0100 [thread overview] Message-ID: <20171006153401.5481-27-marc.zyngier@arm.com> (raw) In-Reply-To: <20171006153401.5481-1-marc.zyngier@arm.com> Yet another braindump so I can free some cells... Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> --- virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c index 68a2ef6df161..b87806fea554 100644 --- a/virt/kvm/arm/vgic/vgic-v4.c +++ b/virt/kvm/arm/vgic/vgic-v4.c @@ -22,6 +22,73 @@ #include "vgic.h" +/* + * How KVM uses GICv4 (insert rude comments here): + * + * The vgic-v4 layer acts as a bridge between several entities: + * - The GICv4 ITS representation offered by the ITS driver + * - VFIO, which is in charge of the PCI endpoint + * - The virtual ITS, which is the only thing the guest sees + * + * The configuration of VLPIs is triggered by a callback from VFIO, + * instructing KVM that a PCI device has been configured to deliver + * MSIs to a vITS. + * + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry, + * and this is used to find the corresponding vITS data structures + * (ITS instance, device, event and irq) using a process that is + * extremely similar to the injection of an MSI. + * + * At this stage, we can link the guest's view of an LPI (uniquely + * identified by the routing entry) and the host irq, using the GICv4 + * driver mapping operation. Should the mapping succeed, we've then + * successfully upgraded the guest's LPI to a VLPI. We can then start + * with updating GICv4's view of the property table and generating an + * INValidation in order to kickstart the delivery of this VLPI to the + * guest directly, without software intervention. Well, almost. + * + * When the PCI endpoint is deconfigured, this operation is reversed + * with VFIO calling kvm_vgic_v4_unset_forwarding(). + * + * Once the VLPI has been mapped, it needs to follow any change the + * guest performs on its LPI through the vITS. For that, a number of + * command handlers have hooks to communicate these changes to the HW: + * - Any invalidation triggers a call to its_prop_update_vlpi() + * - The INT command results in a irq_set_irqchip_state(), which + * generates an INT on the corresponding VLPI. + * - The CLEAR command results in a irq_set_irqchip_state(), which + * generates an CLEAR on the corresponding VLPI. + * - DISCARD translates into an unmap, similar to a call to + * kvm_vgic_v4_unset_forwarding(). + * - MOVI is translated by an update of the existing mapping, changing + * the target vcpu, resulting in a VMOVI being generated. + * - MOVALL is translated by a string of mapping updates (similar to + * the handling of MOVI). MOVALL is horrible. + * + * Note that a DISCARD/MAPTI sequence emitted from the guest without + * reprogramming the PCI endpoint after MAPTI does not result in a + * VLPI being mapped, as there is no callback from VFIO (the guest + * will get the interrupt via the normal SW injection). Fixing this is + * not trivial, and requires some horrible messing with the VFIO + * internals. Not fun. Don't do that. + * + * Then there is the scheduling. Each time a vcpu is about to run on a + * physical CPU, KVM must tell the corresponding redistributor about + * it. And if we've migrated our vcpu from one CPU to another, we must + * tell the ITS (so that the messages reach the right redistributor). + * This is done in two steps: first issue a irq_set_affinity() on the + * irq corresponding to the vcpu, then call its_schedule_vpe(). You + * must be in a non-preemptible context. On exit, another call to + * its_schedule_vpe() tells the redistributor that we're done with the + * vcpu. + * + * Finally, the doorbell handling: Each vcpu is allocated an interrupt + * which will fire each time a VLPI is made pending whilst the vcpu is + * not running. Each time the vcpu gets blocked, the doorbell + * interrupt gets enabled. When the vcpu is unblocked (for whatever + * reason), the doorbell interrupt is disabled. + */ + static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info) { struct kvm_vcpu *vcpu = info; -- 2.14.1
WARNING: multiple messages have this Message-ID (diff)
From: marc.zyngier@arm.com (Marc Zyngier) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 26/26] KVM: arm/arm64: GICv4: Theory of operations Date: Fri, 6 Oct 2017 16:34:01 +0100 [thread overview] Message-ID: <20171006153401.5481-27-marc.zyngier@arm.com> (raw) In-Reply-To: <20171006153401.5481-1-marc.zyngier@arm.com> Yet another braindump so I can free some cells... Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> --- virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c index 68a2ef6df161..b87806fea554 100644 --- a/virt/kvm/arm/vgic/vgic-v4.c +++ b/virt/kvm/arm/vgic/vgic-v4.c @@ -22,6 +22,73 @@ #include "vgic.h" +/* + * How KVM uses GICv4 (insert rude comments here): + * + * The vgic-v4 layer acts as a bridge between several entities: + * - The GICv4 ITS representation offered by the ITS driver + * - VFIO, which is in charge of the PCI endpoint + * - The virtual ITS, which is the only thing the guest sees + * + * The configuration of VLPIs is triggered by a callback from VFIO, + * instructing KVM that a PCI device has been configured to deliver + * MSIs to a vITS. + * + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry, + * and this is used to find the corresponding vITS data structures + * (ITS instance, device, event and irq) using a process that is + * extremely similar to the injection of an MSI. + * + * At this stage, we can link the guest's view of an LPI (uniquely + * identified by the routing entry) and the host irq, using the GICv4 + * driver mapping operation. Should the mapping succeed, we've then + * successfully upgraded the guest's LPI to a VLPI. We can then start + * with updating GICv4's view of the property table and generating an + * INValidation in order to kickstart the delivery of this VLPI to the + * guest directly, without software intervention. Well, almost. + * + * When the PCI endpoint is deconfigured, this operation is reversed + * with VFIO calling kvm_vgic_v4_unset_forwarding(). + * + * Once the VLPI has been mapped, it needs to follow any change the + * guest performs on its LPI through the vITS. For that, a number of + * command handlers have hooks to communicate these changes to the HW: + * - Any invalidation triggers a call to its_prop_update_vlpi() + * - The INT command results in a irq_set_irqchip_state(), which + * generates an INT on the corresponding VLPI. + * - The CLEAR command results in a irq_set_irqchip_state(), which + * generates an CLEAR on the corresponding VLPI. + * - DISCARD translates into an unmap, similar to a call to + * kvm_vgic_v4_unset_forwarding(). + * - MOVI is translated by an update of the existing mapping, changing + * the target vcpu, resulting in a VMOVI being generated. + * - MOVALL is translated by a string of mapping updates (similar to + * the handling of MOVI). MOVALL is horrible. + * + * Note that a DISCARD/MAPTI sequence emitted from the guest without + * reprogramming the PCI endpoint after MAPTI does not result in a + * VLPI being mapped, as there is no callback from VFIO (the guest + * will get the interrupt via the normal SW injection). Fixing this is + * not trivial, and requires some horrible messing with the VFIO + * internals. Not fun. Don't do that. + * + * Then there is the scheduling. Each time a vcpu is about to run on a + * physical CPU, KVM must tell the corresponding redistributor about + * it. And if we've migrated our vcpu from one CPU to another, we must + * tell the ITS (so that the messages reach the right redistributor). + * This is done in two steps: first issue a irq_set_affinity() on the + * irq corresponding to the vcpu, then call its_schedule_vpe(). You + * must be in a non-preemptible context. On exit, another call to + * its_schedule_vpe() tells the redistributor that we're done with the + * vcpu. + * + * Finally, the doorbell handling: Each vcpu is allocated an interrupt + * which will fire each time a VLPI is made pending whilst the vcpu is + * not running. Each time the vcpu gets blocked, the doorbell + * interrupt gets enabled. When the vcpu is unblocked (for whatever + * reason), the doorbell interrupt is disabled. + */ + static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info) { struct kvm_vcpu *vcpu = info; -- 2.14.1
next prev parent reply other threads:[~2017-10-06 15:34 UTC|newest] Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-10-06 15:33 [PATCH v4 00/26] KVM/ARM: Add support for GICv4 Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 01/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64 Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 02/26] KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 14:44 ` Christoffer Dall 2017-10-25 14:44 ` Christoffer Dall 2017-10-26 13:05 ` Marc Zyngier 2017-10-26 13:05 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 03/26] KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 04/26] KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 05/26] KVM: arm/arm64: vITS: Add MSI translation helpers Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 06/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 14:50 ` Christoffer Dall 2017-10-25 14:50 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 07/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 14:54 ` Christoffer Dall 2017-10-25 14:54 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 08/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 15:46 ` Christoffer Dall 2017-10-25 15:46 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 09/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 15:58 ` Christoffer Dall 2017-10-25 15:58 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 10/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 11/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 12/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 16:22 ` Christoffer Dall 2017-10-25 16:22 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 13/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 14/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 16:27 ` Christoffer Dall 2017-10-25 16:27 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 15/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 16/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 17/26] KVM: arm/arm64: GICv4: Propagate VLPI properties at map time Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-25 16:48 ` Christoffer Dall 2017-10-25 16:48 ` Christoffer Dall 2017-10-25 17:01 ` Marc Zyngier 2017-10-25 17:01 ` Marc Zyngier 2017-10-26 8:21 ` Marc Zyngier 2017-10-26 8:21 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-26 14:28 ` Christoffer Dall 2017-10-26 14:28 ` Christoffer Dall 2017-10-26 15:25 ` Christoffer Dall 2017-10-26 15:25 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-26 15:24 ` Christoffer Dall 2017-10-26 15:24 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:33 ` [PATCH v4 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-26 15:28 ` Christoffer Dall 2017-10-26 15:28 ` Christoffer Dall 2017-10-27 13:56 ` Marc Zyngier 2017-10-27 13:56 ` Marc Zyngier 2017-10-30 6:48 ` Christoffer Dall 2017-10-30 6:48 ` Christoffer Dall 2017-10-06 15:33 ` [PATCH v4 24/26] KVM: arm/arm64: GICv4: Enable VLPI support Marc Zyngier 2017-10-06 15:33 ` Marc Zyngier 2017-10-06 15:34 ` [PATCH v4 25/26] KVM: arm/arm64: GICv4: Prevent heterogenous systems from using GICv4 Marc Zyngier 2017-10-06 15:34 ` Marc Zyngier 2017-10-26 15:31 ` Christoffer Dall 2017-10-26 15:31 ` Christoffer Dall 2017-10-26 15:48 ` Mark Rutland 2017-10-26 15:48 ` Mark Rutland 2017-10-27 6:57 ` Marc Zyngier 2017-10-27 6:57 ` Marc Zyngier 2017-10-27 6:57 ` Marc Zyngier 2017-10-27 7:37 ` Mark Rutland 2017-10-27 7:37 ` Mark Rutland 2017-10-27 7:59 ` Marc Zyngier 2017-10-27 7:59 ` Marc Zyngier 2017-10-27 7:59 ` Marc Zyngier 2017-10-27 8:04 ` Mark Rutland 2017-10-27 8:04 ` Mark Rutland 2017-10-27 8:31 ` Marc Zyngier 2017-10-27 8:31 ` Marc Zyngier 2017-10-27 8:31 ` Marc Zyngier 2017-10-29 2:16 ` Christoffer Dall 2017-10-29 2:16 ` Christoffer Dall 2017-10-06 15:34 ` Marc Zyngier [this message] 2017-10-06 15:34 ` [PATCH v4 26/26] KVM: arm/arm64: GICv4: Theory of operations Marc Zyngier 2017-10-26 15:32 ` Christoffer Dall 2017-10-26 15:32 ` Christoffer Dall 2017-10-08 16:45 ` [PATCH v4 00/26] KVM/ARM: Add support for GICv4 Shanker Donthineni 2017-10-08 16:45 ` Shanker Donthineni 2017-10-09 17:52 ` Konrad Rzeszutek Wilk 2017-10-09 17:52 ` Konrad Rzeszutek Wilk 2017-10-09 18:02 ` Marc Zyngier 2017-10-09 18:02 ` Marc Zyngier 2017-10-19 15:48 ` [PATCH v4 27/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity Marc Zyngier 2017-10-19 15:48 ` Marc Zyngier
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20171006153401.5481-27-marc.zyngier@arm.com \ --to=marc.zyngier@arm.com \ --cc=Andre.Przywara@arm.com \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.cs.columbia.edu \ --cc=linux-arm-kernel@lists.infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.