All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v5 00/26] KVM/ARM: Add support for GICv4
@ 2017-10-27 14:28 ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

This series implements full support for GICv4 in KVM, bringing direct
injection of MSIs to arm and arm64, assuming you have the right
hardware (which is quite unlikely).

To get an idea of the design, I'd recommend you start with commit
7954907bedaf as well as patch #26, which try to shed some light on the
approach that I've taken. And before that, please digest some of the
GICv3/GICv4 architecture documentation[1] (less than 800 pages!). Once
you feel reasonably insane, you'll be in the right mood to read the
code (no, I didn't bother changing the above, I already hit LWN/QotW
once).

Now that the low-level irqchip code has been merged, what we have here
is mostly the KVM-specific part. I've lifted two of Eric's patches
(having slightly changed one of them) as they form the base of this
series.

The stack has now been quite heavily bashed on a Huawei D05 box using
an Intel I350 Ethernet card and passing a VF into the guest, using
QEMU as the setup userspace (although this platform needs workarounds
that I've pushed out separately). It has also been *very lightly*
tested on an arm64 model, with a PCI virtio block device passed from
the host to a guest (using kvmtool and Jean-Philippe Brucker's
excellent VFIO support patches[2]).

I've pushed out a branch based on 4.14-rc6 containing the dependencies:

git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/gicv4-kvm

* From v4:
   - Fixed bug where a newly mapped VLPI could fire without having had
     its properties setup yet. Oops.
   - Changed the error code on attempt to save an ITS using VLPIs to
     -ACCES, and documented the new error code.
   - Dropped the patch trying to prevent a mix of GICv3 and GICv4
     CPUs. This is a broken configuration, and trying to detect it
     without making a mess is hard.
   - Added the patch disabling setting the doorbell affinity from
     userspace as part of this series
   - Rebased the series on 4.14-rc6, but it should apply just the
     same on something older

* From v3:
   - Lots of cleanups following Christoffer's review
   - Got rid of all of the irqchip code (well, merged it)
   - Picked the minimal set of patches from Eric to get this working
   - Prevent VM save if using GICv4
   - Rebased on top of 4.14-rc3

* From v2:
  - Lots of fixes all over the map (the doorbell code was amazingly
    wrong, both at the GICv4 level and in KVM)
  - KVM GICv4 enablement is now gated by a command-line option and
    defaults to off
  - Now properly deals with ordering of vITS creation
  - Some debugging features
  - More documentation so that I can forget what this is all about
  - Huawei D05 quirks

* From v1:
  - The bulk of the 30-something initial patches have seen countless
    bugs being fixed, and some key data structures have been subtly
    tweaked (or killed altogether). They are still quite similar to
    what I had in v1 though.
  - The whole KVM code is brand new and as I said above, only lightly
    tested.
  - Collected a bunch a R-bs from Thomas and Eric (many thanks, guys).

[1] https://static.docs.arm.com/ihi0069/c/IHI0069C_gic_architecture_specification.pdf
[2] http://www.spinics.net/lists/kvm/msg151823.html

Eric Auger (2):
  KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
  KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq

Marc Zyngier (24):
  irqchip/gic-v3-its: Setup VLPI properties at map time
  KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS
  KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around
  KVM: arm/arm64: vITS: Add MSI translation helpers
  KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
  KVM: arm/arm64: GICv4: Add property field and per-VM predicate
  KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq
    bypass
  KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
  KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
  KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
  KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
  KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
  KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
  KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
  KVM: arm/arm64: GICv4: Add doorbell interrupt handling
  KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking
    source
  KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
  KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
  KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell
    affinity
  KVM: arm/arm64: GICv4: Enable VLPI support
  KVM: arm/arm64: GICv4: Theory of operations

 Documentation/admin-guide/kernel-parameters.txt    |   4 +
 Documentation/virtual/kvm/devices/arm-vgic-its.txt |   2 +
 arch/arm/kvm/Kconfig                               |   5 +
 arch/arm/kvm/Makefile                              |   1 +
 arch/arm64/kvm/Kconfig                             |   3 +
 arch/arm64/kvm/Makefile                            |   1 +
 drivers/irqchip/irq-gic-v3-its.c                   |  19 +-
 include/kvm/arm_vgic.h                             |  41 ++-
 include/linux/irqchip/arm-gic-v4.h                 |   2 +
 virt/kvm/arm/arch_timer.c                          |  24 +-
 virt/kvm/arm/arm.c                                 |  48 ++-
 virt/kvm/arm/hyp/vgic-v3-sr.c                      |   9 +-
 virt/kvm/arm/vgic/vgic-init.c                      |   9 +
 virt/kvm/arm/vgic/vgic-its.c                       | 193 +++++++----
 virt/kvm/arm/vgic/vgic-mmio-v3.c                   |   5 +
 virt/kvm/arm/vgic/vgic-v3.c                        |  14 +
 virt/kvm/arm/vgic/vgic-v4.c                        | 365 +++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.c                           |  67 +++-
 virt/kvm/arm/vgic/vgic.h                           |  10 +
 19 files changed, 710 insertions(+), 112 deletions(-)
 create mode 100644 virt/kvm/arm/vgic/vgic-v4.c

-- 
2.11.0

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 00/26] KVM/ARM: Add support for GICv4
@ 2017-10-27 14:28 ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

This series implements full support for GICv4 in KVM, bringing direct
injection of MSIs to arm and arm64, assuming you have the right
hardware (which is quite unlikely).

To get an idea of the design, I'd recommend you start with commit
7954907bedaf as well as patch #26, which try to shed some light on the
approach that I've taken. And before that, please digest some of the
GICv3/GICv4 architecture documentation[1] (less than 800 pages!). Once
you feel reasonably insane, you'll be in the right mood to read the
code (no, I didn't bother changing the above, I already hit LWN/QotW
once).

Now that the low-level irqchip code has been merged, what we have here
is mostly the KVM-specific part. I've lifted two of Eric's patches
(having slightly changed one of them) as they form the base of this
series.

The stack has now been quite heavily bashed on a Huawei D05 box using
an Intel I350 Ethernet card and passing a VF into the guest, using
QEMU as the setup userspace (although this platform needs workarounds
that I've pushed out separately). It has also been *very lightly*
tested on an arm64 model, with a PCI virtio block device passed from
the host to a guest (using kvmtool and Jean-Philippe Brucker's
excellent VFIO support patches[2]).

I've pushed out a branch based on 4.14-rc6 containing the dependencies:

git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/gicv4-kvm

* From v4:
   - Fixed bug where a newly mapped VLPI could fire without having had
     its properties setup yet. Oops.
   - Changed the error code on attempt to save an ITS using VLPIs to
     -ACCES, and documented the new error code.
   - Dropped the patch trying to prevent a mix of GICv3 and GICv4
     CPUs. This is a broken configuration, and trying to detect it
     without making a mess is hard.
   - Added the patch disabling setting the doorbell affinity from
     userspace as part of this series
   - Rebased the series on 4.14-rc6, but it should apply just the
     same on something older

* From v3:
   - Lots of cleanups following Christoffer's review
   - Got rid of all of the irqchip code (well, merged it)
   - Picked the minimal set of patches from Eric to get this working
   - Prevent VM save if using GICv4
   - Rebased on top of 4.14-rc3

* From v2:
  - Lots of fixes all over the map (the doorbell code was amazingly
    wrong, both at the GICv4 level and in KVM)
  - KVM GICv4 enablement is now gated by a command-line option and
    defaults to off
  - Now properly deals with ordering of vITS creation
  - Some debugging features
  - More documentation so that I can forget what this is all about
  - Huawei D05 quirks

* From v1:
  - The bulk of the 30-something initial patches have seen countless
    bugs being fixed, and some key data structures have been subtly
    tweaked (or killed altogether). They are still quite similar to
    what I had in v1 though.
  - The whole KVM code is brand new and as I said above, only lightly
    tested.
  - Collected a bunch a R-bs from Thomas and Eric (many thanks, guys).

[1] https://static.docs.arm.com/ihi0069/c/IHI0069C_gic_architecture_specification.pdf
[2] http://www.spinics.net/lists/kvm/msg151823.html

Eric Auger (2):
  KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
  KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq

Marc Zyngier (24):
  irqchip/gic-v3-its: Setup VLPI properties at map time
  KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS
  KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around
  KVM: arm/arm64: vITS: Add MSI translation helpers
  KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
  KVM: arm/arm64: GICv4: Add property field and per-VM predicate
  KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq
    bypass
  KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
  KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
  KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
  KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
  KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
  KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
  KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
  KVM: arm/arm64: GICv4: Add doorbell interrupt handling
  KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking
    source
  KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
  KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
  KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell
    affinity
  KVM: arm/arm64: GICv4: Enable VLPI support
  KVM: arm/arm64: GICv4: Theory of operations

 Documentation/admin-guide/kernel-parameters.txt    |   4 +
 Documentation/virtual/kvm/devices/arm-vgic-its.txt |   2 +
 arch/arm/kvm/Kconfig                               |   5 +
 arch/arm/kvm/Makefile                              |   1 +
 arch/arm64/kvm/Kconfig                             |   3 +
 arch/arm64/kvm/Makefile                            |   1 +
 drivers/irqchip/irq-gic-v3-its.c                   |  19 +-
 include/kvm/arm_vgic.h                             |  41 ++-
 include/linux/irqchip/arm-gic-v4.h                 |   2 +
 virt/kvm/arm/arch_timer.c                          |  24 +-
 virt/kvm/arm/arm.c                                 |  48 ++-
 virt/kvm/arm/hyp/vgic-v3-sr.c                      |   9 +-
 virt/kvm/arm/vgic/vgic-init.c                      |   9 +
 virt/kvm/arm/vgic/vgic-its.c                       | 193 +++++++----
 virt/kvm/arm/vgic/vgic-mmio-v3.c                   |   5 +
 virt/kvm/arm/vgic/vgic-v3.c                        |  14 +
 virt/kvm/arm/vgic/vgic-v4.c                        | 365 +++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.c                           |  67 +++-
 virt/kvm/arm/vgic/vgic.h                           |  10 +
 19 files changed, 710 insertions(+), 112 deletions(-)
 create mode 100644 virt/kvm/arm/vgic/vgic-v4.c

-- 
2.11.0

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 00/26] KVM/ARM: Add support for GICv4
@ 2017-10-27 14:28 ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

This series implements full support for GICv4 in KVM, bringing direct
injection of MSIs to arm and arm64, assuming you have the right
hardware (which is quite unlikely).

To get an idea of the design, I'd recommend you start with commit
7954907bedaf as well as patch #26, which try to shed some light on the
approach that I've taken. And before that, please digest some of the
GICv3/GICv4 architecture documentation[1] (less than 800 pages!). Once
you feel reasonably insane, you'll be in the right mood to read the
code (no, I didn't bother changing the above, I already hit LWN/QotW
once).

Now that the low-level irqchip code has been merged, what we have here
is mostly the KVM-specific part. I've lifted two of Eric's patches
(having slightly changed one of them) as they form the base of this
series.

The stack has now been quite heavily bashed on a Huawei D05 box using
an Intel I350 Ethernet card and passing a VF into the guest, using
QEMU as the setup userspace (although this platform needs workarounds
that I've pushed out separately). It has also been *very lightly*
tested on an arm64 model, with a PCI virtio block device passed from
the host to a guest (using kvmtool and Jean-Philippe Brucker's
excellent VFIO support patches[2]).

I've pushed out a branch based on 4.14-rc6 containing the dependencies:

git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/gicv4-kvm

* From v4:
   - Fixed bug where a newly mapped VLPI could fire without having had
     its properties setup yet. Oops.
   - Changed the error code on attempt to save an ITS using VLPIs to
     -ACCES, and documented the new error code.
   - Dropped the patch trying to prevent a mix of GICv3 and GICv4
     CPUs. This is a broken configuration, and trying to detect it
     without making a mess is hard.
   - Added the patch disabling setting the doorbell affinity from
     userspace as part of this series
   - Rebased the series on 4.14-rc6, but it should apply just the
     same on something older

* From v3:
   - Lots of cleanups following Christoffer's review
   - Got rid of all of the irqchip code (well, merged it)
   - Picked the minimal set of patches from Eric to get this working
   - Prevent VM save if using GICv4
   - Rebased on top of 4.14-rc3

* From v2:
  - Lots of fixes all over the map (the doorbell code was amazingly
    wrong, both at the GICv4 level and in KVM)
  - KVM GICv4 enablement is now gated by a command-line option and
    defaults to off
  - Now properly deals with ordering of vITS creation
  - Some debugging features
  - More documentation so that I can forget what this is all about
  - Huawei D05 quirks

* From v1:
  - The bulk of the 30-something initial patches have seen countless
    bugs being fixed, and some key data structures have been subtly
    tweaked (or killed altogether). They are still quite similar to
    what I had in v1 though.
  - The whole KVM code is brand new and as I said above, only lightly
    tested.
  - Collected a bunch a R-bs from Thomas and Eric (many thanks, guys).

[1] https://static.docs.arm.com/ihi0069/c/IHI0069C_gic_architecture_specification.pdf
[2] http://www.spinics.net/lists/kvm/msg151823.html

Eric Auger (2):
  KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
  KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq

Marc Zyngier (24):
  irqchip/gic-v3-its: Setup VLPI properties at map time
  KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS
  KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around
  KVM: arm/arm64: vITS: Add MSI translation helpers
  KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
  KVM: arm/arm64: GICv4: Add property field and per-VM predicate
  KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq
    bypass
  KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
  KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
  KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
  KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
  KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
  KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
  KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
  KVM: arm/arm64: GICv4: Add doorbell interrupt handling
  KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking
    source
  KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
  KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
  KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell
    affinity
  KVM: arm/arm64: GICv4: Enable VLPI support
  KVM: arm/arm64: GICv4: Theory of operations

 Documentation/admin-guide/kernel-parameters.txt    |   4 +
 Documentation/virtual/kvm/devices/arm-vgic-its.txt |   2 +
 arch/arm/kvm/Kconfig                               |   5 +
 arch/arm/kvm/Makefile                              |   1 +
 arch/arm64/kvm/Kconfig                             |   3 +
 arch/arm64/kvm/Makefile                            |   1 +
 drivers/irqchip/irq-gic-v3-its.c                   |  19 +-
 include/kvm/arm_vgic.h                             |  41 ++-
 include/linux/irqchip/arm-gic-v4.h                 |   2 +
 virt/kvm/arm/arch_timer.c                          |  24 +-
 virt/kvm/arm/arm.c                                 |  48 ++-
 virt/kvm/arm/hyp/vgic-v3-sr.c                      |   9 +-
 virt/kvm/arm/vgic/vgic-init.c                      |   9 +
 virt/kvm/arm/vgic/vgic-its.c                       | 193 +++++++----
 virt/kvm/arm/vgic/vgic-mmio-v3.c                   |   5 +
 virt/kvm/arm/vgic/vgic-v3.c                        |  14 +
 virt/kvm/arm/vgic/vgic-v4.c                        | 365 +++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.c                           |  67 +++-
 virt/kvm/arm/vgic/vgic.h                           |  10 +
 19 files changed, 710 insertions(+), 112 deletions(-)
 create mode 100644 virt/kvm/arm/vgic/vgic-v4.c

-- 
2.11.0

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 01/26] irqchip/gic-v3-its: Setup VLPI properties at map time
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

So far, we require the hypervisor to update the VLPI properties
once the the VLPI mapping has been established. While this
makes it easy for the ITS driver, it creates a window where
an incoming interrupt can be delivered with an unknown set
of properties. Not very nice.

Instead, let's add a "properties" field to the mapping structure,
and use that to configure the VLPI before it actually gets mapped.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
 include/linux/irqchip/arm-gic-v4.h |  2 ++
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 85f47ef4f1f7..2c4074c97454 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1006,9 +1006,15 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 	if (irqd_is_forwarded_to_vcpu(d)) {
 		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 		u32 event = its_get_event_id(d);
+		struct its_vlpi_map *map;
 
 		prop_page = its_dev->event_map.vm->vprop_page;
-		hwirq = its_dev->event_map.vlpi_maps[event].vintid;
+		map = &its_dev->event_map.vlpi_maps[event];
+		hwirq = map->vintid;
+
+		/* Remember the updated property */
+		map->properties &= ~clr;
+		map->properties |= set | LPI_PROP_GROUP1;
 	} else {
 		prop_page = gic_rdists->prop_page;
 		hwirq = d->hwirq;
@@ -1018,6 +1024,7 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 	*cfg &= ~clr;
 	*cfg |= set | LPI_PROP_GROUP1;
 
+
 	/*
 	 * Make the above write visible to the redistributors.
 	 * And yes, we're flushing exactly: One. Single. Byte.
@@ -1238,12 +1245,20 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
 		/* Ensure all the VPEs are mapped on this ITS */
 		its_map_vm(its_dev->its, info->map->vm);
 
+		/*
+		 * Flag the interrupt as forwarded so that we can
+		 * start poking the virtual property table.
+		 */
+		irqd_set_forwarded_to_vcpu(d);
+
+		/* Write out the property to the prop table */
+		lpi_write_config(d, 0xff, info->map->properties);
+
 		/* Drop the physical mapping */
 		its_send_discard(its_dev, event);
 
 		/* and install the virtual one */
 		its_send_vmapti(its_dev, event);
-		irqd_set_forwarded_to_vcpu(d);
 
 		/* Increment the number of VLPIs */
 		its_dev->event_map.nr_vlpis++;
diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
index 43cde15f221b..447da8ca2156 100644
--- a/include/linux/irqchip/arm-gic-v4.h
+++ b/include/linux/irqchip/arm-gic-v4.h
@@ -71,12 +71,14 @@ struct its_vpe {
  * @vm:		Pointer to the GICv4 notion of a VM
  * @vpe:	Pointer to the GICv4 notion of a virtual CPU (VPE)
  * @vintid:	Virtual LPI number
+ * @properties:	Priority and enable bits (as written in the prop table)
  * @db_enabled:	Is the VPE doorbell to be generated?
  */
 struct its_vlpi_map {
 	struct its_vm		*vm;
 	struct its_vpe		*vpe;
 	u32			vintid;
+	u8			properties;
 	bool			db_enabled;
 };
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 01/26] irqchip/gic-v3-its: Setup VLPI properties at map time
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

So far, we require the hypervisor to update the VLPI properties
once the the VLPI mapping has been established. While this
makes it easy for the ITS driver, it creates a window where
an incoming interrupt can be delivered with an unknown set
of properties. Not very nice.

Instead, let's add a "properties" field to the mapping structure,
and use that to configure the VLPI before it actually gets mapped.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
 include/linux/irqchip/arm-gic-v4.h |  2 ++
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 85f47ef4f1f7..2c4074c97454 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1006,9 +1006,15 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 	if (irqd_is_forwarded_to_vcpu(d)) {
 		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 		u32 event = its_get_event_id(d);
+		struct its_vlpi_map *map;
 
 		prop_page = its_dev->event_map.vm->vprop_page;
-		hwirq = its_dev->event_map.vlpi_maps[event].vintid;
+		map = &its_dev->event_map.vlpi_maps[event];
+		hwirq = map->vintid;
+
+		/* Remember the updated property */
+		map->properties &= ~clr;
+		map->properties |= set | LPI_PROP_GROUP1;
 	} else {
 		prop_page = gic_rdists->prop_page;
 		hwirq = d->hwirq;
@@ -1018,6 +1024,7 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 	*cfg &= ~clr;
 	*cfg |= set | LPI_PROP_GROUP1;
 
+
 	/*
 	 * Make the above write visible to the redistributors.
 	 * And yes, we're flushing exactly: One. Single. Byte.
@@ -1238,12 +1245,20 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
 		/* Ensure all the VPEs are mapped on this ITS */
 		its_map_vm(its_dev->its, info->map->vm);
 
+		/*
+		 * Flag the interrupt as forwarded so that we can
+		 * start poking the virtual property table.
+		 */
+		irqd_set_forwarded_to_vcpu(d);
+
+		/* Write out the property to the prop table */
+		lpi_write_config(d, 0xff, info->map->properties);
+
 		/* Drop the physical mapping */
 		its_send_discard(its_dev, event);
 
 		/* and install the virtual one */
 		its_send_vmapti(its_dev, event);
-		irqd_set_forwarded_to_vcpu(d);
 
 		/* Increment the number of VLPIs */
 		its_dev->event_map.nr_vlpis++;
diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
index 43cde15f221b..447da8ca2156 100644
--- a/include/linux/irqchip/arm-gic-v4.h
+++ b/include/linux/irqchip/arm-gic-v4.h
@@ -71,12 +71,14 @@ struct its_vpe {
  * @vm:		Pointer to the GICv4 notion of a VM
  * @vpe:	Pointer to the GICv4 notion of a virtual CPU (VPE)
  * @vintid:	Virtual LPI number
+ * @properties:	Priority and enable bits (as written in the prop table)
  * @db_enabled:	Is the VPE doorbell to be generated?
  */
 struct its_vlpi_map {
 	struct its_vm		*vm;
 	struct its_vpe		*vpe;
 	u32			vintid;
+	u8			properties;
 	bool			db_enabled;
 };
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 01/26] irqchip/gic-v3-its: Setup VLPI properties at map time
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

So far, we require the hypervisor to update the VLPI properties
once the the VLPI mapping has been established. While this
makes it easy for the ITS driver, it creates a window where
an incoming interrupt can be delivered with an unknown set
of properties. Not very nice.

Instead, let's add a "properties" field to the mapping structure,
and use that to configure the VLPI before it actually gets mapped.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
 include/linux/irqchip/arm-gic-v4.h |  2 ++
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 85f47ef4f1f7..2c4074c97454 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1006,9 +1006,15 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 	if (irqd_is_forwarded_to_vcpu(d)) {
 		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 		u32 event = its_get_event_id(d);
+		struct its_vlpi_map *map;
 
 		prop_page = its_dev->event_map.vm->vprop_page;
-		hwirq = its_dev->event_map.vlpi_maps[event].vintid;
+		map = &its_dev->event_map.vlpi_maps[event];
+		hwirq = map->vintid;
+
+		/* Remember the updated property */
+		map->properties &= ~clr;
+		map->properties |= set | LPI_PROP_GROUP1;
 	} else {
 		prop_page = gic_rdists->prop_page;
 		hwirq = d->hwirq;
@@ -1018,6 +1024,7 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 	*cfg &= ~clr;
 	*cfg |= set | LPI_PROP_GROUP1;
 
+
 	/*
 	 * Make the above write visible to the redistributors.
 	 * And yes, we're flushing exactly: One. Single. Byte.
@@ -1238,12 +1245,20 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
 		/* Ensure all the VPEs are mapped on this ITS */
 		its_map_vm(its_dev->its, info->map->vm);
 
+		/*
+		 * Flag the interrupt as forwarded so that we can
+		 * start poking the virtual property table.
+		 */
+		irqd_set_forwarded_to_vcpu(d);
+
+		/* Write out the property to the prop table */
+		lpi_write_config(d, 0xff, info->map->properties);
+
 		/* Drop the physical mapping */
 		its_send_discard(its_dev, event);
 
 		/* and install the virtual one */
 		its_send_vmapti(its_dev, event);
-		irqd_set_forwarded_to_vcpu(d);
 
 		/* Increment the number of VLPIs */
 		its_dev->event_map.nr_vlpis++;
diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
index 43cde15f221b..447da8ca2156 100644
--- a/include/linux/irqchip/arm-gic-v4.h
+++ b/include/linux/irqchip/arm-gic-v4.h
@@ -71,12 +71,14 @@ struct its_vpe {
  * @vm:		Pointer to the GICv4 notion of a VM
  * @vpe:	Pointer to the GICv4 notion of a virtual CPU (VPE)
  * @vintid:	Virtual LPI number
+ * @properties:	Priority and enable bits (as written in the prop table)
  * @db_enabled:	Is the VPE doorbell to be generated?
  */
 struct its_vlpi_map {
 	struct its_vm		*vm;
 	struct its_vpe		*vpe;
 	u32			vintid;
+	u8			properties;
 	bool			db_enabled;
 };
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 02/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

From: Eric Auger <eric.auger@redhat.com>

This patch selects IRQ_BYPASS_MANAGER and HAVE_KVM_IRQ_BYPASS
configs for ARM/ARM64.

kvm_arch_has_irq_bypass() now is implemented and returns true.
As a consequence the irq bypass consumer will be registered for
ARM/ARM64 with the forwarding callbacks:

- stop/start: halt/resume guest execution
- add/del_producer: set/unset forwarding at vgic/irqchip level

We don't have any actual support yet, so nothing gets actually
forwarded.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
[maz: dropped the DEOI stuff for the time being in order to
      reduce the dependency chain, amended commit message]
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Kconfig   |  3 +++
 arch/arm64/kvm/Kconfig |  3 +++
 virt/kvm/arm/arm.c     | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 90d0176fb30d..4e2b192a030a 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -35,6 +36,8 @@ config KVM
 	select HAVE_KVM_IRQCHIP
 	select HAVE_KVM_IRQ_ROUTING
 	select HAVE_KVM_MSI
+	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
 	---help---
 	  Support hosting virtualized guest machines.
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 52cb7ad9b2fd..7e0d6e63cc71 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -35,6 +36,8 @@ config KVM
 	select HAVE_KVM_MSI
 	select HAVE_KVM_IRQCHIP
 	select HAVE_KVM_IRQ_ROUTING
+	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	---help---
 	  Support hosting virtualized guest machines.
 	  We don't support KVM with 16K page tables yet, due to the multiple
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index b9f68e4add71..7ac4ecba9ebd 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -27,6 +27,8 @@
 #include <linux/mman.h>
 #include <linux/sched.h>
 #include <linux/kvm.h>
+#include <linux/kvm_irqfd.h>
+#include <linux/irqbypass.h>
 #include <trace/events/kvm.h>
 #include <kvm/arm_pmu.h>
 
@@ -1449,6 +1451,44 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
 	return NULL;
 }
 
+bool kvm_arch_has_irq_bypass(void)
+{
+	return true;
+}
+
+int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	return 0;
+}
+void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	return;
+}
+
+void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_halt_guest(irqfd->kvm);
+}
+
+void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_resume_guest(irqfd->kvm);
+}
+
 /**
  * Initialize Hyp-mode and memory mappings on all CPUs.
  */
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 02/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

From: Eric Auger <eric.auger@redhat.com>

This patch selects IRQ_BYPASS_MANAGER and HAVE_KVM_IRQ_BYPASS
configs for ARM/ARM64.

kvm_arch_has_irq_bypass() now is implemented and returns true.
As a consequence the irq bypass consumer will be registered for
ARM/ARM64 with the forwarding callbacks:

- stop/start: halt/resume guest execution
- add/del_producer: set/unset forwarding at vgic/irqchip level

We don't have any actual support yet, so nothing gets actually
forwarded.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
[maz: dropped the DEOI stuff for the time being in order to
      reduce the dependency chain, amended commit message]
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Kconfig   |  3 +++
 arch/arm64/kvm/Kconfig |  3 +++
 virt/kvm/arm/arm.c     | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 90d0176fb30d..4e2b192a030a 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -35,6 +36,8 @@ config KVM
 	select HAVE_KVM_IRQCHIP
 	select HAVE_KVM_IRQ_ROUTING
 	select HAVE_KVM_MSI
+	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
 	---help---
 	  Support hosting virtualized guest machines.
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 52cb7ad9b2fd..7e0d6e63cc71 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -35,6 +36,8 @@ config KVM
 	select HAVE_KVM_MSI
 	select HAVE_KVM_IRQCHIP
 	select HAVE_KVM_IRQ_ROUTING
+	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	---help---
 	  Support hosting virtualized guest machines.
 	  We don't support KVM with 16K page tables yet, due to the multiple
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index b9f68e4add71..7ac4ecba9ebd 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -27,6 +27,8 @@
 #include <linux/mman.h>
 #include <linux/sched.h>
 #include <linux/kvm.h>
+#include <linux/kvm_irqfd.h>
+#include <linux/irqbypass.h>
 #include <trace/events/kvm.h>
 #include <kvm/arm_pmu.h>
 
@@ -1449,6 +1451,44 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
 	return NULL;
 }
 
+bool kvm_arch_has_irq_bypass(void)
+{
+	return true;
+}
+
+int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	return 0;
+}
+void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	return;
+}
+
+void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_halt_guest(irqfd->kvm);
+}
+
+void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_resume_guest(irqfd->kvm);
+}
+
 /**
  * Initialize Hyp-mode and memory mappings on all CPUs.
  */
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 02/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

From: Eric Auger <eric.auger@redhat.com>

This patch selects IRQ_BYPASS_MANAGER and HAVE_KVM_IRQ_BYPASS
configs for ARM/ARM64.

kvm_arch_has_irq_bypass() now is implemented and returns true.
As a consequence the irq bypass consumer will be registered for
ARM/ARM64 with the forwarding callbacks:

- stop/start: halt/resume guest execution
- add/del_producer: set/unset forwarding at vgic/irqchip level

We don't have any actual support yet, so nothing gets actually
forwarded.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
[maz: dropped the DEOI stuff for the time being in order to
      reduce the dependency chain, amended commit message]
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Kconfig   |  3 +++
 arch/arm64/kvm/Kconfig |  3 +++
 virt/kvm/arm/arm.c     | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 90d0176fb30d..4e2b192a030a 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -35,6 +36,8 @@ config KVM
 	select HAVE_KVM_IRQCHIP
 	select HAVE_KVM_IRQ_ROUTING
 	select HAVE_KVM_MSI
+	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
 	---help---
 	  Support hosting virtualized guest machines.
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 52cb7ad9b2fd..7e0d6e63cc71 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -35,6 +36,8 @@ config KVM
 	select HAVE_KVM_MSI
 	select HAVE_KVM_IRQCHIP
 	select HAVE_KVM_IRQ_ROUTING
+	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	---help---
 	  Support hosting virtualized guest machines.
 	  We don't support KVM with 16K page tables yet, due to the multiple
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index b9f68e4add71..7ac4ecba9ebd 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -27,6 +27,8 @@
 #include <linux/mman.h>
 #include <linux/sched.h>
 #include <linux/kvm.h>
+#include <linux/kvm_irqfd.h>
+#include <linux/irqbypass.h>
 #include <trace/events/kvm.h>
 #include <kvm/arm_pmu.h>
 
@@ -1449,6 +1451,44 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
 	return NULL;
 }
 
+bool kvm_arch_has_irq_bypass(void)
+{
+	return true;
+}
+
+int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	return 0;
+}
+void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	return;
+}
+
+void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_halt_guest(irqfd->kvm);
+}
+
+void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_resume_guest(irqfd->kvm);
+}
+
 /**
  * Initialize Hyp-mode and memory mappings on all CPUs.
  */
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 03/26] KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

From: Eric Auger <eric.auger@redhat.com>

We want to reuse the core of the map/unmap functions for IRQ
forwarding. Let's move the computation of the hwirq in
kvm_vgic_map_phys_irq and pass the linux IRQ as parameter.
the host_irq is added to struct vgic_irq.

We introduce kvm_vgic_map/unmap_irq which take a struct vgic_irq
handle as a parameter.

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h    |  8 ++++---
 virt/kvm/arm/arch_timer.c | 24 +------------------
 virt/kvm/arm/vgic/vgic.c  | 60 +++++++++++++++++++++++++++++++++++------------
 3 files changed, 51 insertions(+), 41 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 34dba516ef24..53f631bdec75 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -116,6 +116,7 @@ struct vgic_irq {
 	bool hw;			/* Tied to HW IRQ */
 	struct kref refcount;		/* Used for LPIs */
 	u32 hwintid;			/* HW INTID number */
+	unsigned int host_irq;		/* linux irq corresponding to hwintid */
 	union {
 		u8 targets;			/* GICv2 target VCPUs mask */
 		u32 mpidr;			/* GICv3 target VCPU */
@@ -307,9 +308,10 @@ void kvm_vgic_init_cpu_hardware(void);
 
 int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
 			bool level, void *owner);
-int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq);
-int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq);
-bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq);
+int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
+			  u32 vintid);
+int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid);
+bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid);
 
 int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
 
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 8e89d63005c7..b24e2f7b363a 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -647,9 +647,6 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
-	struct irq_desc *desc;
-	struct irq_data *data;
-	int phys_irq;
 	int ret;
 
 	if (timer->enabled)
@@ -667,26 +664,7 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 	}
 
-	/*
-	 * Find the physical IRQ number corresponding to the host_vtimer_irq
-	 */
-	desc = irq_to_desc(host_vtimer_irq);
-	if (!desc) {
-		kvm_err("%s: no interrupt descriptor\n", __func__);
-		return -EINVAL;
-	}
-
-	data = irq_desc_get_irq_data(desc);
-	while (data->parent_data)
-		data = data->parent_data;
-
-	phys_irq = data->hwirq;
-
-	/*
-	 * Tell the VGIC that the virtual interrupt is tied to a
-	 * physical interrupt. We do that once per VCPU.
-	 */
-	ret = kvm_vgic_map_phys_irq(vcpu, vtimer->irq.irq, phys_irq);
+	ret = kvm_vgic_map_phys_irq(vcpu, host_vtimer_irq, vtimer->irq.irq);
 	if (ret)
 		return ret;
 
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index fed717e07938..9d557efd1f7d 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -17,6 +17,8 @@
 #include <linux/kvm.h>
 #include <linux/kvm_host.h>
 #include <linux/list_sort.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
 
 #include "vgic.h"
 
@@ -403,38 +405,66 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
 	return 0;
 }
 
-int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq)
+/* @irq->irq_lock must be held */
+static int kvm_vgic_map_irq(struct kvm_vcpu *vcpu, struct vgic_irq *irq,
+			    unsigned int host_irq)
 {
-	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	struct irq_desc *desc;
+	struct irq_data *data;
 
-	BUG_ON(!irq);
-
-	spin_lock(&irq->irq_lock);
+	/*
+	 * Find the physical IRQ number corresponding to @host_irq
+	 */
+	desc = irq_to_desc(host_irq);
+	if (!desc) {
+		kvm_err("%s: no interrupt descriptor\n", __func__);
+		return -EINVAL;
+	}
+	data = irq_desc_get_irq_data(desc);
+	while (data->parent_data)
+		data = data->parent_data;
 
 	irq->hw = true;
-	irq->hwintid = phys_irq;
+	irq->host_irq = host_irq;
+	irq->hwintid = data->hwirq;
+	return 0;
+}
+
+/* @irq->irq_lock must be held */
+static inline void kvm_vgic_unmap_irq(struct vgic_irq *irq)
+{
+	irq->hw = false;
+	irq->hwintid = 0;
+}
+
+int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
+			  u32 vintid)
+{
+	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
+	int ret;
 
+	BUG_ON(!irq);
+
+	spin_lock(&irq->irq_lock);
+	ret = kvm_vgic_map_irq(vcpu, irq, host_irq);
 	spin_unlock(&irq->irq_lock);
 	vgic_put_irq(vcpu->kvm, irq);
 
-	return 0;
+	return ret;
 }
 
-int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq)
+int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid)
 {
 	struct vgic_irq *irq;
 
 	if (!vgic_initialized(vcpu->kvm))
 		return -EAGAIN;
 
-	irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
 	BUG_ON(!irq);
 
 	spin_lock(&irq->irq_lock);
-
-	irq->hw = false;
-	irq->hwintid = 0;
-
+	kvm_vgic_unmap_irq(irq);
 	spin_unlock(&irq->irq_lock);
 	vgic_put_irq(vcpu->kvm, irq);
 
@@ -772,9 +802,9 @@ void vgic_kick_vcpus(struct kvm *kvm)
 	}
 }
 
-bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq)
+bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid)
 {
-	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
 	bool map_is_active;
 
 	spin_lock(&irq->irq_lock);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 03/26] KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

From: Eric Auger <eric.auger@redhat.com>

We want to reuse the core of the map/unmap functions for IRQ
forwarding. Let's move the computation of the hwirq in
kvm_vgic_map_phys_irq and pass the linux IRQ as parameter.
the host_irq is added to struct vgic_irq.

We introduce kvm_vgic_map/unmap_irq which take a struct vgic_irq
handle as a parameter.

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h    |  8 ++++---
 virt/kvm/arm/arch_timer.c | 24 +------------------
 virt/kvm/arm/vgic/vgic.c  | 60 +++++++++++++++++++++++++++++++++++------------
 3 files changed, 51 insertions(+), 41 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 34dba516ef24..53f631bdec75 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -116,6 +116,7 @@ struct vgic_irq {
 	bool hw;			/* Tied to HW IRQ */
 	struct kref refcount;		/* Used for LPIs */
 	u32 hwintid;			/* HW INTID number */
+	unsigned int host_irq;		/* linux irq corresponding to hwintid */
 	union {
 		u8 targets;			/* GICv2 target VCPUs mask */
 		u32 mpidr;			/* GICv3 target VCPU */
@@ -307,9 +308,10 @@ void kvm_vgic_init_cpu_hardware(void);
 
 int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
 			bool level, void *owner);
-int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq);
-int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq);
-bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq);
+int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
+			  u32 vintid);
+int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid);
+bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid);
 
 int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
 
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 8e89d63005c7..b24e2f7b363a 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -647,9 +647,6 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
-	struct irq_desc *desc;
-	struct irq_data *data;
-	int phys_irq;
 	int ret;
 
 	if (timer->enabled)
@@ -667,26 +664,7 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 	}
 
-	/*
-	 * Find the physical IRQ number corresponding to the host_vtimer_irq
-	 */
-	desc = irq_to_desc(host_vtimer_irq);
-	if (!desc) {
-		kvm_err("%s: no interrupt descriptor\n", __func__);
-		return -EINVAL;
-	}
-
-	data = irq_desc_get_irq_data(desc);
-	while (data->parent_data)
-		data = data->parent_data;
-
-	phys_irq = data->hwirq;
-
-	/*
-	 * Tell the VGIC that the virtual interrupt is tied to a
-	 * physical interrupt. We do that once per VCPU.
-	 */
-	ret = kvm_vgic_map_phys_irq(vcpu, vtimer->irq.irq, phys_irq);
+	ret = kvm_vgic_map_phys_irq(vcpu, host_vtimer_irq, vtimer->irq.irq);
 	if (ret)
 		return ret;
 
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index fed717e07938..9d557efd1f7d 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -17,6 +17,8 @@
 #include <linux/kvm.h>
 #include <linux/kvm_host.h>
 #include <linux/list_sort.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
 
 #include "vgic.h"
 
@@ -403,38 +405,66 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
 	return 0;
 }
 
-int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq)
+/* @irq->irq_lock must be held */
+static int kvm_vgic_map_irq(struct kvm_vcpu *vcpu, struct vgic_irq *irq,
+			    unsigned int host_irq)
 {
-	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	struct irq_desc *desc;
+	struct irq_data *data;
 
-	BUG_ON(!irq);
-
-	spin_lock(&irq->irq_lock);
+	/*
+	 * Find the physical IRQ number corresponding to @host_irq
+	 */
+	desc = irq_to_desc(host_irq);
+	if (!desc) {
+		kvm_err("%s: no interrupt descriptor\n", __func__);
+		return -EINVAL;
+	}
+	data = irq_desc_get_irq_data(desc);
+	while (data->parent_data)
+		data = data->parent_data;
 
 	irq->hw = true;
-	irq->hwintid = phys_irq;
+	irq->host_irq = host_irq;
+	irq->hwintid = data->hwirq;
+	return 0;
+}
+
+/* @irq->irq_lock must be held */
+static inline void kvm_vgic_unmap_irq(struct vgic_irq *irq)
+{
+	irq->hw = false;
+	irq->hwintid = 0;
+}
+
+int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
+			  u32 vintid)
+{
+	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
+	int ret;
 
+	BUG_ON(!irq);
+
+	spin_lock(&irq->irq_lock);
+	ret = kvm_vgic_map_irq(vcpu, irq, host_irq);
 	spin_unlock(&irq->irq_lock);
 	vgic_put_irq(vcpu->kvm, irq);
 
-	return 0;
+	return ret;
 }
 
-int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq)
+int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid)
 {
 	struct vgic_irq *irq;
 
 	if (!vgic_initialized(vcpu->kvm))
 		return -EAGAIN;
 
-	irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
 	BUG_ON(!irq);
 
 	spin_lock(&irq->irq_lock);
-
-	irq->hw = false;
-	irq->hwintid = 0;
-
+	kvm_vgic_unmap_irq(irq);
 	spin_unlock(&irq->irq_lock);
 	vgic_put_irq(vcpu->kvm, irq);
 
@@ -772,9 +802,9 @@ void vgic_kick_vcpus(struct kvm *kvm)
 	}
 }
 
-bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq)
+bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid)
 {
-	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
 	bool map_is_active;
 
 	spin_lock(&irq->irq_lock);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 03/26] KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

From: Eric Auger <eric.auger@redhat.com>

We want to reuse the core of the map/unmap functions for IRQ
forwarding. Let's move the computation of the hwirq in
kvm_vgic_map_phys_irq and pass the linux IRQ as parameter.
the host_irq is added to struct vgic_irq.

We introduce kvm_vgic_map/unmap_irq which take a struct vgic_irq
handle as a parameter.

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h    |  8 ++++---
 virt/kvm/arm/arch_timer.c | 24 +------------------
 virt/kvm/arm/vgic/vgic.c  | 60 +++++++++++++++++++++++++++++++++++------------
 3 files changed, 51 insertions(+), 41 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 34dba516ef24..53f631bdec75 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -116,6 +116,7 @@ struct vgic_irq {
 	bool hw;			/* Tied to HW IRQ */
 	struct kref refcount;		/* Used for LPIs */
 	u32 hwintid;			/* HW INTID number */
+	unsigned int host_irq;		/* linux irq corresponding to hwintid */
 	union {
 		u8 targets;			/* GICv2 target VCPUs mask */
 		u32 mpidr;			/* GICv3 target VCPU */
@@ -307,9 +308,10 @@ void kvm_vgic_init_cpu_hardware(void);
 
 int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
 			bool level, void *owner);
-int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq);
-int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq);
-bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq);
+int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
+			  u32 vintid);
+int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid);
+bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid);
 
 int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
 
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 8e89d63005c7..b24e2f7b363a 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -647,9 +647,6 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
-	struct irq_desc *desc;
-	struct irq_data *data;
-	int phys_irq;
 	int ret;
 
 	if (timer->enabled)
@@ -667,26 +664,7 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 	}
 
-	/*
-	 * Find the physical IRQ number corresponding to the host_vtimer_irq
-	 */
-	desc = irq_to_desc(host_vtimer_irq);
-	if (!desc) {
-		kvm_err("%s: no interrupt descriptor\n", __func__);
-		return -EINVAL;
-	}
-
-	data = irq_desc_get_irq_data(desc);
-	while (data->parent_data)
-		data = data->parent_data;
-
-	phys_irq = data->hwirq;
-
-	/*
-	 * Tell the VGIC that the virtual interrupt is tied to a
-	 * physical interrupt. We do that once per VCPU.
-	 */
-	ret = kvm_vgic_map_phys_irq(vcpu, vtimer->irq.irq, phys_irq);
+	ret = kvm_vgic_map_phys_irq(vcpu, host_vtimer_irq, vtimer->irq.irq);
 	if (ret)
 		return ret;
 
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index fed717e07938..9d557efd1f7d 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -17,6 +17,8 @@
 #include <linux/kvm.h>
 #include <linux/kvm_host.h>
 #include <linux/list_sort.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
 
 #include "vgic.h"
 
@@ -403,38 +405,66 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,
 	return 0;
 }
 
-int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq)
+/* @irq->irq_lock must be held */
+static int kvm_vgic_map_irq(struct kvm_vcpu *vcpu, struct vgic_irq *irq,
+			    unsigned int host_irq)
 {
-	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	struct irq_desc *desc;
+	struct irq_data *data;
 
-	BUG_ON(!irq);
-
-	spin_lock(&irq->irq_lock);
+	/*
+	 * Find the physical IRQ number corresponding to @host_irq
+	 */
+	desc = irq_to_desc(host_irq);
+	if (!desc) {
+		kvm_err("%s: no interrupt descriptor\n", __func__);
+		return -EINVAL;
+	}
+	data = irq_desc_get_irq_data(desc);
+	while (data->parent_data)
+		data = data->parent_data;
 
 	irq->hw = true;
-	irq->hwintid = phys_irq;
+	irq->host_irq = host_irq;
+	irq->hwintid = data->hwirq;
+	return 0;
+}
+
+/* @irq->irq_lock must be held */
+static inline void kvm_vgic_unmap_irq(struct vgic_irq *irq)
+{
+	irq->hw = false;
+	irq->hwintid = 0;
+}
+
+int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
+			  u32 vintid)
+{
+	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
+	int ret;
 
+	BUG_ON(!irq);
+
+	spin_lock(&irq->irq_lock);
+	ret = kvm_vgic_map_irq(vcpu, irq, host_irq);
 	spin_unlock(&irq->irq_lock);
 	vgic_put_irq(vcpu->kvm, irq);
 
-	return 0;
+	return ret;
 }
 
-int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq)
+int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid)
 {
 	struct vgic_irq *irq;
 
 	if (!vgic_initialized(vcpu->kvm))
 		return -EAGAIN;
 
-	irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
 	BUG_ON(!irq);
 
 	spin_lock(&irq->irq_lock);
-
-	irq->hw = false;
-	irq->hwintid = 0;
-
+	kvm_vgic_unmap_irq(irq);
 	spin_unlock(&irq->irq_lock);
 	vgic_put_irq(vcpu->kvm, irq);
 
@@ -772,9 +802,9 @@ void vgic_kick_vcpus(struct kvm *kvm)
 	}
 }
 
-bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq)
+bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid)
 {
-	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);
+	struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
 	bool map_is_active;
 
 	spin_lock(&irq->irq_lock);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 04/26] KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

The GICv4 support introduces a hard dependency between the KVM
core and the ITS infrastructure. arm64 already selects it at
the architecture level, but 32bit doesn't. In order to avoid
littering the kernel with #ifdefs, let's just select the whole
of the GICv3 suport code.

You know you want it.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 4e2b192a030a..52b50af04e3b 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -23,6 +23,8 @@ config KVM
 	select PREEMPT_NOTIFIERS
 	select ANON_INODES
 	select ARM_GIC
+	select ARM_GIC_V3
+	select ARM_GIC_V3_ITS
 	select HAVE_KVM_CPU_RELAX_INTERCEPT
 	select HAVE_KVM_ARCH_TLB_FLUSH_ALL
 	select KVM_MMIO
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 04/26] KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

The GICv4 support introduces a hard dependency between the KVM
core and the ITS infrastructure. arm64 already selects it at
the architecture level, but 32bit doesn't. In order to avoid
littering the kernel with #ifdefs, let's just select the whole
of the GICv3 suport code.

You know you want it.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 4e2b192a030a..52b50af04e3b 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -23,6 +23,8 @@ config KVM
 	select PREEMPT_NOTIFIERS
 	select ANON_INODES
 	select ARM_GIC
+	select ARM_GIC_V3
+	select ARM_GIC_V3_ITS
 	select HAVE_KVM_CPU_RELAX_INTERCEPT
 	select HAVE_KVM_ARCH_TLB_FLUSH_ALL
 	select KVM_MMIO
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 04/26] KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

The GICv4 support introduces a hard dependency between the KVM
core and the ITS infrastructure. arm64 already selects it at
the architecture level, but 32bit doesn't. In order to avoid
littering the kernel with #ifdefs, let's just select the whole
of the GICv3 suport code.

You know you want it.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 4e2b192a030a..52b50af04e3b 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -23,6 +23,8 @@ config KVM
 	select PREEMPT_NOTIFIERS
 	select ANON_INODES
 	select ARM_GIC
+	select ARM_GIC_V3
+	select ARM_GIC_V3_ITS
 	select HAVE_KVM_CPU_RELAX_INTERCEPT
 	select HAVE_KVM_ARCH_TLB_FLUSH_ALL
 	select KVM_MMIO
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 05/26] KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

The way we call kvm_vgic_destroy is a bit bizarre. We call it
*after* having freed the vcpus, which sort of defeats the point
of cleaning up things before that point.

Let's move kvm_vgic_destroy towards the beginning of kvm_arch_destroy_vm,
which seems more sensible.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 7ac4ecba9ebd..5d5218ecd547 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -177,6 +177,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 {
 	int i;
 
+	kvm_vgic_destroy(kvm);
+
 	free_percpu(kvm->arch.last_vcpu_ran);
 	kvm->arch.last_vcpu_ran = NULL;
 
@@ -186,8 +188,6 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 			kvm->vcpus[i] = NULL;
 		}
 	}
-
-	kvm_vgic_destroy(kvm);
 }
 
 int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 05/26] KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

The way we call kvm_vgic_destroy is a bit bizarre. We call it
*after* having freed the vcpus, which sort of defeats the point
of cleaning up things before that point.

Let's move kvm_vgic_destroy towards the beginning of kvm_arch_destroy_vm,
which seems more sensible.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 7ac4ecba9ebd..5d5218ecd547 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -177,6 +177,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 {
 	int i;
 
+	kvm_vgic_destroy(kvm);
+
 	free_percpu(kvm->arch.last_vcpu_ran);
 	kvm->arch.last_vcpu_ran = NULL;
 
@@ -186,8 +188,6 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 			kvm->vcpus[i] = NULL;
 		}
 	}
-
-	kvm_vgic_destroy(kvm);
 }
 
 int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 05/26] KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

The way we call kvm_vgic_destroy is a bit bizarre. We call it
*after* having freed the vcpus, which sort of defeats the point
of cleaning up things before that point.

Let's move kvm_vgic_destroy towards the beginning of kvm_arch_destroy_vm,
which seems more sensible.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 7ac4ecba9ebd..5d5218ecd547 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -177,6 +177,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 {
 	int i;
 
+	kvm_vgic_destroy(kvm);
+
 	free_percpu(kvm->arch.last_vcpu_ran);
 	kvm->arch.last_vcpu_ran = NULL;
 
@@ -186,8 +188,6 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 			kvm->vcpus[i] = NULL;
 		}
 	}
-
-	kvm_vgic_destroy(kvm);
 }
 
 int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 06/26] KVM: arm/arm64: vITS: Add MSI translation helpers
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

The whole MSI injection process is fairly monolithic. An MSI write
gets turned into an injected LPI in one swift go. But this is actually
a more fine-grained process:

- First, a virtual ITS gets selected using the doorbell address
- Then the DevID/EventID pair gets translated into an LPI
- Finally the LPI is injected

Since the GICv4 code needs the first two steps in order to match
an IRQ routing entry to an LPI, let's expose them as helpers,
and refactor the existing code to use them

Reviewed-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 93 +++++++++++++++++++++++++-------------------
 virt/kvm/arm/vgic/vgic.h     |  4 ++
 2 files changed, 57 insertions(+), 40 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index f51c1e1b3f70..475dffd0a44c 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -503,15 +503,8 @@ static unsigned long vgic_mmio_read_its_idregs(struct kvm *kvm,
 	return 0;
 }
 
-/*
- * Find the target VCPU and the LPI number for a given devid/eventid pair
- * and make this IRQ pending, possibly injecting it.
- * Must be called with the its_lock mutex held.
- * Returns 0 on success, a positive error value for any ITS mapping
- * related errors and negative error values for generic errors.
- */
-static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
-				u32 devid, u32 eventid)
+int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
+			 u32 devid, u32 eventid, struct vgic_irq **irq)
 {
 	struct kvm_vcpu *vcpu;
 	struct its_ite *ite;
@@ -530,26 +523,60 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
 	if (!vcpu->arch.vgic_cpu.lpis_enabled)
 		return -EBUSY;
 
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->pending_latch = true;
-	vgic_queue_irq_unlock(kvm, ite->irq);
-
+	*irq = ite->irq;
 	return 0;
 }
 
-static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
+struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
 {
+	u64 address;
+	struct kvm_io_device *kvm_io_dev;
 	struct vgic_io_device *iodev;
 
-	if (dev->ops != &kvm_io_gic_ops)
-		return NULL;
+	if (!vgic_has_its(kvm))
+		return ERR_PTR(-ENODEV);
 
-	iodev = container_of(dev, struct vgic_io_device, dev);
+	if (!(msi->flags & KVM_MSI_VALID_DEVID))
+		return ERR_PTR(-EINVAL);
+
+	address = (u64)msi->address_hi << 32 | msi->address_lo;
+
+	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
+	if (!kvm_io_dev)
+		return ERR_PTR(-EINVAL);
 
+	if (kvm_io_dev->ops != &kvm_io_gic_ops)
+		return ERR_PTR(-EINVAL);
+
+	iodev = container_of(kvm_io_dev, struct vgic_io_device, dev);
 	if (iodev->iodev_type != IODEV_ITS)
-		return NULL;
+		return ERR_PTR(-EINVAL);
 
-	return iodev;
+	return iodev->its;
+}
+
+/*
+ * Find the target VCPU and the LPI number for a given devid/eventid pair
+ * and make this IRQ pending, possibly injecting it.
+ * Must be called with the its_lock mutex held.
+ * Returns 0 on success, a positive error value for any ITS mapping
+ * related errors and negative error values for generic errors.
+ */
+static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
+				u32 devid, u32 eventid)
+{
+	struct vgic_irq *irq = NULL;
+	int err;
+
+	err = vgic_its_resolve_lpi(kvm, its, devid, eventid, &irq);
+	if (err)
+		return err;
+
+	spin_lock(&irq->irq_lock);
+	irq->pending_latch = true;
+	vgic_queue_irq_unlock(kvm, irq);
+
+	return 0;
 }
 
 /*
@@ -560,30 +587,16 @@ static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
  */
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
 {
-	u64 address;
-	struct kvm_io_device *kvm_io_dev;
-	struct vgic_io_device *iodev;
+	struct vgic_its *its;
 	int ret;
 
-	if (!vgic_has_its(kvm))
-		return -ENODEV;
-
-	if (!(msi->flags & KVM_MSI_VALID_DEVID))
-		return -EINVAL;
-
-	address = (u64)msi->address_hi << 32 | msi->address_lo;
-
-	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
-	if (!kvm_io_dev)
-		return -EINVAL;
+	its = vgic_msi_to_its(kvm, msi);
+	if (IS_ERR(its))
+		return PTR_ERR(its);
 
-	iodev = vgic_get_its_iodev(kvm_io_dev);
-	if (!iodev)
-		return -EINVAL;
-
-	mutex_lock(&iodev->its->its_lock);
-	ret = vgic_its_trigger_msi(kvm, iodev->its, msi->devid, msi->data);
-	mutex_unlock(&iodev->its->its_lock);
+	mutex_lock(&its->its_lock);
+	ret = vgic_its_trigger_msi(kvm, its, msi->devid, msi->data);
+	mutex_unlock(&its->its_lock);
 
 	if (ret < 0)
 		return ret;
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index bf9ceab67c77..31b70326b966 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -236,4 +236,8 @@ static inline int vgic_v3_max_apr_idx(struct kvm_vcpu *vcpu)
 	}
 }
 
+int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
+			 u32 devid, u32 eventid, struct vgic_irq **irq);
+struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
+
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 06/26] KVM: arm/arm64: vITS: Add MSI translation helpers
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

The whole MSI injection process is fairly monolithic. An MSI write
gets turned into an injected LPI in one swift go. But this is actually
a more fine-grained process:

- First, a virtual ITS gets selected using the doorbell address
- Then the DevID/EventID pair gets translated into an LPI
- Finally the LPI is injected

Since the GICv4 code needs the first two steps in order to match
an IRQ routing entry to an LPI, let's expose them as helpers,
and refactor the existing code to use them

Reviewed-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 93 +++++++++++++++++++++++++-------------------
 virt/kvm/arm/vgic/vgic.h     |  4 ++
 2 files changed, 57 insertions(+), 40 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index f51c1e1b3f70..475dffd0a44c 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -503,15 +503,8 @@ static unsigned long vgic_mmio_read_its_idregs(struct kvm *kvm,
 	return 0;
 }
 
-/*
- * Find the target VCPU and the LPI number for a given devid/eventid pair
- * and make this IRQ pending, possibly injecting it.
- * Must be called with the its_lock mutex held.
- * Returns 0 on success, a positive error value for any ITS mapping
- * related errors and negative error values for generic errors.
- */
-static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
-				u32 devid, u32 eventid)
+int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
+			 u32 devid, u32 eventid, struct vgic_irq **irq)
 {
 	struct kvm_vcpu *vcpu;
 	struct its_ite *ite;
@@ -530,26 +523,60 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
 	if (!vcpu->arch.vgic_cpu.lpis_enabled)
 		return -EBUSY;
 
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->pending_latch = true;
-	vgic_queue_irq_unlock(kvm, ite->irq);
-
+	*irq = ite->irq;
 	return 0;
 }
 
-static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
+struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
 {
+	u64 address;
+	struct kvm_io_device *kvm_io_dev;
 	struct vgic_io_device *iodev;
 
-	if (dev->ops != &kvm_io_gic_ops)
-		return NULL;
+	if (!vgic_has_its(kvm))
+		return ERR_PTR(-ENODEV);
 
-	iodev = container_of(dev, struct vgic_io_device, dev);
+	if (!(msi->flags & KVM_MSI_VALID_DEVID))
+		return ERR_PTR(-EINVAL);
+
+	address = (u64)msi->address_hi << 32 | msi->address_lo;
+
+	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
+	if (!kvm_io_dev)
+		return ERR_PTR(-EINVAL);
 
+	if (kvm_io_dev->ops != &kvm_io_gic_ops)
+		return ERR_PTR(-EINVAL);
+
+	iodev = container_of(kvm_io_dev, struct vgic_io_device, dev);
 	if (iodev->iodev_type != IODEV_ITS)
-		return NULL;
+		return ERR_PTR(-EINVAL);
 
-	return iodev;
+	return iodev->its;
+}
+
+/*
+ * Find the target VCPU and the LPI number for a given devid/eventid pair
+ * and make this IRQ pending, possibly injecting it.
+ * Must be called with the its_lock mutex held.
+ * Returns 0 on success, a positive error value for any ITS mapping
+ * related errors and negative error values for generic errors.
+ */
+static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
+				u32 devid, u32 eventid)
+{
+	struct vgic_irq *irq = NULL;
+	int err;
+
+	err = vgic_its_resolve_lpi(kvm, its, devid, eventid, &irq);
+	if (err)
+		return err;
+
+	spin_lock(&irq->irq_lock);
+	irq->pending_latch = true;
+	vgic_queue_irq_unlock(kvm, irq);
+
+	return 0;
 }
 
 /*
@@ -560,30 +587,16 @@ static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
  */
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
 {
-	u64 address;
-	struct kvm_io_device *kvm_io_dev;
-	struct vgic_io_device *iodev;
+	struct vgic_its *its;
 	int ret;
 
-	if (!vgic_has_its(kvm))
-		return -ENODEV;
-
-	if (!(msi->flags & KVM_MSI_VALID_DEVID))
-		return -EINVAL;
-
-	address = (u64)msi->address_hi << 32 | msi->address_lo;
-
-	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
-	if (!kvm_io_dev)
-		return -EINVAL;
+	its = vgic_msi_to_its(kvm, msi);
+	if (IS_ERR(its))
+		return PTR_ERR(its);
 
-	iodev = vgic_get_its_iodev(kvm_io_dev);
-	if (!iodev)
-		return -EINVAL;
-
-	mutex_lock(&iodev->its->its_lock);
-	ret = vgic_its_trigger_msi(kvm, iodev->its, msi->devid, msi->data);
-	mutex_unlock(&iodev->its->its_lock);
+	mutex_lock(&its->its_lock);
+	ret = vgic_its_trigger_msi(kvm, its, msi->devid, msi->data);
+	mutex_unlock(&its->its_lock);
 
 	if (ret < 0)
 		return ret;
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index bf9ceab67c77..31b70326b966 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -236,4 +236,8 @@ static inline int vgic_v3_max_apr_idx(struct kvm_vcpu *vcpu)
 	}
 }
 
+int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
+			 u32 devid, u32 eventid, struct vgic_irq **irq);
+struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
+
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 06/26] KVM: arm/arm64: vITS: Add MSI translation helpers
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

The whole MSI injection process is fairly monolithic. An MSI write
gets turned into an injected LPI in one swift go. But this is actually
a more fine-grained process:

- First, a virtual ITS gets selected using the doorbell address
- Then the DevID/EventID pair gets translated into an LPI
- Finally the LPI is injected

Since the GICv4 code needs the first two steps in order to match
an IRQ routing entry to an LPI, let's expose them as helpers,
and refactor the existing code to use them

Reviewed-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 93 +++++++++++++++++++++++++-------------------
 virt/kvm/arm/vgic/vgic.h     |  4 ++
 2 files changed, 57 insertions(+), 40 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index f51c1e1b3f70..475dffd0a44c 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -503,15 +503,8 @@ static unsigned long vgic_mmio_read_its_idregs(struct kvm *kvm,
 	return 0;
 }
 
-/*
- * Find the target VCPU and the LPI number for a given devid/eventid pair
- * and make this IRQ pending, possibly injecting it.
- * Must be called with the its_lock mutex held.
- * Returns 0 on success, a positive error value for any ITS mapping
- * related errors and negative error values for generic errors.
- */
-static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
-				u32 devid, u32 eventid)
+int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
+			 u32 devid, u32 eventid, struct vgic_irq **irq)
 {
 	struct kvm_vcpu *vcpu;
 	struct its_ite *ite;
@@ -530,26 +523,60 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
 	if (!vcpu->arch.vgic_cpu.lpis_enabled)
 		return -EBUSY;
 
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->pending_latch = true;
-	vgic_queue_irq_unlock(kvm, ite->irq);
-
+	*irq = ite->irq;
 	return 0;
 }
 
-static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
+struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
 {
+	u64 address;
+	struct kvm_io_device *kvm_io_dev;
 	struct vgic_io_device *iodev;
 
-	if (dev->ops != &kvm_io_gic_ops)
-		return NULL;
+	if (!vgic_has_its(kvm))
+		return ERR_PTR(-ENODEV);
 
-	iodev = container_of(dev, struct vgic_io_device, dev);
+	if (!(msi->flags & KVM_MSI_VALID_DEVID))
+		return ERR_PTR(-EINVAL);
+
+	address = (u64)msi->address_hi << 32 | msi->address_lo;
+
+	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
+	if (!kvm_io_dev)
+		return ERR_PTR(-EINVAL);
 
+	if (kvm_io_dev->ops != &kvm_io_gic_ops)
+		return ERR_PTR(-EINVAL);
+
+	iodev = container_of(kvm_io_dev, struct vgic_io_device, dev);
 	if (iodev->iodev_type != IODEV_ITS)
-		return NULL;
+		return ERR_PTR(-EINVAL);
 
-	return iodev;
+	return iodev->its;
+}
+
+/*
+ * Find the target VCPU and the LPI number for a given devid/eventid pair
+ * and make this IRQ pending, possibly injecting it.
+ * Must be called with the its_lock mutex held.
+ * Returns 0 on success, a positive error value for any ITS mapping
+ * related errors and negative error values for generic errors.
+ */
+static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
+				u32 devid, u32 eventid)
+{
+	struct vgic_irq *irq = NULL;
+	int err;
+
+	err = vgic_its_resolve_lpi(kvm, its, devid, eventid, &irq);
+	if (err)
+		return err;
+
+	spin_lock(&irq->irq_lock);
+	irq->pending_latch = true;
+	vgic_queue_irq_unlock(kvm, irq);
+
+	return 0;
 }
 
 /*
@@ -560,30 +587,16 @@ static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
  */
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
 {
-	u64 address;
-	struct kvm_io_device *kvm_io_dev;
-	struct vgic_io_device *iodev;
+	struct vgic_its *its;
 	int ret;
 
-	if (!vgic_has_its(kvm))
-		return -ENODEV;
-
-	if (!(msi->flags & KVM_MSI_VALID_DEVID))
-		return -EINVAL;
-
-	address = (u64)msi->address_hi << 32 | msi->address_lo;
-
-	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
-	if (!kvm_io_dev)
-		return -EINVAL;
+	its = vgic_msi_to_its(kvm, msi);
+	if (IS_ERR(its))
+		return PTR_ERR(its);
 
-	iodev = vgic_get_its_iodev(kvm_io_dev);
-	if (!iodev)
-		return -EINVAL;
-
-	mutex_lock(&iodev->its->its_lock);
-	ret = vgic_its_trigger_msi(kvm, iodev->its, msi->devid, msi->data);
-	mutex_unlock(&iodev->its->its_lock);
+	mutex_lock(&its->its_lock);
+	ret = vgic_its_trigger_msi(kvm, its, msi->devid, msi->data);
+	mutex_unlock(&its->its_lock);
 
 	if (ret < 0)
 		return ret;
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index bf9ceab67c77..31b70326b966 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -236,4 +236,8 @@ static inline int vgic_v3_max_apr_idx(struct kvm_vcpu *vcpu)
 	}
 }
 
+int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
+			 u32 devid, u32 eventid, struct vgic_irq **irq);
+struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
+
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 07/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

In order help integrating the vITS code with GICv4, let's add
a new helper that deals with updating the affinity of an LPI,
which will later be augmented with super duper extra GICv4
goodness.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 475dffd0a44c..8ee03f1e89fc 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -335,6 +335,15 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
 	return i;
 }
 
+static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
+{
+	spin_lock(&irq->irq_lock);
+	irq->target_vcpu = vcpu;
+	spin_unlock(&irq->irq_lock);
+
+	return 0;
+}
+
 /*
  * Promotes the ITS view of affinity of an ITTE (which redistributor this LPI
  * is targeting) to the VGIC's view, which deals with target VCPUs.
@@ -349,10 +358,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
 		return;
 
 	vcpu = kvm_get_vcpu(kvm, ite->collection->target_addr);
-
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->target_vcpu = vcpu;
-	spin_unlock(&ite->irq->irq_lock);
+	update_affinity(ite->irq, vcpu);
 }
 
 /*
@@ -693,11 +699,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
 	ite->collection = collection;
 	vcpu = kvm_get_vcpu(kvm, collection->target_addr);
 
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->target_vcpu = vcpu;
-	spin_unlock(&ite->irq->irq_lock);
-
-	return 0;
+	return update_affinity(ite->irq, vcpu);
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 07/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

In order help integrating the vITS code with GICv4, let's add
a new helper that deals with updating the affinity of an LPI,
which will later be augmented with super duper extra GICv4
goodness.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 475dffd0a44c..8ee03f1e89fc 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -335,6 +335,15 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
 	return i;
 }
 
+static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
+{
+	spin_lock(&irq->irq_lock);
+	irq->target_vcpu = vcpu;
+	spin_unlock(&irq->irq_lock);
+
+	return 0;
+}
+
 /*
  * Promotes the ITS view of affinity of an ITTE (which redistributor this LPI
  * is targeting) to the VGIC's view, which deals with target VCPUs.
@@ -349,10 +358,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
 		return;
 
 	vcpu = kvm_get_vcpu(kvm, ite->collection->target_addr);
-
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->target_vcpu = vcpu;
-	spin_unlock(&ite->irq->irq_lock);
+	update_affinity(ite->irq, vcpu);
 }
 
 /*
@@ -693,11 +699,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
 	ite->collection = collection;
 	vcpu = kvm_get_vcpu(kvm, collection->target_addr);
 
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->target_vcpu = vcpu;
-	spin_unlock(&ite->irq->irq_lock);
-
-	return 0;
+	return update_affinity(ite->irq, vcpu);
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 07/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

In order help integrating the vITS code with GICv4, let's add
a new helper that deals with updating the affinity of an LPI,
which will later be augmented with super duper extra GICv4
goodness.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 475dffd0a44c..8ee03f1e89fc 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -335,6 +335,15 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
 	return i;
 }
 
+static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
+{
+	spin_lock(&irq->irq_lock);
+	irq->target_vcpu = vcpu;
+	spin_unlock(&irq->irq_lock);
+
+	return 0;
+}
+
 /*
  * Promotes the ITS view of affinity of an ITTE (which redistributor this LPI
  * is targeting) to the VGIC's view, which deals with target VCPUs.
@@ -349,10 +358,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
 		return;
 
 	vcpu = kvm_get_vcpu(kvm, ite->collection->target_addr);
-
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->target_vcpu = vcpu;
-	spin_unlock(&ite->irq->irq_lock);
+	update_affinity(ite->irq, vcpu);
 }
 
 /*
@@ -693,11 +699,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
 	ite->collection = collection;
 	vcpu = kvm_get_vcpu(kvm, collection->target_addr);
 
-	spin_lock(&ite->irq->irq_lock);
-	ite->irq->target_vcpu = vcpu;
-	spin_unlock(&ite->irq->irq_lock);
-
-	return 0;
+	return update_affinity(ite->irq, vcpu);
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 08/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Add a new has_gicv4 field in the global VGIC state that indicates
whether the HW is GICv4 capable, as a per-VM predicate indicating
if there is a possibility for a VM to support direct injection
(the above being true and the VM having an ITS).

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h           | 3 +++
 virt/kvm/arm/vgic/vgic-mmio-v3.c | 5 +++++
 virt/kvm/arm/vgic/vgic.h         | 2 ++
 3 files changed, 10 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 53f631bdec75..ba9fb450aa1b 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -73,6 +73,9 @@ struct vgic_global {
 	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
 	bool			can_emulate_gicv2;
 
+	/* Hardware has GICv4? */
+	bool			has_gicv4;
+
 	/* GIC system register CPU interface */
 	struct static_key_false gicv3_cpuif;
 
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 408ef06638fc..f87fd17b2eb9 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -54,6 +54,11 @@ bool vgic_has_its(struct kvm *kvm)
 	return dist->has_its;
 }
 
+bool vgic_supports_direct_msis(struct kvm *kvm)
+{
+	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
+}
+
 static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
 {
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 31b70326b966..e67ccb6a6250 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -240,4 +240,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
 			 u32 devid, u32 eventid, struct vgic_irq **irq);
 struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 
+bool vgic_supports_direct_msis(struct kvm *kvm);
+
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 08/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

Add a new has_gicv4 field in the global VGIC state that indicates
whether the HW is GICv4 capable, as a per-VM predicate indicating
if there is a possibility for a VM to support direct injection
(the above being true and the VM having an ITS).

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h           | 3 +++
 virt/kvm/arm/vgic/vgic-mmio-v3.c | 5 +++++
 virt/kvm/arm/vgic/vgic.h         | 2 ++
 3 files changed, 10 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 53f631bdec75..ba9fb450aa1b 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -73,6 +73,9 @@ struct vgic_global {
 	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
 	bool			can_emulate_gicv2;
 
+	/* Hardware has GICv4? */
+	bool			has_gicv4;
+
 	/* GIC system register CPU interface */
 	struct static_key_false gicv3_cpuif;
 
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 408ef06638fc..f87fd17b2eb9 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -54,6 +54,11 @@ bool vgic_has_its(struct kvm *kvm)
 	return dist->has_its;
 }
 
+bool vgic_supports_direct_msis(struct kvm *kvm)
+{
+	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
+}
+
 static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
 {
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 31b70326b966..e67ccb6a6250 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -240,4 +240,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
 			 u32 devid, u32 eventid, struct vgic_irq **irq);
 struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 
+bool vgic_supports_direct_msis(struct kvm *kvm);
+
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 08/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

Add a new has_gicv4 field in the global VGIC state that indicates
whether the HW is GICv4 capable, as a per-VM predicate indicating
if there is a possibility for a VM to support direct injection
(the above being true and the VM having an ITS).

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h           | 3 +++
 virt/kvm/arm/vgic/vgic-mmio-v3.c | 5 +++++
 virt/kvm/arm/vgic/vgic.h         | 2 ++
 3 files changed, 10 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 53f631bdec75..ba9fb450aa1b 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -73,6 +73,9 @@ struct vgic_global {
 	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
 	bool			can_emulate_gicv2;
 
+	/* Hardware has GICv4? */
+	bool			has_gicv4;
+
 	/* GIC system register CPU interface */
 	struct static_key_false gicv3_cpuif;
 
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 408ef06638fc..f87fd17b2eb9 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -54,6 +54,11 @@ bool vgic_has_its(struct kvm *kvm)
 	return dist->has_its;
 }
 
+bool vgic_supports_direct_msis(struct kvm *kvm)
+{
+	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
+}
+
 static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
 {
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 31b70326b966..e67ccb6a6250 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -240,4 +240,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
 			 u32 devid, u32 eventid, struct vgic_irq **irq);
 struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 
+bool vgic_supports_direct_msis(struct kvm *kvm);
+
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

In order to control the GICv4 view of virtual CPUs, we rely
on an irqdomain allocated for that purpose. Let's add a couple
of helpers to that effect.

At the same time, the vgic data structures gain new fields to
track all this... erm... wonderful stuff.

The way we hook into the vgic init is slightly convoluted. We
need the vgic to be initialized (in order to guarantee that
the number of vcpus is now fixed), and we must have a vITS
(otherwise this is all very pointless). So we end-up calling
the init from both vgic_init and vgic_its_create.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Makefile         |  1 +
 arch/arm64/kvm/Makefile       |  1 +
 include/kvm/arm_vgic.h        | 19 ++++++++++
 virt/kvm/arm/vgic/vgic-init.c |  9 +++++
 virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
 virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.h      |  2 ++
 7 files changed, 123 insertions(+)
 create mode 100644 virt/kvm/arm/vgic/vgic-v4.c

diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index d9beee652d36..0a1dd2cdb928 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
 obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
 obj-y += $(KVM)/arm/vgic/vgic-v2.o
 obj-y += $(KVM)/arm/vgic/vgic-v3.o
+obj-y += $(KVM)/arm/vgic/vgic-v4.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 5d9810086c25..c30fd388ef80 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index ba9fb450aa1b..7eeb6c2a2f9c 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -26,6 +26,8 @@
 #include <linux/list.h>
 #include <linux/jump_label.h>
 
+#include <linux/irqchip/arm-gic-v4.h>
+
 #define VGIC_V3_MAX_CPUS	255
 #define VGIC_V2_MAX_CPUS	8
 #define VGIC_NR_IRQS_LEGACY     256
@@ -236,6 +238,15 @@ struct vgic_dist {
 
 	/* used by vgic-debug */
 	struct vgic_state_iter *iter;
+
+	/*
+	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
+	 * array, the property table pointer as well as allocation
+	 * data. This essentially ties the Linux IRQ core and ITS
+	 * together, and avoids leaking KVM's data structures anywhere
+	 * else.
+	 */
+	struct its_vm		its_vm;
 };
 
 struct vgic_v2_cpu_if {
@@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
 	u32		vgic_ap0r[4];
 	u32		vgic_ap1r[4];
 	u64		vgic_lr[VGIC_V3_MAX_LRS];
+
+	/*
+	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
+	 * pending table pointer, the its_vm pointer and a few other
+	 * HW specific things. As for the its_vm structure, this is
+	 * linking the Linux IRQ subsystem and the ITS together.
+	 */
+	struct its_vpe	its_vpe;
 };
 
 struct vgic_cpu {
diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 5801261f3add..40be908da238 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
 	if (ret)
 		goto out;
 
+	if (vgic_supports_direct_msis(kvm)) {
+		ret = vgic_v4_init(kvm);
+		if (ret)
+			goto out;
+	}
+
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_enable(vcpu);
 
@@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
 
 	kfree(dist->spis);
 	dist->nr_spis = 0;
+
+	if (vgic_supports_direct_msis(kvm))
+		vgic_v4_teardown(kvm);
 }
 
 void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 8ee03f1e89fc..89768d2b6a91 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
 	if (!its)
 		return -ENOMEM;
 
+	if (vgic_initialized(dev->kvm)) {
+		int ret = vgic_v4_init(dev->kvm);
+		if (ret) {
+			kfree(its);
+			return ret;
+		}
+	}
+	
 	mutex_init(&its->its_lock);
 	mutex_init(&its->cmd_lock);
 
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
new file mode 100644
index 000000000000..c794f0cef09e
--- /dev/null
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -0,0 +1,83 @@
+/*
+ * Copyright (C) 2017 ARM Ltd.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/kvm_host.h>
+
+#include "vgic.h"
+
+/**
+ * vgic_v4_init - Initialize the GICv4 data structures
+ * @kvm:	Pointer to the VM being initialized
+ *
+ * We may be called each time a vITS is created, or when the
+ * vgic is initialized. This relies on kvm->lock to be
+ * held. In both cases, the number of vcpus should now be
+ * fixed.
+ */
+int vgic_v4_init(struct kvm *kvm)
+{
+	struct vgic_dist *dist = &kvm->arch.vgic;
+	struct kvm_vcpu *vcpu;
+	int i, nr_vcpus, ret;
+
+	if (dist->its_vm.vpes)
+		return 0;
+
+	nr_vcpus = atomic_read(&kvm->online_vcpus);
+
+	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
+				    GFP_KERNEL);
+	if (!dist->its_vm.vpes)
+		return -ENOMEM;
+
+	dist->its_vm.nr_vpes = nr_vcpus;
+
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
+
+	ret = its_alloc_vcpu_irqs(&dist->its_vm);
+	if (ret < 0) {
+		kvm_err("VPE IRQ allocation failure\n");
+		kfree(dist->its_vm.vpes);
+		dist->its_vm.nr_vpes = 0;
+		dist->its_vm.vpes = NULL;
+		return ret;
+	}
+
+	return ret;
+}
+
+/**
+ * vgic_v4_teardown - Free the GICv4 data structures
+ * @kvm:	Pointer to the VM being destroyed
+ *
+ * Relies on kvm->lock to be held.
+ */
+void vgic_v4_teardown(struct kvm *kvm)
+{
+	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
+
+	if (!its_vm->vpes)
+		return;
+
+	its_free_vcpu_irqs(its_vm);
+	kfree(its_vm->vpes);
+	its_vm->nr_vpes = 0;
+	its_vm->vpes = NULL;
+}
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index e67ccb6a6250..c4105f613f57 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
 struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 
 bool vgic_supports_direct_msis(struct kvm *kvm);
+int vgic_v4_init(struct kvm *kvm);
+void vgic_v4_teardown(struct kvm *kvm);
 
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

In order to control the GICv4 view of virtual CPUs, we rely
on an irqdomain allocated for that purpose. Let's add a couple
of helpers to that effect.

At the same time, the vgic data structures gain new fields to
track all this... erm... wonderful stuff.

The way we hook into the vgic init is slightly convoluted. We
need the vgic to be initialized (in order to guarantee that
the number of vcpus is now fixed), and we must have a vITS
(otherwise this is all very pointless). So we end-up calling
the init from both vgic_init and vgic_its_create.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Makefile         |  1 +
 arch/arm64/kvm/Makefile       |  1 +
 include/kvm/arm_vgic.h        | 19 ++++++++++
 virt/kvm/arm/vgic/vgic-init.c |  9 +++++
 virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
 virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.h      |  2 ++
 7 files changed, 123 insertions(+)
 create mode 100644 virt/kvm/arm/vgic/vgic-v4.c

diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index d9beee652d36..0a1dd2cdb928 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
 obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
 obj-y += $(KVM)/arm/vgic/vgic-v2.o
 obj-y += $(KVM)/arm/vgic/vgic-v3.o
+obj-y += $(KVM)/arm/vgic/vgic-v4.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 5d9810086c25..c30fd388ef80 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index ba9fb450aa1b..7eeb6c2a2f9c 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -26,6 +26,8 @@
 #include <linux/list.h>
 #include <linux/jump_label.h>
 
+#include <linux/irqchip/arm-gic-v4.h>
+
 #define VGIC_V3_MAX_CPUS	255
 #define VGIC_V2_MAX_CPUS	8
 #define VGIC_NR_IRQS_LEGACY     256
@@ -236,6 +238,15 @@ struct vgic_dist {
 
 	/* used by vgic-debug */
 	struct vgic_state_iter *iter;
+
+	/*
+	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
+	 * array, the property table pointer as well as allocation
+	 * data. This essentially ties the Linux IRQ core and ITS
+	 * together, and avoids leaking KVM's data structures anywhere
+	 * else.
+	 */
+	struct its_vm		its_vm;
 };
 
 struct vgic_v2_cpu_if {
@@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
 	u32		vgic_ap0r[4];
 	u32		vgic_ap1r[4];
 	u64		vgic_lr[VGIC_V3_MAX_LRS];
+
+	/*
+	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
+	 * pending table pointer, the its_vm pointer and a few other
+	 * HW specific things. As for the its_vm structure, this is
+	 * linking the Linux IRQ subsystem and the ITS together.
+	 */
+	struct its_vpe	its_vpe;
 };
 
 struct vgic_cpu {
diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 5801261f3add..40be908da238 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
 	if (ret)
 		goto out;
 
+	if (vgic_supports_direct_msis(kvm)) {
+		ret = vgic_v4_init(kvm);
+		if (ret)
+			goto out;
+	}
+
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_enable(vcpu);
 
@@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
 
 	kfree(dist->spis);
 	dist->nr_spis = 0;
+
+	if (vgic_supports_direct_msis(kvm))
+		vgic_v4_teardown(kvm);
 }
 
 void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 8ee03f1e89fc..89768d2b6a91 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
 	if (!its)
 		return -ENOMEM;
 
+	if (vgic_initialized(dev->kvm)) {
+		int ret = vgic_v4_init(dev->kvm);
+		if (ret) {
+			kfree(its);
+			return ret;
+		}
+	}
+	
 	mutex_init(&its->its_lock);
 	mutex_init(&its->cmd_lock);
 
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
new file mode 100644
index 000000000000..c794f0cef09e
--- /dev/null
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -0,0 +1,83 @@
+/*
+ * Copyright (C) 2017 ARM Ltd.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/kvm_host.h>
+
+#include "vgic.h"
+
+/**
+ * vgic_v4_init - Initialize the GICv4 data structures
+ * @kvm:	Pointer to the VM being initialized
+ *
+ * We may be called each time a vITS is created, or when the
+ * vgic is initialized. This relies on kvm->lock to be
+ * held. In both cases, the number of vcpus should now be
+ * fixed.
+ */
+int vgic_v4_init(struct kvm *kvm)
+{
+	struct vgic_dist *dist = &kvm->arch.vgic;
+	struct kvm_vcpu *vcpu;
+	int i, nr_vcpus, ret;
+
+	if (dist->its_vm.vpes)
+		return 0;
+
+	nr_vcpus = atomic_read(&kvm->online_vcpus);
+
+	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
+				    GFP_KERNEL);
+	if (!dist->its_vm.vpes)
+		return -ENOMEM;
+
+	dist->its_vm.nr_vpes = nr_vcpus;
+
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
+
+	ret = its_alloc_vcpu_irqs(&dist->its_vm);
+	if (ret < 0) {
+		kvm_err("VPE IRQ allocation failure\n");
+		kfree(dist->its_vm.vpes);
+		dist->its_vm.nr_vpes = 0;
+		dist->its_vm.vpes = NULL;
+		return ret;
+	}
+
+	return ret;
+}
+
+/**
+ * vgic_v4_teardown - Free the GICv4 data structures
+ * @kvm:	Pointer to the VM being destroyed
+ *
+ * Relies on kvm->lock to be held.
+ */
+void vgic_v4_teardown(struct kvm *kvm)
+{
+	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
+
+	if (!its_vm->vpes)
+		return;
+
+	its_free_vcpu_irqs(its_vm);
+	kfree(its_vm->vpes);
+	its_vm->nr_vpes = 0;
+	its_vm->vpes = NULL;
+}
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index e67ccb6a6250..c4105f613f57 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
 struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 
 bool vgic_supports_direct_msis(struct kvm *kvm);
+int vgic_v4_init(struct kvm *kvm);
+void vgic_v4_teardown(struct kvm *kvm);
 
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

In order to control the GICv4 view of virtual CPUs, we rely
on an irqdomain allocated for that purpose. Let's add a couple
of helpers to that effect.

At the same time, the vgic data structures gain new fields to
track all this... erm... wonderful stuff.

The way we hook into the vgic init is slightly convoluted. We
need the vgic to be initialized (in order to guarantee that
the number of vcpus is now fixed), and we must have a vITS
(otherwise this is all very pointless). So we end-up calling
the init from both vgic_init and vgic_its_create.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/Makefile         |  1 +
 arch/arm64/kvm/Makefile       |  1 +
 include/kvm/arm_vgic.h        | 19 ++++++++++
 virt/kvm/arm/vgic/vgic-init.c |  9 +++++
 virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
 virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.h      |  2 ++
 7 files changed, 123 insertions(+)
 create mode 100644 virt/kvm/arm/vgic/vgic-v4.c

diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index d9beee652d36..0a1dd2cdb928 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
 obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
 obj-y += $(KVM)/arm/vgic/vgic-v2.o
 obj-y += $(KVM)/arm/vgic/vgic-v3.o
+obj-y += $(KVM)/arm/vgic/vgic-v4.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 5d9810086c25..c30fd388ef80 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index ba9fb450aa1b..7eeb6c2a2f9c 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -26,6 +26,8 @@
 #include <linux/list.h>
 #include <linux/jump_label.h>
 
+#include <linux/irqchip/arm-gic-v4.h>
+
 #define VGIC_V3_MAX_CPUS	255
 #define VGIC_V2_MAX_CPUS	8
 #define VGIC_NR_IRQS_LEGACY     256
@@ -236,6 +238,15 @@ struct vgic_dist {
 
 	/* used by vgic-debug */
 	struct vgic_state_iter *iter;
+
+	/*
+	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
+	 * array, the property table pointer as well as allocation
+	 * data. This essentially ties the Linux IRQ core and ITS
+	 * together, and avoids leaking KVM's data structures anywhere
+	 * else.
+	 */
+	struct its_vm		its_vm;
 };
 
 struct vgic_v2_cpu_if {
@@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
 	u32		vgic_ap0r[4];
 	u32		vgic_ap1r[4];
 	u64		vgic_lr[VGIC_V3_MAX_LRS];
+
+	/*
+	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
+	 * pending table pointer, the its_vm pointer and a few other
+	 * HW specific things. As for the its_vm structure, this is
+	 * linking the Linux IRQ subsystem and the ITS together.
+	 */
+	struct its_vpe	its_vpe;
 };
 
 struct vgic_cpu {
diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 5801261f3add..40be908da238 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
 	if (ret)
 		goto out;
 
+	if (vgic_supports_direct_msis(kvm)) {
+		ret = vgic_v4_init(kvm);
+		if (ret)
+			goto out;
+	}
+
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_enable(vcpu);
 
@@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
 
 	kfree(dist->spis);
 	dist->nr_spis = 0;
+
+	if (vgic_supports_direct_msis(kvm))
+		vgic_v4_teardown(kvm);
 }
 
 void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 8ee03f1e89fc..89768d2b6a91 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
 	if (!its)
 		return -ENOMEM;
 
+	if (vgic_initialized(dev->kvm)) {
+		int ret = vgic_v4_init(dev->kvm);
+		if (ret) {
+			kfree(its);
+			return ret;
+		}
+	}
+	
 	mutex_init(&its->its_lock);
 	mutex_init(&its->cmd_lock);
 
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
new file mode 100644
index 000000000000..c794f0cef09e
--- /dev/null
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -0,0 +1,83 @@
+/*
+ * Copyright (C) 2017 ARM Ltd.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/kvm_host.h>
+
+#include "vgic.h"
+
+/**
+ * vgic_v4_init - Initialize the GICv4 data structures
+ * @kvm:	Pointer to the VM being initialized
+ *
+ * We may be called each time a vITS is created, or when the
+ * vgic is initialized. This relies on kvm->lock to be
+ * held. In both cases, the number of vcpus should now be
+ * fixed.
+ */
+int vgic_v4_init(struct kvm *kvm)
+{
+	struct vgic_dist *dist = &kvm->arch.vgic;
+	struct kvm_vcpu *vcpu;
+	int i, nr_vcpus, ret;
+
+	if (dist->its_vm.vpes)
+		return 0;
+
+	nr_vcpus = atomic_read(&kvm->online_vcpus);
+
+	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
+				    GFP_KERNEL);
+	if (!dist->its_vm.vpes)
+		return -ENOMEM;
+
+	dist->its_vm.nr_vpes = nr_vcpus;
+
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
+
+	ret = its_alloc_vcpu_irqs(&dist->its_vm);
+	if (ret < 0) {
+		kvm_err("VPE IRQ allocation failure\n");
+		kfree(dist->its_vm.vpes);
+		dist->its_vm.nr_vpes = 0;
+		dist->its_vm.vpes = NULL;
+		return ret;
+	}
+
+	return ret;
+}
+
+/**
+ * vgic_v4_teardown - Free the GICv4 data structures
+ * @kvm:	Pointer to the VM being destroyed
+ *
+ * Relies on kvm->lock to be held.
+ */
+void vgic_v4_teardown(struct kvm *kvm)
+{
+	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
+
+	if (!its_vm->vpes)
+		return;
+
+	its_free_vcpu_irqs(its_vm);
+	kfree(its_vm->vpes);
+	its_vm->nr_vpes = 0;
+	its_vm->vpes = NULL;
+}
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index e67ccb6a6250..c4105f613f57 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
 struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 
 bool vgic_supports_direct_msis(struct kvm *kvm);
+int vgic_v4_init(struct kvm *kvm);
+void vgic_v4_teardown(struct kvm *kvm);
 
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Let's use the irq bypass mechanism introduced for platform device
interrupts to intercept the virtual PCIe endpoint configuration
and establish our LPI->VLPI mapping.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h      |   8 ++++
 virt/kvm/arm/arm.c          |   6 ++-
 virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+), 2 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 7eeb6c2a2f9c..2f750c770bf2 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
 
 int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
 
+struct kvm_kernel_irq_routing_entry;
+
+int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry);
+
+int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
+				 struct kvm_kernel_irq_routing_entry *irq_entry);
+
 #endif /* __KVM_ARM_VGIC_H */
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 5d5218ecd547..8388c1cc23f6 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(cons, struct kvm_kernel_irqfd, consumer);
 
-	return 0;
+	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
+					  &irqfd->irq_entry);
 }
 void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
 				      struct irq_bypass_producer *prod)
@@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(cons, struct kvm_kernel_irqfd, consumer);
 
-	return;
+	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
+				     &irqfd->irq_entry);
 }
 
 void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index c794f0cef09e..01a2889b7b7c 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -18,6 +18,7 @@
 #include <linux/interrupt.h>
 #include <linux/irqdomain.h>
 #include <linux/kvm_host.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include "vgic.h"
 
@@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->nr_vpes = 0;
 	its_vm->vpes = NULL;
 }
+
+static struct vgic_its *vgic_get_its(struct kvm *kvm,
+				     struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct kvm_msi msi  = (struct kvm_msi) {
+		.address_lo	= irq_entry->msi.address_lo,
+		.address_hi	= irq_entry->msi.address_hi,
+		.data		= irq_entry->msi.data,
+		.flags		= irq_entry->msi.flags,
+		.devid		= irq_entry->msi.devid,
+	};
+
+	/*
+	 * Get a reference on the LPI. If NULL, this is not a valid
+	 * translation for any of our vITSs.
+	 */
+	return vgic_msi_to_its(kvm, &msi);
+}
+
+int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct vgic_its *its;
+	struct vgic_irq *irq;
+	struct its_vlpi_map map;
+	int ret;
+
+	if (!vgic_supports_direct_msis(kvm))
+		return 0;
+
+	/*
+	 * Get the ITS, and escape early on error (not a valid
+	 * doorbell for any of our vITSs).
+	 */
+	its = vgic_get_its(kvm, irq_entry);
+	if (IS_ERR(its))
+		return 0;
+
+	mutex_lock(&its->its_lock);
+
+	/* Perform then actual DevID/EventID -> LPI translation. */
+	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
+				   irq_entry->msi.data, &irq);
+	if (ret)
+		goto out;
+
+	/*
+	 * Emit the mapping request. If it fails, the ITS probably
+	 * isn't v4 compatible, so let's silently bail out. Holding
+	 * the ITS lock should ensure that nothing can modify the
+	 * target vcpu.
+	 */
+	map = (struct its_vlpi_map) {
+		.vm		= &kvm->arch.vgic.its_vm,
+		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
+		.vintid		= irq->intid,
+		.properties	= ((irq->priority & 0xfc) |
+				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
+				   LPI_PROP_GROUP1),
+		.db_enabled	= true,
+	};
+
+	ret = its_map_vlpi(virq, &map);
+	if (ret)
+		goto out;
+
+	irq->hw		= true;
+	irq->host_irq	= virq;
+
+out:
+	mutex_unlock(&its->its_lock);
+	return ret;
+}
+
+int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
+				 struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct vgic_its *its;
+	struct vgic_irq *irq;
+	int ret;
+
+	if (!vgic_supports_direct_msis(kvm))
+		return 0;
+
+	/*
+	 * Get the ITS, and escape early on error (not a valid
+	 * doorbell for any of our vITSs).
+	 */
+	its = vgic_get_its(kvm, irq_entry);
+	if (IS_ERR(its))
+		return 0;
+
+	mutex_lock(&its->its_lock);
+
+	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
+				   irq_entry->msi.data, &irq);
+	if (ret)
+		goto out;
+
+	WARN_ON(!(irq->hw && irq->host_irq == virq));
+	irq->hw = false;
+	ret = its_unmap_vlpi(virq);
+
+out:
+	mutex_unlock(&its->its_lock);
+	return ret;
+}
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

Let's use the irq bypass mechanism introduced for platform device
interrupts to intercept the virtual PCIe endpoint configuration
and establish our LPI->VLPI mapping.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h      |   8 ++++
 virt/kvm/arm/arm.c          |   6 ++-
 virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+), 2 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 7eeb6c2a2f9c..2f750c770bf2 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
 
 int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
 
+struct kvm_kernel_irq_routing_entry;
+
+int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry);
+
+int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
+				 struct kvm_kernel_irq_routing_entry *irq_entry);
+
 #endif /* __KVM_ARM_VGIC_H */
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 5d5218ecd547..8388c1cc23f6 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(cons, struct kvm_kernel_irqfd, consumer);
 
-	return 0;
+	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
+					  &irqfd->irq_entry);
 }
 void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
 				      struct irq_bypass_producer *prod)
@@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(cons, struct kvm_kernel_irqfd, consumer);
 
-	return;
+	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
+				     &irqfd->irq_entry);
 }
 
 void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index c794f0cef09e..01a2889b7b7c 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -18,6 +18,7 @@
 #include <linux/interrupt.h>
 #include <linux/irqdomain.h>
 #include <linux/kvm_host.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include "vgic.h"
 
@@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->nr_vpes = 0;
 	its_vm->vpes = NULL;
 }
+
+static struct vgic_its *vgic_get_its(struct kvm *kvm,
+				     struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct kvm_msi msi  = (struct kvm_msi) {
+		.address_lo	= irq_entry->msi.address_lo,
+		.address_hi	= irq_entry->msi.address_hi,
+		.data		= irq_entry->msi.data,
+		.flags		= irq_entry->msi.flags,
+		.devid		= irq_entry->msi.devid,
+	};
+
+	/*
+	 * Get a reference on the LPI. If NULL, this is not a valid
+	 * translation for any of our vITSs.
+	 */
+	return vgic_msi_to_its(kvm, &msi);
+}
+
+int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct vgic_its *its;
+	struct vgic_irq *irq;
+	struct its_vlpi_map map;
+	int ret;
+
+	if (!vgic_supports_direct_msis(kvm))
+		return 0;
+
+	/*
+	 * Get the ITS, and escape early on error (not a valid
+	 * doorbell for any of our vITSs).
+	 */
+	its = vgic_get_its(kvm, irq_entry);
+	if (IS_ERR(its))
+		return 0;
+
+	mutex_lock(&its->its_lock);
+
+	/* Perform then actual DevID/EventID -> LPI translation. */
+	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
+				   irq_entry->msi.data, &irq);
+	if (ret)
+		goto out;
+
+	/*
+	 * Emit the mapping request. If it fails, the ITS probably
+	 * isn't v4 compatible, so let's silently bail out. Holding
+	 * the ITS lock should ensure that nothing can modify the
+	 * target vcpu.
+	 */
+	map = (struct its_vlpi_map) {
+		.vm		= &kvm->arch.vgic.its_vm,
+		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
+		.vintid		= irq->intid,
+		.properties	= ((irq->priority & 0xfc) |
+				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
+				   LPI_PROP_GROUP1),
+		.db_enabled	= true,
+	};
+
+	ret = its_map_vlpi(virq, &map);
+	if (ret)
+		goto out;
+
+	irq->hw		= true;
+	irq->host_irq	= virq;
+
+out:
+	mutex_unlock(&its->its_lock);
+	return ret;
+}
+
+int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
+				 struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct vgic_its *its;
+	struct vgic_irq *irq;
+	int ret;
+
+	if (!vgic_supports_direct_msis(kvm))
+		return 0;
+
+	/*
+	 * Get the ITS, and escape early on error (not a valid
+	 * doorbell for any of our vITSs).
+	 */
+	its = vgic_get_its(kvm, irq_entry);
+	if (IS_ERR(its))
+		return 0;
+
+	mutex_lock(&its->its_lock);
+
+	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
+				   irq_entry->msi.data, &irq);
+	if (ret)
+		goto out;
+
+	WARN_ON(!(irq->hw && irq->host_irq == virq));
+	irq->hw = false;
+	ret = its_unmap_vlpi(virq);
+
+out:
+	mutex_unlock(&its->its_lock);
+	return ret;
+}
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

Let's use the irq bypass mechanism introduced for platform device
interrupts to intercept the virtual PCIe endpoint configuration
and establish our LPI->VLPI mapping.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h      |   8 ++++
 virt/kvm/arm/arm.c          |   6 ++-
 virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+), 2 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 7eeb6c2a2f9c..2f750c770bf2 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
 
 int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
 
+struct kvm_kernel_irq_routing_entry;
+
+int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry);
+
+int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
+				 struct kvm_kernel_irq_routing_entry *irq_entry);
+
 #endif /* __KVM_ARM_VGIC_H */
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 5d5218ecd547..8388c1cc23f6 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(cons, struct kvm_kernel_irqfd, consumer);
 
-	return 0;
+	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
+					  &irqfd->irq_entry);
 }
 void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
 				      struct irq_bypass_producer *prod)
@@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(cons, struct kvm_kernel_irqfd, consumer);
 
-	return;
+	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
+				     &irqfd->irq_entry);
 }
 
 void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index c794f0cef09e..01a2889b7b7c 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -18,6 +18,7 @@
 #include <linux/interrupt.h>
 #include <linux/irqdomain.h>
 #include <linux/kvm_host.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include "vgic.h"
 
@@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->nr_vpes = 0;
 	its_vm->vpes = NULL;
 }
+
+static struct vgic_its *vgic_get_its(struct kvm *kvm,
+				     struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct kvm_msi msi  = (struct kvm_msi) {
+		.address_lo	= irq_entry->msi.address_lo,
+		.address_hi	= irq_entry->msi.address_hi,
+		.data		= irq_entry->msi.data,
+		.flags		= irq_entry->msi.flags,
+		.devid		= irq_entry->msi.devid,
+	};
+
+	/*
+	 * Get a reference on the LPI. If NULL, this is not a valid
+	 * translation for any of our vITSs.
+	 */
+	return vgic_msi_to_its(kvm, &msi);
+}
+
+int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct vgic_its *its;
+	struct vgic_irq *irq;
+	struct its_vlpi_map map;
+	int ret;
+
+	if (!vgic_supports_direct_msis(kvm))
+		return 0;
+
+	/*
+	 * Get the ITS, and escape early on error (not a valid
+	 * doorbell for any of our vITSs).
+	 */
+	its = vgic_get_its(kvm, irq_entry);
+	if (IS_ERR(its))
+		return 0;
+
+	mutex_lock(&its->its_lock);
+
+	/* Perform then actual DevID/EventID -> LPI translation. */
+	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
+				   irq_entry->msi.data, &irq);
+	if (ret)
+		goto out;
+
+	/*
+	 * Emit the mapping request. If it fails, the ITS probably
+	 * isn't v4 compatible, so let's silently bail out. Holding
+	 * the ITS lock should ensure that nothing can modify the
+	 * target vcpu.
+	 */
+	map = (struct its_vlpi_map) {
+		.vm		= &kvm->arch.vgic.its_vm,
+		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
+		.vintid		= irq->intid,
+		.properties	= ((irq->priority & 0xfc) |
+				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
+				   LPI_PROP_GROUP1),
+		.db_enabled	= true,
+	};
+
+	ret = its_map_vlpi(virq, &map);
+	if (ret)
+		goto out;
+
+	irq->hw		= true;
+	irq->host_irq	= virq;
+
+out:
+	mutex_unlock(&its->its_lock);
+	return ret;
+}
+
+int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
+				 struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct vgic_its *its;
+	struct vgic_irq *irq;
+	int ret;
+
+	if (!vgic_supports_direct_msis(kvm))
+		return 0;
+
+	/*
+	 * Get the ITS, and escape early on error (not a valid
+	 * doorbell for any of our vITSs).
+	 */
+	its = vgic_get_its(kvm, irq_entry);
+	if (IS_ERR(its))
+		return 0;
+
+	mutex_lock(&its->its_lock);
+
+	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
+				   irq_entry->msi.data, &irq);
+	if (ret)
+		goto out;
+
+	WARN_ON(!(irq->hw && irq->host_irq == virq));
+	irq->hw = false;
+	ret = its_unmap_vlpi(virq);
+
+out:
+	mutex_unlock(&its->its_lock);
+	return ret;
+}
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

If the guest issues an INT command targetting a VLPI, let's
call into the irq_set_irqchip_state() helper to make it pending
on the physical side.

This works just as well if userspace decides to inject an interrupt
using the normal userspace API...

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 89768d2b6a91..b2a678d131d0 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
 	if (err)
 		return err;
 
+	if (irq->hw)
+		return irq_set_irqchip_state(irq->host_irq,
+					     IRQCHIP_STATE_PENDING, true);
+
 	spin_lock(&irq->irq_lock);
 	irq->pending_latch = true;
 	vgic_queue_irq_unlock(kvm, irq);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

If the guest issues an INT command targetting a VLPI, let's
call into the irq_set_irqchip_state() helper to make it pending
on the physical side.

This works just as well if userspace decides to inject an interrupt
using the normal userspace API...

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 89768d2b6a91..b2a678d131d0 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
 	if (err)
 		return err;
 
+	if (irq->hw)
+		return irq_set_irqchip_state(irq->host_irq,
+					     IRQCHIP_STATE_PENDING, true);
+
 	spin_lock(&irq->irq_lock);
 	irq->pending_latch = true;
 	vgic_queue_irq_unlock(kvm, irq);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

If the guest issues an INT command targetting a VLPI, let's
call into the irq_set_irqchip_state() helper to make it pending
on the physical side.

This works just as well if userspace decides to inject an interrupt
using the normal userspace API...

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 89768d2b6a91..b2a678d131d0 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
 	if (err)
 		return err;
 
+	if (irq->hw)
+		return irq_set_irqchip_state(irq->host_irq,
+					     IRQCHIP_STATE_PENDING, true);
+
 	spin_lock(&irq->irq_lock);
 	irq->pending_latch = true;
 	vgic_queue_irq_unlock(kvm, irq);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

When freeing an LPI (on a DISCARD command, for example), we need
to unmap the VLPI down to the physical ITS level.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index b2a678d131d0..c9b1c0967426 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
 	list_del(&ite->ite_list);
 
 	/* This put matches the get in vgic_add_lpi. */
-	if (ite->irq)
+	if (ite->irq) {
+		if (ite->irq->hw)
+			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
+
 		vgic_put_irq(kvm, ite->irq);
+	}
 
 	kfree(ite);
 }
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

When freeing an LPI (on a DISCARD command, for example), we need
to unmap the VLPI down to the physical ITS level.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index b2a678d131d0..c9b1c0967426 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
 	list_del(&ite->ite_list);
 
 	/* This put matches the get in vgic_add_lpi. */
-	if (ite->irq)
+	if (ite->irq) {
+		if (ite->irq->hw)
+			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
+
 		vgic_put_irq(kvm, ite->irq);
+	}
 
 	kfree(ite);
 }
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

When freeing an LPI (on a DISCARD command, for example), we need
to unmap the VLPI down to the physical ITS level.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index b2a678d131d0..c9b1c0967426 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
 	list_del(&ite->ite_list);
 
 	/* This put matches the get in vgic_add_lpi. */
-	if (ite->irq)
+	if (ite->irq) {
+		if (ite->irq->hw)
+			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
+
 		vgic_put_irq(kvm, ite->irq);
+	}
 
 	kfree(ite);
 }
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

When the guest issues an affinity change, we need to tell the physical
ITS that we're now targetting a new vcpu.  This is done by extracting
the current mapping, updating the target, and reapplying the mapping.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index c9b1c0967426..42ffb9084bb7 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
 
 static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
 {
+	int ret = 0;
+
 	spin_lock(&irq->irq_lock);
 	irq->target_vcpu = vcpu;
 	spin_unlock(&irq->irq_lock);
 
-	return 0;
+	if (irq->hw) {
+		struct its_vlpi_map map;
+
+		ret = its_get_vlpi(irq->host_irq, &map);
+		if (ret)
+			return ret;
+
+		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
+
+		ret = its_map_vlpi(irq->host_irq, &map);
+	}
+
+	return ret;
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

When the guest issues an affinity change, we need to tell the physical
ITS that we're now targetting a new vcpu.  This is done by extracting
the current mapping, updating the target, and reapplying the mapping.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index c9b1c0967426..42ffb9084bb7 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
 
 static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
 {
+	int ret = 0;
+
 	spin_lock(&irq->irq_lock);
 	irq->target_vcpu = vcpu;
 	spin_unlock(&irq->irq_lock);
 
-	return 0;
+	if (irq->hw) {
+		struct its_vlpi_map map;
+
+		ret = its_get_vlpi(irq->host_irq, &map);
+		if (ret)
+			return ret;
+
+		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
+
+		ret = its_map_vlpi(irq->host_irq, &map);
+	}
+
+	return ret;
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

When the guest issues an affinity change, we need to tell the physical
ITS that we're now targetting a new vcpu.  This is done by extracting
the current mapping, updating the target, and reapplying the mapping.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index c9b1c0967426..42ffb9084bb7 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
 
 static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
 {
+	int ret = 0;
+
 	spin_lock(&irq->irq_lock);
 	irq->target_vcpu = vcpu;
 	spin_unlock(&irq->irq_lock);
 
-	return 0;
+	if (irq->hw) {
+		struct its_vlpi_map map;
+
+		ret = its_get_vlpi(irq->host_irq, &map);
+		if (ret)
+			return ret;
+
+		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
+
+		ret = its_map_vlpi(irq->host_irq, &map);
+	}
+
+	return ret;
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 14/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Handling CLEAR is pretty easy. Just ask the ITS driver to clear
the corresponding pending bit (which will turn into a CLEAR
command on the physical side).

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 42ffb9084bb7..5778b50911e8 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1070,6 +1070,10 @@ static int vgic_its_cmd_handle_clear(struct kvm *kvm, struct vgic_its *its,
 
 	ite->irq->pending_latch = false;
 
+	if (ite->irq->hw)
+		return irq_set_irqchip_state(ite->irq->host_irq,
+					     IRQCHIP_STATE_PENDING, false);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 14/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

Handling CLEAR is pretty easy. Just ask the ITS driver to clear
the corresponding pending bit (which will turn into a CLEAR
command on the physical side).

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 42ffb9084bb7..5778b50911e8 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1070,6 +1070,10 @@ static int vgic_its_cmd_handle_clear(struct kvm *kvm, struct vgic_its *its,
 
 	ite->irq->pending_latch = false;
 
+	if (ite->irq->hw)
+		return irq_set_irqchip_state(ite->irq->host_irq,
+					     IRQCHIP_STATE_PENDING, false);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 14/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

Handling CLEAR is pretty easy. Just ask the ITS driver to clear
the corresponding pending bit (which will turn into a CLEAR
command on the physical side).

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 42ffb9084bb7..5778b50911e8 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1070,6 +1070,10 @@ static int vgic_its_cmd_handle_clear(struct kvm *kvm, struct vgic_its *its,
 
 	ite->irq->pending_latch = false;
 
+	if (ite->irq->hw)
+		return irq_set_irqchip_state(ite->irq->host_irq,
+					     IRQCHIP_STATE_PENDING, false);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 15/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

The current implementation of MOVALL doesn't allow us to call
into the core ITS code as we hold a number of spinlocks.

Let's try a method used in other parts of the code, were we copy
the intids of the candicate interrupts, and then do whatever
we need to do with them outside of the critical section.

This allows us to move the interrupts one by one, at the expense
of a bit of CPU time. Who cares? MOVALL is such a stupid command
anyway...

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 5778b50911e8..0b7e648e7a0c 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1148,11 +1148,12 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
 static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 				      u64 *its_cmd)
 {
-	struct vgic_dist *dist = &kvm->arch.vgic;
 	u32 target1_addr = its_cmd_get_target_addr(its_cmd);
 	u32 target2_addr = its_cmd_mask_field(its_cmd, 3, 16, 32);
 	struct kvm_vcpu *vcpu1, *vcpu2;
 	struct vgic_irq *irq;
+	u32 *intids;
+	int irq_count, i;
 
 	if (target1_addr >= atomic_read(&kvm->online_vcpus) ||
 	    target2_addr >= atomic_read(&kvm->online_vcpus))
@@ -1164,19 +1165,19 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 	vcpu1 = kvm_get_vcpu(kvm, target1_addr);
 	vcpu2 = kvm_get_vcpu(kvm, target2_addr);
 
-	spin_lock(&dist->lpi_list_lock);
+	irq_count = vgic_copy_lpi_list(vcpu1, &intids);
+	if (irq_count < 0)
+		return irq_count;
 
-	list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
-		spin_lock(&irq->irq_lock);
+	for (i = 0; i < irq_count; i++) {
+		irq = vgic_get_irq(kvm, NULL, intids[i]);
 
-		if (irq->target_vcpu == vcpu1)
-			irq->target_vcpu = vcpu2;
+		update_affinity(irq, vcpu2);
 
-		spin_unlock(&irq->irq_lock);
+		vgic_put_irq(kvm, irq);
 	}
 
-	spin_unlock(&dist->lpi_list_lock);
-
+	kfree(intids);
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 15/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

The current implementation of MOVALL doesn't allow us to call
into the core ITS code as we hold a number of spinlocks.

Let's try a method used in other parts of the code, were we copy
the intids of the candicate interrupts, and then do whatever
we need to do with them outside of the critical section.

This allows us to move the interrupts one by one, at the expense
of a bit of CPU time. Who cares? MOVALL is such a stupid command
anyway...

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 5778b50911e8..0b7e648e7a0c 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1148,11 +1148,12 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
 static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 				      u64 *its_cmd)
 {
-	struct vgic_dist *dist = &kvm->arch.vgic;
 	u32 target1_addr = its_cmd_get_target_addr(its_cmd);
 	u32 target2_addr = its_cmd_mask_field(its_cmd, 3, 16, 32);
 	struct kvm_vcpu *vcpu1, *vcpu2;
 	struct vgic_irq *irq;
+	u32 *intids;
+	int irq_count, i;
 
 	if (target1_addr >= atomic_read(&kvm->online_vcpus) ||
 	    target2_addr >= atomic_read(&kvm->online_vcpus))
@@ -1164,19 +1165,19 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 	vcpu1 = kvm_get_vcpu(kvm, target1_addr);
 	vcpu2 = kvm_get_vcpu(kvm, target2_addr);
 
-	spin_lock(&dist->lpi_list_lock);
+	irq_count = vgic_copy_lpi_list(vcpu1, &intids);
+	if (irq_count < 0)
+		return irq_count;
 
-	list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
-		spin_lock(&irq->irq_lock);
+	for (i = 0; i < irq_count; i++) {
+		irq = vgic_get_irq(kvm, NULL, intids[i]);
 
-		if (irq->target_vcpu == vcpu1)
-			irq->target_vcpu = vcpu2;
+		update_affinity(irq, vcpu2);
 
-		spin_unlock(&irq->irq_lock);
+		vgic_put_irq(kvm, irq);
 	}
 
-	spin_unlock(&dist->lpi_list_lock);
-
+	kfree(intids);
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 15/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

The current implementation of MOVALL doesn't allow us to call
into the core ITS code as we hold a number of spinlocks.

Let's try a method used in other parts of the code, were we copy
the intids of the candicate interrupts, and then do whatever
we need to do with them outside of the critical section.

This allows us to move the interrupts one by one, at the expense
of a bit of CPU time. Who cares? MOVALL is such a stupid command
anyway...

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 5778b50911e8..0b7e648e7a0c 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1148,11 +1148,12 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
 static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 				      u64 *its_cmd)
 {
-	struct vgic_dist *dist = &kvm->arch.vgic;
 	u32 target1_addr = its_cmd_get_target_addr(its_cmd);
 	u32 target2_addr = its_cmd_mask_field(its_cmd, 3, 16, 32);
 	struct kvm_vcpu *vcpu1, *vcpu2;
 	struct vgic_irq *irq;
+	u32 *intids;
+	int irq_count, i;
 
 	if (target1_addr >= atomic_read(&kvm->online_vcpus) ||
 	    target2_addr >= atomic_read(&kvm->online_vcpus))
@@ -1164,19 +1165,19 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 	vcpu1 = kvm_get_vcpu(kvm, target1_addr);
 	vcpu2 = kvm_get_vcpu(kvm, target2_addr);
 
-	spin_lock(&dist->lpi_list_lock);
+	irq_count = vgic_copy_lpi_list(vcpu1, &intids);
+	if (irq_count < 0)
+		return irq_count;
 
-	list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
-		spin_lock(&irq->irq_lock);
+	for (i = 0; i < irq_count; i++) {
+		irq = vgic_get_irq(kvm, NULL, intids[i]);
 
-		if (irq->target_vcpu == vcpu1)
-			irq->target_vcpu = vcpu2;
+		update_affinity(irq, vcpu2);
 
-		spin_unlock(&irq->irq_lock);
+		vgic_put_irq(kvm, irq);
 	}
 
-	spin_unlock(&dist->lpi_list_lock);
-
+	kfree(intids);
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Upon updating a property, we propagate it all the way to the physical
ITS, and ask for an INV command to be executed there.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 0b7e648e7a0c..2e77c7c83942 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 		spin_unlock(&irq->irq_lock);
 	}
 
+	if (irq->hw)
+		return its_prop_update_vlpi(irq->host_irq, prop, true);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

Upon updating a property, we propagate it all the way to the physical
ITS, and ask for an INV command to be executed there.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 0b7e648e7a0c..2e77c7c83942 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 		spin_unlock(&irq->irq_lock);
 	}
 
+	if (irq->hw)
+		return its_prop_update_vlpi(irq->host_irq, prop, true);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

Upon updating a property, we propagate it all the way to the physical
ITS, and ask for an INV command to be executed there.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 0b7e648e7a0c..2e77c7c83942 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 		spin_unlock(&irq->irq_lock);
 	}
 
+	if (irq->hw)
+		return its_prop_update_vlpi(irq->host_irq, prop, true);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Since when updating the properties one LPI at a time, there is no
need to perform an INV each time we read one. Instead, we rely
on the final VINVALL that gets sent to the ITS to do the work.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 2e77c7c83942..eb72eb027060 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -38,7 +38,7 @@ static int vgic_its_save_tables_v0(struct vgic_its *its);
 static int vgic_its_restore_tables_v0(struct vgic_its *its);
 static int vgic_its_commit_v0(struct vgic_its *its);
 static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
-			     struct kvm_vcpu *filter_vcpu);
+			     struct kvm_vcpu *filter_vcpu, bool needs_inv);
 
 /*
  * Creates a new (reference to a) struct vgic_irq for a given LPI.
@@ -106,7 +106,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
 	 * However we only have those structs for mapped IRQs, so we read in
 	 * the respective config data from memory here upon mapping the LPI.
 	 */
-	ret = update_lpi_config(kvm, irq, NULL);
+	ret = update_lpi_config(kvm, irq, NULL, false);
 	if (ret)
 		return ERR_PTR(ret);
 
@@ -273,7 +273,7 @@ static struct its_collection *find_collection(struct vgic_its *its, int coll_id)
  * VCPU. Unconditionally applies if filter_vcpu is NULL.
  */
 static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
-			     struct kvm_vcpu *filter_vcpu)
+			     struct kvm_vcpu *filter_vcpu, bool needs_inv)
 {
 	u64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);
 	u8 prop;
@@ -297,7 +297,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 	}
 
 	if (irq->hw)
-		return its_prop_update_vlpi(irq->host_irq, prop, true);
+		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
 
 	return 0;
 }
@@ -1096,7 +1096,7 @@ static int vgic_its_cmd_handle_inv(struct kvm *kvm, struct vgic_its *its,
 	if (!ite)
 		return E_ITS_INV_UNMAPPED_INTERRUPT;
 
-	return update_lpi_config(kvm, ite->irq, NULL);
+	return update_lpi_config(kvm, ite->irq, NULL, true);
 }
 
 /*
@@ -1131,12 +1131,15 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
 		irq = vgic_get_irq(kvm, NULL, intids[i]);
 		if (!irq)
 			continue;
-		update_lpi_config(kvm, irq, vcpu);
+		update_lpi_config(kvm, irq, vcpu, false);
 		vgic_put_irq(kvm, irq);
 	}
 
 	kfree(intids);
 
+	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.its_vm)
+		its_invall_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Since when updating the properties one LPI at a time, there is no
need to perform an INV each time we read one. Instead, we rely
on the final VINVALL that gets sent to the ITS to do the work.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 2e77c7c83942..eb72eb027060 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -38,7 +38,7 @@ static int vgic_its_save_tables_v0(struct vgic_its *its);
 static int vgic_its_restore_tables_v0(struct vgic_its *its);
 static int vgic_its_commit_v0(struct vgic_its *its);
 static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
-			     struct kvm_vcpu *filter_vcpu);
+			     struct kvm_vcpu *filter_vcpu, bool needs_inv);
 
 /*
  * Creates a new (reference to a) struct vgic_irq for a given LPI.
@@ -106,7 +106,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
 	 * However we only have those structs for mapped IRQs, so we read in
 	 * the respective config data from memory here upon mapping the LPI.
 	 */
-	ret = update_lpi_config(kvm, irq, NULL);
+	ret = update_lpi_config(kvm, irq, NULL, false);
 	if (ret)
 		return ERR_PTR(ret);
 
@@ -273,7 +273,7 @@ static struct its_collection *find_collection(struct vgic_its *its, int coll_id)
  * VCPU. Unconditionally applies if filter_vcpu is NULL.
  */
 static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
-			     struct kvm_vcpu *filter_vcpu)
+			     struct kvm_vcpu *filter_vcpu, bool needs_inv)
 {
 	u64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);
 	u8 prop;
@@ -297,7 +297,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 	}
 
 	if (irq->hw)
-		return its_prop_update_vlpi(irq->host_irq, prop, true);
+		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
 
 	return 0;
 }
@@ -1096,7 +1096,7 @@ static int vgic_its_cmd_handle_inv(struct kvm *kvm, struct vgic_its *its,
 	if (!ite)
 		return E_ITS_INV_UNMAPPED_INTERRUPT;
 
-	return update_lpi_config(kvm, ite->irq, NULL);
+	return update_lpi_config(kvm, ite->irq, NULL, true);
 }
 
 /*
@@ -1131,12 +1131,15 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
 		irq = vgic_get_irq(kvm, NULL, intids[i]);
 		if (!irq)
 			continue;
-		update_lpi_config(kvm, irq, vcpu);
+		update_lpi_config(kvm, irq, vcpu, false);
 		vgic_put_irq(kvm, irq);
 	}
 
 	kfree(intids);
 
+	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.its_vm)
+		its_invall_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

Since when updating the properties one LPI at a time, there is no
need to perform an INV each time we read one. Instead, we rely
on the final VINVALL that gets sent to the ITS to do the work.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-its.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 2e77c7c83942..eb72eb027060 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -38,7 +38,7 @@ static int vgic_its_save_tables_v0(struct vgic_its *its);
 static int vgic_its_restore_tables_v0(struct vgic_its *its);
 static int vgic_its_commit_v0(struct vgic_its *its);
 static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
-			     struct kvm_vcpu *filter_vcpu);
+			     struct kvm_vcpu *filter_vcpu, bool needs_inv);
 
 /*
  * Creates a new (reference to a) struct vgic_irq for a given LPI.
@@ -106,7 +106,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
 	 * However we only have those structs for mapped IRQs, so we read in
 	 * the respective config data from memory here upon mapping the LPI.
 	 */
-	ret = update_lpi_config(kvm, irq, NULL);
+	ret = update_lpi_config(kvm, irq, NULL, false);
 	if (ret)
 		return ERR_PTR(ret);
 
@@ -273,7 +273,7 @@ static struct its_collection *find_collection(struct vgic_its *its, int coll_id)
  * VCPU. Unconditionally applies if filter_vcpu is NULL.
  */
 static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
-			     struct kvm_vcpu *filter_vcpu)
+			     struct kvm_vcpu *filter_vcpu, bool needs_inv)
 {
 	u64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);
 	u8 prop;
@@ -297,7 +297,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 	}
 
 	if (irq->hw)
-		return its_prop_update_vlpi(irq->host_irq, prop, true);
+		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
 
 	return 0;
 }
@@ -1096,7 +1096,7 @@ static int vgic_its_cmd_handle_inv(struct kvm *kvm, struct vgic_its *its,
 	if (!ite)
 		return E_ITS_INV_UNMAPPED_INTERRUPT;
 
-	return update_lpi_config(kvm, ite->irq, NULL);
+	return update_lpi_config(kvm, ite->irq, NULL, true);
 }
 
 /*
@@ -1131,12 +1131,15 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
 		irq = vgic_get_irq(kvm, NULL, intids[i]);
 		if (!irq)
 			continue;
-		update_lpi_config(kvm, irq, vcpu);
+		update_lpi_config(kvm, irq, vcpu, false);
 		vgic_put_irq(kvm, irq);
 	}
 
 	kfree(intids);
 
+	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.its_vm)
+		its_invall_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe);
+
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

When a vPE exits, the pending_last flag is set when there are
pending VLPIs stored in the pending table. Similarily, we set
this flag when a doorbell interrupt fires, as it indicates the
same condition.

Let's update kvm_vgic_vcpu_pending_irq() to account for that
flag as well, making a vcpu runnable when set.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 9d557efd1f7d..0691a2250949 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -769,6 +769,9 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
 	if (!vcpu->kvm->arch.vgic.enabled)
 		return false;
 
+	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last)
+		return true;
+
 	spin_lock(&vgic_cpu->ap_list_lock);
 
 	list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

When a vPE exits, the pending_last flag is set when there are
pending VLPIs stored in the pending table. Similarily, we set
this flag when a doorbell interrupt fires, as it indicates the
same condition.

Let's update kvm_vgic_vcpu_pending_irq() to account for that
flag as well, making a vcpu runnable when set.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 9d557efd1f7d..0691a2250949 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -769,6 +769,9 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
 	if (!vcpu->kvm->arch.vgic.enabled)
 		return false;
 
+	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last)
+		return true;
+
 	spin_lock(&vgic_cpu->ap_list_lock);
 
 	list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

When a vPE exits, the pending_last flag is set when there are
pending VLPIs stored in the pending table. Similarily, we set
this flag when a doorbell interrupt fires, as it indicates the
same condition.

Let's update kvm_vgic_vcpu_pending_irq() to account for that
flag as well, making a vcpu runnable when set.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 9d557efd1f7d..0691a2250949 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -769,6 +769,9 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
 	if (!vcpu->kvm->arch.vgic.enabled)
 		return false;
 
+	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last)
+		return true;
+
 	spin_lock(&vgic_cpu->ap_list_lock);
 
 	list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

When a vPE is not running, a VLPI being made pending results in a
doorbell interrupt being delivered. Let's handle this interrupt
and update the pending_last flag that indicates that VLPIs are
pending. The corresponding vcpu is also kicked into action.

Special care is taken to prevent the doorbell from being enabled
at request time (this is controlled separately), and to make
the disabling on the interrupt non-lazy.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index 01a2889b7b7c..ea898af1a7a9 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -16,12 +16,24 @@
  */
 
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/kvm_host.h>
 #include <linux/irqchip/arm-gic-v3.h>
 
 #include "vgic.h"
 
+static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
+{
+	struct kvm_vcpu *vcpu = info;
+
+	vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last = true;
+	kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
+	kvm_vcpu_kick(vcpu);
+
+	return IRQ_HANDLED;
+}
+
 /**
  * vgic_v4_init - Initialize the GICv4 data structures
  * @kvm:	Pointer to the VM being initialized
@@ -61,6 +73,33 @@ int vgic_v4_init(struct kvm *kvm)
 		return ret;
 	}
 
+	kvm_for_each_vcpu(i, vcpu, kvm) {
+		int irq = dist->its_vm.vpes[i]->irq;
+
+		/*
+		 * Don't automatically enable the doorbell, as we're
+		 * flipping it back and forth when the vcpu gets
+		 * blocked. Also disable the lazy disabling, as the
+		 * doorbell could kick us out of the guest too
+		 * early...
+		 */
+		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		ret = request_irq(irq, vgic_v4_doorbell_handler,
+				  0, "vcpu", vcpu);
+		if (ret) {
+			kvm_err("failed to allocate vcpu IRQ%d\n", irq);
+			/*
+			 * Trick: adjust the number of vpes so we know
+			 * how many to nuke on teardown...
+			 */
+			dist->its_vm.nr_vpes = i;
+			break;
+		}
+	}
+
+	if (ret)
+		vgic_v4_teardown(kvm);
+
 	return ret;
 }
 
@@ -73,10 +112,19 @@ int vgic_v4_init(struct kvm *kvm)
 void vgic_v4_teardown(struct kvm *kvm)
 {
 	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
+	int i;
 
 	if (!its_vm->vpes)
 		return;
 
+	for (i = 0; i < its_vm->nr_vpes; i++) {
+		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
+		int irq = its_vm->vpes[i]->irq;
+
+		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		free_irq(irq, vcpu);
+	}
+
 	its_free_vcpu_irqs(its_vm);
 	kfree(its_vm->vpes);
 	its_vm->nr_vpes = 0;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

When a vPE is not running, a VLPI being made pending results in a
doorbell interrupt being delivered. Let's handle this interrupt
and update the pending_last flag that indicates that VLPIs are
pending. The corresponding vcpu is also kicked into action.

Special care is taken to prevent the doorbell from being enabled
at request time (this is controlled separately), and to make
the disabling on the interrupt non-lazy.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index 01a2889b7b7c..ea898af1a7a9 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -16,12 +16,24 @@
  */
 
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/kvm_host.h>
 #include <linux/irqchip/arm-gic-v3.h>
 
 #include "vgic.h"
 
+static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
+{
+	struct kvm_vcpu *vcpu = info;
+
+	vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last = true;
+	kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
+	kvm_vcpu_kick(vcpu);
+
+	return IRQ_HANDLED;
+}
+
 /**
  * vgic_v4_init - Initialize the GICv4 data structures
  * @kvm:	Pointer to the VM being initialized
@@ -61,6 +73,33 @@ int vgic_v4_init(struct kvm *kvm)
 		return ret;
 	}
 
+	kvm_for_each_vcpu(i, vcpu, kvm) {
+		int irq = dist->its_vm.vpes[i]->irq;
+
+		/*
+		 * Don't automatically enable the doorbell, as we're
+		 * flipping it back and forth when the vcpu gets
+		 * blocked. Also disable the lazy disabling, as the
+		 * doorbell could kick us out of the guest too
+		 * early...
+		 */
+		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		ret = request_irq(irq, vgic_v4_doorbell_handler,
+				  0, "vcpu", vcpu);
+		if (ret) {
+			kvm_err("failed to allocate vcpu IRQ%d\n", irq);
+			/*
+			 * Trick: adjust the number of vpes so we know
+			 * how many to nuke on teardown...
+			 */
+			dist->its_vm.nr_vpes = i;
+			break;
+		}
+	}
+
+	if (ret)
+		vgic_v4_teardown(kvm);
+
 	return ret;
 }
 
@@ -73,10 +112,19 @@ int vgic_v4_init(struct kvm *kvm)
 void vgic_v4_teardown(struct kvm *kvm)
 {
 	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
+	int i;
 
 	if (!its_vm->vpes)
 		return;
 
+	for (i = 0; i < its_vm->nr_vpes; i++) {
+		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
+		int irq = its_vm->vpes[i]->irq;
+
+		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		free_irq(irq, vcpu);
+	}
+
 	its_free_vcpu_irqs(its_vm);
 	kfree(its_vm->vpes);
 	its_vm->nr_vpes = 0;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

When a vPE is not running, a VLPI being made pending results in a
doorbell interrupt being delivered. Let's handle this interrupt
and update the pending_last flag that indicates that VLPIs are
pending. The corresponding vcpu is also kicked into action.

Special care is taken to prevent the doorbell from being enabled
at request time (this is controlled separately), and to make
the disabling on the interrupt non-lazy.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index 01a2889b7b7c..ea898af1a7a9 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -16,12 +16,24 @@
  */
 
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/kvm_host.h>
 #include <linux/irqchip/arm-gic-v3.h>
 
 #include "vgic.h"
 
+static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
+{
+	struct kvm_vcpu *vcpu = info;
+
+	vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last = true;
+	kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
+	kvm_vcpu_kick(vcpu);
+
+	return IRQ_HANDLED;
+}
+
 /**
  * vgic_v4_init - Initialize the GICv4 data structures
  * @kvm:	Pointer to the VM being initialized
@@ -61,6 +73,33 @@ int vgic_v4_init(struct kvm *kvm)
 		return ret;
 	}
 
+	kvm_for_each_vcpu(i, vcpu, kvm) {
+		int irq = dist->its_vm.vpes[i]->irq;
+
+		/*
+		 * Don't automatically enable the doorbell, as we're
+		 * flipping it back and forth when the vcpu gets
+		 * blocked. Also disable the lazy disabling, as the
+		 * doorbell could kick us out of the guest too
+		 * early...
+		 */
+		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		ret = request_irq(irq, vgic_v4_doorbell_handler,
+				  0, "vcpu", vcpu);
+		if (ret) {
+			kvm_err("failed to allocate vcpu IRQ%d\n", irq);
+			/*
+			 * Trick: adjust the number of vpes so we know
+			 * how many to nuke on teardown...
+			 */
+			dist->its_vm.nr_vpes = i;
+			break;
+		}
+	}
+
+	if (ret)
+		vgic_v4_teardown(kvm);
+
 	return ret;
 }
 
@@ -73,10 +112,19 @@ int vgic_v4_init(struct kvm *kvm)
 void vgic_v4_teardown(struct kvm *kvm)
 {
 	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
+	int i;
 
 	if (!its_vm->vpes)
 		return;
 
+	for (i = 0; i < its_vm->nr_vpes; i++) {
+		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
+		int irq = its_vm->vpes[i]->irq;
+
+		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		free_irq(irq, vcpu);
+	}
+
 	its_free_vcpu_irqs(its_vm);
 	kfree(its_vm->vpes);
 	its_vm->nr_vpes = 0;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

The doorbell interrupt is only useful if the vcpu is blocked on WFI.
In all other cases, recieving a doorbell interrupt is just a waste
of cycles.

So let's only enable the doorbell if a vcpu is getting blocked,
and disable it when it is unblocked. This is very similar to
what we're doing for the background timer.

Reviewed-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h      |  3 +++
 virt/kvm/arm/arm.c          |  2 ++
 virt/kvm/arm/vgic/vgic-v4.c | 18 ++++++++++++++++++
 3 files changed, 23 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 2f750c770bf2..8c896540a72c 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -381,4 +381,7 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
 int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
 				 struct kvm_kernel_irq_routing_entry *irq_entry);
 
+void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu);
+void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu);
+
 #endif /* __KVM_ARM_VGIC_H */
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 8388c1cc23f6..092a4ec03e57 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -316,11 +316,13 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
 {
 	kvm_timer_schedule(vcpu);
+	kvm_vgic_v4_enable_doorbell(vcpu);
 }
 
 void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
 {
 	kvm_timer_unschedule(vcpu);
+	kvm_vgic_v4_disable_doorbell(vcpu);
 }
 
 int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index ea898af1a7a9..1a2f2fcdfa67 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -237,3 +237,21 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
 	mutex_unlock(&its->its_lock);
 	return ret;
 }
+
+void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu)
+{
+	if (vgic_supports_direct_msis(vcpu->kvm)) {
+		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+		if (irq)
+			enable_irq(irq);
+	}
+}
+
+void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu)
+{
+	if (vgic_supports_direct_msis(vcpu->kvm)) {
+		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+		if (irq)
+			disable_irq(irq);
+	}
+}
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

The doorbell interrupt is only useful if the vcpu is blocked on WFI.
In all other cases, recieving a doorbell interrupt is just a waste
of cycles.

So let's only enable the doorbell if a vcpu is getting blocked,
and disable it when it is unblocked. This is very similar to
what we're doing for the background timer.

Reviewed-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h      |  3 +++
 virt/kvm/arm/arm.c          |  2 ++
 virt/kvm/arm/vgic/vgic-v4.c | 18 ++++++++++++++++++
 3 files changed, 23 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 2f750c770bf2..8c896540a72c 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -381,4 +381,7 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
 int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
 				 struct kvm_kernel_irq_routing_entry *irq_entry);
 
+void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu);
+void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu);
+
 #endif /* __KVM_ARM_VGIC_H */
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 8388c1cc23f6..092a4ec03e57 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -316,11 +316,13 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
 {
 	kvm_timer_schedule(vcpu);
+	kvm_vgic_v4_enable_doorbell(vcpu);
 }
 
 void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
 {
 	kvm_timer_unschedule(vcpu);
+	kvm_vgic_v4_disable_doorbell(vcpu);
 }
 
 int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index ea898af1a7a9..1a2f2fcdfa67 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -237,3 +237,21 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
 	mutex_unlock(&its->its_lock);
 	return ret;
 }
+
+void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu)
+{
+	if (vgic_supports_direct_msis(vcpu->kvm)) {
+		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+		if (irq)
+			enable_irq(irq);
+	}
+}
+
+void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu)
+{
+	if (vgic_supports_direct_msis(vcpu->kvm)) {
+		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+		if (irq)
+			disable_irq(irq);
+	}
+}
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

The doorbell interrupt is only useful if the vcpu is blocked on WFI.
In all other cases, recieving a doorbell interrupt is just a waste
of cycles.

So let's only enable the doorbell if a vcpu is getting blocked,
and disable it when it is unblocked. This is very similar to
what we're doing for the background timer.

Reviewed-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_vgic.h      |  3 +++
 virt/kvm/arm/arm.c          |  2 ++
 virt/kvm/arm/vgic/vgic-v4.c | 18 ++++++++++++++++++
 3 files changed, 23 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 2f750c770bf2..8c896540a72c 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -381,4 +381,7 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
 int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
 				 struct kvm_kernel_irq_routing_entry *irq_entry);
 
+void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu);
+void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu);
+
 #endif /* __KVM_ARM_VGIC_H */
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 8388c1cc23f6..092a4ec03e57 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -316,11 +316,13 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
 {
 	kvm_timer_schedule(vcpu);
+	kvm_vgic_v4_enable_doorbell(vcpu);
 }
 
 void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
 {
 	kvm_timer_unschedule(vcpu);
+	kvm_vgic_v4_disable_doorbell(vcpu);
 }
 
 int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index ea898af1a7a9..1a2f2fcdfa67 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -237,3 +237,21 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
 	mutex_unlock(&its->its_lock);
 	return ret;
 }
+
+void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu)
+{
+	if (vgic_supports_direct_msis(vcpu->kvm)) {
+		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+		if (irq)
+			enable_irq(irq);
+	}
+}
+
+void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu)
+{
+	if (vgic_supports_direct_msis(vcpu->kvm)) {
+		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+		if (irq)
+			disable_irq(irq);
+	}
+}
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

The redistributor needs to be told which vPE is about to be run,
and tells us whether there is any pending VLPI on exit.

Let's add the scheduling calls to the vgic flush/sync functions,
allowing the VLPIs to be delivered to the guest.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.c    |  4 ++++
 virt/kvm/arm/vgic/vgic.h    |  2 ++
 3 files changed, 45 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index 1a2f2fcdfa67..d7fe610bb1f5 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->vpes = NULL;
 }
 
+int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
+{
+	if (!vgic_supports_direct_msis(vcpu->kvm))
+		return 0;
+
+	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
+}
+
+int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
+{
+	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+	int err;
+
+	if (!vgic_supports_direct_msis(vcpu->kvm))
+		return 0;
+
+	/*
+	 * Before making the VPE resident, make sure the redistributor
+	 * corresponding to our current CPU expects us here. See the
+	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
+	 * turns into a VMOVP command at the ITS level.
+	 */
+	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
+	if (err)
+		return err;
+
+	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
+	if (err)
+		return err;
+
+	/*
+	 * Now that the VPE is resident, let's get rid of a potential
+	 * doorbell interrupt that would still be pending.
+	 */
+	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
+
+	return err;
+}
+
 static struct vgic_its *vgic_get_its(struct kvm *kvm,
 				     struct kvm_kernel_irq_routing_entry *irq_entry)
 {
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 0691a2250949..71ca0ff1b35a 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
 
+	WARN_ON(vgic_v4_sync_hwstate(vcpu));
+
 	/* An empty ap_list_head implies used_lrs == 0 */
 	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
 		return;
@@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 /* Flush our emulation state into the GIC hardware before entering the guest. */
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 {
+	WARN_ON(vgic_v4_flush_hwstate(vcpu));
+
 	/*
 	 * If there are no virtual interrupts active or pending for this
 	 * VCPU, then there is no work to do and we can bail out without
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index c4105f613f57..9cafb61b79af 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 bool vgic_supports_direct_msis(struct kvm *kvm);
 int vgic_v4_init(struct kvm *kvm);
 void vgic_v4_teardown(struct kvm *kvm);
+int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
+int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
 
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

The redistributor needs to be told which vPE is about to be run,
and tells us whether there is any pending VLPI on exit.

Let's add the scheduling calls to the vgic flush/sync functions,
allowing the VLPIs to be delivered to the guest.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.c    |  4 ++++
 virt/kvm/arm/vgic/vgic.h    |  2 ++
 3 files changed, 45 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index 1a2f2fcdfa67..d7fe610bb1f5 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->vpes = NULL;
 }
 
+int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
+{
+	if (!vgic_supports_direct_msis(vcpu->kvm))
+		return 0;
+
+	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
+}
+
+int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
+{
+	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+	int err;
+
+	if (!vgic_supports_direct_msis(vcpu->kvm))
+		return 0;
+
+	/*
+	 * Before making the VPE resident, make sure the redistributor
+	 * corresponding to our current CPU expects us here. See the
+	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
+	 * turns into a VMOVP command at the ITS level.
+	 */
+	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
+	if (err)
+		return err;
+
+	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
+	if (err)
+		return err;
+
+	/*
+	 * Now that the VPE is resident, let's get rid of a potential
+	 * doorbell interrupt that would still be pending.
+	 */
+	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
+
+	return err;
+}
+
 static struct vgic_its *vgic_get_its(struct kvm *kvm,
 				     struct kvm_kernel_irq_routing_entry *irq_entry)
 {
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 0691a2250949..71ca0ff1b35a 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
 
+	WARN_ON(vgic_v4_sync_hwstate(vcpu));
+
 	/* An empty ap_list_head implies used_lrs == 0 */
 	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
 		return;
@@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 /* Flush our emulation state into the GIC hardware before entering the guest. */
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 {
+	WARN_ON(vgic_v4_flush_hwstate(vcpu));
+
 	/*
 	 * If there are no virtual interrupts active or pending for this
 	 * VCPU, then there is no work to do and we can bail out without
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index c4105f613f57..9cafb61b79af 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 bool vgic_supports_direct_msis(struct kvm *kvm);
 int vgic_v4_init(struct kvm *kvm);
 void vgic_v4_teardown(struct kvm *kvm);
+int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
+int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
 
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

The redistributor needs to be told which vPE is about to be run,
and tells us whether there is any pending VLPI on exit.

Let's add the scheduling calls to the vgic flush/sync functions,
allowing the VLPIs to be delivered to the guest.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.c    |  4 ++++
 virt/kvm/arm/vgic/vgic.h    |  2 ++
 3 files changed, 45 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index 1a2f2fcdfa67..d7fe610bb1f5 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->vpes = NULL;
 }
 
+int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
+{
+	if (!vgic_supports_direct_msis(vcpu->kvm))
+		return 0;
+
+	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
+}
+
+int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
+{
+	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
+	int err;
+
+	if (!vgic_supports_direct_msis(vcpu->kvm))
+		return 0;
+
+	/*
+	 * Before making the VPE resident, make sure the redistributor
+	 * corresponding to our current CPU expects us here. See the
+	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
+	 * turns into a VMOVP command at the ITS level.
+	 */
+	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
+	if (err)
+		return err;
+
+	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
+	if (err)
+		return err;
+
+	/*
+	 * Now that the VPE is resident, let's get rid of a potential
+	 * doorbell interrupt that would still be pending.
+	 */
+	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
+
+	return err;
+}
+
 static struct vgic_its *vgic_get_its(struct kvm *kvm,
 				     struct kvm_kernel_irq_routing_entry *irq_entry)
 {
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 0691a2250949..71ca0ff1b35a 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
 
+	WARN_ON(vgic_v4_sync_hwstate(vcpu));
+
 	/* An empty ap_list_head implies used_lrs == 0 */
 	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
 		return;
@@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 /* Flush our emulation state into the GIC hardware before entering the guest. */
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 {
+	WARN_ON(vgic_v4_flush_hwstate(vcpu));
+
 	/*
 	 * If there are no virtual interrupts active or pending for this
 	 * VCPU, then there is no work to do and we can bail out without
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index c4105f613f57..9cafb61b79af 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 bool vgic_supports_direct_msis(struct kvm *kvm);
 int vgic_v4_init(struct kvm *kvm);
 void vgic_v4_teardown(struct kvm *kvm);
+int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
+int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
 
 #endif
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

In order for VLPIs to be delivered to the guest, we must make
sure that the cpuif is always enabled, irrespective of the
presence of virtual interrupt in the LRs.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/hyp/vgic-v3-sr.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index 91728faa13fd..f5c3d6d7019e 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -258,7 +258,8 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
 			cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
 		}
 	} else {
-		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
+		    cpu_if->its_vpe.its_vm)
 			write_gicreg(0, ICH_HCR_EL2);
 
 		cpu_if->vgic_elrsr = 0xffff;
@@ -337,9 +338,11 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
 		/*
 		 * If we need to trap system registers, we must write
 		 * ICH_HCR_EL2 anyway, even if no interrupts are being
-		 * injected,
+		 * injected. Same thing if GICv4 is used, as VLPI
+		 * delivery is gated by ICH_HCR_EL2.En.
 		 */
-		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
+		    cpu_if->its_vpe.its_vm)
 			write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

In order for VLPIs to be delivered to the guest, we must make
sure that the cpuif is always enabled, irrespective of the
presence of virtual interrupt in the LRs.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/hyp/vgic-v3-sr.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index 91728faa13fd..f5c3d6d7019e 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -258,7 +258,8 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
 			cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
 		}
 	} else {
-		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
+		    cpu_if->its_vpe.its_vm)
 			write_gicreg(0, ICH_HCR_EL2);
 
 		cpu_if->vgic_elrsr = 0xffff;
@@ -337,9 +338,11 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
 		/*
 		 * If we need to trap system registers, we must write
 		 * ICH_HCR_EL2 anyway, even if no interrupts are being
-		 * injected,
+		 * injected. Same thing if GICv4 is used, as VLPI
+		 * delivery is gated by ICH_HCR_EL2.En.
 		 */
-		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
+		    cpu_if->its_vpe.its_vm)
 			write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

In order for VLPIs to be delivered to the guest, we must make
sure that the cpuif is always enabled, irrespective of the
presence of virtual interrupt in the LRs.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/hyp/vgic-v3-sr.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index 91728faa13fd..f5c3d6d7019e 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -258,7 +258,8 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
 			cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
 		}
 	} else {
-		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
+		    cpu_if->its_vpe.its_vm)
 			write_gicreg(0, ICH_HCR_EL2);
 
 		cpu_if->vgic_elrsr = 0xffff;
@@ -337,9 +338,11 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
 		/*
 		 * If we need to trap system registers, we must write
 		 * ICH_HCR_EL2 anyway, even if no interrupts are being
-		 * injected,
+		 * injected. Same thing if GICv4 is used, as VLPI
+		 * delivery is gated by ICH_HCR_EL2.En.
 		 */
-		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
+		    cpu_if->its_vpe.its_vm)
 			write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

The GICv4 architecture doesn't make it easy for save/restore to
work, as it doesn't give any guarantee that the pending state
is written into the pending table.

So let's not take any chance, and let's return an error if
we encounter any LPI that has the HW bit set. In order for
userspace to distinguish this error from other failure modes,
use -EACCES as an error code.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/virtual/kvm/devices/arm-vgic-its.txt | 2 ++
 virt/kvm/arm/vgic/vgic-its.c                       | 9 +++++++++
 2 files changed, 11 insertions(+)

diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
index eb06beb75960..1eec7bcb2d52 100644
--- a/Documentation/virtual/kvm/devices/arm-vgic-its.txt
+++ b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
@@ -60,6 +60,8 @@ Groups:
     -EINVAL: Inconsistent restored data
     -EFAULT: Invalid guest ram access
     -EBUSY:  One or more VCPUS are running
+    -EACCES: The virtual ITS is backed by a physical GICv4 ITS, and the
+    	     state is not available
 
   KVM_DEV_ARM_VGIC_GRP_ITS_REGS
   Attributes:
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index eb72eb027060..2710834d0920 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1989,6 +1989,15 @@ static int vgic_its_save_itt(struct vgic_its *its, struct its_device *device)
 	list_for_each_entry(ite, &device->itt_head, ite_list) {
 		gpa_t gpa = base + ite->event_id * ite_esz;
 
+		/*
+		 * If an LPI carries the HW bit, this means that this
+		 * interrupt is controlled by GICv4, and we do not
+		 * have direct access to that state. Let's simply fail
+		 * the save operation...
+		 */
+		if (ite->irq->hw)
+			return -EACCES;
+
 		ret = vgic_its_save_ite(its, device, ite, gpa, ite_esz);
 		if (ret)
 			return ret;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

The GICv4 architecture doesn't make it easy for save/restore to
work, as it doesn't give any guarantee that the pending state
is written into the pending table.

So let's not take any chance, and let's return an error if
we encounter any LPI that has the HW bit set. In order for
userspace to distinguish this error from other failure modes,
use -EACCES as an error code.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/virtual/kvm/devices/arm-vgic-its.txt | 2 ++
 virt/kvm/arm/vgic/vgic-its.c                       | 9 +++++++++
 2 files changed, 11 insertions(+)

diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
index eb06beb75960..1eec7bcb2d52 100644
--- a/Documentation/virtual/kvm/devices/arm-vgic-its.txt
+++ b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
@@ -60,6 +60,8 @@ Groups:
     -EINVAL: Inconsistent restored data
     -EFAULT: Invalid guest ram access
     -EBUSY:  One or more VCPUS are running
+    -EACCES: The virtual ITS is backed by a physical GICv4 ITS, and the
+    	     state is not available
 
   KVM_DEV_ARM_VGIC_GRP_ITS_REGS
   Attributes:
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index eb72eb027060..2710834d0920 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1989,6 +1989,15 @@ static int vgic_its_save_itt(struct vgic_its *its, struct its_device *device)
 	list_for_each_entry(ite, &device->itt_head, ite_list) {
 		gpa_t gpa = base + ite->event_id * ite_esz;
 
+		/*
+		 * If an LPI carries the HW bit, this means that this
+		 * interrupt is controlled by GICv4, and we do not
+		 * have direct access to that state. Let's simply fail
+		 * the save operation...
+		 */
+		if (ite->irq->hw)
+			return -EACCES;
+
 		ret = vgic_its_save_ite(its, device, ite, gpa, ite_esz);
 		if (ret)
 			return ret;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

The GICv4 architecture doesn't make it easy for save/restore to
work, as it doesn't give any guarantee that the pending state
is written into the pending table.

So let's not take any chance, and let's return an error if
we encounter any LPI that has the HW bit set. In order for
userspace to distinguish this error from other failure modes,
use -EACCES as an error code.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/virtual/kvm/devices/arm-vgic-its.txt | 2 ++
 virt/kvm/arm/vgic/vgic-its.c                       | 9 +++++++++
 2 files changed, 11 insertions(+)

diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
index eb06beb75960..1eec7bcb2d52 100644
--- a/Documentation/virtual/kvm/devices/arm-vgic-its.txt
+++ b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
@@ -60,6 +60,8 @@ Groups:
     -EINVAL: Inconsistent restored data
     -EFAULT: Invalid guest ram access
     -EBUSY:  One or more VCPUS are running
+    -EACCES: The virtual ITS is backed by a physical GICv4 ITS, and the
+    	     state is not available
 
   KVM_DEV_ARM_VGIC_GRP_ITS_REGS
   Attributes:
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index eb72eb027060..2710834d0920 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1989,6 +1989,15 @@ static int vgic_its_save_itt(struct vgic_its *its, struct its_device *device)
 	list_for_each_entry(ite, &device->itt_head, ite_list) {
 		gpa_t gpa = base + ite->event_id * ite_esz;
 
+		/*
+		 * If an LPI carries the HW bit, this means that this
+		 * interrupt is controlled by GICv4, and we do not
+		 * have direct access to that state. Let's simply fail
+		 * the save operation...
+		 */
+		if (ite->irq->hw)
+			return -EACCES;
+
 		ret = vgic_its_save_ite(its, device, ite, gpa, ite_esz);
 		if (ret)
 			return ret;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

We so far allocate the doorbell interrupts without taking any
special measure regarding the affinity of these interrupts. We
simply move them around as required when the vcpu gets scheduled
on a different CPU.

But that's counting without userspace (and the evil irqbalance) that
can try and move the VPE interrupt around, causing the ITS code
to emit VMOVP commands and remap the doorbell to another redistributor.
Worse, this can happen while the vcpu is running, causing all kind
of trouble if the VPE is already resident, and we end-up in UNPRED
territory.

So let's take a definitive action and prevent userspace from messing
with us. This is just a matter of adding IRQ_NO_BALANCING to the
set of flags we already have, letting the kernel in sole control
of the affinity.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index d7fe610bb1f5..d10e18eabd3b 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -23,6 +23,8 @@
 
 #include "vgic.h"
 
+#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
+
 static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
 {
 	struct kvm_vcpu *vcpu = info;
@@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
 		 * doorbell could kick us out of the guest too
 		 * early...
 		 */
-		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		irq_set_status_flags(irq, DB_IRQ_FLAGS);
 		ret = request_irq(irq, vgic_v4_doorbell_handler,
 				  0, "vcpu", vcpu);
 		if (ret) {
@@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
 		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
 		int irq = its_vm->vpes[i]->irq;
 
-		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
 		free_irq(irq, vcpu);
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

We so far allocate the doorbell interrupts without taking any
special measure regarding the affinity of these interrupts. We
simply move them around as required when the vcpu gets scheduled
on a different CPU.

But that's counting without userspace (and the evil irqbalance) that
can try and move the VPE interrupt around, causing the ITS code
to emit VMOVP commands and remap the doorbell to another redistributor.
Worse, this can happen while the vcpu is running, causing all kind
of trouble if the VPE is already resident, and we end-up in UNPRED
territory.

So let's take a definitive action and prevent userspace from messing
with us. This is just a matter of adding IRQ_NO_BALANCING to the
set of flags we already have, letting the kernel in sole control
of the affinity.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index d7fe610bb1f5..d10e18eabd3b 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -23,6 +23,8 @@
 
 #include "vgic.h"
 
+#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
+
 static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
 {
 	struct kvm_vcpu *vcpu = info;
@@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
 		 * doorbell could kick us out of the guest too
 		 * early...
 		 */
-		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		irq_set_status_flags(irq, DB_IRQ_FLAGS);
 		ret = request_irq(irq, vgic_v4_doorbell_handler,
 				  0, "vcpu", vcpu);
 		if (ret) {
@@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
 		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
 		int irq = its_vm->vpes[i]->irq;
 
-		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
 		free_irq(irq, vcpu);
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

We so far allocate the doorbell interrupts without taking any
special measure regarding the affinity of these interrupts. We
simply move them around as required when the vcpu gets scheduled
on a different CPU.

But that's counting without userspace (and the evil irqbalance) that
can try and move the VPE interrupt around, causing the ITS code
to emit VMOVP commands and remap the doorbell to another redistributor.
Worse, this can happen while the vcpu is running, causing all kind
of trouble if the VPE is already resident, and we end-up in UNPRED
territory.

So let's take a definitive action and prevent userspace from messing
with us. This is just a matter of adding IRQ_NO_BALANCING to the
set of flags we already have, letting the kernel in sole control
of the affinity.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index d7fe610bb1f5..d10e18eabd3b 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -23,6 +23,8 @@
 
 #include "vgic.h"
 
+#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
+
 static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
 {
 	struct kvm_vcpu *vcpu = info;
@@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
 		 * doorbell could kick us out of the guest too
 		 * early...
 		 */
-		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		irq_set_status_flags(irq, DB_IRQ_FLAGS);
 		ret = request_irq(irq, vgic_v4_doorbell_handler,
 				  0, "vcpu", vcpu);
 		if (ret) {
@@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
 		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
 		int irq = its_vm->vpes[i]->irq;
 
-		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
+		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
 		free_irq(irq, vcpu);
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

All it takes is the has_v4 flag to be set in gic_kvm_info
as well as "kvm-arm.vgic_v4_enable=1" being passed on the
command line for GICv4 to be enabled in KVM.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  4 ++++
 virt/kvm/arm/vgic/vgic-v3.c                     | 14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 05496622b4ef..93c8fff399eb 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1874,6 +1874,10 @@
 			[KVM,ARM] Trap guest accesses to GICv3 common
 			system registers
 
+	kvm-arm.vgic_v4_enable=
+			[KVM,ARM] Allow use of GICv4 for direct injection of
+			LPIs.
+
 	kvm-intel.ept=	[KVM,Intel] Disable extended page tables
 			(virtualized MMU) support on capable Intel chips.
 			Default is 1 (enabled)
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 96ea597db0e7..405733678c2f 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -24,6 +24,7 @@
 static bool group0_trap;
 static bool group1_trap;
 static bool common_trap;
+static bool gicv4_enable;
 
 void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
 {
@@ -459,6 +460,12 @@ static int __init early_common_trap_cfg(char *buf)
 }
 early_param("kvm-arm.vgic_v3_common_trap", early_common_trap_cfg);
 
+static int __init early_gicv4_enable(char *buf)
+{
+	return strtobool(buf, &gicv4_enable);
+}
+early_param("kvm-arm.vgic_v4_enable", early_gicv4_enable);
+
 /**
  * vgic_v3_probe - probe for a GICv3 compatible interrupt controller in DT
  * @node:	pointer to the DT node
@@ -478,6 +485,13 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
 	kvm_vgic_global_state.can_emulate_gicv2 = false;
 	kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
 
+	/* GICv4 support? */
+	if (info->has_v4) {
+		kvm_vgic_global_state.has_gicv4 = gicv4_enable;
+		kvm_info("GICv4 support %sabled\n",
+			 gicv4_enable ? "en" : "dis");
+	}
+
 	if (!info->vcpu.start) {
 		kvm_info("GICv3: no GICV resource entry\n");
 		kvm_vgic_global_state.vcpu_base = 0;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

All it takes is the has_v4 flag to be set in gic_kvm_info
as well as "kvm-arm.vgic_v4_enable=1" being passed on the
command line for GICv4 to be enabled in KVM.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  4 ++++
 virt/kvm/arm/vgic/vgic-v3.c                     | 14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 05496622b4ef..93c8fff399eb 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1874,6 +1874,10 @@
 			[KVM,ARM] Trap guest accesses to GICv3 common
 			system registers
 
+	kvm-arm.vgic_v4_enable=
+			[KVM,ARM] Allow use of GICv4 for direct injection of
+			LPIs.
+
 	kvm-intel.ept=	[KVM,Intel] Disable extended page tables
 			(virtualized MMU) support on capable Intel chips.
 			Default is 1 (enabled)
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 96ea597db0e7..405733678c2f 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -24,6 +24,7 @@
 static bool group0_trap;
 static bool group1_trap;
 static bool common_trap;
+static bool gicv4_enable;
 
 void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
 {
@@ -459,6 +460,12 @@ static int __init early_common_trap_cfg(char *buf)
 }
 early_param("kvm-arm.vgic_v3_common_trap", early_common_trap_cfg);
 
+static int __init early_gicv4_enable(char *buf)
+{
+	return strtobool(buf, &gicv4_enable);
+}
+early_param("kvm-arm.vgic_v4_enable", early_gicv4_enable);
+
 /**
  * vgic_v3_probe - probe for a GICv3 compatible interrupt controller in DT
  * @node:	pointer to the DT node
@@ -478,6 +485,13 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
 	kvm_vgic_global_state.can_emulate_gicv2 = false;
 	kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
 
+	/* GICv4 support? */
+	if (info->has_v4) {
+		kvm_vgic_global_state.has_gicv4 = gicv4_enable;
+		kvm_info("GICv4 support %sabled\n",
+			 gicv4_enable ? "en" : "dis");
+	}
+
 	if (!info->vcpu.start) {
 		kvm_info("GICv3: no GICV resource entry\n");
 		kvm_vgic_global_state.vcpu_base = 0;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

All it takes is the has_v4 flag to be set in gic_kvm_info
as well as "kvm-arm.vgic_v4_enable=1" being passed on the
command line for GICv4 to be enabled in KVM.

Acked-by: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  4 ++++
 virt/kvm/arm/vgic/vgic-v3.c                     | 14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 05496622b4ef..93c8fff399eb 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1874,6 +1874,10 @@
 			[KVM,ARM] Trap guest accesses to GICv3 common
 			system registers
 
+	kvm-arm.vgic_v4_enable=
+			[KVM,ARM] Allow use of GICv4 for direct injection of
+			LPIs.
+
 	kvm-intel.ept=	[KVM,Intel] Disable extended page tables
 			(virtualized MMU) support on capable Intel chips.
 			Default is 1 (enabled)
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 96ea597db0e7..405733678c2f 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -24,6 +24,7 @@
 static bool group0_trap;
 static bool group1_trap;
 static bool common_trap;
+static bool gicv4_enable;
 
 void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
 {
@@ -459,6 +460,12 @@ static int __init early_common_trap_cfg(char *buf)
 }
 early_param("kvm-arm.vgic_v3_common_trap", early_common_trap_cfg);
 
+static int __init early_gicv4_enable(char *buf)
+{
+	return strtobool(buf, &gicv4_enable);
+}
+early_param("kvm-arm.vgic_v4_enable", early_gicv4_enable);
+
 /**
  * vgic_v3_probe - probe for a GICv3 compatible interrupt controller in DT
  * @node:	pointer to the DT node
@@ -478,6 +485,13 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
 	kvm_vgic_global_state.can_emulate_gicv2 = false;
 	kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
 
+	/* GICv4 support? */
+	if (info->has_v4) {
+		kvm_vgic_global_state.has_gicv4 = gicv4_enable;
+		kvm_info("GICv4 support %sabled\n",
+			 gicv4_enable ? "en" : "dis");
+	}
+
 	if (!info->vcpu.start) {
 		kvm_info("GICv3: no GICV resource entry\n");
 		kvm_vgic_global_state.vcpu_base = 0;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
  2017-10-27 14:28 ` Marc Zyngier
  (?)
@ 2017-10-27 14:28   ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Yet another braindump so I can free some cells...

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index d10e18eabd3b..e367d65a0ebe 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -23,6 +23,73 @@
 
 #include "vgic.h"
 
+/*
+ * How KVM uses GICv4 (insert rude comments here):
+ *
+ * The vgic-v4 layer acts as a bridge between several entities:
+ * - The GICv4 ITS representation offered by the ITS driver
+ * - VFIO, which is in charge of the PCI endpoint
+ * - The virtual ITS, which is the only thing the guest sees
+ *
+ * The configuration of VLPIs is triggered by a callback from VFIO,
+ * instructing KVM that a PCI device has been configured to deliver
+ * MSIs to a vITS.
+ *
+ * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
+ * and this is used to find the corresponding vITS data structures
+ * (ITS instance, device, event and irq) using a process that is
+ * extremely similar to the injection of an MSI.
+ *
+ * At this stage, we can link the guest's view of an LPI (uniquely
+ * identified by the routing entry) and the host irq, using the GICv4
+ * driver mapping operation. Should the mapping succeed, we've then
+ * successfully upgraded the guest's LPI to a VLPI. We can then start
+ * with updating GICv4's view of the property table and generating an
+ * INValidation in order to kickstart the delivery of this VLPI to the
+ * guest directly, without software intervention. Well, almost.
+ *
+ * When the PCI endpoint is deconfigured, this operation is reversed
+ * with VFIO calling kvm_vgic_v4_unset_forwarding().
+ *
+ * Once the VLPI has been mapped, it needs to follow any change the
+ * guest performs on its LPI through the vITS. For that, a number of
+ * command handlers have hooks to communicate these changes to the HW:
+ * - Any invalidation triggers a call to its_prop_update_vlpi()
+ * - The INT command results in a irq_set_irqchip_state(), which
+ *   generates an INT on the corresponding VLPI.
+ * - The CLEAR command results in a irq_set_irqchip_state(), which
+ *   generates an CLEAR on the corresponding VLPI.
+ * - DISCARD translates into an unmap, similar to a call to
+ *   kvm_vgic_v4_unset_forwarding().
+ * - MOVI is translated by an update of the existing mapping, changing
+ *   the target vcpu, resulting in a VMOVI being generated.
+ * - MOVALL is translated by a string of mapping updates (similar to
+ *   the handling of MOVI). MOVALL is horrible.
+ *
+ * Note that a DISCARD/MAPTI sequence emitted from the guest without
+ * reprogramming the PCI endpoint after MAPTI does not result in a
+ * VLPI being mapped, as there is no callback from VFIO (the guest
+ * will get the interrupt via the normal SW injection). Fixing this is
+ * not trivial, and requires some horrible messing with the VFIO
+ * internals. Not fun. Don't do that.
+ *
+ * Then there is the scheduling. Each time a vcpu is about to run on a
+ * physical CPU, KVM must tell the corresponding redistributor about
+ * it. And if we've migrated our vcpu from one CPU to another, we must
+ * tell the ITS (so that the messages reach the right redistributor).
+ * This is done in two steps: first issue a irq_set_affinity() on the
+ * irq corresponding to the vcpu, then call its_schedule_vpe(). You
+ * must be in a non-preemptible context. On exit, another call to
+ * its_schedule_vpe() tells the redistributor that we're done with the
+ * vcpu.
+ *
+ * Finally, the doorbell handling: Each vcpu is allocated an interrupt
+ * which will fire each time a VLPI is made pending whilst the vcpu is
+ * not running. Each time the vcpu gets blocked, the doorbell
+ * interrupt gets enabled. When the vcpu is unblocked (for whatever
+ * reason), the doorbell interrupt is disabled.
+ */
+
 #define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
 
 static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Yet another braindump so I can free some cells...

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index d10e18eabd3b..e367d65a0ebe 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -23,6 +23,73 @@
 
 #include "vgic.h"
 
+/*
+ * How KVM uses GICv4 (insert rude comments here):
+ *
+ * The vgic-v4 layer acts as a bridge between several entities:
+ * - The GICv4 ITS representation offered by the ITS driver
+ * - VFIO, which is in charge of the PCI endpoint
+ * - The virtual ITS, which is the only thing the guest sees
+ *
+ * The configuration of VLPIs is triggered by a callback from VFIO,
+ * instructing KVM that a PCI device has been configured to deliver
+ * MSIs to a vITS.
+ *
+ * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
+ * and this is used to find the corresponding vITS data structures
+ * (ITS instance, device, event and irq) using a process that is
+ * extremely similar to the injection of an MSI.
+ *
+ * At this stage, we can link the guest's view of an LPI (uniquely
+ * identified by the routing entry) and the host irq, using the GICv4
+ * driver mapping operation. Should the mapping succeed, we've then
+ * successfully upgraded the guest's LPI to a VLPI. We can then start
+ * with updating GICv4's view of the property table and generating an
+ * INValidation in order to kickstart the delivery of this VLPI to the
+ * guest directly, without software intervention. Well, almost.
+ *
+ * When the PCI endpoint is deconfigured, this operation is reversed
+ * with VFIO calling kvm_vgic_v4_unset_forwarding().
+ *
+ * Once the VLPI has been mapped, it needs to follow any change the
+ * guest performs on its LPI through the vITS. For that, a number of
+ * command handlers have hooks to communicate these changes to the HW:
+ * - Any invalidation triggers a call to its_prop_update_vlpi()
+ * - The INT command results in a irq_set_irqchip_state(), which
+ *   generates an INT on the corresponding VLPI.
+ * - The CLEAR command results in a irq_set_irqchip_state(), which
+ *   generates an CLEAR on the corresponding VLPI.
+ * - DISCARD translates into an unmap, similar to a call to
+ *   kvm_vgic_v4_unset_forwarding().
+ * - MOVI is translated by an update of the existing mapping, changing
+ *   the target vcpu, resulting in a VMOVI being generated.
+ * - MOVALL is translated by a string of mapping updates (similar to
+ *   the handling of MOVI). MOVALL is horrible.
+ *
+ * Note that a DISCARD/MAPTI sequence emitted from the guest without
+ * reprogramming the PCI endpoint after MAPTI does not result in a
+ * VLPI being mapped, as there is no callback from VFIO (the guest
+ * will get the interrupt via the normal SW injection). Fixing this is
+ * not trivial, and requires some horrible messing with the VFIO
+ * internals. Not fun. Don't do that.
+ *
+ * Then there is the scheduling. Each time a vcpu is about to run on a
+ * physical CPU, KVM must tell the corresponding redistributor about
+ * it. And if we've migrated our vcpu from one CPU to another, we must
+ * tell the ITS (so that the messages reach the right redistributor).
+ * This is done in two steps: first issue a irq_set_affinity() on the
+ * irq corresponding to the vcpu, then call its_schedule_vpe(). You
+ * must be in a non-preemptible context. On exit, another call to
+ * its_schedule_vpe() tells the redistributor that we're done with the
+ * vcpu.
+ *
+ * Finally, the doorbell handling: Each vcpu is allocated an interrupt
+ * which will fire each time a VLPI is made pending whilst the vcpu is
+ * not running. Each time the vcpu gets blocked, the doorbell
+ * interrupt gets enabled. When the vcpu is unblocked (for whatever
+ * reason), the doorbell interrupt is disabled.
+ */
+
 #define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
 
 static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
@ 2017-10-27 14:28   ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-10-27 14:28 UTC (permalink / raw)
  To: linux-arm-kernel

Yet another braindump so I can free some cells...

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index d10e18eabd3b..e367d65a0ebe 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -23,6 +23,73 @@
 
 #include "vgic.h"
 
+/*
+ * How KVM uses GICv4 (insert rude comments here):
+ *
+ * The vgic-v4 layer acts as a bridge between several entities:
+ * - The GICv4 ITS representation offered by the ITS driver
+ * - VFIO, which is in charge of the PCI endpoint
+ * - The virtual ITS, which is the only thing the guest sees
+ *
+ * The configuration of VLPIs is triggered by a callback from VFIO,
+ * instructing KVM that a PCI device has been configured to deliver
+ * MSIs to a vITS.
+ *
+ * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
+ * and this is used to find the corresponding vITS data structures
+ * (ITS instance, device, event and irq) using a process that is
+ * extremely similar to the injection of an MSI.
+ *
+ * At this stage, we can link the guest's view of an LPI (uniquely
+ * identified by the routing entry) and the host irq, using the GICv4
+ * driver mapping operation. Should the mapping succeed, we've then
+ * successfully upgraded the guest's LPI to a VLPI. We can then start
+ * with updating GICv4's view of the property table and generating an
+ * INValidation in order to kickstart the delivery of this VLPI to the
+ * guest directly, without software intervention. Well, almost.
+ *
+ * When the PCI endpoint is deconfigured, this operation is reversed
+ * with VFIO calling kvm_vgic_v4_unset_forwarding().
+ *
+ * Once the VLPI has been mapped, it needs to follow any change the
+ * guest performs on its LPI through the vITS. For that, a number of
+ * command handlers have hooks to communicate these changes to the HW:
+ * - Any invalidation triggers a call to its_prop_update_vlpi()
+ * - The INT command results in a irq_set_irqchip_state(), which
+ *   generates an INT on the corresponding VLPI.
+ * - The CLEAR command results in a irq_set_irqchip_state(), which
+ *   generates an CLEAR on the corresponding VLPI.
+ * - DISCARD translates into an unmap, similar to a call to
+ *   kvm_vgic_v4_unset_forwarding().
+ * - MOVI is translated by an update of the existing mapping, changing
+ *   the target vcpu, resulting in a VMOVI being generated.
+ * - MOVALL is translated by a string of mapping updates (similar to
+ *   the handling of MOVI). MOVALL is horrible.
+ *
+ * Note that a DISCARD/MAPTI sequence emitted from the guest without
+ * reprogramming the PCI endpoint after MAPTI does not result in a
+ * VLPI being mapped, as there is no callback from VFIO (the guest
+ * will get the interrupt via the normal SW injection). Fixing this is
+ * not trivial, and requires some horrible messing with the VFIO
+ * internals. Not fun. Don't do that.
+ *
+ * Then there is the scheduling. Each time a vcpu is about to run on a
+ * physical CPU, KVM must tell the corresponding redistributor about
+ * it. And if we've migrated our vcpu from one CPU to another, we must
+ * tell the ITS (so that the messages reach the right redistributor).
+ * This is done in two steps: first issue a irq_set_affinity() on the
+ * irq corresponding to the vcpu, then call its_schedule_vpe(). You
+ * must be in a non-preemptible context. On exit, another call to
+ * its_schedule_vpe() tells the redistributor that we're done with the
+ * vcpu.
+ *
+ * Finally, the doorbell handling: Each vcpu is allocated an interrupt
+ * which will fire each time a VLPI is made pending whilst the vcpu is
+ * not running. Each time the vcpu gets blocked, the doorbell
+ * interrupt gets enabled. When the vcpu is unblocked (for whatever
+ * reason), the doorbell interrupt is disabled.
+ */
+
 #define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
 
 static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 01/26] irqchip/gic-v3-its: Setup VLPI properties at map time
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-10-30  6:46     ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:46 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: linux-arm-kernel, kvmarm, kvm, linux-kernel, Christoffer Dall,
	Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

On Fri, Oct 27, 2017 at 03:28:30PM +0100, Marc Zyngier wrote:
> So far, we require the hypervisor to update the VLPI properties
> once the the VLPI mapping has been established. While this
> makes it easy for the ITS driver, it creates a window where
> an incoming interrupt can be delivered with an unknown set
> of properties. Not very nice.
> 
> Instead, let's add a "properties" field to the mapping structure,
> and use that to configure the VLPI before it actually gets mapped.
> 

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
>  include/linux/irqchip/arm-gic-v4.h |  2 ++
>  2 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 85f47ef4f1f7..2c4074c97454 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -1006,9 +1006,15 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
>  	if (irqd_is_forwarded_to_vcpu(d)) {
>  		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
>  		u32 event = its_get_event_id(d);
> +		struct its_vlpi_map *map;
>  
>  		prop_page = its_dev->event_map.vm->vprop_page;
> -		hwirq = its_dev->event_map.vlpi_maps[event].vintid;
> +		map = &its_dev->event_map.vlpi_maps[event];
> +		hwirq = map->vintid;
> +
> +		/* Remember the updated property */
> +		map->properties &= ~clr;
> +		map->properties |= set | LPI_PROP_GROUP1;
>  	} else {
>  		prop_page = gic_rdists->prop_page;
>  		hwirq = d->hwirq;
> @@ -1018,6 +1024,7 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
>  	*cfg &= ~clr;
>  	*cfg |= set | LPI_PROP_GROUP1;
>  
> +
>  	/*
>  	 * Make the above write visible to the redistributors.
>  	 * And yes, we're flushing exactly: One. Single. Byte.
> @@ -1238,12 +1245,20 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
>  		/* Ensure all the VPEs are mapped on this ITS */
>  		its_map_vm(its_dev->its, info->map->vm);
>  
> +		/*
> +		 * Flag the interrupt as forwarded so that we can
> +		 * start poking the virtual property table.
> +		 */
> +		irqd_set_forwarded_to_vcpu(d);
> +
> +		/* Write out the property to the prop table */
> +		lpi_write_config(d, 0xff, info->map->properties);
> +
>  		/* Drop the physical mapping */
>  		its_send_discard(its_dev, event);
>  
>  		/* and install the virtual one */
>  		its_send_vmapti(its_dev, event);
> -		irqd_set_forwarded_to_vcpu(d);
>  
>  		/* Increment the number of VLPIs */
>  		its_dev->event_map.nr_vlpis++;
> diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
> index 43cde15f221b..447da8ca2156 100644
> --- a/include/linux/irqchip/arm-gic-v4.h
> +++ b/include/linux/irqchip/arm-gic-v4.h
> @@ -71,12 +71,14 @@ struct its_vpe {
>   * @vm:		Pointer to the GICv4 notion of a VM
>   * @vpe:	Pointer to the GICv4 notion of a virtual CPU (VPE)
>   * @vintid:	Virtual LPI number
> + * @properties:	Priority and enable bits (as written in the prop table)
>   * @db_enabled:	Is the VPE doorbell to be generated?
>   */
>  struct its_vlpi_map {
>  	struct its_vm		*vm;
>  	struct its_vpe		*vpe;
>  	u32			vintid;
> +	u8			properties;
>  	bool			db_enabled;
>  };
>  
> -- 
> 2.11.0
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 01/26] irqchip/gic-v3-its: Setup VLPI properties at map time
@ 2017-10-30  6:46     ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:46 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: kvm, Andre Przywara, linux-kernel, linux-arm-kernel, kvmarm

On Fri, Oct 27, 2017 at 03:28:30PM +0100, Marc Zyngier wrote:
> So far, we require the hypervisor to update the VLPI properties
> once the the VLPI mapping has been established. While this
> makes it easy for the ITS driver, it creates a window where
> an incoming interrupt can be delivered with an unknown set
> of properties. Not very nice.
> 
> Instead, let's add a "properties" field to the mapping structure,
> and use that to configure the VLPI before it actually gets mapped.
> 

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
>  include/linux/irqchip/arm-gic-v4.h |  2 ++
>  2 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 85f47ef4f1f7..2c4074c97454 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -1006,9 +1006,15 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
>  	if (irqd_is_forwarded_to_vcpu(d)) {
>  		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
>  		u32 event = its_get_event_id(d);
> +		struct its_vlpi_map *map;
>  
>  		prop_page = its_dev->event_map.vm->vprop_page;
> -		hwirq = its_dev->event_map.vlpi_maps[event].vintid;
> +		map = &its_dev->event_map.vlpi_maps[event];
> +		hwirq = map->vintid;
> +
> +		/* Remember the updated property */
> +		map->properties &= ~clr;
> +		map->properties |= set | LPI_PROP_GROUP1;
>  	} else {
>  		prop_page = gic_rdists->prop_page;
>  		hwirq = d->hwirq;
> @@ -1018,6 +1024,7 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
>  	*cfg &= ~clr;
>  	*cfg |= set | LPI_PROP_GROUP1;
>  
> +
>  	/*
>  	 * Make the above write visible to the redistributors.
>  	 * And yes, we're flushing exactly: One. Single. Byte.
> @@ -1238,12 +1245,20 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
>  		/* Ensure all the VPEs are mapped on this ITS */
>  		its_map_vm(its_dev->its, info->map->vm);
>  
> +		/*
> +		 * Flag the interrupt as forwarded so that we can
> +		 * start poking the virtual property table.
> +		 */
> +		irqd_set_forwarded_to_vcpu(d);
> +
> +		/* Write out the property to the prop table */
> +		lpi_write_config(d, 0xff, info->map->properties);
> +
>  		/* Drop the physical mapping */
>  		its_send_discard(its_dev, event);
>  
>  		/* and install the virtual one */
>  		its_send_vmapti(its_dev, event);
> -		irqd_set_forwarded_to_vcpu(d);
>  
>  		/* Increment the number of VLPIs */
>  		its_dev->event_map.nr_vlpis++;
> diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
> index 43cde15f221b..447da8ca2156 100644
> --- a/include/linux/irqchip/arm-gic-v4.h
> +++ b/include/linux/irqchip/arm-gic-v4.h
> @@ -71,12 +71,14 @@ struct its_vpe {
>   * @vm:		Pointer to the GICv4 notion of a VM
>   * @vpe:	Pointer to the GICv4 notion of a virtual CPU (VPE)
>   * @vintid:	Virtual LPI number
> + * @properties:	Priority and enable bits (as written in the prop table)
>   * @db_enabled:	Is the VPE doorbell to be generated?
>   */
>  struct its_vlpi_map {
>  	struct its_vm		*vm;
>  	struct its_vpe		*vpe;
>  	u32			vintid;
> +	u8			properties;
>  	bool			db_enabled;
>  };
>  
> -- 
> 2.11.0
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 01/26] irqchip/gic-v3-its: Setup VLPI properties at map time
@ 2017-10-30  6:46     ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:46 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Oct 27, 2017 at 03:28:30PM +0100, Marc Zyngier wrote:
> So far, we require the hypervisor to update the VLPI properties
> once the the VLPI mapping has been established. While this
> makes it easy for the ITS driver, it creates a window where
> an incoming interrupt can be delivered with an unknown set
> of properties. Not very nice.
> 
> Instead, let's add a "properties" field to the mapping structure,
> and use that to configure the VLPI before it actually gets mapped.
> 

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
>  include/linux/irqchip/arm-gic-v4.h |  2 ++
>  2 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 85f47ef4f1f7..2c4074c97454 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -1006,9 +1006,15 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
>  	if (irqd_is_forwarded_to_vcpu(d)) {
>  		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
>  		u32 event = its_get_event_id(d);
> +		struct its_vlpi_map *map;
>  
>  		prop_page = its_dev->event_map.vm->vprop_page;
> -		hwirq = its_dev->event_map.vlpi_maps[event].vintid;
> +		map = &its_dev->event_map.vlpi_maps[event];
> +		hwirq = map->vintid;
> +
> +		/* Remember the updated property */
> +		map->properties &= ~clr;
> +		map->properties |= set | LPI_PROP_GROUP1;
>  	} else {
>  		prop_page = gic_rdists->prop_page;
>  		hwirq = d->hwirq;
> @@ -1018,6 +1024,7 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
>  	*cfg &= ~clr;
>  	*cfg |= set | LPI_PROP_GROUP1;
>  
> +
>  	/*
>  	 * Make the above write visible to the redistributors.
>  	 * And yes, we're flushing exactly: One. Single. Byte.
> @@ -1238,12 +1245,20 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
>  		/* Ensure all the VPEs are mapped on this ITS */
>  		its_map_vm(its_dev->its, info->map->vm);
>  
> +		/*
> +		 * Flag the interrupt as forwarded so that we can
> +		 * start poking the virtual property table.
> +		 */
> +		irqd_set_forwarded_to_vcpu(d);
> +
> +		/* Write out the property to the prop table */
> +		lpi_write_config(d, 0xff, info->map->properties);
> +
>  		/* Drop the physical mapping */
>  		its_send_discard(its_dev, event);
>  
>  		/* and install the virtual one */
>  		its_send_vmapti(its_dev, event);
> -		irqd_set_forwarded_to_vcpu(d);
>  
>  		/* Increment the number of VLPIs */
>  		its_dev->event_map.nr_vlpis++;
> diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
> index 43cde15f221b..447da8ca2156 100644
> --- a/include/linux/irqchip/arm-gic-v4.h
> +++ b/include/linux/irqchip/arm-gic-v4.h
> @@ -71,12 +71,14 @@ struct its_vpe {
>   * @vm:		Pointer to the GICv4 notion of a VM
>   * @vpe:	Pointer to the GICv4 notion of a virtual CPU (VPE)
>   * @vintid:	Virtual LPI number
> + * @properties:	Priority and enable bits (as written in the prop table)
>   * @db_enabled:	Is the VPE doorbell to be generated?
>   */
>  struct its_vlpi_map {
>  	struct its_vm		*vm;
>  	struct its_vpe		*vpe;
>  	u32			vintid;
> +	u8			properties;
>  	bool			db_enabled;
>  };
>  
> -- 
> 2.11.0
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 02/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-10-30  6:47     ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:47 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: linux-arm-kernel, kvmarm, kvm, linux-kernel, Christoffer Dall,
	Eric Auger, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

On Fri, Oct 27, 2017 at 03:28:31PM +0100, Marc Zyngier wrote:
> From: Eric Auger <eric.auger@redhat.com>
> 
> This patch selects IRQ_BYPASS_MANAGER and HAVE_KVM_IRQ_BYPASS
> configs for ARM/ARM64.
> 
> kvm_arch_has_irq_bypass() now is implemented and returns true.
> As a consequence the irq bypass consumer will be registered for
> ARM/ARM64 with the forwarding callbacks:
> 
> - stop/start: halt/resume guest execution
> - add/del_producer: set/unset forwarding at vgic/irqchip level
> 
> We don't have any actual support yet, so nothing gets actually
> forwarded.
> 

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> [maz: dropped the DEOI stuff for the time being in order to
>       reduce the dependency chain, amended commit message]
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Kconfig   |  3 +++
>  arch/arm64/kvm/Kconfig |  3 +++
>  virt/kvm/arm/arm.c     | 40 ++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 46 insertions(+)
> 
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> index 90d0176fb30d..4e2b192a030a 100644
> --- a/arch/arm/kvm/Kconfig
> +++ b/arch/arm/kvm/Kconfig
> @@ -3,6 +3,7 @@
>  #
>  
>  source "virt/kvm/Kconfig"
> +source "virt/lib/Kconfig"
>  
>  menuconfig VIRTUALIZATION
>  	bool "Virtualization"
> @@ -35,6 +36,8 @@ config KVM
>  	select HAVE_KVM_IRQCHIP
>  	select HAVE_KVM_IRQ_ROUTING
>  	select HAVE_KVM_MSI
> +	select IRQ_BYPASS_MANAGER
> +	select HAVE_KVM_IRQ_BYPASS
>  	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
>  	---help---
>  	  Support hosting virtualized guest machines.
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 52cb7ad9b2fd..7e0d6e63cc71 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -3,6 +3,7 @@
>  #
>  
>  source "virt/kvm/Kconfig"
> +source "virt/lib/Kconfig"
>  
>  menuconfig VIRTUALIZATION
>  	bool "Virtualization"
> @@ -35,6 +36,8 @@ config KVM
>  	select HAVE_KVM_MSI
>  	select HAVE_KVM_IRQCHIP
>  	select HAVE_KVM_IRQ_ROUTING
> +	select IRQ_BYPASS_MANAGER
> +	select HAVE_KVM_IRQ_BYPASS
>  	---help---
>  	  Support hosting virtualized guest machines.
>  	  We don't support KVM with 16K page tables yet, due to the multiple
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index b9f68e4add71..7ac4ecba9ebd 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -27,6 +27,8 @@
>  #include <linux/mman.h>
>  #include <linux/sched.h>
>  #include <linux/kvm.h>
> +#include <linux/kvm_irqfd.h>
> +#include <linux/irqbypass.h>
>  #include <trace/events/kvm.h>
>  #include <kvm/arm_pmu.h>
>  
> @@ -1449,6 +1451,44 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
>  	return NULL;
>  }
>  
> +bool kvm_arch_has_irq_bypass(void)
> +{
> +	return true;
> +}
> +
> +int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> +				      struct irq_bypass_producer *prod)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	return 0;
> +}
> +void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> +				      struct irq_bypass_producer *prod)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	return;
> +}
> +
> +void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	kvm_arm_halt_guest(irqfd->kvm);
> +}
> +
> +void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	kvm_arm_resume_guest(irqfd->kvm);
> +}
> +
>  /**
>   * Initialize Hyp-mode and memory mappings on all CPUs.
>   */
> -- 
> 2.11.0
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 02/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
@ 2017-10-30  6:47     ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:47 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: kvm, Andre Przywara, linux-kernel, linux-arm-kernel, kvmarm

On Fri, Oct 27, 2017 at 03:28:31PM +0100, Marc Zyngier wrote:
> From: Eric Auger <eric.auger@redhat.com>
> 
> This patch selects IRQ_BYPASS_MANAGER and HAVE_KVM_IRQ_BYPASS
> configs for ARM/ARM64.
> 
> kvm_arch_has_irq_bypass() now is implemented and returns true.
> As a consequence the irq bypass consumer will be registered for
> ARM/ARM64 with the forwarding callbacks:
> 
> - stop/start: halt/resume guest execution
> - add/del_producer: set/unset forwarding at vgic/irqchip level
> 
> We don't have any actual support yet, so nothing gets actually
> forwarded.
> 

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> [maz: dropped the DEOI stuff for the time being in order to
>       reduce the dependency chain, amended commit message]
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Kconfig   |  3 +++
>  arch/arm64/kvm/Kconfig |  3 +++
>  virt/kvm/arm/arm.c     | 40 ++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 46 insertions(+)
> 
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> index 90d0176fb30d..4e2b192a030a 100644
> --- a/arch/arm/kvm/Kconfig
> +++ b/arch/arm/kvm/Kconfig
> @@ -3,6 +3,7 @@
>  #
>  
>  source "virt/kvm/Kconfig"
> +source "virt/lib/Kconfig"
>  
>  menuconfig VIRTUALIZATION
>  	bool "Virtualization"
> @@ -35,6 +36,8 @@ config KVM
>  	select HAVE_KVM_IRQCHIP
>  	select HAVE_KVM_IRQ_ROUTING
>  	select HAVE_KVM_MSI
> +	select IRQ_BYPASS_MANAGER
> +	select HAVE_KVM_IRQ_BYPASS
>  	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
>  	---help---
>  	  Support hosting virtualized guest machines.
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 52cb7ad9b2fd..7e0d6e63cc71 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -3,6 +3,7 @@
>  #
>  
>  source "virt/kvm/Kconfig"
> +source "virt/lib/Kconfig"
>  
>  menuconfig VIRTUALIZATION
>  	bool "Virtualization"
> @@ -35,6 +36,8 @@ config KVM
>  	select HAVE_KVM_MSI
>  	select HAVE_KVM_IRQCHIP
>  	select HAVE_KVM_IRQ_ROUTING
> +	select IRQ_BYPASS_MANAGER
> +	select HAVE_KVM_IRQ_BYPASS
>  	---help---
>  	  Support hosting virtualized guest machines.
>  	  We don't support KVM with 16K page tables yet, due to the multiple
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index b9f68e4add71..7ac4ecba9ebd 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -27,6 +27,8 @@
>  #include <linux/mman.h>
>  #include <linux/sched.h>
>  #include <linux/kvm.h>
> +#include <linux/kvm_irqfd.h>
> +#include <linux/irqbypass.h>
>  #include <trace/events/kvm.h>
>  #include <kvm/arm_pmu.h>
>  
> @@ -1449,6 +1451,44 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
>  	return NULL;
>  }
>  
> +bool kvm_arch_has_irq_bypass(void)
> +{
> +	return true;
> +}
> +
> +int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> +				      struct irq_bypass_producer *prod)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	return 0;
> +}
> +void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> +				      struct irq_bypass_producer *prod)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	return;
> +}
> +
> +void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	kvm_arm_halt_guest(irqfd->kvm);
> +}
> +
> +void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	kvm_arm_resume_guest(irqfd->kvm);
> +}
> +
>  /**
>   * Initialize Hyp-mode and memory mappings on all CPUs.
>   */
> -- 
> 2.11.0
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 02/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64
@ 2017-10-30  6:47     ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:47 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Oct 27, 2017 at 03:28:31PM +0100, Marc Zyngier wrote:
> From: Eric Auger <eric.auger@redhat.com>
> 
> This patch selects IRQ_BYPASS_MANAGER and HAVE_KVM_IRQ_BYPASS
> configs for ARM/ARM64.
> 
> kvm_arch_has_irq_bypass() now is implemented and returns true.
> As a consequence the irq bypass consumer will be registered for
> ARM/ARM64 with the forwarding callbacks:
> 
> - stop/start: halt/resume guest execution
> - add/del_producer: set/unset forwarding at vgic/irqchip level
> 
> We don't have any actual support yet, so nothing gets actually
> forwarded.
> 

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> [maz: dropped the DEOI stuff for the time being in order to
>       reduce the dependency chain, amended commit message]
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Kconfig   |  3 +++
>  arch/arm64/kvm/Kconfig |  3 +++
>  virt/kvm/arm/arm.c     | 40 ++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 46 insertions(+)
> 
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> index 90d0176fb30d..4e2b192a030a 100644
> --- a/arch/arm/kvm/Kconfig
> +++ b/arch/arm/kvm/Kconfig
> @@ -3,6 +3,7 @@
>  #
>  
>  source "virt/kvm/Kconfig"
> +source "virt/lib/Kconfig"
>  
>  menuconfig VIRTUALIZATION
>  	bool "Virtualization"
> @@ -35,6 +36,8 @@ config KVM
>  	select HAVE_KVM_IRQCHIP
>  	select HAVE_KVM_IRQ_ROUTING
>  	select HAVE_KVM_MSI
> +	select IRQ_BYPASS_MANAGER
> +	select HAVE_KVM_IRQ_BYPASS
>  	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
>  	---help---
>  	  Support hosting virtualized guest machines.
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 52cb7ad9b2fd..7e0d6e63cc71 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -3,6 +3,7 @@
>  #
>  
>  source "virt/kvm/Kconfig"
> +source "virt/lib/Kconfig"
>  
>  menuconfig VIRTUALIZATION
>  	bool "Virtualization"
> @@ -35,6 +36,8 @@ config KVM
>  	select HAVE_KVM_MSI
>  	select HAVE_KVM_IRQCHIP
>  	select HAVE_KVM_IRQ_ROUTING
> +	select IRQ_BYPASS_MANAGER
> +	select HAVE_KVM_IRQ_BYPASS
>  	---help---
>  	  Support hosting virtualized guest machines.
>  	  We don't support KVM with 16K page tables yet, due to the multiple
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index b9f68e4add71..7ac4ecba9ebd 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -27,6 +27,8 @@
>  #include <linux/mman.h>
>  #include <linux/sched.h>
>  #include <linux/kvm.h>
> +#include <linux/kvm_irqfd.h>
> +#include <linux/irqbypass.h>
>  #include <trace/events/kvm.h>
>  #include <kvm/arm_pmu.h>
>  
> @@ -1449,6 +1451,44 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
>  	return NULL;
>  }
>  
> +bool kvm_arch_has_irq_bypass(void)
> +{
> +	return true;
> +}
> +
> +int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> +				      struct irq_bypass_producer *prod)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	return 0;
> +}
> +void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> +				      struct irq_bypass_producer *prod)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	return;
> +}
> +
> +void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	kvm_arm_halt_guest(irqfd->kvm);
> +}
> +
> +void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
> +{
> +	struct kvm_kernel_irqfd *irqfd =
> +		container_of(cons, struct kvm_kernel_irqfd, consumer);
> +
> +	kvm_arm_resume_guest(irqfd->kvm);
> +}
> +
>  /**
>   * Initialize Hyp-mode and memory mappings on all CPUs.
>   */
> -- 
> 2.11.0
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-10-30  6:51     ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:51 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: linux-arm-kernel, kvmarm, kvm, linux-kernel, Andre Przywara

On Fri, Oct 27, 2017 at 03:28:53PM +0100, Marc Zyngier wrote:
> We so far allocate the doorbell interrupts without taking any
> special measure regarding the affinity of these interrupts. We
> simply move them around as required when the vcpu gets scheduled
> on a different CPU.
> 
> But that's counting without userspace (and the evil irqbalance) that
> can try and move the VPE interrupt around, causing the ITS code
> to emit VMOVP commands and remap the doorbell to another redistributor.
> Worse, this can happen while the vcpu is running, causing all kind
> of trouble if the VPE is already resident, and we end-up in UNPRED
> territory.
> 
> So let's take a definitive action and prevent userspace from messing
> with us. This is just a matter of adding IRQ_NO_BALANCING to the
> set of flags we already have, letting the kernel in sole control
> of the affinity.
> 

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d7fe610bb1f5..d10e18eabd3b 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,8 @@
>  
>  #include "vgic.h"
>  
> +#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
> +
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
>  {
>  	struct kvm_vcpu *vcpu = info;
> @@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
>  		 * doorbell could kick us out of the guest too
>  		 * early...
>  		 */
> -		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_set_status_flags(irq, DB_IRQ_FLAGS);
>  		ret = request_irq(irq, vgic_v4_doorbell_handler,
>  				  0, "vcpu", vcpu);
>  		if (ret) {
> @@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
>  		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
>  		int irq = its_vm->vpes[i]->irq;
>  
> -		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
>  		free_irq(irq, vcpu);
>  	}
>  
> -- 
> 2.11.0
> 
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
@ 2017-10-30  6:51     ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:51 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: linux-kernel, Andre Przywara, kvmarm, linux-arm-kernel, kvm

On Fri, Oct 27, 2017 at 03:28:53PM +0100, Marc Zyngier wrote:
> We so far allocate the doorbell interrupts without taking any
> special measure regarding the affinity of these interrupts. We
> simply move them around as required when the vcpu gets scheduled
> on a different CPU.
> 
> But that's counting without userspace (and the evil irqbalance) that
> can try and move the VPE interrupt around, causing the ITS code
> to emit VMOVP commands and remap the doorbell to another redistributor.
> Worse, this can happen while the vcpu is running, causing all kind
> of trouble if the VPE is already resident, and we end-up in UNPRED
> territory.
> 
> So let's take a definitive action and prevent userspace from messing
> with us. This is just a matter of adding IRQ_NO_BALANCING to the
> set of flags we already have, letting the kernel in sole control
> of the affinity.
> 

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d7fe610bb1f5..d10e18eabd3b 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,8 @@
>  
>  #include "vgic.h"
>  
> +#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
> +
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
>  {
>  	struct kvm_vcpu *vcpu = info;
> @@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
>  		 * doorbell could kick us out of the guest too
>  		 * early...
>  		 */
> -		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_set_status_flags(irq, DB_IRQ_FLAGS);
>  		ret = request_irq(irq, vgic_v4_doorbell_handler,
>  				  0, "vcpu", vcpu);
>  		if (ret) {
> @@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
>  		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
>  		int irq = its_vm->vpes[i]->irq;
>  
> -		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
>  		free_irq(irq, vcpu);
>  	}
>  
> -- 
> 2.11.0
> 
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
@ 2017-10-30  6:51     ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-10-30  6:51 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Oct 27, 2017 at 03:28:53PM +0100, Marc Zyngier wrote:
> We so far allocate the doorbell interrupts without taking any
> special measure regarding the affinity of these interrupts. We
> simply move them around as required when the vcpu gets scheduled
> on a different CPU.
> 
> But that's counting without userspace (and the evil irqbalance) that
> can try and move the VPE interrupt around, causing the ITS code
> to emit VMOVP commands and remap the doorbell to another redistributor.
> Worse, this can happen while the vcpu is running, causing all kind
> of trouble if the VPE is already resident, and we end-up in UNPRED
> territory.
> 
> So let's take a definitive action and prevent userspace from messing
> with us. This is just a matter of adding IRQ_NO_BALANCING to the
> set of flags we already have, letting the kernel in sole control
> of the affinity.
> 

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d7fe610bb1f5..d10e18eabd3b 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,8 @@
>  
>  #include "vgic.h"
>  
> +#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
> +
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
>  {
>  	struct kvm_vcpu *vcpu = info;
> @@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
>  		 * doorbell could kick us out of the guest too
>  		 * early...
>  		 */
> -		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_set_status_flags(irq, DB_IRQ_FLAGS);
>  		ret = request_irq(irq, vgic_v4_doorbell_handler,
>  				  0, "vcpu", vcpu);
>  		if (ret) {
> @@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
>  		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
>  		int irq = its_vm->vpes[i]->irq;
>  
> -		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
>  		free_irq(irq, vcpu);
>  	}
>  
> -- 
> 2.11.0
> 
> _______________________________________________
> kvmarm mailing list
> kvmarm at lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 13:06     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:06 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Let's use the irq bypass mechanism introduced for platform device
> interrupts
nit: I would remove "introduced for platform device interrupts"
as this is not upstream yet. x86 posted interrupts also use it.

>
 and establish our LPI->VLPI mapping.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  include/kvm/arm_vgic.h      |   8 ++++
>  virt/kvm/arm/arm.c          |   6 ++-
>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 120 insertions(+), 2 deletions(-)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 7eeb6c2a2f9c..2f750c770bf2 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>  
>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>  
> +struct kvm_kernel_irq_routing_entry;
> +
> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
> +
> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
> +
>  #endif /* __KVM_ARM_VGIC_H */
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 5d5218ecd547..8388c1cc23f6 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>  	struct kvm_kernel_irqfd *irqfd =
>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>  
> -	return 0;
> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
> +					  &irqfd->irq_entry);
>  }
>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>  				      struct irq_bypass_producer *prod)
> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>  	struct kvm_kernel_irqfd *irqfd =
>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>  
> -	return;
> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
> +				     &irqfd->irq_entry);
>  }
>  
>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index c794f0cef09e..01a2889b7b7c 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -18,6 +18,7 @@
>  #include <linux/interrupt.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kvm_host.h>
> +#include <linux/irqchip/arm-gic-v3.h>
>  
>  #include "vgic.h"
>  
> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>  	its_vm->nr_vpes = 0;
>  	its_vm->vpes = NULL;
>  }
> +
> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct kvm_msi msi  = (struct kvm_msi) {
> +		.address_lo	= irq_entry->msi.address_lo,
> +		.address_hi	= irq_entry->msi.address_hi,
> +		.data		= irq_entry->msi.data,
> +		.flags		= irq_entry->msi.flags,
> +		.devid		= irq_entry->msi.devid,
> +	};
> +
> +	/*
> +	 * Get a reference on the LPI. If NULL, this is not a valid
> +	 * translation for any of our vITSs.
> +	 */
I don't understand the relevance of the above comment.
> +	return vgic_msi_to_its(kvm, &msi);
> +}
> +
> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
@virq is the host linux irq. virq naming is a bit confusing to me.
> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct vgic_its *its;
> +	struct vgic_irq *irq;
> +	struct its_vlpi_map map;
> +	int ret;
> +
Don't you need to check that the linux irq (virq) is an LPI? You may
encounter some VFIO "producers" for irq that are not LPIs, typically if
we eventually upstream SPI forwarding.
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0;
> +
> +	/*
> +	 * Get the ITS, and escape early on error (not a valid
> +	 * doorbell for any of our vITSs).
> +	 */
> +	its = vgic_get_its(kvm, irq_entry);
> +	if (IS_ERR(its))
> +		return 0;
> +
> +	mutex_lock(&its->its_lock);
wouldn't it be safer to take the its_lock before the get_its()? and even
before the vgic_supports_direct_msis, in an unlikely case where the its
would disappear?
> +
> +	/* Perform then actual DevID/EventID -> LPI translation. */
> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
> +				   irq_entry->msi.data, &irq);
> +	if (ret)
> +		goto out;
> +
> +	/*
> +	 * Emit the mapping request. If it fails, the ITS probably
> +	 * isn't v4 compatible, so let's silently bail out. Holding
> +	 * the ITS lock should ensure that nothing can modify the
> +	 * target vcpu.
> +	 */
> +	map = (struct its_vlpi_map) {
> +		.vm		= &kvm->arch.vgic.its_vm,
> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
> +		.vintid		= irq->intid,
> +		.properties	= ((irq->priority & 0xfc) |
> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
> +				   LPI_PROP_GROUP1),
there is an inconsistency between the comment in irqchip/arm-gic-v4.h
and this setting:

* @properties: Priority and enable bits (as written in the prop table)

Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?
> +		.db_enabled	= true,
> +	};
> +
> +	ret = its_map_vlpi(virq, &map);
Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
fails, will you get a change to turn  IRQ_DISABLE_LAZY again.
> +	if (ret)
> +		goto out;
> +
> +	irq->hw		= true;
> +	irq->host_irq	= virq;
Shouldn't we theoretically hold the irq lock here?
> +
> +out:
> +	mutex_unlock(&its->its_lock);
> +	return ret;
> +}
> +
> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
same
> +				 struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct vgic_its *its;
> +	struct vgic_irq *irq;
> +	int ret;
> +
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0;
> +
> +	/*
> +	 * Get the ITS, and escape early on error (not a valid
> +	 * doorbell for any of our vITSs).
> +	 */
> +	its = vgic_get_its(kvm, irq_entry);
> +	if (IS_ERR(its))
> +		return 0;
> +
> +	mutex_lock(&its->its_lock);
same?

Thanks

Eric
> +
> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
> +				   irq_entry->msi.data, &irq);
> +	if (ret)
> +		goto out;
> +
> +	WARN_ON(!(irq->hw && irq->host_irq == virq));
> +	irq->hw = false;
> +	ret = its_unmap_vlpi(virq);
> +
> +out:
> +	mutex_unlock(&its->its_lock);
> +	return ret;
> +}
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-07 13:06     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:06 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Let's use the irq bypass mechanism introduced for platform device
> interrupts
nit: I would remove "introduced for platform device interrupts"
as this is not upstream yet. x86 posted interrupts also use it.

>
 and establish our LPI->VLPI mapping.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  include/kvm/arm_vgic.h      |   8 ++++
>  virt/kvm/arm/arm.c          |   6 ++-
>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 120 insertions(+), 2 deletions(-)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 7eeb6c2a2f9c..2f750c770bf2 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>  
>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>  
> +struct kvm_kernel_irq_routing_entry;
> +
> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
> +
> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
> +
>  #endif /* __KVM_ARM_VGIC_H */
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 5d5218ecd547..8388c1cc23f6 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>  	struct kvm_kernel_irqfd *irqfd =
>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>  
> -	return 0;
> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
> +					  &irqfd->irq_entry);
>  }
>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>  				      struct irq_bypass_producer *prod)
> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>  	struct kvm_kernel_irqfd *irqfd =
>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>  
> -	return;
> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
> +				     &irqfd->irq_entry);
>  }
>  
>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index c794f0cef09e..01a2889b7b7c 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -18,6 +18,7 @@
>  #include <linux/interrupt.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kvm_host.h>
> +#include <linux/irqchip/arm-gic-v3.h>
>  
>  #include "vgic.h"
>  
> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>  	its_vm->nr_vpes = 0;
>  	its_vm->vpes = NULL;
>  }
> +
> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct kvm_msi msi  = (struct kvm_msi) {
> +		.address_lo	= irq_entry->msi.address_lo,
> +		.address_hi	= irq_entry->msi.address_hi,
> +		.data		= irq_entry->msi.data,
> +		.flags		= irq_entry->msi.flags,
> +		.devid		= irq_entry->msi.devid,
> +	};
> +
> +	/*
> +	 * Get a reference on the LPI. If NULL, this is not a valid
> +	 * translation for any of our vITSs.
> +	 */
I don't understand the relevance of the above comment.
> +	return vgic_msi_to_its(kvm, &msi);
> +}
> +
> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
@virq is the host linux irq. virq naming is a bit confusing to me.
> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct vgic_its *its;
> +	struct vgic_irq *irq;
> +	struct its_vlpi_map map;
> +	int ret;
> +
Don't you need to check that the linux irq (virq) is an LPI? You may
encounter some VFIO "producers" for irq that are not LPIs, typically if
we eventually upstream SPI forwarding.
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0;
> +
> +	/*
> +	 * Get the ITS, and escape early on error (not a valid
> +	 * doorbell for any of our vITSs).
> +	 */
> +	its = vgic_get_its(kvm, irq_entry);
> +	if (IS_ERR(its))
> +		return 0;
> +
> +	mutex_lock(&its->its_lock);
wouldn't it be safer to take the its_lock before the get_its()? and even
before the vgic_supports_direct_msis, in an unlikely case where the its
would disappear?
> +
> +	/* Perform then actual DevID/EventID -> LPI translation. */
> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
> +				   irq_entry->msi.data, &irq);
> +	if (ret)
> +		goto out;
> +
> +	/*
> +	 * Emit the mapping request. If it fails, the ITS probably
> +	 * isn't v4 compatible, so let's silently bail out. Holding
> +	 * the ITS lock should ensure that nothing can modify the
> +	 * target vcpu.
> +	 */
> +	map = (struct its_vlpi_map) {
> +		.vm		= &kvm->arch.vgic.its_vm,
> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
> +		.vintid		= irq->intid,
> +		.properties	= ((irq->priority & 0xfc) |
> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
> +				   LPI_PROP_GROUP1),
there is an inconsistency between the comment in irqchip/arm-gic-v4.h
and this setting:

* @properties: Priority and enable bits (as written in the prop table)

Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?
> +		.db_enabled	= true,
> +	};
> +
> +	ret = its_map_vlpi(virq, &map);
Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
fails, will you get a change to turn  IRQ_DISABLE_LAZY again.
> +	if (ret)
> +		goto out;
> +
> +	irq->hw		= true;
> +	irq->host_irq	= virq;
Shouldn't we theoretically hold the irq lock here?
> +
> +out:
> +	mutex_unlock(&its->its_lock);
> +	return ret;
> +}
> +
> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
same
> +				 struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct vgic_its *its;
> +	struct vgic_irq *irq;
> +	int ret;
> +
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0;
> +
> +	/*
> +	 * Get the ITS, and escape early on error (not a valid
> +	 * doorbell for any of our vITSs).
> +	 */
> +	its = vgic_get_its(kvm, irq_entry);
> +	if (IS_ERR(its))
> +		return 0;
> +
> +	mutex_lock(&its->its_lock);
same?

Thanks

Eric
> +
> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
> +				   irq_entry->msi.data, &irq);
> +	if (ret)
> +		goto out;
> +
> +	WARN_ON(!(irq->hw && irq->host_irq == virq));
> +	irq->hw = false;
> +	ret = its_unmap_vlpi(virq);
> +
> +out:
> +	mutex_unlock(&its->its_lock);
> +	return ret;
> +}
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-07 13:06     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:06 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Let's use the irq bypass mechanism introduced for platform device
> interrupts
nit: I would remove "introduced for platform device interrupts"
as this is not upstream yet. x86 posted interrupts also use it.

>
 and establish our LPI->VLPI mapping.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  include/kvm/arm_vgic.h      |   8 ++++
>  virt/kvm/arm/arm.c          |   6 ++-
>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 120 insertions(+), 2 deletions(-)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 7eeb6c2a2f9c..2f750c770bf2 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>  
>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>  
> +struct kvm_kernel_irq_routing_entry;
> +
> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
> +
> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
> +
>  #endif /* __KVM_ARM_VGIC_H */
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 5d5218ecd547..8388c1cc23f6 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>  	struct kvm_kernel_irqfd *irqfd =
>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>  
> -	return 0;
> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
> +					  &irqfd->irq_entry);
>  }
>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>  				      struct irq_bypass_producer *prod)
> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>  	struct kvm_kernel_irqfd *irqfd =
>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>  
> -	return;
> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
> +				     &irqfd->irq_entry);
>  }
>  
>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index c794f0cef09e..01a2889b7b7c 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -18,6 +18,7 @@
>  #include <linux/interrupt.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kvm_host.h>
> +#include <linux/irqchip/arm-gic-v3.h>
>  
>  #include "vgic.h"
>  
> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>  	its_vm->nr_vpes = 0;
>  	its_vm->vpes = NULL;
>  }
> +
> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct kvm_msi msi  = (struct kvm_msi) {
> +		.address_lo	= irq_entry->msi.address_lo,
> +		.address_hi	= irq_entry->msi.address_hi,
> +		.data		= irq_entry->msi.data,
> +		.flags		= irq_entry->msi.flags,
> +		.devid		= irq_entry->msi.devid,
> +	};
> +
> +	/*
> +	 * Get a reference on the LPI. If NULL, this is not a valid
> +	 * translation for any of our vITSs.
> +	 */
I don't understand the relevance of the above comment.
> +	return vgic_msi_to_its(kvm, &msi);
> +}
> +
> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
@virq is the host linux irq. virq naming is a bit confusing to me.
> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct vgic_its *its;
> +	struct vgic_irq *irq;
> +	struct its_vlpi_map map;
> +	int ret;
> +
Don't you need to check that the linux irq (virq) is an LPI? You may
encounter some VFIO "producers" for irq that are not LPIs, typically if
we eventually upstream SPI forwarding.
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0;
> +
> +	/*
> +	 * Get the ITS, and escape early on error (not a valid
> +	 * doorbell for any of our vITSs).
> +	 */
> +	its = vgic_get_its(kvm, irq_entry);
> +	if (IS_ERR(its))
> +		return 0;
> +
> +	mutex_lock(&its->its_lock);
wouldn't it be safer to take the its_lock before the get_its()? and even
before the vgic_supports_direct_msis, in an unlikely case where the its
would disappear?
> +
> +	/* Perform then actual DevID/EventID -> LPI translation. */
> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
> +				   irq_entry->msi.data, &irq);
> +	if (ret)
> +		goto out;
> +
> +	/*
> +	 * Emit the mapping request. If it fails, the ITS probably
> +	 * isn't v4 compatible, so let's silently bail out. Holding
> +	 * the ITS lock should ensure that nothing can modify the
> +	 * target vcpu.
> +	 */
> +	map = (struct its_vlpi_map) {
> +		.vm		= &kvm->arch.vgic.its_vm,
> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
> +		.vintid		= irq->intid,
> +		.properties	= ((irq->priority & 0xfc) |
> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
> +				   LPI_PROP_GROUP1),
there is an inconsistency between the comment in irqchip/arm-gic-v4.h
and this setting:

* @properties: Priority and enable bits (as written in the prop table)

Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?
> +		.db_enabled	= true,
> +	};
> +
> +	ret = its_map_vlpi(virq, &map);
Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
fails, will you get a change to turn  IRQ_DISABLE_LAZY again.
> +	if (ret)
> +		goto out;
> +
> +	irq->hw		= true;
> +	irq->host_irq	= virq;
Shouldn't we theoretically hold the irq lock here?
> +
> +out:
> +	mutex_unlock(&its->its_lock);
> +	return ret;
> +}
> +
> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
same
> +				 struct kvm_kernel_irq_routing_entry *irq_entry)
> +{
> +	struct vgic_its *its;
> +	struct vgic_irq *irq;
> +	int ret;
> +
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0;
> +
> +	/*
> +	 * Get the ITS, and escape early on error (not a valid
> +	 * doorbell for any of our vITSs).
> +	 */
> +	its = vgic_get_its(kvm, irq_entry);
> +	if (IS_ERR(its))
> +		return 0;
> +
> +	mutex_lock(&its->its_lock);
same?

Thanks

Eric
> +
> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
> +				   irq_entry->msi.data, &irq);
> +	if (ret)
> +		goto out;
> +
> +	WARN_ON(!(irq->hw && irq->host_irq == virq));
> +	irq->hw = false;
> +	ret = its_unmap_vlpi(virq);
> +
> +out:
> +	mutex_unlock(&its->its_lock);
> +	return ret;
> +}
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 13:08     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:08 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order to control the GICv4 view of virtual CPUs, we rely
> on an irqdomain allocated for that purpose. Let's add a couple
> of helpers to that effect.
> 
> At the same time, the vgic data structures gain new fields to
> track all this... erm... wonderful stuff.
> 
> The way we hook into the vgic init is slightly convoluted. We
> need the vgic to be initialized (in order to guarantee that
> the number of vcpus is now fixed), and we must have a vITS
> (otherwise this is all very pointless). So we end-up calling
> the init from both vgic_init and vgic_its_create.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Makefile         |  1 +
>  arch/arm64/kvm/Makefile       |  1 +
>  include/kvm/arm_vgic.h        | 19 ++++++++++
>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>  7 files changed, 123 insertions(+)
>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> 
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index d9beee652d36..0a1dd2cdb928 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 5d9810086c25..c30fd388ef80 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index ba9fb450aa1b..7eeb6c2a2f9c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -26,6 +26,8 @@
>  #include <linux/list.h>
>  #include <linux/jump_label.h>
>  
> +#include <linux/irqchip/arm-gic-v4.h>
> +
>  #define VGIC_V3_MAX_CPUS	255
>  #define VGIC_V2_MAX_CPUS	8
>  #define VGIC_NR_IRQS_LEGACY     256
> @@ -236,6 +238,15 @@ struct vgic_dist {
>  
>  	/* used by vgic-debug */
>  	struct vgic_state_iter *iter;
> +
> +	/*
> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> +	 * array, the property table pointer as well as allocation
> +	 * data. This essentially ties the Linux IRQ core and ITS
> +	 * together, and avoids leaking KVM's data structures anywhere
> +	 * else.
> +	 */
> +	struct its_vm		its_vm;
>  };
>  
>  struct vgic_v2_cpu_if {
> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> +
> +	/*
> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> +	 * pending table pointer, the its_vm pointer and a few other
> +	 * HW specific things. As for the its_vm structure, this is
> +	 * linking the Linux IRQ subsystem and the ITS together.
> +	 */
> +	struct its_vpe	its_vpe;
>  };
>  
>  struct vgic_cpu {
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f3add..40be908da238 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> +	if (vgic_supports_direct_msis(kvm)) {
> +		ret = vgic_v4_init(kvm);
> +		if (ret)
> +			goto out;
> +	}
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
>  
> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>  
>  	kfree(dist->spis);
>  	dist->nr_spis = 0;
> +
> +	if (vgic_supports_direct_msis(kvm))
> +		vgic_v4_teardown(kvm);
>  }
>  
>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 8ee03f1e89fc..89768d2b6a91 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  	if (!its)
>  		return -ENOMEM;
>  
> +	if (vgic_initialized(dev->kvm)) {
Don't we need to test vgic_supports_direct_msis() on this path too?

Thanks

Eric
> +		int ret = vgic_v4_init(dev->kvm);
> +		if (ret) {
> +			kfree(its);
> +			return ret;
> +		}
> +	}
> +	
>  	mutex_init(&its->its_lock);
>  	mutex_init(&its->cmd_lock);
>  
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> new file mode 100644
> index 000000000000..c794f0cef09e
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -0,0 +1,83 @@
> +/*
> + * Copyright (C) 2017 ARM Ltd.
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/irqdomain.h>
> +#include <linux/kvm_host.h>
> +
> +#include "vgic.h"
> +
> +/**
> + * vgic_v4_init - Initialize the GICv4 data structures
> + * @kvm:	Pointer to the VM being initialized
> + *
> + * We may be called each time a vITS is created, or when the
> + * vgic is initialized. This relies on kvm->lock to be
> + * held. In both cases, the number of vcpus should now be
> + * fixed.
> + */
> +int vgic_v4_init(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct kvm_vcpu *vcpu;
> +	int i, nr_vcpus, ret;
> +
> +	if (dist->its_vm.vpes)
> +		return 0;
> +
> +	nr_vcpus = atomic_read(&kvm->online_vcpus);
> +
> +	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
> +				    GFP_KERNEL);
> +	if (!dist->its_vm.vpes)
> +		return -ENOMEM;
> +
> +	dist->its_vm.nr_vpes = nr_vcpus;
> +
> +	kvm_for_each_vcpu(i, vcpu, kvm)
> +		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +	ret = its_alloc_vcpu_irqs(&dist->its_vm);
> +	if (ret < 0) {
> +		kvm_err("VPE IRQ allocation failure\n");
> +		kfree(dist->its_vm.vpes);
> +		dist->its_vm.nr_vpes = 0;
> +		dist->its_vm.vpes = NULL;
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * vgic_v4_teardown - Free the GICv4 data structures
> + * @kvm:	Pointer to the VM being destroyed
> + *
> + * Relies on kvm->lock to be held.
> + */
> +void vgic_v4_teardown(struct kvm *kvm)
> +{
> +	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +
> +	if (!its_vm->vpes)
> +		return;
> +
> +	its_free_vcpu_irqs(its_vm);
> +	kfree(its_vm->vpes);
> +	its_vm->nr_vpes = 0;
> +	its_vm->vpes = NULL;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index e67ccb6a6250..c4105f613f57 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
>  bool vgic_supports_direct_msis(struct kvm *kvm);
> +int vgic_v4_init(struct kvm *kvm);
> +void vgic_v4_teardown(struct kvm *kvm);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-07 13:08     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:08 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order to control the GICv4 view of virtual CPUs, we rely
> on an irqdomain allocated for that purpose. Let's add a couple
> of helpers to that effect.
> 
> At the same time, the vgic data structures gain new fields to
> track all this... erm... wonderful stuff.
> 
> The way we hook into the vgic init is slightly convoluted. We
> need the vgic to be initialized (in order to guarantee that
> the number of vcpus is now fixed), and we must have a vITS
> (otherwise this is all very pointless). So we end-up calling
> the init from both vgic_init and vgic_its_create.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Makefile         |  1 +
>  arch/arm64/kvm/Makefile       |  1 +
>  include/kvm/arm_vgic.h        | 19 ++++++++++
>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>  7 files changed, 123 insertions(+)
>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> 
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index d9beee652d36..0a1dd2cdb928 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 5d9810086c25..c30fd388ef80 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index ba9fb450aa1b..7eeb6c2a2f9c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -26,6 +26,8 @@
>  #include <linux/list.h>
>  #include <linux/jump_label.h>
>  
> +#include <linux/irqchip/arm-gic-v4.h>
> +
>  #define VGIC_V3_MAX_CPUS	255
>  #define VGIC_V2_MAX_CPUS	8
>  #define VGIC_NR_IRQS_LEGACY     256
> @@ -236,6 +238,15 @@ struct vgic_dist {
>  
>  	/* used by vgic-debug */
>  	struct vgic_state_iter *iter;
> +
> +	/*
> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> +	 * array, the property table pointer as well as allocation
> +	 * data. This essentially ties the Linux IRQ core and ITS
> +	 * together, and avoids leaking KVM's data structures anywhere
> +	 * else.
> +	 */
> +	struct its_vm		its_vm;
>  };
>  
>  struct vgic_v2_cpu_if {
> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> +
> +	/*
> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> +	 * pending table pointer, the its_vm pointer and a few other
> +	 * HW specific things. As for the its_vm structure, this is
> +	 * linking the Linux IRQ subsystem and the ITS together.
> +	 */
> +	struct its_vpe	its_vpe;
>  };
>  
>  struct vgic_cpu {
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f3add..40be908da238 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> +	if (vgic_supports_direct_msis(kvm)) {
> +		ret = vgic_v4_init(kvm);
> +		if (ret)
> +			goto out;
> +	}
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
>  
> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>  
>  	kfree(dist->spis);
>  	dist->nr_spis = 0;
> +
> +	if (vgic_supports_direct_msis(kvm))
> +		vgic_v4_teardown(kvm);
>  }
>  
>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 8ee03f1e89fc..89768d2b6a91 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  	if (!its)
>  		return -ENOMEM;
>  
> +	if (vgic_initialized(dev->kvm)) {
Don't we need to test vgic_supports_direct_msis() on this path too?

Thanks

Eric
> +		int ret = vgic_v4_init(dev->kvm);
> +		if (ret) {
> +			kfree(its);
> +			return ret;
> +		}
> +	}
> +	
>  	mutex_init(&its->its_lock);
>  	mutex_init(&its->cmd_lock);
>  
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> new file mode 100644
> index 000000000000..c794f0cef09e
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -0,0 +1,83 @@
> +/*
> + * Copyright (C) 2017 ARM Ltd.
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/irqdomain.h>
> +#include <linux/kvm_host.h>
> +
> +#include "vgic.h"
> +
> +/**
> + * vgic_v4_init - Initialize the GICv4 data structures
> + * @kvm:	Pointer to the VM being initialized
> + *
> + * We may be called each time a vITS is created, or when the
> + * vgic is initialized. This relies on kvm->lock to be
> + * held. In both cases, the number of vcpus should now be
> + * fixed.
> + */
> +int vgic_v4_init(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct kvm_vcpu *vcpu;
> +	int i, nr_vcpus, ret;
> +
> +	if (dist->its_vm.vpes)
> +		return 0;
> +
> +	nr_vcpus = atomic_read(&kvm->online_vcpus);
> +
> +	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
> +				    GFP_KERNEL);
> +	if (!dist->its_vm.vpes)
> +		return -ENOMEM;
> +
> +	dist->its_vm.nr_vpes = nr_vcpus;
> +
> +	kvm_for_each_vcpu(i, vcpu, kvm)
> +		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +	ret = its_alloc_vcpu_irqs(&dist->its_vm);
> +	if (ret < 0) {
> +		kvm_err("VPE IRQ allocation failure\n");
> +		kfree(dist->its_vm.vpes);
> +		dist->its_vm.nr_vpes = 0;
> +		dist->its_vm.vpes = NULL;
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * vgic_v4_teardown - Free the GICv4 data structures
> + * @kvm:	Pointer to the VM being destroyed
> + *
> + * Relies on kvm->lock to be held.
> + */
> +void vgic_v4_teardown(struct kvm *kvm)
> +{
> +	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +
> +	if (!its_vm->vpes)
> +		return;
> +
> +	its_free_vcpu_irqs(its_vm);
> +	kfree(its_vm->vpes);
> +	its_vm->nr_vpes = 0;
> +	its_vm->vpes = NULL;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index e67ccb6a6250..c4105f613f57 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
>  bool vgic_supports_direct_msis(struct kvm *kvm);
> +int vgic_v4_init(struct kvm *kvm);
> +void vgic_v4_teardown(struct kvm *kvm);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-07 13:08     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:08 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order to control the GICv4 view of virtual CPUs, we rely
> on an irqdomain allocated for that purpose. Let's add a couple
> of helpers to that effect.
> 
> At the same time, the vgic data structures gain new fields to
> track all this... erm... wonderful stuff.
> 
> The way we hook into the vgic init is slightly convoluted. We
> need the vgic to be initialized (in order to guarantee that
> the number of vcpus is now fixed), and we must have a vITS
> (otherwise this is all very pointless). So we end-up calling
> the init from both vgic_init and vgic_its_create.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Makefile         |  1 +
>  arch/arm64/kvm/Makefile       |  1 +
>  include/kvm/arm_vgic.h        | 19 ++++++++++
>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>  7 files changed, 123 insertions(+)
>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> 
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index d9beee652d36..0a1dd2cdb928 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 5d9810086c25..c30fd388ef80 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index ba9fb450aa1b..7eeb6c2a2f9c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -26,6 +26,8 @@
>  #include <linux/list.h>
>  #include <linux/jump_label.h>
>  
> +#include <linux/irqchip/arm-gic-v4.h>
> +
>  #define VGIC_V3_MAX_CPUS	255
>  #define VGIC_V2_MAX_CPUS	8
>  #define VGIC_NR_IRQS_LEGACY     256
> @@ -236,6 +238,15 @@ struct vgic_dist {
>  
>  	/* used by vgic-debug */
>  	struct vgic_state_iter *iter;
> +
> +	/*
> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> +	 * array, the property table pointer as well as allocation
> +	 * data. This essentially ties the Linux IRQ core and ITS
> +	 * together, and avoids leaking KVM's data structures anywhere
> +	 * else.
> +	 */
> +	struct its_vm		its_vm;
>  };
>  
>  struct vgic_v2_cpu_if {
> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> +
> +	/*
> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> +	 * pending table pointer, the its_vm pointer and a few other
> +	 * HW specific things. As for the its_vm structure, this is
> +	 * linking the Linux IRQ subsystem and the ITS together.
> +	 */
> +	struct its_vpe	its_vpe;
>  };
>  
>  struct vgic_cpu {
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f3add..40be908da238 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> +	if (vgic_supports_direct_msis(kvm)) {
> +		ret = vgic_v4_init(kvm);
> +		if (ret)
> +			goto out;
> +	}
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
>  
> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>  
>  	kfree(dist->spis);
>  	dist->nr_spis = 0;
> +
> +	if (vgic_supports_direct_msis(kvm))
> +		vgic_v4_teardown(kvm);
>  }
>  
>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 8ee03f1e89fc..89768d2b6a91 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  	if (!its)
>  		return -ENOMEM;
>  
> +	if (vgic_initialized(dev->kvm)) {
Don't we need to test vgic_supports_direct_msis() on this path too?

Thanks

Eric
> +		int ret = vgic_v4_init(dev->kvm);
> +		if (ret) {
> +			kfree(its);
> +			return ret;
> +		}
> +	}
> +	
>  	mutex_init(&its->its_lock);
>  	mutex_init(&its->cmd_lock);
>  
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> new file mode 100644
> index 000000000000..c794f0cef09e
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -0,0 +1,83 @@
> +/*
> + * Copyright (C) 2017 ARM Ltd.
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/irqdomain.h>
> +#include <linux/kvm_host.h>
> +
> +#include "vgic.h"
> +
> +/**
> + * vgic_v4_init - Initialize the GICv4 data structures
> + * @kvm:	Pointer to the VM being initialized
> + *
> + * We may be called each time a vITS is created, or when the
> + * vgic is initialized. This relies on kvm->lock to be
> + * held. In both cases, the number of vcpus should now be
> + * fixed.
> + */
> +int vgic_v4_init(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct kvm_vcpu *vcpu;
> +	int i, nr_vcpus, ret;
> +
> +	if (dist->its_vm.vpes)
> +		return 0;
> +
> +	nr_vcpus = atomic_read(&kvm->online_vcpus);
> +
> +	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
> +				    GFP_KERNEL);
> +	if (!dist->its_vm.vpes)
> +		return -ENOMEM;
> +
> +	dist->its_vm.nr_vpes = nr_vcpus;
> +
> +	kvm_for_each_vcpu(i, vcpu, kvm)
> +		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +	ret = its_alloc_vcpu_irqs(&dist->its_vm);
> +	if (ret < 0) {
> +		kvm_err("VPE IRQ allocation failure\n");
> +		kfree(dist->its_vm.vpes);
> +		dist->its_vm.nr_vpes = 0;
> +		dist->its_vm.vpes = NULL;
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * vgic_v4_teardown - Free the GICv4 data structures
> + * @kvm:	Pointer to the VM being destroyed
> + *
> + * Relies on kvm->lock to be held.
> + */
> +void vgic_v4_teardown(struct kvm *kvm)
> +{
> +	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +
> +	if (!its_vm->vpes)
> +		return;
> +
> +	its_free_vcpu_irqs(its_vm);
> +	kfree(its_vm->vpes);
> +	its_vm->nr_vpes = 0;
> +	its_vm->vpes = NULL;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index e67ccb6a6250..c4105f613f57 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
>  bool vgic_supports_direct_msis(struct kvm *kvm);
> +int vgic_v4_init(struct kvm *kvm);
> +void vgic_v4_teardown(struct kvm *kvm);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 13:09     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:09 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order to control the GICv4 view of virtual CPUs, we rely
> on an irqdomain allocated for that purpose. Let's add a couple
> of helpers to that effect.
> 
> At the same time, the vgic data structures gain new fields to
> track all this... erm... wonderful stuff.
> 
> The way we hook into the vgic init is slightly convoluted. We
> need the vgic to be initialized (in order to guarantee that
> the number of vcpus is now fixed), and we must have a vITS
> (otherwise this is all very pointless). So we end-up calling
> the init from both vgic_init and vgic_its_create.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Makefile         |  1 +
>  arch/arm64/kvm/Makefile       |  1 +
>  include/kvm/arm_vgic.h        | 19 ++++++++++
>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>  7 files changed, 123 insertions(+)
>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> 
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index d9beee652d36..0a1dd2cdb928 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 5d9810086c25..c30fd388ef80 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index ba9fb450aa1b..7eeb6c2a2f9c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -26,6 +26,8 @@
>  #include <linux/list.h>
>  #include <linux/jump_label.h>
>  
> +#include <linux/irqchip/arm-gic-v4.h>
> +
>  #define VGIC_V3_MAX_CPUS	255
>  #define VGIC_V2_MAX_CPUS	8
>  #define VGIC_NR_IRQS_LEGACY     256
> @@ -236,6 +238,15 @@ struct vgic_dist {
>  
>  	/* used by vgic-debug */
>  	struct vgic_state_iter *iter;
> +
> +	/*
> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> +	 * array, the property table pointer as well as allocation
> +	 * data. This essentially ties the Linux IRQ core and ITS
> +	 * together, and avoids leaking KVM's data structures anywhere
> +	 * else.
> +	 */
> +	struct its_vm		its_vm;
>  };
>  
>  struct vgic_v2_cpu_if {
> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> +
> +	/*
> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> +	 * pending table pointer, the its_vm pointer and a few other
> +	 * HW specific things. As for the its_vm structure, this is
> +	 * linking the Linux IRQ subsystem and the ITS together.
> +	 */
> +	struct its_vpe	its_vpe;
>  };
>  
>  struct vgic_cpu {
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f3add..40be908da238 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> +	if (vgic_supports_direct_msis(kvm)) {
> +		ret = vgic_v4_init(kvm);
> +		if (ret)
> +			goto out;
> +	}
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
>  
> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>  
>  	kfree(dist->spis);
>  	dist->nr_spis = 0;
> +
> +	if (vgic_supports_direct_msis(kvm))
> +		vgic_v4_teardown(kvm);
>  }
>  
>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 8ee03f1e89fc..89768d2b6a91 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  	if (!its)
>  		return -ENOMEM;
>  
> +	if (vgic_initialized(dev->kvm)) {
> +		int ret = vgic_v4_init(dev->kvm);

If you respin I forgot 2 nits below:
nit: line to be added
> +		if (ret) {
> +			kfree(its);
> +			return ret;
> +		}
> +	}
> +	
>  	mutex_init(&its->its_lock);
>  	mutex_init(&its->cmd_lock);
>  
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> new file mode 100644
> index 000000000000..c794f0cef09e
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -0,0 +1,83 @@
> +/*
> + * Copyright (C) 2017 ARM Ltd.
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/irqdomain.h>
> +#include <linux/kvm_host.h>
> +
> +#include "vgic.h"
> +
> +/**
> + * vgic_v4_init - Initialize the GICv4 data structures
> + * @kvm:	Pointer to the VM being initialized
> + *
> + * We may be called each time a vITS is created, or when the
> + * vgic is initialized. This relies on kvm->lock to be
> + * held. In both cases, the number of vcpus should now be
> + * fixed.
> + */
> +int vgic_v4_init(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct kvm_vcpu *vcpu;
> +	int i, nr_vcpus, ret;
> +
> +	if (dist->its_vm.vpes)
> +		return 0;
> +
> +	nr_vcpus = atomic_read(&kvm->online_vcpus);
> +
> +	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
> +				    GFP_KERNEL);
> +	if (!dist->its_vm.vpes)
> +		return -ENOMEM;
> +
> +	dist->its_vm.nr_vpes = nr_vcpus;
> +
> +	kvm_for_each_vcpu(i, vcpu, kvm)
> +		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +	ret = its_alloc_vcpu_irqs(&dist->its_vm);
> +	if (ret < 0) {
> +		kvm_err("VPE IRQ allocation failure\n");
> +		kfree(dist->its_vm.vpes);
> +		dist->its_vm.nr_vpes = 0;
> +		dist->its_vm.vpes = NULL;
> +		return ret;
nit: not requested

Thanks

Eric
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * vgic_v4_teardown - Free the GICv4 data structures
> + * @kvm:	Pointer to the VM being destroyed
> + *
> + * Relies on kvm->lock to be held.
> + */
> +void vgic_v4_teardown(struct kvm *kvm)
> +{
> +	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +
> +	if (!its_vm->vpes)
> +		return;
> +
> +	its_free_vcpu_irqs(its_vm);
> +	kfree(its_vm->vpes);
> +	its_vm->nr_vpes = 0;
> +	its_vm->vpes = NULL;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index e67ccb6a6250..c4105f613f57 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
>  bool vgic_supports_direct_msis(struct kvm *kvm);
> +int vgic_v4_init(struct kvm *kvm);
> +void vgic_v4_teardown(struct kvm *kvm);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-07 13:09     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:09 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order to control the GICv4 view of virtual CPUs, we rely
> on an irqdomain allocated for that purpose. Let's add a couple
> of helpers to that effect.
> 
> At the same time, the vgic data structures gain new fields to
> track all this... erm... wonderful stuff.
> 
> The way we hook into the vgic init is slightly convoluted. We
> need the vgic to be initialized (in order to guarantee that
> the number of vcpus is now fixed), and we must have a vITS
> (otherwise this is all very pointless). So we end-up calling
> the init from both vgic_init and vgic_its_create.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Makefile         |  1 +
>  arch/arm64/kvm/Makefile       |  1 +
>  include/kvm/arm_vgic.h        | 19 ++++++++++
>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>  7 files changed, 123 insertions(+)
>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> 
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index d9beee652d36..0a1dd2cdb928 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 5d9810086c25..c30fd388ef80 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index ba9fb450aa1b..7eeb6c2a2f9c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -26,6 +26,8 @@
>  #include <linux/list.h>
>  #include <linux/jump_label.h>
>  
> +#include <linux/irqchip/arm-gic-v4.h>
> +
>  #define VGIC_V3_MAX_CPUS	255
>  #define VGIC_V2_MAX_CPUS	8
>  #define VGIC_NR_IRQS_LEGACY     256
> @@ -236,6 +238,15 @@ struct vgic_dist {
>  
>  	/* used by vgic-debug */
>  	struct vgic_state_iter *iter;
> +
> +	/*
> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> +	 * array, the property table pointer as well as allocation
> +	 * data. This essentially ties the Linux IRQ core and ITS
> +	 * together, and avoids leaking KVM's data structures anywhere
> +	 * else.
> +	 */
> +	struct its_vm		its_vm;
>  };
>  
>  struct vgic_v2_cpu_if {
> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> +
> +	/*
> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> +	 * pending table pointer, the its_vm pointer and a few other
> +	 * HW specific things. As for the its_vm structure, this is
> +	 * linking the Linux IRQ subsystem and the ITS together.
> +	 */
> +	struct its_vpe	its_vpe;
>  };
>  
>  struct vgic_cpu {
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f3add..40be908da238 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> +	if (vgic_supports_direct_msis(kvm)) {
> +		ret = vgic_v4_init(kvm);
> +		if (ret)
> +			goto out;
> +	}
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
>  
> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>  
>  	kfree(dist->spis);
>  	dist->nr_spis = 0;
> +
> +	if (vgic_supports_direct_msis(kvm))
> +		vgic_v4_teardown(kvm);
>  }
>  
>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 8ee03f1e89fc..89768d2b6a91 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  	if (!its)
>  		return -ENOMEM;
>  
> +	if (vgic_initialized(dev->kvm)) {
> +		int ret = vgic_v4_init(dev->kvm);

If you respin I forgot 2 nits below:
nit: line to be added
> +		if (ret) {
> +			kfree(its);
> +			return ret;
> +		}
> +	}
> +	
>  	mutex_init(&its->its_lock);
>  	mutex_init(&its->cmd_lock);
>  
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> new file mode 100644
> index 000000000000..c794f0cef09e
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -0,0 +1,83 @@
> +/*
> + * Copyright (C) 2017 ARM Ltd.
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/irqdomain.h>
> +#include <linux/kvm_host.h>
> +
> +#include "vgic.h"
> +
> +/**
> + * vgic_v4_init - Initialize the GICv4 data structures
> + * @kvm:	Pointer to the VM being initialized
> + *
> + * We may be called each time a vITS is created, or when the
> + * vgic is initialized. This relies on kvm->lock to be
> + * held. In both cases, the number of vcpus should now be
> + * fixed.
> + */
> +int vgic_v4_init(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct kvm_vcpu *vcpu;
> +	int i, nr_vcpus, ret;
> +
> +	if (dist->its_vm.vpes)
> +		return 0;
> +
> +	nr_vcpus = atomic_read(&kvm->online_vcpus);
> +
> +	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
> +				    GFP_KERNEL);
> +	if (!dist->its_vm.vpes)
> +		return -ENOMEM;
> +
> +	dist->its_vm.nr_vpes = nr_vcpus;
> +
> +	kvm_for_each_vcpu(i, vcpu, kvm)
> +		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +	ret = its_alloc_vcpu_irqs(&dist->its_vm);
> +	if (ret < 0) {
> +		kvm_err("VPE IRQ allocation failure\n");
> +		kfree(dist->its_vm.vpes);
> +		dist->its_vm.nr_vpes = 0;
> +		dist->its_vm.vpes = NULL;
> +		return ret;
nit: not requested

Thanks

Eric
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * vgic_v4_teardown - Free the GICv4 data structures
> + * @kvm:	Pointer to the VM being destroyed
> + *
> + * Relies on kvm->lock to be held.
> + */
> +void vgic_v4_teardown(struct kvm *kvm)
> +{
> +	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +
> +	if (!its_vm->vpes)
> +		return;
> +
> +	its_free_vcpu_irqs(its_vm);
> +	kfree(its_vm->vpes);
> +	its_vm->nr_vpes = 0;
> +	its_vm->vpes = NULL;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index e67ccb6a6250..c4105f613f57 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
>  bool vgic_supports_direct_msis(struct kvm *kvm);
> +int vgic_v4_init(struct kvm *kvm);
> +void vgic_v4_teardown(struct kvm *kvm);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-07 13:09     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:09 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order to control the GICv4 view of virtual CPUs, we rely
> on an irqdomain allocated for that purpose. Let's add a couple
> of helpers to that effect.
> 
> At the same time, the vgic data structures gain new fields to
> track all this... erm... wonderful stuff.
> 
> The way we hook into the vgic init is slightly convoluted. We
> need the vgic to be initialized (in order to guarantee that
> the number of vcpus is now fixed), and we must have a vITS
> (otherwise this is all very pointless). So we end-up calling
> the init from both vgic_init and vgic_its_create.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm/kvm/Makefile         |  1 +
>  arch/arm64/kvm/Makefile       |  1 +
>  include/kvm/arm_vgic.h        | 19 ++++++++++
>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>  7 files changed, 123 insertions(+)
>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> 
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index d9beee652d36..0a1dd2cdb928 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 5d9810086c25..c30fd388ef80 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index ba9fb450aa1b..7eeb6c2a2f9c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -26,6 +26,8 @@
>  #include <linux/list.h>
>  #include <linux/jump_label.h>
>  
> +#include <linux/irqchip/arm-gic-v4.h>
> +
>  #define VGIC_V3_MAX_CPUS	255
>  #define VGIC_V2_MAX_CPUS	8
>  #define VGIC_NR_IRQS_LEGACY     256
> @@ -236,6 +238,15 @@ struct vgic_dist {
>  
>  	/* used by vgic-debug */
>  	struct vgic_state_iter *iter;
> +
> +	/*
> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> +	 * array, the property table pointer as well as allocation
> +	 * data. This essentially ties the Linux IRQ core and ITS
> +	 * together, and avoids leaking KVM's data structures anywhere
> +	 * else.
> +	 */
> +	struct its_vm		its_vm;
>  };
>  
>  struct vgic_v2_cpu_if {
> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> +
> +	/*
> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> +	 * pending table pointer, the its_vm pointer and a few other
> +	 * HW specific things. As for the its_vm structure, this is
> +	 * linking the Linux IRQ subsystem and the ITS together.
> +	 */
> +	struct its_vpe	its_vpe;
>  };
>  
>  struct vgic_cpu {
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f3add..40be908da238 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> +	if (vgic_supports_direct_msis(kvm)) {
> +		ret = vgic_v4_init(kvm);
> +		if (ret)
> +			goto out;
> +	}
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
>  
> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>  
>  	kfree(dist->spis);
>  	dist->nr_spis = 0;
> +
> +	if (vgic_supports_direct_msis(kvm))
> +		vgic_v4_teardown(kvm);
>  }
>  
>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 8ee03f1e89fc..89768d2b6a91 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  	if (!its)
>  		return -ENOMEM;
>  
> +	if (vgic_initialized(dev->kvm)) {
> +		int ret = vgic_v4_init(dev->kvm);

If you respin I forgot 2 nits below:
nit: line to be added
> +		if (ret) {
> +			kfree(its);
> +			return ret;
> +		}
> +	}
> +	
>  	mutex_init(&its->its_lock);
>  	mutex_init(&its->cmd_lock);
>  
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> new file mode 100644
> index 000000000000..c794f0cef09e
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -0,0 +1,83 @@
> +/*
> + * Copyright (C) 2017 ARM Ltd.
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/interrupt.h>
> +#include <linux/irqdomain.h>
> +#include <linux/kvm_host.h>
> +
> +#include "vgic.h"
> +
> +/**
> + * vgic_v4_init - Initialize the GICv4 data structures
> + * @kvm:	Pointer to the VM being initialized
> + *
> + * We may be called each time a vITS is created, or when the
> + * vgic is initialized. This relies on kvm->lock to be
> + * held. In both cases, the number of vcpus should now be
> + * fixed.
> + */
> +int vgic_v4_init(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct kvm_vcpu *vcpu;
> +	int i, nr_vcpus, ret;
> +
> +	if (dist->its_vm.vpes)
> +		return 0;
> +
> +	nr_vcpus = atomic_read(&kvm->online_vcpus);
> +
> +	dist->its_vm.vpes = kzalloc(sizeof(*dist->its_vm.vpes) * nr_vcpus,
> +				    GFP_KERNEL);
> +	if (!dist->its_vm.vpes)
> +		return -ENOMEM;
> +
> +	dist->its_vm.nr_vpes = nr_vcpus;
> +
> +	kvm_for_each_vcpu(i, vcpu, kvm)
> +		dist->its_vm.vpes[i] = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +	ret = its_alloc_vcpu_irqs(&dist->its_vm);
> +	if (ret < 0) {
> +		kvm_err("VPE IRQ allocation failure\n");
> +		kfree(dist->its_vm.vpes);
> +		dist->its_vm.nr_vpes = 0;
> +		dist->its_vm.vpes = NULL;
> +		return ret;
nit: not requested

Thanks

Eric
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * vgic_v4_teardown - Free the GICv4 data structures
> + * @kvm:	Pointer to the VM being destroyed
> + *
> + * Relies on kvm->lock to be held.
> + */
> +void vgic_v4_teardown(struct kvm *kvm)
> +{
> +	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +
> +	if (!its_vm->vpes)
> +		return;
> +
> +	its_free_vcpu_irqs(its_vm);
> +	kfree(its_vm->vpes);
> +	its_vm->nr_vpes = 0;
> +	its_vm->vpes = NULL;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index e67ccb6a6250..c4105f613f57 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -241,5 +241,7 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
>  bool vgic_supports_direct_msis(struct kvm *kvm);
> +int vgic_v4_init(struct kvm *kvm);
> +void vgic_v4_teardown(struct kvm *kvm);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 07/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 13:44     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:44 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order help integrating the vITS code with GICv4, let's add
in order to
> a new helper that deals with updating the affinity of an LPI,
> which will later be augmented with super duper extra GICv4
> goodness.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 20 +++++++++++---------
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 475dffd0a44c..8ee03f1e89fc 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -335,6 +335,15 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>  	return i;
>  }
>  
> +static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
> +{
> +	spin_lock(&irq->irq_lock);
> +	irq->target_vcpu = vcpu;
> +	spin_unlock(&irq->irq_lock);
> +
> +	return 0;
> +}
> +
>  /*
>   * Promotes the ITS view of affinity of an ITTE (which redistributor this LPI
>   * is targeting) to the VGIC's view, which deals with target VCPUs.
> @@ -349,10 +358,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
>  		return;
>  
>  	vcpu = kvm_get_vcpu(kvm, ite->collection->target_addr);
> -
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->target_vcpu = vcpu;
> -	spin_unlock(&ite->irq->irq_lock);
> +	update_affinity(ite->irq, vcpu);
>  }
>  
>  /*
> @@ -693,11 +699,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
>  	ite->collection = collection;
>  	vcpu = kvm_get_vcpu(kvm, collection->target_addr);
>  
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->target_vcpu = vcpu;
> -	spin_unlock(&ite->irq->irq_lock);
> -
> -	return 0;
> +	return update_affinity(ite->irq, vcpu);
>  }
>  
>  /*
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 07/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
@ 2017-11-07 13:44     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:44 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order help integrating the vITS code with GICv4, let's add
in order to
> a new helper that deals with updating the affinity of an LPI,
> which will later be augmented with super duper extra GICv4
> goodness.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 20 +++++++++++---------
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 475dffd0a44c..8ee03f1e89fc 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -335,6 +335,15 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>  	return i;
>  }
>  
> +static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
> +{
> +	spin_lock(&irq->irq_lock);
> +	irq->target_vcpu = vcpu;
> +	spin_unlock(&irq->irq_lock);
> +
> +	return 0;
> +}
> +
>  /*
>   * Promotes the ITS view of affinity of an ITTE (which redistributor this LPI
>   * is targeting) to the VGIC's view, which deals with target VCPUs.
> @@ -349,10 +358,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
>  		return;
>  
>  	vcpu = kvm_get_vcpu(kvm, ite->collection->target_addr);
> -
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->target_vcpu = vcpu;
> -	spin_unlock(&ite->irq->irq_lock);
> +	update_affinity(ite->irq, vcpu);
>  }
>  
>  /*
> @@ -693,11 +699,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
>  	ite->collection = collection;
>  	vcpu = kvm_get_vcpu(kvm, collection->target_addr);
>  
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->target_vcpu = vcpu;
> -	spin_unlock(&ite->irq->irq_lock);
> -
> -	return 0;
> +	return update_affinity(ite->irq, vcpu);
>  }
>  
>  /*
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 07/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI
@ 2017-11-07 13:44     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 13:44 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> In order help integrating the vITS code with GICv4, let's add
in order to
> a new helper that deals with updating the affinity of an LPI,
> which will later be augmented with super duper extra GICv4
> goodness.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 20 +++++++++++---------
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 475dffd0a44c..8ee03f1e89fc 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -335,6 +335,15 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>  	return i;
>  }
>  
> +static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
> +{
> +	spin_lock(&irq->irq_lock);
> +	irq->target_vcpu = vcpu;
> +	spin_unlock(&irq->irq_lock);
> +
> +	return 0;
> +}
> +
>  /*
>   * Promotes the ITS view of affinity of an ITTE (which redistributor this LPI
>   * is targeting) to the VGIC's view, which deals with target VCPUs.
> @@ -349,10 +358,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
>  		return;
>  
>  	vcpu = kvm_get_vcpu(kvm, ite->collection->target_addr);
> -
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->target_vcpu = vcpu;
> -	spin_unlock(&ite->irq->irq_lock);
> +	update_affinity(ite->irq, vcpu);
>  }
>  
>  /*
> @@ -693,11 +699,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
>  	ite->collection = collection;
>  	vcpu = kvm_get_vcpu(kvm, collection->target_addr);
>  
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->target_vcpu = vcpu;
> -	spin_unlock(&ite->irq->irq_lock);
> -
> -	return 0;
> +	return update_affinity(ite->irq, vcpu);
>  }
>  
>  /*
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-11-07 13:06     ` Auger Eric
  (?)
@ 2017-11-07 14:42       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 14:42 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi Eric,

On 07/11/17 13:06, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Let's use the irq bypass mechanism introduced for platform device
>> interrupts
> nit: I would remove "introduced for platform device interrupts"
> as this is not upstream yet. x86 posted interrupts also use it.
> 
>>
>  and establish our LPI->VLPI mapping.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  include/kvm/arm_vgic.h      |   8 ++++
>>  virt/kvm/arm/arm.c          |   6 ++-
>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>  
>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>  
>> +struct kvm_kernel_irq_routing_entry;
>> +
>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>> +
>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>> +
>>  #endif /* __KVM_ARM_VGIC_H */
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index 5d5218ecd547..8388c1cc23f6 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>  	struct kvm_kernel_irqfd *irqfd =
>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>  
>> -	return 0;
>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>> +					  &irqfd->irq_entry);
>>  }
>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>  				      struct irq_bypass_producer *prod)
>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>  	struct kvm_kernel_irqfd *irqfd =
>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>  
>> -	return;
>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>> +				     &irqfd->irq_entry);
>>  }
>>  
>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index c794f0cef09e..01a2889b7b7c 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -18,6 +18,7 @@
>>  #include <linux/interrupt.h>
>>  #include <linux/irqdomain.h>
>>  #include <linux/kvm_host.h>
>> +#include <linux/irqchip/arm-gic-v3.h>
>>  
>>  #include "vgic.h"
>>  
>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>  	its_vm->nr_vpes = 0;
>>  	its_vm->vpes = NULL;
>>  }
>> +
>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>> +{
>> +	struct kvm_msi msi  = (struct kvm_msi) {
>> +		.address_lo	= irq_entry->msi.address_lo,
>> +		.address_hi	= irq_entry->msi.address_hi,
>> +		.data		= irq_entry->msi.data,
>> +		.flags		= irq_entry->msi.flags,
>> +		.devid		= irq_entry->msi.devid,
>> +	};
>> +
>> +	/*
>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>> +	 * translation for any of our vITSs.
>> +	 */
> I don't understand the relevance of the above comment.

Hmmm. The first part looks like an outdated leftover, as the ITS is not
refcounted, and we don't deal with LPIs here.

>> +	return vgic_msi_to_its(kvm, &msi);
>> +}
>> +
>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
> @virq is the host linux irq. virq naming is a bit confusing to me.

There is plenty of irq-related code that uses this naming. In this
context, we tend to use irq for the vgic view, hwirq for the irqchip
view. How would you call this one?

>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>> +{
>> +	struct vgic_its *its;
>> +	struct vgic_irq *irq;
>> +	struct its_vlpi_map map;
>> +	int ret;
>> +
> Don't you need to check that the linux irq (virq) is an LPI? You may
> encounter some VFIO "producers" for irq that are not LPIs, typically if
> we eventually upstream SPI forwarding.

That's indeed a concern. The issue is that we don't really have a way to
check this, other than following the irq_data pointers and check that
the hwirq is within a certain range. And even that doesn't guarantee
anything.

>> +	if (!vgic_supports_direct_msis(kvm))
>> +		return 0;
>> +
>> +	/*
>> +	 * Get the ITS, and escape early on error (not a valid
>> +	 * doorbell for any of our vITSs).
>> +	 */
>> +	its = vgic_get_its(kvm, irq_entry);
>> +	if (IS_ERR(its))
>> +		return 0;
>> +
>> +	mutex_lock(&its->its_lock);
> wouldn't it be safer to take the its_lock before the get_its()? and even
> before the vgic_supports_direct_msis, in an unlikely case where the its
> would disappear?

How do you get a lock on the ITS before getting a pointer to it? Also,
aren't we in the context of a vcpu here (we trap to userspace, come back
via a VFIO ioctl, and arrive here)? Could the ITS actually vanish from
under our feet here? The only way to do it would be to destroy the VM.


>> +
>> +	/* Perform then actual DevID/EventID -> LPI translation. */
>> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
>> +				   irq_entry->msi.data, &irq);
>> +	if (ret)
>> +		goto out;
>> +
>> +	/*
>> +	 * Emit the mapping request. If it fails, the ITS probably
>> +	 * isn't v4 compatible, so let's silently bail out. Holding
>> +	 * the ITS lock should ensure that nothing can modify the
>> +	 * target vcpu.
>> +	 */
>> +	map = (struct its_vlpi_map) {
>> +		.vm		= &kvm->arch.vgic.its_vm,
>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>> +		.vintid		= irq->intid,
>> +		.properties	= ((irq->priority & 0xfc) |
>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>> +				   LPI_PROP_GROUP1),
> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
> and this setting:
> 
> * @properties: Priority and enable bits (as written in the prop table)

Which inconsistency?

> Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?

We could. But given that irq->priority is assigned with
LPI_PROP_PRIORITY(), we could drop the 0xfc altogether.

>> +		.db_enabled	= true,
>> +	};
>> +
>> +	ret = its_map_vlpi(virq, &map);
> Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
> fails, will you get a change to turn  IRQ_DISABLE_LAZY again.

Good point. I'll fix this.

>> +	if (ret)
>> +		goto out;
>> +
>> +	irq->hw		= true;
>> +	irq->host_irq	= virq;
> Shouldn't we theoretically hold the irq lock here?

We hold the ITS lock. The only way to DISCARD the irq would be to issue
another command, which is not possible until we actually release the
lock. Or am I missing something?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-07 14:42       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 14:42 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi Eric,

On 07/11/17 13:06, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Let's use the irq bypass mechanism introduced for platform device
>> interrupts
> nit: I would remove "introduced for platform device interrupts"
> as this is not upstream yet. x86 posted interrupts also use it.
> 
>>
>  and establish our LPI->VLPI mapping.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  include/kvm/arm_vgic.h      |   8 ++++
>>  virt/kvm/arm/arm.c          |   6 ++-
>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>  
>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>  
>> +struct kvm_kernel_irq_routing_entry;
>> +
>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>> +
>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>> +
>>  #endif /* __KVM_ARM_VGIC_H */
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index 5d5218ecd547..8388c1cc23f6 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>  	struct kvm_kernel_irqfd *irqfd =
>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>  
>> -	return 0;
>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>> +					  &irqfd->irq_entry);
>>  }
>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>  				      struct irq_bypass_producer *prod)
>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>  	struct kvm_kernel_irqfd *irqfd =
>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>  
>> -	return;
>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>> +				     &irqfd->irq_entry);
>>  }
>>  
>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index c794f0cef09e..01a2889b7b7c 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -18,6 +18,7 @@
>>  #include <linux/interrupt.h>
>>  #include <linux/irqdomain.h>
>>  #include <linux/kvm_host.h>
>> +#include <linux/irqchip/arm-gic-v3.h>
>>  
>>  #include "vgic.h"
>>  
>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>  	its_vm->nr_vpes = 0;
>>  	its_vm->vpes = NULL;
>>  }
>> +
>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>> +{
>> +	struct kvm_msi msi  = (struct kvm_msi) {
>> +		.address_lo	= irq_entry->msi.address_lo,
>> +		.address_hi	= irq_entry->msi.address_hi,
>> +		.data		= irq_entry->msi.data,
>> +		.flags		= irq_entry->msi.flags,
>> +		.devid		= irq_entry->msi.devid,
>> +	};
>> +
>> +	/*
>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>> +	 * translation for any of our vITSs.
>> +	 */
> I don't understand the relevance of the above comment.

Hmmm. The first part looks like an outdated leftover, as the ITS is not
refcounted, and we don't deal with LPIs here.

>> +	return vgic_msi_to_its(kvm, &msi);
>> +}
>> +
>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
> @virq is the host linux irq. virq naming is a bit confusing to me.

There is plenty of irq-related code that uses this naming. In this
context, we tend to use irq for the vgic view, hwirq for the irqchip
view. How would you call this one?

>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>> +{
>> +	struct vgic_its *its;
>> +	struct vgic_irq *irq;
>> +	struct its_vlpi_map map;
>> +	int ret;
>> +
> Don't you need to check that the linux irq (virq) is an LPI? You may
> encounter some VFIO "producers" for irq that are not LPIs, typically if
> we eventually upstream SPI forwarding.

That's indeed a concern. The issue is that we don't really have a way to
check this, other than following the irq_data pointers and check that
the hwirq is within a certain range. And even that doesn't guarantee
anything.

>> +	if (!vgic_supports_direct_msis(kvm))
>> +		return 0;
>> +
>> +	/*
>> +	 * Get the ITS, and escape early on error (not a valid
>> +	 * doorbell for any of our vITSs).
>> +	 */
>> +	its = vgic_get_its(kvm, irq_entry);
>> +	if (IS_ERR(its))
>> +		return 0;
>> +
>> +	mutex_lock(&its->its_lock);
> wouldn't it be safer to take the its_lock before the get_its()? and even
> before the vgic_supports_direct_msis, in an unlikely case where the its
> would disappear?

How do you get a lock on the ITS before getting a pointer to it? Also,
aren't we in the context of a vcpu here (we trap to userspace, come back
via a VFIO ioctl, and arrive here)? Could the ITS actually vanish from
under our feet here? The only way to do it would be to destroy the VM.


>> +
>> +	/* Perform then actual DevID/EventID -> LPI translation. */
>> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
>> +				   irq_entry->msi.data, &irq);
>> +	if (ret)
>> +		goto out;
>> +
>> +	/*
>> +	 * Emit the mapping request. If it fails, the ITS probably
>> +	 * isn't v4 compatible, so let's silently bail out. Holding
>> +	 * the ITS lock should ensure that nothing can modify the
>> +	 * target vcpu.
>> +	 */
>> +	map = (struct its_vlpi_map) {
>> +		.vm		= &kvm->arch.vgic.its_vm,
>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>> +		.vintid		= irq->intid,
>> +		.properties	= ((irq->priority & 0xfc) |
>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>> +				   LPI_PROP_GROUP1),
> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
> and this setting:
> 
> * @properties: Priority and enable bits (as written in the prop table)

Which inconsistency?

> Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?

We could. But given that irq->priority is assigned with
LPI_PROP_PRIORITY(), we could drop the 0xfc altogether.

>> +		.db_enabled	= true,
>> +	};
>> +
>> +	ret = its_map_vlpi(virq, &map);
> Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
> fails, will you get a change to turn  IRQ_DISABLE_LAZY again.

Good point. I'll fix this.

>> +	if (ret)
>> +		goto out;
>> +
>> +	irq->hw		= true;
>> +	irq->host_irq	= virq;
> Shouldn't we theoretically hold the irq lock here?

We hold the ITS lock. The only way to DISCARD the irq would be to issue
another command, which is not possible until we actually release the
lock. Or am I missing something?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-07 14:42       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 14:42 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Eric,

On 07/11/17 13:06, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Let's use the irq bypass mechanism introduced for platform device
>> interrupts
> nit: I would remove "introduced for platform device interrupts"
> as this is not upstream yet. x86 posted interrupts also use it.
> 
>>
>  and establish our LPI->VLPI mapping.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  include/kvm/arm_vgic.h      |   8 ++++
>>  virt/kvm/arm/arm.c          |   6 ++-
>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>  
>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>  
>> +struct kvm_kernel_irq_routing_entry;
>> +
>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>> +
>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>> +
>>  #endif /* __KVM_ARM_VGIC_H */
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index 5d5218ecd547..8388c1cc23f6 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>  	struct kvm_kernel_irqfd *irqfd =
>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>  
>> -	return 0;
>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>> +					  &irqfd->irq_entry);
>>  }
>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>  				      struct irq_bypass_producer *prod)
>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>  	struct kvm_kernel_irqfd *irqfd =
>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>  
>> -	return;
>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>> +				     &irqfd->irq_entry);
>>  }
>>  
>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index c794f0cef09e..01a2889b7b7c 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -18,6 +18,7 @@
>>  #include <linux/interrupt.h>
>>  #include <linux/irqdomain.h>
>>  #include <linux/kvm_host.h>
>> +#include <linux/irqchip/arm-gic-v3.h>
>>  
>>  #include "vgic.h"
>>  
>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>  	its_vm->nr_vpes = 0;
>>  	its_vm->vpes = NULL;
>>  }
>> +
>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>> +{
>> +	struct kvm_msi msi  = (struct kvm_msi) {
>> +		.address_lo	= irq_entry->msi.address_lo,
>> +		.address_hi	= irq_entry->msi.address_hi,
>> +		.data		= irq_entry->msi.data,
>> +		.flags		= irq_entry->msi.flags,
>> +		.devid		= irq_entry->msi.devid,
>> +	};
>> +
>> +	/*
>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>> +	 * translation for any of our vITSs.
>> +	 */
> I don't understand the relevance of the above comment.

Hmmm. The first part looks like an outdated leftover, as the ITS is not
refcounted, and we don't deal with LPIs here.

>> +	return vgic_msi_to_its(kvm, &msi);
>> +}
>> +
>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
> @virq is the host linux irq. virq naming is a bit confusing to me.

There is plenty of irq-related code that uses this naming. In this
context, we tend to use irq for the vgic view, hwirq for the irqchip
view. How would you call this one?

>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>> +{
>> +	struct vgic_its *its;
>> +	struct vgic_irq *irq;
>> +	struct its_vlpi_map map;
>> +	int ret;
>> +
> Don't you need to check that the linux irq (virq) is an LPI? You may
> encounter some VFIO "producers" for irq that are not LPIs, typically if
> we eventually upstream SPI forwarding.

That's indeed a concern. The issue is that we don't really have a way to
check this, other than following the irq_data pointers and check that
the hwirq is within a certain range. And even that doesn't guarantee
anything.

>> +	if (!vgic_supports_direct_msis(kvm))
>> +		return 0;
>> +
>> +	/*
>> +	 * Get the ITS, and escape early on error (not a valid
>> +	 * doorbell for any of our vITSs).
>> +	 */
>> +	its = vgic_get_its(kvm, irq_entry);
>> +	if (IS_ERR(its))
>> +		return 0;
>> +
>> +	mutex_lock(&its->its_lock);
> wouldn't it be safer to take the its_lock before the get_its()? and even
> before the vgic_supports_direct_msis, in an unlikely case where the its
> would disappear?

How do you get a lock on the ITS before getting a pointer to it? Also,
aren't we in the context of a vcpu here (we trap to userspace, come back
via a VFIO ioctl, and arrive here)? Could the ITS actually vanish from
under our feet here? The only way to do it would be to destroy the VM.


>> +
>> +	/* Perform then actual DevID/EventID -> LPI translation. */
>> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
>> +				   irq_entry->msi.data, &irq);
>> +	if (ret)
>> +		goto out;
>> +
>> +	/*
>> +	 * Emit the mapping request. If it fails, the ITS probably
>> +	 * isn't v4 compatible, so let's silently bail out. Holding
>> +	 * the ITS lock should ensure that nothing can modify the
>> +	 * target vcpu.
>> +	 */
>> +	map = (struct its_vlpi_map) {
>> +		.vm		= &kvm->arch.vgic.its_vm,
>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>> +		.vintid		= irq->intid,
>> +		.properties	= ((irq->priority & 0xfc) |
>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>> +				   LPI_PROP_GROUP1),
> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
> and this setting:
> 
> * @properties: Priority and enable bits (as written in the prop table)

Which inconsistency?

> Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?

We could. But given that irq->priority is assigned with
LPI_PROP_PRIORITY(), we could drop the 0xfc altogether.

>> +		.db_enabled	= true,
>> +	};
>> +
>> +	ret = its_map_vlpi(virq, &map);
> Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
> fails, will you get a change to turn  IRQ_DISABLE_LAZY again.

Good point. I'll fix this.

>> +	if (ret)
>> +		goto out;
>> +
>> +	irq->hw		= true;
>> +	irq->host_irq	= virq;
> Shouldn't we theoretically hold the irq lock here?

We hold the ITS lock. The only way to DISCARD the irq would be to issue
another command, which is not possible until we actually release the
lock. Or am I missing something?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 15:24     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 15:24 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

Hi Marc,
On 27/10/2017 16:28, Marc Zyngier wrote:
> The GICv4 architecture doesn't make it easy for save/restore to
> work, as it doesn't give any guarantee that the pending state
> is written into the pending table.

I don't understand where does the limitation exactly come from. Can't we
use the GICR_VPENDBASER table data?

Thanks

Eric
> 
> So let's not take any chance, and let's return an error if
> we encounter any LPI that has the HW bit set. In order for
> userspace to distinguish this error from other failure modes,
> use -EACCES as an error code.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  Documentation/virtual/kvm/devices/arm-vgic-its.txt | 2 ++
>  virt/kvm/arm/vgic/vgic-its.c                       | 9 +++++++++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> index eb06beb75960..1eec7bcb2d52 100644
> --- a/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> +++ b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> @@ -60,6 +60,8 @@ Groups:
>      -EINVAL: Inconsistent restored data
>      -EFAULT: Invalid guest ram access
>      -EBUSY:  One or more VCPUS are running
> +    -EACCES: The virtual ITS is backed by a physical GICv4 ITS, and the
> +    	     state is not available
>  
>    KVM_DEV_ARM_VGIC_GRP_ITS_REGS
>    Attributes:
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index eb72eb027060..2710834d0920 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1989,6 +1989,15 @@ static int vgic_its_save_itt(struct vgic_its *its, struct its_device *device)
>  	list_for_each_entry(ite, &device->itt_head, ite_list) {
>  		gpa_t gpa = base + ite->event_id * ite_esz;
>  
> +		/*
> +		 * If an LPI carries the HW bit, this means that this
> +		 * interrupt is controlled by GICv4, and we do not
> +		 * have direct access to that state. Let's simply fail
> +		 * the save operation...
> +		 */
> +		if (ite->irq->hw)
> +			return -EACCES;
> +
>  		ret = vgic_its_save_ite(its, device, ite, gpa, ite_esz);
>  		if (ret)
>  			return ret;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 15:24     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 15:24 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

Hi Marc,
On 27/10/2017 16:28, Marc Zyngier wrote:
> The GICv4 architecture doesn't make it easy for save/restore to
> work, as it doesn't give any guarantee that the pending state
> is written into the pending table.

I don't understand where does the limitation exactly come from. Can't we
use the GICR_VPENDBASER table data?

Thanks

Eric
> 
> So let's not take any chance, and let's return an error if
> we encounter any LPI that has the HW bit set. In order for
> userspace to distinguish this error from other failure modes,
> use -EACCES as an error code.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  Documentation/virtual/kvm/devices/arm-vgic-its.txt | 2 ++
>  virt/kvm/arm/vgic/vgic-its.c                       | 9 +++++++++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> index eb06beb75960..1eec7bcb2d52 100644
> --- a/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> +++ b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> @@ -60,6 +60,8 @@ Groups:
>      -EINVAL: Inconsistent restored data
>      -EFAULT: Invalid guest ram access
>      -EBUSY:  One or more VCPUS are running
> +    -EACCES: The virtual ITS is backed by a physical GICv4 ITS, and the
> +    	     state is not available
>  
>    KVM_DEV_ARM_VGIC_GRP_ITS_REGS
>    Attributes:
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index eb72eb027060..2710834d0920 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1989,6 +1989,15 @@ static int vgic_its_save_itt(struct vgic_its *its, struct its_device *device)
>  	list_for_each_entry(ite, &device->itt_head, ite_list) {
>  		gpa_t gpa = base + ite->event_id * ite_esz;
>  
> +		/*
> +		 * If an LPI carries the HW bit, this means that this
> +		 * interrupt is controlled by GICv4, and we do not
> +		 * have direct access to that state. Let's simply fail
> +		 * the save operation...
> +		 */
> +		if (ite->irq->hw)
> +			return -EACCES;
> +
>  		ret = vgic_its_save_ite(its, device, ite, gpa, ite_esz);
>  		if (ret)
>  			return ret;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 15:24     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 15:24 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

Hi Marc,
On 27/10/2017 16:28, Marc Zyngier wrote:
> The GICv4 architecture doesn't make it easy for save/restore to
> work, as it doesn't give any guarantee that the pending state
> is written into the pending table.

I don't understand where does the limitation exactly come from. Can't we
use the GICR_VPENDBASER table data?

Thanks

Eric
> 
> So let's not take any chance, and let's return an error if
> we encounter any LPI that has the HW bit set. In order for
> userspace to distinguish this error from other failure modes,
> use -EACCES as an error code.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  Documentation/virtual/kvm/devices/arm-vgic-its.txt | 2 ++
>  virt/kvm/arm/vgic/vgic-its.c                       | 9 +++++++++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> index eb06beb75960..1eec7bcb2d52 100644
> --- a/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> +++ b/Documentation/virtual/kvm/devices/arm-vgic-its.txt
> @@ -60,6 +60,8 @@ Groups:
>      -EINVAL: Inconsistent restored data
>      -EFAULT: Invalid guest ram access
>      -EBUSY:  One or more VCPUS are running
> +    -EACCES: The virtual ITS is backed by a physical GICv4 ITS, and the
> +    	     state is not available
>  
>    KVM_DEV_ARM_VGIC_GRP_ITS_REGS
>    Attributes:
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index eb72eb027060..2710834d0920 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1989,6 +1989,15 @@ static int vgic_its_save_itt(struct vgic_its *its, struct its_device *device)
>  	list_for_each_entry(ite, &device->itt_head, ite_list) {
>  		gpa_t gpa = base + ite->event_id * ite_esz;
>  
> +		/*
> +		 * If an LPI carries the HW bit, this means that this
> +		 * interrupt is controlled by GICv4, and we do not
> +		 * have direct access to that state. Let's simply fail
> +		 * the save operation...
> +		 */
> +		if (ite->irq->hw)
> +			return -EACCES;
> +
>  		ret = vgic_its_save_ite(its, device, ite, gpa, ite_esz);
>  		if (ret)
>  			return ret;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  2017-11-07 15:24     ` Auger Eric
  (?)
@ 2017-11-07 15:38       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 15:38 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On 07/11/17 15:24, Auger Eric wrote:
> Hi Marc,
> 
> Hi Marc,
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> The GICv4 architecture doesn't make it easy for save/restore to
>> work, as it doesn't give any guarantee that the pending state
>> is written into the pending table.
> 
> I don't understand where does the limitation exactly come from. Can't we
> use the GICR_VPENDBASER table data?
You can't. None of the tables that are written by either the ITS or the
redistributors are architected. All you know is that there is one bit
per vLPI, but that's it (you don't even know which one is which).

But that's not a big deal. I don't think you can realistically migrate a
VM that has a directly assigned device anyway. Or can we?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 15:38       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 15:38 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On 07/11/17 15:24, Auger Eric wrote:
> Hi Marc,
> 
> Hi Marc,
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> The GICv4 architecture doesn't make it easy for save/restore to
>> work, as it doesn't give any guarantee that the pending state
>> is written into the pending table.
> 
> I don't understand where does the limitation exactly come from. Can't we
> use the GICR_VPENDBASER table data?
You can't. None of the tables that are written by either the ITS or the
redistributors are architected. All you know is that there is one bit
per vLPI, but that's it (you don't even know which one is which).

But that's not a big deal. I don't think you can realistically migrate a
VM that has a directly assigned device anyway. Or can we?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 15:38       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 15:38 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/11/17 15:24, Auger Eric wrote:
> Hi Marc,
> 
> Hi Marc,
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> The GICv4 architecture doesn't make it easy for save/restore to
>> work, as it doesn't give any guarantee that the pending state
>> is written into the pending table.
> 
> I don't understand where does the limitation exactly come from. Can't we
> use the GICR_VPENDBASER table data?
You can't. None of the tables that are written by either the ITS or the
redistributors are architected. All you know is that there is one bit
per vLPI, but that's it (you don't even know which one is which).

But that's not a big deal. I don't think you can realistically migrate a
VM that has a directly assigned device anyway. Or can we?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-11-07 14:42       ` Marc Zyngier
  (?)
@ 2017-11-07 15:59         ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 15:59 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi,

On 07/11/2017 15:42, Marc Zyngier wrote:
> Hi Eric,
> 
> On 07/11/17 13:06, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> Let's use the irq bypass mechanism introduced for platform device
>>> interrupts
>> nit: I would remove "introduced for platform device interrupts"
>> as this is not upstream yet. x86 posted interrupts also use it.
>>
>>>
>>  and establish our LPI->VLPI mapping.
>>>
>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>> --- a/include/kvm/arm_vgic.h
>>> +++ b/include/kvm/arm_vgic.h
>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>  
>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>  
>>> +struct kvm_kernel_irq_routing_entry;
>>> +
>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>> +
>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>> +
>>>  #endif /* __KVM_ARM_VGIC_H */
>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>> index 5d5218ecd547..8388c1cc23f6 100644
>>> --- a/virt/kvm/arm/arm.c
>>> +++ b/virt/kvm/arm/arm.c
>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>  	struct kvm_kernel_irqfd *irqfd =
>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>  
>>> -	return 0;
>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>> +					  &irqfd->irq_entry);
>>>  }
>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>  				      struct irq_bypass_producer *prod)
>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>  	struct kvm_kernel_irqfd *irqfd =
>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>  
>>> -	return;
>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>> +				     &irqfd->irq_entry);
>>>  }
>>>  
>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>> index c794f0cef09e..01a2889b7b7c 100644
>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>> @@ -18,6 +18,7 @@
>>>  #include <linux/interrupt.h>
>>>  #include <linux/irqdomain.h>
>>>  #include <linux/kvm_host.h>
>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>  
>>>  #include "vgic.h"
>>>  
>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>  	its_vm->nr_vpes = 0;
>>>  	its_vm->vpes = NULL;
>>>  }
>>> +
>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>> +{
>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>> +		.address_lo	= irq_entry->msi.address_lo,
>>> +		.address_hi	= irq_entry->msi.address_hi,
>>> +		.data		= irq_entry->msi.data,
>>> +		.flags		= irq_entry->msi.flags,
>>> +		.devid		= irq_entry->msi.devid,
>>> +	};
>>> +
>>> +	/*
>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>> +	 * translation for any of our vITSs.
>>> +	 */
>> I don't understand the relevance of the above comment.
> 
> Hmmm. The first part looks like an outdated leftover, as the ITS is not
> refcounted, and we don't deal with LPIs here.
> 
>>> +	return vgic_msi_to_its(kvm, &msi);
>>> +}
>>> +
>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
>> @virq is the host linux irq. virq naming is a bit confusing to me.
> 
> There is plenty of irq-related code that uses this naming. In this
> context, we tend to use irq for the vgic view, hwirq for the irqchip
> view. How would you call this one?
OK
> 
>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>>> +{
>>> +	struct vgic_its *its;
>>> +	struct vgic_irq *irq;
>>> +	struct its_vlpi_map map;
>>> +	int ret;
>>> +
>> Don't you need to check that the linux irq (virq) is an LPI? You may
>> encounter some VFIO "producers" for irq that are not LPIs, typically if
>> we eventually upstream SPI forwarding.
> 
> That's indeed a concern. The issue is that we don't really have a way to
> check this, other than following the irq_data pointers and check that
> the hwirq is within a certain range. And even that doesn't guarantee
> anything.
OK. But somehow this means the userspace can setup forwarding between an
SPI and a vLPI, right?
> 
>>> +	if (!vgic_supports_direct_msis(kvm))
>>> +		return 0;
>>> +
>>> +	/*
>>> +	 * Get the ITS, and escape early on error (not a valid
>>> +	 * doorbell for any of our vITSs).
>>> +	 */
>>> +	its = vgic_get_its(kvm, irq_entry);
>>> +	if (IS_ERR(its))
>>> +		return 0;
>>> +
>>> +	mutex_lock(&its->its_lock);
>> wouldn't it be safer to take the its_lock before the get_its()? and even
>> before the vgic_supports_direct_msis, in an unlikely case where the its
>> would disappear?
> 
> How do you get a lock on the ITS before getting a pointer to it?
Oh forget it
 Also,
> aren't we in the context of a vcpu here (we trap to userspace, come back
> via a VFIO ioctl, and arrive here)? Could the ITS actually vanish from
> under our feet here? The only way to do it would be to destroy the VM.
correct, sorry for the noise.
> 
> 
>>> +
>>> +	/* Perform then actual DevID/EventID -> LPI translation. */
>>> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
>>> +				   irq_entry->msi.data, &irq);
>>> +	if (ret)
>>> +		goto out;
>>> +
>>> +	/*
>>> +	 * Emit the mapping request. If it fails, the ITS probably
>>> +	 * isn't v4 compatible, so let's silently bail out. Holding
>>> +	 * the ITS lock should ensure that nothing can modify the
>>> +	 * target vcpu.
>>> +	 */
>>> +	map = (struct its_vlpi_map) {
>>> +		.vm		= &kvm->arch.vgic.its_vm,
>>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>>> +		.vintid		= irq->intid,
>>> +		.properties	= ((irq->priority & 0xfc) |
>>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>>> +				   LPI_PROP_GROUP1),
>> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
>> and this setting:
>>
>> * @properties: Priority and enable bits (as written in the prop table)
> 
> Which inconsistency?
I was confused by LPI_PROP_GROUP1 which was not documented in the
comment. But looking more carefully in the spec it corresponds to [1] =
RES1.
> 
>> Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?
> 
> We could. But given that irq->priority is assigned with
> LPI_PROP_PRIORITY(), we could drop the 0xfc altogether.
> 
>>> +		.db_enabled	= true,
>>> +	};
>>> +
>>> +	ret = its_map_vlpi(virq, &map);
>> Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
>> fails, will you get a change to turn  IRQ_DISABLE_LAZY again.
> 
> Good point. I'll fix this.
> 
>>> +	if (ret)
>>> +		goto out;
>>> +
>>> +	irq->hw		= true;
>>> +	irq->host_irq	= virq;
>> Shouldn't we theoretically hold the irq lock here?
> 
> We hold the ITS lock. The only way to DISCARD the irq would be to issue
> another command, which is not possible until we actually release the
> lock. Or am I missing something?
No you don't. I was rather anticipating other (future) callers, for
instance other bypass functions which could mess with those fields.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-07 15:59         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 15:59 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi,

On 07/11/2017 15:42, Marc Zyngier wrote:
> Hi Eric,
> 
> On 07/11/17 13:06, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> Let's use the irq bypass mechanism introduced for platform device
>>> interrupts
>> nit: I would remove "introduced for platform device interrupts"
>> as this is not upstream yet. x86 posted interrupts also use it.
>>
>>>
>>  and establish our LPI->VLPI mapping.
>>>
>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>> --- a/include/kvm/arm_vgic.h
>>> +++ b/include/kvm/arm_vgic.h
>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>  
>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>  
>>> +struct kvm_kernel_irq_routing_entry;
>>> +
>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>> +
>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>> +
>>>  #endif /* __KVM_ARM_VGIC_H */
>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>> index 5d5218ecd547..8388c1cc23f6 100644
>>> --- a/virt/kvm/arm/arm.c
>>> +++ b/virt/kvm/arm/arm.c
>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>  	struct kvm_kernel_irqfd *irqfd =
>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>  
>>> -	return 0;
>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>> +					  &irqfd->irq_entry);
>>>  }
>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>  				      struct irq_bypass_producer *prod)
>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>  	struct kvm_kernel_irqfd *irqfd =
>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>  
>>> -	return;
>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>> +				     &irqfd->irq_entry);
>>>  }
>>>  
>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>> index c794f0cef09e..01a2889b7b7c 100644
>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>> @@ -18,6 +18,7 @@
>>>  #include <linux/interrupt.h>
>>>  #include <linux/irqdomain.h>
>>>  #include <linux/kvm_host.h>
>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>  
>>>  #include "vgic.h"
>>>  
>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>  	its_vm->nr_vpes = 0;
>>>  	its_vm->vpes = NULL;
>>>  }
>>> +
>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>> +{
>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>> +		.address_lo	= irq_entry->msi.address_lo,
>>> +		.address_hi	= irq_entry->msi.address_hi,
>>> +		.data		= irq_entry->msi.data,
>>> +		.flags		= irq_entry->msi.flags,
>>> +		.devid		= irq_entry->msi.devid,
>>> +	};
>>> +
>>> +	/*
>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>> +	 * translation for any of our vITSs.
>>> +	 */
>> I don't understand the relevance of the above comment.
> 
> Hmmm. The first part looks like an outdated leftover, as the ITS is not
> refcounted, and we don't deal with LPIs here.
> 
>>> +	return vgic_msi_to_its(kvm, &msi);
>>> +}
>>> +
>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
>> @virq is the host linux irq. virq naming is a bit confusing to me.
> 
> There is plenty of irq-related code that uses this naming. In this
> context, we tend to use irq for the vgic view, hwirq for the irqchip
> view. How would you call this one?
OK
> 
>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>>> +{
>>> +	struct vgic_its *its;
>>> +	struct vgic_irq *irq;
>>> +	struct its_vlpi_map map;
>>> +	int ret;
>>> +
>> Don't you need to check that the linux irq (virq) is an LPI? You may
>> encounter some VFIO "producers" for irq that are not LPIs, typically if
>> we eventually upstream SPI forwarding.
> 
> That's indeed a concern. The issue is that we don't really have a way to
> check this, other than following the irq_data pointers and check that
> the hwirq is within a certain range. And even that doesn't guarantee
> anything.
OK. But somehow this means the userspace can setup forwarding between an
SPI and a vLPI, right?
> 
>>> +	if (!vgic_supports_direct_msis(kvm))
>>> +		return 0;
>>> +
>>> +	/*
>>> +	 * Get the ITS, and escape early on error (not a valid
>>> +	 * doorbell for any of our vITSs).
>>> +	 */
>>> +	its = vgic_get_its(kvm, irq_entry);
>>> +	if (IS_ERR(its))
>>> +		return 0;
>>> +
>>> +	mutex_lock(&its->its_lock);
>> wouldn't it be safer to take the its_lock before the get_its()? and even
>> before the vgic_supports_direct_msis, in an unlikely case where the its
>> would disappear?
> 
> How do you get a lock on the ITS before getting a pointer to it?
Oh forget it
 Also,
> aren't we in the context of a vcpu here (we trap to userspace, come back
> via a VFIO ioctl, and arrive here)? Could the ITS actually vanish from
> under our feet here? The only way to do it would be to destroy the VM.
correct, sorry for the noise.
> 
> 
>>> +
>>> +	/* Perform then actual DevID/EventID -> LPI translation. */
>>> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
>>> +				   irq_entry->msi.data, &irq);
>>> +	if (ret)
>>> +		goto out;
>>> +
>>> +	/*
>>> +	 * Emit the mapping request. If it fails, the ITS probably
>>> +	 * isn't v4 compatible, so let's silently bail out. Holding
>>> +	 * the ITS lock should ensure that nothing can modify the
>>> +	 * target vcpu.
>>> +	 */
>>> +	map = (struct its_vlpi_map) {
>>> +		.vm		= &kvm->arch.vgic.its_vm,
>>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>>> +		.vintid		= irq->intid,
>>> +		.properties	= ((irq->priority & 0xfc) |
>>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>>> +				   LPI_PROP_GROUP1),
>> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
>> and this setting:
>>
>> * @properties: Priority and enable bits (as written in the prop table)
> 
> Which inconsistency?
I was confused by LPI_PROP_GROUP1 which was not documented in the
comment. But looking more carefully in the spec it corresponds to [1] =
RES1.
> 
>> Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?
> 
> We could. But given that irq->priority is assigned with
> LPI_PROP_PRIORITY(), we could drop the 0xfc altogether.
> 
>>> +		.db_enabled	= true,
>>> +	};
>>> +
>>> +	ret = its_map_vlpi(virq, &map);
>> Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
>> fails, will you get a change to turn  IRQ_DISABLE_LAZY again.
> 
> Good point. I'll fix this.
> 
>>> +	if (ret)
>>> +		goto out;
>>> +
>>> +	irq->hw		= true;
>>> +	irq->host_irq	= virq;
>> Shouldn't we theoretically hold the irq lock here?
> 
> We hold the ITS lock. The only way to DISCARD the irq would be to issue
> another command, which is not possible until we actually release the
> lock. Or am I missing something?
No you don't. I was rather anticipating other (future) callers, for
instance other bypass functions which could mess with those fields.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-07 15:59         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 15:59 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 07/11/2017 15:42, Marc Zyngier wrote:
> Hi Eric,
> 
> On 07/11/17 13:06, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> Let's use the irq bypass mechanism introduced for platform device
>>> interrupts
>> nit: I would remove "introduced for platform device interrupts"
>> as this is not upstream yet. x86 posted interrupts also use it.
>>
>>>
>>  and establish our LPI->VLPI mapping.
>>>
>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>> --- a/include/kvm/arm_vgic.h
>>> +++ b/include/kvm/arm_vgic.h
>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>  
>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>  
>>> +struct kvm_kernel_irq_routing_entry;
>>> +
>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>> +
>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>> +
>>>  #endif /* __KVM_ARM_VGIC_H */
>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>> index 5d5218ecd547..8388c1cc23f6 100644
>>> --- a/virt/kvm/arm/arm.c
>>> +++ b/virt/kvm/arm/arm.c
>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>  	struct kvm_kernel_irqfd *irqfd =
>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>  
>>> -	return 0;
>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>> +					  &irqfd->irq_entry);
>>>  }
>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>  				      struct irq_bypass_producer *prod)
>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>  	struct kvm_kernel_irqfd *irqfd =
>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>  
>>> -	return;
>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>> +				     &irqfd->irq_entry);
>>>  }
>>>  
>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>> index c794f0cef09e..01a2889b7b7c 100644
>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>> @@ -18,6 +18,7 @@
>>>  #include <linux/interrupt.h>
>>>  #include <linux/irqdomain.h>
>>>  #include <linux/kvm_host.h>
>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>  
>>>  #include "vgic.h"
>>>  
>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>  	its_vm->nr_vpes = 0;
>>>  	its_vm->vpes = NULL;
>>>  }
>>> +
>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>> +{
>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>> +		.address_lo	= irq_entry->msi.address_lo,
>>> +		.address_hi	= irq_entry->msi.address_hi,
>>> +		.data		= irq_entry->msi.data,
>>> +		.flags		= irq_entry->msi.flags,
>>> +		.devid		= irq_entry->msi.devid,
>>> +	};
>>> +
>>> +	/*
>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>> +	 * translation for any of our vITSs.
>>> +	 */
>> I don't understand the relevance of the above comment.
> 
> Hmmm. The first part looks like an outdated leftover, as the ITS is not
> refcounted, and we don't deal with LPIs here.
> 
>>> +	return vgic_msi_to_its(kvm, &msi);
>>> +}
>>> +
>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
>> @virq is the host linux irq. virq naming is a bit confusing to me.
> 
> There is plenty of irq-related code that uses this naming. In this
> context, we tend to use irq for the vgic view, hwirq for the irqchip
> view. How would you call this one?
OK
> 
>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>>> +{
>>> +	struct vgic_its *its;
>>> +	struct vgic_irq *irq;
>>> +	struct its_vlpi_map map;
>>> +	int ret;
>>> +
>> Don't you need to check that the linux irq (virq) is an LPI? You may
>> encounter some VFIO "producers" for irq that are not LPIs, typically if
>> we eventually upstream SPI forwarding.
> 
> That's indeed a concern. The issue is that we don't really have a way to
> check this, other than following the irq_data pointers and check that
> the hwirq is within a certain range. And even that doesn't guarantee
> anything.
OK. But somehow this means the userspace can setup forwarding between an
SPI and a vLPI, right?
> 
>>> +	if (!vgic_supports_direct_msis(kvm))
>>> +		return 0;
>>> +
>>> +	/*
>>> +	 * Get the ITS, and escape early on error (not a valid
>>> +	 * doorbell for any of our vITSs).
>>> +	 */
>>> +	its = vgic_get_its(kvm, irq_entry);
>>> +	if (IS_ERR(its))
>>> +		return 0;
>>> +
>>> +	mutex_lock(&its->its_lock);
>> wouldn't it be safer to take the its_lock before the get_its()? and even
>> before the vgic_supports_direct_msis, in an unlikely case where the its
>> would disappear?
> 
> How do you get a lock on the ITS before getting a pointer to it?
Oh forget it
 Also,
> aren't we in the context of a vcpu here (we trap to userspace, come back
> via a VFIO ioctl, and arrive here)? Could the ITS actually vanish from
> under our feet here? The only way to do it would be to destroy the VM.
correct, sorry for the noise.
> 
> 
>>> +
>>> +	/* Perform then actual DevID/EventID -> LPI translation. */
>>> +	ret = vgic_its_resolve_lpi(kvm, its, irq_entry->msi.devid,
>>> +				   irq_entry->msi.data, &irq);
>>> +	if (ret)
>>> +		goto out;
>>> +
>>> +	/*
>>> +	 * Emit the mapping request. If it fails, the ITS probably
>>> +	 * isn't v4 compatible, so let's silently bail out. Holding
>>> +	 * the ITS lock should ensure that nothing can modify the
>>> +	 * target vcpu.
>>> +	 */
>>> +	map = (struct its_vlpi_map) {
>>> +		.vm		= &kvm->arch.vgic.its_vm,
>>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>>> +		.vintid		= irq->intid,
>>> +		.properties	= ((irq->priority & 0xfc) |
>>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>>> +				   LPI_PROP_GROUP1),
>> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
>> and this setting:
>>
>> * @properties: Priority and enable bits (as written in the prop table)
> 
> Which inconsistency?
I was confused by LPI_PROP_GROUP1 which was not documented in the
comment. But looking more carefully in the spec it corresponds to [1] =
RES1.
> 
>> Also maybe we could use LPI_PROP_PRIORITY macro instead of 0xfc?
> 
> We could. But given that irq->priority is assigned with
> LPI_PROP_PRIORITY(), we could drop the 0xfc altogether.
> 
>>> +		.db_enabled	= true,
>>> +	};
>>> +
>>> +	ret = its_map_vlpi(virq, &map);
>> Looking at its_map_lpi implementation, assuming irq_set_vcpu_affinity()
>> fails, will you get a change to turn  IRQ_DISABLE_LAZY again.
> 
> Good point. I'll fix this.
> 
>>> +	if (ret)
>>> +		goto out;
>>> +
>>> +	irq->hw		= true;
>>> +	irq->host_irq	= virq;
>> Shouldn't we theoretically hold the irq lock here?
> 
> We hold the ITS lock. The only way to DISCARD the irq would be to issue
> another command, which is not possible until we actually release the
> lock. Or am I missing something?
No you don't. I was rather anticipating other (future) callers, for
instance other bypass functions which could mess with those fields.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  2017-11-07 15:38       ` Marc Zyngier
  (?)
@ 2017-11-07 16:12         ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 16:12 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 07/11/2017 16:38, Marc Zyngier wrote:
> On 07/11/17 15:24, Auger Eric wrote:
>> Hi Marc,
>>
>> Hi Marc,
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> The GICv4 architecture doesn't make it easy for save/restore to
>>> work, as it doesn't give any guarantee that the pending state
>>> is written into the pending table.
>>
>> I don't understand where does the limitation exactly come from. Can't we
>> use the GICR_VPENDBASER table data?
> You can't. None of the tables that are written by either the ITS or the
> redistributors are architected. All you know is that there is one bit
> per vLPI, but that's it (you don't even know which one is which).
Oh I thought the redistributor config and pending tables were fully
specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
table.
> 
> But that's not a big deal. I don't think you can realistically migrate a
> VM that has a directly assigned device anyway. Or can we?
No we can't except for mediated devices for which migration can be
supported.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 16:12         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 16:12 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 07/11/2017 16:38, Marc Zyngier wrote:
> On 07/11/17 15:24, Auger Eric wrote:
>> Hi Marc,
>>
>> Hi Marc,
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> The GICv4 architecture doesn't make it easy for save/restore to
>>> work, as it doesn't give any guarantee that the pending state
>>> is written into the pending table.
>>
>> I don't understand where does the limitation exactly come from. Can't we
>> use the GICR_VPENDBASER table data?
> You can't. None of the tables that are written by either the ITS or the
> redistributors are architected. All you know is that there is one bit
> per vLPI, but that's it (you don't even know which one is which).
Oh I thought the redistributor config and pending tables were fully
specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
table.
> 
> But that's not a big deal. I don't think you can realistically migrate a
> VM that has a directly assigned device anyway. Or can we?
No we can't except for mediated devices for which migration can be
supported.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 16:12         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 16:12 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 07/11/2017 16:38, Marc Zyngier wrote:
> On 07/11/17 15:24, Auger Eric wrote:
>> Hi Marc,
>>
>> Hi Marc,
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> The GICv4 architecture doesn't make it easy for save/restore to
>>> work, as it doesn't give any guarantee that the pending state
>>> is written into the pending table.
>>
>> I don't understand where does the limitation exactly come from. Can't we
>> use the GICR_VPENDBASER table data?
> You can't. None of the tables that are written by either the ITS or the
> redistributors are architected. All you know is that there is one bit
> per vLPI, but that's it (you don't even know which one is which).
Oh I thought the redistributor config and pending tables were fully
specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
table.
> 
> But that's not a big deal. I don't think you can realistically migrate a
> VM that has a directly assigned device anyway. Or can we?
No we can't except for mediated devices for which migration can be
supported.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  2017-11-07 16:12         ` Auger Eric
  (?)
@ 2017-11-07 16:34           ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 16:34 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On 07/11/17 16:12, Auger Eric wrote:
> Hi Marc,
> 
> On 07/11/2017 16:38, Marc Zyngier wrote:
>> On 07/11/17 15:24, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> Hi Marc,
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>> work, as it doesn't give any guarantee that the pending state
>>>> is written into the pending table.
>>>
>>> I don't understand where does the limitation exactly come from. Can't we
>>> use the GICR_VPENDBASER table data?
>> You can't. None of the tables that are written by either the ITS or the
>> redistributors are architected. All you know is that there is one bit
>> per vLPI, but that's it (you don't even know which one is which).
> Oh I thought the redistributor config and pending tables were fully
> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
> table.

The property table is definitely architected. It is a lot less clear for
the pending table. The main issue is that you cannot really know when
the various bits have actually been flushed all the way from the
redistributor caches to memory to be introspected. Yes, it sucks.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 16:34           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 16:34 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

On 07/11/17 16:12, Auger Eric wrote:
> Hi Marc,
> 
> On 07/11/2017 16:38, Marc Zyngier wrote:
>> On 07/11/17 15:24, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> Hi Marc,
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>> work, as it doesn't give any guarantee that the pending state
>>>> is written into the pending table.
>>>
>>> I don't understand where does the limitation exactly come from. Can't we
>>> use the GICR_VPENDBASER table data?
>> You can't. None of the tables that are written by either the ITS or the
>> redistributors are architected. All you know is that there is one bit
>> per vLPI, but that's it (you don't even know which one is which).
> Oh I thought the redistributor config and pending tables were fully
> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
> table.

The property table is definitely architected. It is a lot less clear for
the pending table. The main issue is that you cannot really know when
the various bits have actually been flushed all the way from the
redistributor caches to memory to be introspected. Yes, it sucks.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 16:34           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-07 16:34 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/11/17 16:12, Auger Eric wrote:
> Hi Marc,
> 
> On 07/11/2017 16:38, Marc Zyngier wrote:
>> On 07/11/17 15:24, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> Hi Marc,
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>> work, as it doesn't give any guarantee that the pending state
>>>> is written into the pending table.
>>>
>>> I don't understand where does the limitation exactly come from. Can't we
>>> use the GICR_VPENDBASER table data?
>> You can't. None of the tables that are written by either the ITS or the
>> redistributors are architected. All you know is that there is one bit
>> per vLPI, but that's it (you don't even know which one is which).
> Oh I thought the redistributor config and pending tables were fully
> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
> table.

The property table is definitely architected. It is a lot less clear for
the pending table. The main issue is that you cannot really know when
the various bits have actually been flushed all the way from the
redistributor caches to memory to be introspected. Yes, it sucks.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 20:15     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:15 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> If the guest issues an INT command targetting a VLPI, let's
> call into the irq_set_irqchip_state() helper to make it pending
> on the physical side.
> 
> This works just as well if userspace decides to inject an interrupt
> using the normal userspace API...
There is also another path:
KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
vgic_its_inject_msi / vgic_its_trigger_msi

I wonder whether we shouldn't prevent the userspace from messing up with
the host irq pending state?

Thanks

Eric
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 89768d2b6a91..b2a678d131d0 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>  	if (err)
>  		return err;
>  
> +	if (irq->hw)
> +		return irq_set_irqchip_state(irq->host_irq,
> +					     IRQCHIP_STATE_PENDING, true);
> +
>  	spin_lock(&irq->irq_lock);
>  	irq->pending_latch = true;
>  	vgic_queue_irq_unlock(kvm, irq);
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-11-07 20:15     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:15 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> If the guest issues an INT command targetting a VLPI, let's
> call into the irq_set_irqchip_state() helper to make it pending
> on the physical side.
> 
> This works just as well if userspace decides to inject an interrupt
> using the normal userspace API...
There is also another path:
KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
vgic_its_inject_msi / vgic_its_trigger_msi

I wonder whether we shouldn't prevent the userspace from messing up with
the host irq pending state?

Thanks

Eric
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 89768d2b6a91..b2a678d131d0 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>  	if (err)
>  		return err;
>  
> +	if (irq->hw)
> +		return irq_set_irqchip_state(irq->host_irq,
> +					     IRQCHIP_STATE_PENDING, true);
> +
>  	spin_lock(&irq->irq_lock);
>  	irq->pending_latch = true;
>  	vgic_queue_irq_unlock(kvm, irq);
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-11-07 20:15     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:15 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> If the guest issues an INT command targetting a VLPI, let's
> call into the irq_set_irqchip_state() helper to make it pending
> on the physical side.
> 
> This works just as well if userspace decides to inject an interrupt
> using the normal userspace API...
There is also another path:
KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
vgic_its_inject_msi / vgic_its_trigger_msi

I wonder whether we shouldn't prevent the userspace from messing up with
the host irq pending state?

Thanks

Eric
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 89768d2b6a91..b2a678d131d0 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>  	if (err)
>  		return err;
>  
> +	if (irq->hw)
> +		return irq_set_irqchip_state(irq->host_irq,
> +					     IRQCHIP_STATE_PENDING, true);
> +
>  	spin_lock(&irq->irq_lock);
>  	irq->pending_latch = true;
>  	vgic_queue_irq_unlock(kvm, irq);
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 20:28     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:28 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When freeing an LPI (on a DISCARD command, for example), we need
> to unmap the VLPI down to the physical ITS level.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index b2a678d131d0..c9b1c0967426 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>  	list_del(&ite->ite_list);
>  
>  	/* This put matches the get in vgic_add_lpi. */
> -	if (ite->irq)
> +	if (ite->irq) {
> +		if (ite->irq->hw)
> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
> +
>  		vgic_put_irq(kvm, ite->irq);
You could have put the its_unmap_vlpi() directly in vgic_put_irq which
is meant to decr the ref/release the LPI irq.

Thanks

Eric
> +	}
>  
>  	kfree(ite);
>  }
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-11-07 20:28     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:28 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When freeing an LPI (on a DISCARD command, for example), we need
> to unmap the VLPI down to the physical ITS level.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index b2a678d131d0..c9b1c0967426 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>  	list_del(&ite->ite_list);
>  
>  	/* This put matches the get in vgic_add_lpi. */
> -	if (ite->irq)
> +	if (ite->irq) {
> +		if (ite->irq->hw)
> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
> +
>  		vgic_put_irq(kvm, ite->irq);
You could have put the its_unmap_vlpi() directly in vgic_put_irq which
is meant to decr the ref/release the LPI irq.

Thanks

Eric
> +	}
>  
>  	kfree(ite);
>  }
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-11-07 20:28     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:28 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When freeing an LPI (on a DISCARD command, for example), we need
> to unmap the VLPI down to the physical ITS level.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index b2a678d131d0..c9b1c0967426 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>  	list_del(&ite->ite_list);
>  
>  	/* This put matches the get in vgic_add_lpi. */
> -	if (ite->irq)
> +	if (ite->irq) {
> +		if (ite->irq->hw)
> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
> +
>  		vgic_put_irq(kvm, ite->irq);
You could have put the its_unmap_vlpi() directly in vgic_put_irq which
is meant to decr the ref/release the LPI irq.

Thanks

Eric
> +	}
>  
>  	kfree(ite);
>  }
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 08/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 20:30     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:30 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> Add a new has_gicv4 field in the global VGIC state that indicates
> whether the HW is GICv4 capable, as a per-VM predicate indicating
> if there is a possibility for a VM to support direct injection
> (the above being true and the VM having an ITS).
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  include/kvm/arm_vgic.h           | 3 +++
>  virt/kvm/arm/vgic/vgic-mmio-v3.c | 5 +++++
>  virt/kvm/arm/vgic/vgic.h         | 2 ++
>  3 files changed, 10 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 53f631bdec75..ba9fb450aa1b 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -73,6 +73,9 @@ struct vgic_global {
>  	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
>  	bool			can_emulate_gicv2;
>  
> +	/* Hardware has GICv4? */
> +	bool			has_gicv4;
> +
>  	/* GIC system register CPU interface */
>  	struct static_key_false gicv3_cpuif;
>  
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 408ef06638fc..f87fd17b2eb9 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -54,6 +54,11 @@ bool vgic_has_its(struct kvm *kvm)
>  	return dist->has_its;
>  }
>  
> +bool vgic_supports_direct_msis(struct kvm *kvm)
> +{
> +	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
> +}
> +
>  static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
>  					    gpa_t addr, unsigned int len)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 31b70326b966..e67ccb6a6250 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -240,4 +240,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  			 u32 devid, u32 eventid, struct vgic_irq **irq);
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
> +bool vgic_supports_direct_msis(struct kvm *kvm);
> +
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 08/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate
@ 2017-11-07 20:30     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:30 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> Add a new has_gicv4 field in the global VGIC state that indicates
> whether the HW is GICv4 capable, as a per-VM predicate indicating
> if there is a possibility for a VM to support direct injection
> (the above being true and the VM having an ITS).
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  include/kvm/arm_vgic.h           | 3 +++
>  virt/kvm/arm/vgic/vgic-mmio-v3.c | 5 +++++
>  virt/kvm/arm/vgic/vgic.h         | 2 ++
>  3 files changed, 10 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 53f631bdec75..ba9fb450aa1b 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -73,6 +73,9 @@ struct vgic_global {
>  	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
>  	bool			can_emulate_gicv2;
>  
> +	/* Hardware has GICv4? */
> +	bool			has_gicv4;
> +
>  	/* GIC system register CPU interface */
>  	struct static_key_false gicv3_cpuif;
>  
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 408ef06638fc..f87fd17b2eb9 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -54,6 +54,11 @@ bool vgic_has_its(struct kvm *kvm)
>  	return dist->has_its;
>  }
>  
> +bool vgic_supports_direct_msis(struct kvm *kvm)
> +{
> +	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
> +}
> +
>  static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
>  					    gpa_t addr, unsigned int len)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 31b70326b966..e67ccb6a6250 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -240,4 +240,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  			 u32 devid, u32 eventid, struct vgic_irq **irq);
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
> +bool vgic_supports_direct_msis(struct kvm *kvm);
> +
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 08/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate
@ 2017-11-07 20:30     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:30 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> Add a new has_gicv4 field in the global VGIC state that indicates
> whether the HW is GICv4 capable, as a per-VM predicate indicating
> if there is a possibility for a VM to support direct injection
> (the above being true and the VM having an ITS).
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  include/kvm/arm_vgic.h           | 3 +++
>  virt/kvm/arm/vgic/vgic-mmio-v3.c | 5 +++++
>  virt/kvm/arm/vgic/vgic.h         | 2 ++
>  3 files changed, 10 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 53f631bdec75..ba9fb450aa1b 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -73,6 +73,9 @@ struct vgic_global {
>  	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
>  	bool			can_emulate_gicv2;
>  
> +	/* Hardware has GICv4? */
> +	bool			has_gicv4;
> +
>  	/* GIC system register CPU interface */
>  	struct static_key_false gicv3_cpuif;
>  
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 408ef06638fc..f87fd17b2eb9 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -54,6 +54,11 @@ bool vgic_has_its(struct kvm *kvm)
>  	return dist->has_its;
>  }
>  
> +bool vgic_supports_direct_msis(struct kvm *kvm)
> +{
> +	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
> +}
> +
>  static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
>  					    gpa_t addr, unsigned int len)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 31b70326b966..e67ccb6a6250 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -240,4 +240,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
>  			 u32 devid, u32 eventid, struct vgic_irq **irq);
>  struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  
> +bool vgic_supports_direct_msis(struct kvm *kvm);
> +
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 06/26] KVM: arm/arm64: vITS: Add MSI translation helpers
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 20:34     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:34 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The whole MSI injection process is fairly monolithic. An MSI write
> gets turned into an injected LPI in one swift go. But this is actually
> a more fine-grained process:
> 
> - First, a virtual ITS gets selected using the doorbell address
> - Then the DevID/EventID pair gets translated into an LPI
> - Finally the LPI is injected
> 
> Since the GICv4 code needs the first two steps in order to match
> an IRQ routing entry to an LPI, let's expose them as helpers,
> and refactor the existing code to use them
> 
> Reviewed-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 93 +++++++++++++++++++++++++-------------------
>  virt/kvm/arm/vgic/vgic.h     |  4 ++
>  2 files changed, 57 insertions(+), 40 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index f51c1e1b3f70..475dffd0a44c 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -503,15 +503,8 @@ static unsigned long vgic_mmio_read_its_idregs(struct kvm *kvm,
>  	return 0;
>  }
>  
> -/*
> - * Find the target VCPU and the LPI number for a given devid/eventid pair
> - * and make this IRQ pending, possibly injecting it.
> - * Must be called with the its_lock mutex held.
> - * Returns 0 on success, a positive error value for any ITS mapping
> - * related errors and negative error values for generic errors.
> - */
> -static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
> -				u32 devid, u32 eventid)
> +int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
> +			 u32 devid, u32 eventid, struct vgic_irq **irq)
>  {
>  	struct kvm_vcpu *vcpu;
>  	struct its_ite *ite;
> @@ -530,26 +523,60 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>  	if (!vcpu->arch.vgic_cpu.lpis_enabled)
>  		return -EBUSY;
>  
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->pending_latch = true;
> -	vgic_queue_irq_unlock(kvm, ite->irq);
> -
> +	*irq = ite->irq;
>  	return 0;
>  }
>  
> -static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
> +struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
>  {
> +	u64 address;
> +	struct kvm_io_device *kvm_io_dev;
>  	struct vgic_io_device *iodev;
>  
> -	if (dev->ops != &kvm_io_gic_ops)
> -		return NULL;
> +	if (!vgic_has_its(kvm))
> +		return ERR_PTR(-ENODEV);
>  
> -	iodev = container_of(dev, struct vgic_io_device, dev);
> +	if (!(msi->flags & KVM_MSI_VALID_DEVID))
> +		return ERR_PTR(-EINVAL);
> +
> +	address = (u64)msi->address_hi << 32 | msi->address_lo;
> +
> +	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
> +	if (!kvm_io_dev)
> +		return ERR_PTR(-EINVAL);
>  
> +	if (kvm_io_dev->ops != &kvm_io_gic_ops)
> +		return ERR_PTR(-EINVAL);
> +
> +	iodev = container_of(kvm_io_dev, struct vgic_io_device, dev);
>  	if (iodev->iodev_type != IODEV_ITS)
> -		return NULL;
> +		return ERR_PTR(-EINVAL);
>  
> -	return iodev;
> +	return iodev->its;
> +}
> +
> +/*
> + * Find the target VCPU and the LPI number for a given devid/eventid pair
> + * and make this IRQ pending, possibly injecting it.
> + * Must be called with the its_lock mutex held.
> + * Returns 0 on success, a positive error value for any ITS mapping
> + * related errors and negative error values for generic errors.
> + */
> +static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
> +				u32 devid, u32 eventid)
> +{
> +	struct vgic_irq *irq = NULL;
> +	int err;
> +
> +	err = vgic_its_resolve_lpi(kvm, its, devid, eventid, &irq);
> +	if (err)
> +		return err;
> +
> +	spin_lock(&irq->irq_lock);
> +	irq->pending_latch = true;
> +	vgic_queue_irq_unlock(kvm, irq);
> +
> +	return 0;
>  }
>  
>  /*
> @@ -560,30 +587,16 @@ static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
>   */
>  int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
>  {
> -	u64 address;
> -	struct kvm_io_device *kvm_io_dev;
> -	struct vgic_io_device *iodev;
> +	struct vgic_its *its;
>  	int ret;
>  
> -	if (!vgic_has_its(kvm))
> -		return -ENODEV;
> -
> -	if (!(msi->flags & KVM_MSI_VALID_DEVID))
> -		return -EINVAL;
> -
> -	address = (u64)msi->address_hi << 32 | msi->address_lo;
> -
> -	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
> -	if (!kvm_io_dev)
> -		return -EINVAL;
> +	its = vgic_msi_to_its(kvm, msi);
> +	if (IS_ERR(its))
> +		return PTR_ERR(its);
>  
> -	iodev = vgic_get_its_iodev(kvm_io_dev);
> -	if (!iodev)
> -		return -EINVAL;
> -
> -	mutex_lock(&iodev->its->its_lock);
> -	ret = vgic_its_trigger_msi(kvm, iodev->its, msi->devid, msi->data);
> -	mutex_unlock(&iodev->its->its_lock);
> +	mutex_lock(&its->its_lock);
> +	ret = vgic_its_trigger_msi(kvm, its, msi->devid, msi->data);
> +	mutex_unlock(&its->its_lock);
>  
>  	if (ret < 0)
>  		return ret;
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index bf9ceab67c77..31b70326b966 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -236,4 +236,8 @@ static inline int vgic_v3_max_apr_idx(struct kvm_vcpu *vcpu)
>  	}
>  }
>  
> +int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
> +			 u32 devid, u32 eventid, struct vgic_irq **irq);
> +struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
> +
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 06/26] KVM: arm/arm64: vITS: Add MSI translation helpers
@ 2017-11-07 20:34     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:34 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The whole MSI injection process is fairly monolithic. An MSI write
> gets turned into an injected LPI in one swift go. But this is actually
> a more fine-grained process:
> 
> - First, a virtual ITS gets selected using the doorbell address
> - Then the DevID/EventID pair gets translated into an LPI
> - Finally the LPI is injected
> 
> Since the GICv4 code needs the first two steps in order to match
> an IRQ routing entry to an LPI, let's expose them as helpers,
> and refactor the existing code to use them
> 
> Reviewed-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 93 +++++++++++++++++++++++++-------------------
>  virt/kvm/arm/vgic/vgic.h     |  4 ++
>  2 files changed, 57 insertions(+), 40 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index f51c1e1b3f70..475dffd0a44c 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -503,15 +503,8 @@ static unsigned long vgic_mmio_read_its_idregs(struct kvm *kvm,
>  	return 0;
>  }
>  
> -/*
> - * Find the target VCPU and the LPI number for a given devid/eventid pair
> - * and make this IRQ pending, possibly injecting it.
> - * Must be called with the its_lock mutex held.
> - * Returns 0 on success, a positive error value for any ITS mapping
> - * related errors and negative error values for generic errors.
> - */
> -static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
> -				u32 devid, u32 eventid)
> +int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
> +			 u32 devid, u32 eventid, struct vgic_irq **irq)
>  {
>  	struct kvm_vcpu *vcpu;
>  	struct its_ite *ite;
> @@ -530,26 +523,60 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>  	if (!vcpu->arch.vgic_cpu.lpis_enabled)
>  		return -EBUSY;
>  
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->pending_latch = true;
> -	vgic_queue_irq_unlock(kvm, ite->irq);
> -
> +	*irq = ite->irq;
>  	return 0;
>  }
>  
> -static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
> +struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
>  {
> +	u64 address;
> +	struct kvm_io_device *kvm_io_dev;
>  	struct vgic_io_device *iodev;
>  
> -	if (dev->ops != &kvm_io_gic_ops)
> -		return NULL;
> +	if (!vgic_has_its(kvm))
> +		return ERR_PTR(-ENODEV);
>  
> -	iodev = container_of(dev, struct vgic_io_device, dev);
> +	if (!(msi->flags & KVM_MSI_VALID_DEVID))
> +		return ERR_PTR(-EINVAL);
> +
> +	address = (u64)msi->address_hi << 32 | msi->address_lo;
> +
> +	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
> +	if (!kvm_io_dev)
> +		return ERR_PTR(-EINVAL);
>  
> +	if (kvm_io_dev->ops != &kvm_io_gic_ops)
> +		return ERR_PTR(-EINVAL);
> +
> +	iodev = container_of(kvm_io_dev, struct vgic_io_device, dev);
>  	if (iodev->iodev_type != IODEV_ITS)
> -		return NULL;
> +		return ERR_PTR(-EINVAL);
>  
> -	return iodev;
> +	return iodev->its;
> +}
> +
> +/*
> + * Find the target VCPU and the LPI number for a given devid/eventid pair
> + * and make this IRQ pending, possibly injecting it.
> + * Must be called with the its_lock mutex held.
> + * Returns 0 on success, a positive error value for any ITS mapping
> + * related errors and negative error values for generic errors.
> + */
> +static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
> +				u32 devid, u32 eventid)
> +{
> +	struct vgic_irq *irq = NULL;
> +	int err;
> +
> +	err = vgic_its_resolve_lpi(kvm, its, devid, eventid, &irq);
> +	if (err)
> +		return err;
> +
> +	spin_lock(&irq->irq_lock);
> +	irq->pending_latch = true;
> +	vgic_queue_irq_unlock(kvm, irq);
> +
> +	return 0;
>  }
>  
>  /*
> @@ -560,30 +587,16 @@ static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
>   */
>  int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
>  {
> -	u64 address;
> -	struct kvm_io_device *kvm_io_dev;
> -	struct vgic_io_device *iodev;
> +	struct vgic_its *its;
>  	int ret;
>  
> -	if (!vgic_has_its(kvm))
> -		return -ENODEV;
> -
> -	if (!(msi->flags & KVM_MSI_VALID_DEVID))
> -		return -EINVAL;
> -
> -	address = (u64)msi->address_hi << 32 | msi->address_lo;
> -
> -	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
> -	if (!kvm_io_dev)
> -		return -EINVAL;
> +	its = vgic_msi_to_its(kvm, msi);
> +	if (IS_ERR(its))
> +		return PTR_ERR(its);
>  
> -	iodev = vgic_get_its_iodev(kvm_io_dev);
> -	if (!iodev)
> -		return -EINVAL;
> -
> -	mutex_lock(&iodev->its->its_lock);
> -	ret = vgic_its_trigger_msi(kvm, iodev->its, msi->devid, msi->data);
> -	mutex_unlock(&iodev->its->its_lock);
> +	mutex_lock(&its->its_lock);
> +	ret = vgic_its_trigger_msi(kvm, its, msi->devid, msi->data);
> +	mutex_unlock(&its->its_lock);
>  
>  	if (ret < 0)
>  		return ret;
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index bf9ceab67c77..31b70326b966 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -236,4 +236,8 @@ static inline int vgic_v3_max_apr_idx(struct kvm_vcpu *vcpu)
>  	}
>  }
>  
> +int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
> +			 u32 devid, u32 eventid, struct vgic_irq **irq);
> +struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
> +
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 06/26] KVM: arm/arm64: vITS: Add MSI translation helpers
@ 2017-11-07 20:34     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 20:34 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The whole MSI injection process is fairly monolithic. An MSI write
> gets turned into an injected LPI in one swift go. But this is actually
> a more fine-grained process:
> 
> - First, a virtual ITS gets selected using the doorbell address
> - Then the DevID/EventID pair gets translated into an LPI
> - Finally the LPI is injected
> 
> Since the GICv4 code needs the first two steps in order to match
> an IRQ routing entry to an LPI, let's expose them as helpers,
> and refactor the existing code to use them
> 
> Reviewed-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 93 +++++++++++++++++++++++++-------------------
>  virt/kvm/arm/vgic/vgic.h     |  4 ++
>  2 files changed, 57 insertions(+), 40 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index f51c1e1b3f70..475dffd0a44c 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -503,15 +503,8 @@ static unsigned long vgic_mmio_read_its_idregs(struct kvm *kvm,
>  	return 0;
>  }
>  
> -/*
> - * Find the target VCPU and the LPI number for a given devid/eventid pair
> - * and make this IRQ pending, possibly injecting it.
> - * Must be called with the its_lock mutex held.
> - * Returns 0 on success, a positive error value for any ITS mapping
> - * related errors and negative error values for generic errors.
> - */
> -static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
> -				u32 devid, u32 eventid)
> +int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
> +			 u32 devid, u32 eventid, struct vgic_irq **irq)
>  {
>  	struct kvm_vcpu *vcpu;
>  	struct its_ite *ite;
> @@ -530,26 +523,60 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>  	if (!vcpu->arch.vgic_cpu.lpis_enabled)
>  		return -EBUSY;
>  
> -	spin_lock(&ite->irq->irq_lock);
> -	ite->irq->pending_latch = true;
> -	vgic_queue_irq_unlock(kvm, ite->irq);
> -
> +	*irq = ite->irq;
>  	return 0;
>  }
>  
> -static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
> +struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
>  {
> +	u64 address;
> +	struct kvm_io_device *kvm_io_dev;
>  	struct vgic_io_device *iodev;
>  
> -	if (dev->ops != &kvm_io_gic_ops)
> -		return NULL;
> +	if (!vgic_has_its(kvm))
> +		return ERR_PTR(-ENODEV);
>  
> -	iodev = container_of(dev, struct vgic_io_device, dev);
> +	if (!(msi->flags & KVM_MSI_VALID_DEVID))
> +		return ERR_PTR(-EINVAL);
> +
> +	address = (u64)msi->address_hi << 32 | msi->address_lo;
> +
> +	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
> +	if (!kvm_io_dev)
> +		return ERR_PTR(-EINVAL);
>  
> +	if (kvm_io_dev->ops != &kvm_io_gic_ops)
> +		return ERR_PTR(-EINVAL);
> +
> +	iodev = container_of(kvm_io_dev, struct vgic_io_device, dev);
>  	if (iodev->iodev_type != IODEV_ITS)
> -		return NULL;
> +		return ERR_PTR(-EINVAL);
>  
> -	return iodev;
> +	return iodev->its;
> +}
> +
> +/*
> + * Find the target VCPU and the LPI number for a given devid/eventid pair
> + * and make this IRQ pending, possibly injecting it.
> + * Must be called with the its_lock mutex held.
> + * Returns 0 on success, a positive error value for any ITS mapping
> + * related errors and negative error values for generic errors.
> + */
> +static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
> +				u32 devid, u32 eventid)
> +{
> +	struct vgic_irq *irq = NULL;
> +	int err;
> +
> +	err = vgic_its_resolve_lpi(kvm, its, devid, eventid, &irq);
> +	if (err)
> +		return err;
> +
> +	spin_lock(&irq->irq_lock);
> +	irq->pending_latch = true;
> +	vgic_queue_irq_unlock(kvm, irq);
> +
> +	return 0;
>  }
>  
>  /*
> @@ -560,30 +587,16 @@ static struct vgic_io_device *vgic_get_its_iodev(struct kvm_io_device *dev)
>   */
>  int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
>  {
> -	u64 address;
> -	struct kvm_io_device *kvm_io_dev;
> -	struct vgic_io_device *iodev;
> +	struct vgic_its *its;
>  	int ret;
>  
> -	if (!vgic_has_its(kvm))
> -		return -ENODEV;
> -
> -	if (!(msi->flags & KVM_MSI_VALID_DEVID))
> -		return -EINVAL;
> -
> -	address = (u64)msi->address_hi << 32 | msi->address_lo;
> -
> -	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
> -	if (!kvm_io_dev)
> -		return -EINVAL;
> +	its = vgic_msi_to_its(kvm, msi);
> +	if (IS_ERR(its))
> +		return PTR_ERR(its);
>  
> -	iodev = vgic_get_its_iodev(kvm_io_dev);
> -	if (!iodev)
> -		return -EINVAL;
> -
> -	mutex_lock(&iodev->its->its_lock);
> -	ret = vgic_its_trigger_msi(kvm, iodev->its, msi->devid, msi->data);
> -	mutex_unlock(&iodev->its->its_lock);
> +	mutex_lock(&its->its_lock);
> +	ret = vgic_its_trigger_msi(kvm, its, msi->devid, msi->data);
> +	mutex_unlock(&its->its_lock);
>  
>  	if (ret < 0)
>  		return ret;
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index bf9ceab67c77..31b70326b966 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -236,4 +236,8 @@ static inline int vgic_v3_max_apr_idx(struct kvm_vcpu *vcpu)
>  	}
>  }
>  
> +int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,
> +			 u32 devid, u32 eventid, struct vgic_irq **irq);
> +struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
> +
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:01     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:01 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When the guest issues an affinity change, we need to tell the physical
> ITS that we're now targetting a new vcpu.  This is done by extracting
> the current mapping, updating the target, and reapplying the mapping.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index c9b1c0967426..42ffb9084bb7 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>  
>  static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
>  {
> +	int ret = 0;
> +
>  	spin_lock(&irq->irq_lock);
>  	irq->target_vcpu = vcpu;
>  	spin_unlock(&irq->irq_lock);
>  
> -	return 0;
> +	if (irq->hw) {
> +		struct its_vlpi_map map;
> +
> +		ret = its_get_vlpi(irq->host_irq, &map);
> +		if (ret)
> +			return ret;
> +
> +		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +		ret = its_map_vlpi(irq->host_irq, &map);
Do we check somewhere the virtual LPI ID does not exceed the capacity of
the VPT. I fail to find that in the irq-gic-v3-its.c driver.

Also in case the hw part fails, should we still have the irq->target_cpu
set to the new value. In other words shouldn't we first do the hw block
and upon success set the target_vcpu?

Thanks

Eric
> +	}
> +
> +	return ret;
>  }
>  
>  /*
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
@ 2017-11-07 21:01     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:01 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When the guest issues an affinity change, we need to tell the physical
> ITS that we're now targetting a new vcpu.  This is done by extracting
> the current mapping, updating the target, and reapplying the mapping.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index c9b1c0967426..42ffb9084bb7 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>  
>  static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
>  {
> +	int ret = 0;
> +
>  	spin_lock(&irq->irq_lock);
>  	irq->target_vcpu = vcpu;
>  	spin_unlock(&irq->irq_lock);
>  
> -	return 0;
> +	if (irq->hw) {
> +		struct its_vlpi_map map;
> +
> +		ret = its_get_vlpi(irq->host_irq, &map);
> +		if (ret)
> +			return ret;
> +
> +		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +		ret = its_map_vlpi(irq->host_irq, &map);
Do we check somewhere the virtual LPI ID does not exceed the capacity of
the VPT. I fail to find that in the irq-gic-v3-its.c driver.

Also in case the hw part fails, should we still have the irq->target_cpu
set to the new value. In other words shouldn't we first do the hw block
and upon success set the target_vcpu?

Thanks

Eric
> +	}
> +
> +	return ret;
>  }
>  
>  /*
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
@ 2017-11-07 21:01     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:01 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When the guest issues an affinity change, we need to tell the physical
> ITS that we're now targetting a new vcpu.  This is done by extracting
> the current mapping, updating the target, and reapplying the mapping.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index c9b1c0967426..42ffb9084bb7 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>  
>  static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
>  {
> +	int ret = 0;
> +
>  	spin_lock(&irq->irq_lock);
>  	irq->target_vcpu = vcpu;
>  	spin_unlock(&irq->irq_lock);
>  
> -	return 0;
> +	if (irq->hw) {
> +		struct its_vlpi_map map;
> +
> +		ret = its_get_vlpi(irq->host_irq, &map);
> +		if (ret)
> +			return ret;
> +
> +		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
> +
> +		ret = its_map_vlpi(irq->host_irq, &map);
Do we check somewhere the virtual LPI ID does not exceed the capacity of
the VPT. I fail to find that in the irq-gic-v3-its.c driver.

Also in case the hw part fails, should we still have the irq->target_cpu
set to the new value. In other words shouldn't we first do the hw block
and upon success set the target_vcpu?

Thanks

Eric
> +	}
> +
> +	return ret;
>  }
>  
>  /*
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 14/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:04     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:04 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> Handling CLEAR is pretty easy. Just ask the ITS driver to clear
> the corresponding pending bit (which will turn into a CLEAR
> command on the physical side).
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 42ffb9084bb7..5778b50911e8 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1070,6 +1070,10 @@ static int vgic_its_cmd_handle_clear(struct kvm *kvm, struct vgic_its *its,
>  
>  	ite->irq->pending_latch = false;
>  
> +	if (ite->irq->hw)
> +		return irq_set_irqchip_state(ite->irq->host_irq,
> +					     IRQCHIP_STATE_PENDING, false);
> +
nit: could be done before the pending_latch setting as this latter is
not used in that case.

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric


>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 14/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
@ 2017-11-07 21:04     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:04 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> Handling CLEAR is pretty easy. Just ask the ITS driver to clear
> the corresponding pending bit (which will turn into a CLEAR
> command on the physical side).
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 42ffb9084bb7..5778b50911e8 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1070,6 +1070,10 @@ static int vgic_its_cmd_handle_clear(struct kvm *kvm, struct vgic_its *its,
>  
>  	ite->irq->pending_latch = false;
>  
> +	if (ite->irq->hw)
> +		return irq_set_irqchip_state(ite->irq->host_irq,
> +					     IRQCHIP_STATE_PENDING, false);
> +
nit: could be done before the pending_latch setting as this latter is
not used in that case.

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric


>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 14/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI
@ 2017-11-07 21:04     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:04 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> Handling CLEAR is pretty easy. Just ask the ITS driver to clear
> the corresponding pending bit (which will turn into a CLEAR
> command on the physical side).
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 42ffb9084bb7..5778b50911e8 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1070,6 +1070,10 @@ static int vgic_its_cmd_handle_clear(struct kvm *kvm, struct vgic_its *its,
>  
>  	ite->irq->pending_latch = false;
>  
> +	if (ite->irq->hw)
> +		return irq_set_irqchip_state(ite->irq->host_irq,
> +					     IRQCHIP_STATE_PENDING, false);
> +
nit: could be done before the pending_latch setting as this latter is
not used in that case.

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric


>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 15/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:06     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:06 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,
On 27/10/2017 16:28, Marc Zyngier wrote:
> The current implementation of MOVALL doesn't allow us to call
> into the core ITS code as we hold a number of spinlocks.
> 
> Let's try a method used in other parts of the code, were we copy
nit: where
> the intids of the candicate interrupts, and then do whatever
candidate
> we need to do with them outside of the critical section.
> 
> This allows us to move the interrupts one by one, at the expense
> of a bit of CPU time. Who cares? MOVALL is such a stupid command
> anyway...
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 19 ++++++++++---------
>  1 file changed, 10 insertions(+), 9 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 5778b50911e8..0b7e648e7a0c 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1148,11 +1148,12 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
>  static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
>  				      u64 *its_cmd)
>  {
> -	struct vgic_dist *dist = &kvm->arch.vgic;
>  	u32 target1_addr = its_cmd_get_target_addr(its_cmd);
>  	u32 target2_addr = its_cmd_mask_field(its_cmd, 3, 16, 32);
>  	struct kvm_vcpu *vcpu1, *vcpu2;
>  	struct vgic_irq *irq;
> +	u32 *intids;
> +	int irq_count, i;
>  
>  	if (target1_addr >= atomic_read(&kvm->online_vcpus) ||
>  	    target2_addr >= atomic_read(&kvm->online_vcpus))
> @@ -1164,19 +1165,19 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
>  	vcpu1 = kvm_get_vcpu(kvm, target1_addr);
>  	vcpu2 = kvm_get_vcpu(kvm, target2_addr);
>  
> -	spin_lock(&dist->lpi_list_lock);
> +	irq_count = vgic_copy_lpi_list(vcpu1, &intids);
> +	if (irq_count < 0)
nit: <=?

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

> +		return irq_count;
>  
> -	list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
> -		spin_lock(&irq->irq_lock);
> +	for (i = 0; i < irq_count; i++) {
> +		irq = vgic_get_irq(kvm, NULL, intids[i]);
>  
> -		if (irq->target_vcpu == vcpu1)
> -			irq->target_vcpu = vcpu2;
> +		update_affinity(irq, vcpu2);
>  
> -		spin_unlock(&irq->irq_lock);
> +		vgic_put_irq(kvm, irq);
>  	}
>  
> -	spin_unlock(&dist->lpi_list_lock);
> -
> +	kfree(intids);
>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 15/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
@ 2017-11-07 21:06     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:06 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,
On 27/10/2017 16:28, Marc Zyngier wrote:
> The current implementation of MOVALL doesn't allow us to call
> into the core ITS code as we hold a number of spinlocks.
> 
> Let's try a method used in other parts of the code, were we copy
nit: where
> the intids of the candicate interrupts, and then do whatever
candidate
> we need to do with them outside of the critical section.
> 
> This allows us to move the interrupts one by one, at the expense
> of a bit of CPU time. Who cares? MOVALL is such a stupid command
> anyway...
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 19 ++++++++++---------
>  1 file changed, 10 insertions(+), 9 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 5778b50911e8..0b7e648e7a0c 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1148,11 +1148,12 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
>  static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
>  				      u64 *its_cmd)
>  {
> -	struct vgic_dist *dist = &kvm->arch.vgic;
>  	u32 target1_addr = its_cmd_get_target_addr(its_cmd);
>  	u32 target2_addr = its_cmd_mask_field(its_cmd, 3, 16, 32);
>  	struct kvm_vcpu *vcpu1, *vcpu2;
>  	struct vgic_irq *irq;
> +	u32 *intids;
> +	int irq_count, i;
>  
>  	if (target1_addr >= atomic_read(&kvm->online_vcpus) ||
>  	    target2_addr >= atomic_read(&kvm->online_vcpus))
> @@ -1164,19 +1165,19 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
>  	vcpu1 = kvm_get_vcpu(kvm, target1_addr);
>  	vcpu2 = kvm_get_vcpu(kvm, target2_addr);
>  
> -	spin_lock(&dist->lpi_list_lock);
> +	irq_count = vgic_copy_lpi_list(vcpu1, &intids);
> +	if (irq_count < 0)
nit: <=?

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

> +		return irq_count;
>  
> -	list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
> -		spin_lock(&irq->irq_lock);
> +	for (i = 0; i < irq_count; i++) {
> +		irq = vgic_get_irq(kvm, NULL, intids[i]);
>  
> -		if (irq->target_vcpu == vcpu1)
> -			irq->target_vcpu = vcpu2;
> +		update_affinity(irq, vcpu2);
>  
> -		spin_unlock(&irq->irq_lock);
> +		vgic_put_irq(kvm, irq);
>  	}
>  
> -	spin_unlock(&dist->lpi_list_lock);
> -
> +	kfree(intids);
>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 15/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE
@ 2017-11-07 21:06     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:06 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,
On 27/10/2017 16:28, Marc Zyngier wrote:
> The current implementation of MOVALL doesn't allow us to call
> into the core ITS code as we hold a number of spinlocks.
> 
> Let's try a method used in other parts of the code, were we copy
nit: where
> the intids of the candicate interrupts, and then do whatever
candidate
> we need to do with them outside of the critical section.
> 
> This allows us to move the interrupts one by one, at the expense
> of a bit of CPU time. Who cares? MOVALL is such a stupid command
> anyway...
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 19 ++++++++++---------
>  1 file changed, 10 insertions(+), 9 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 5778b50911e8..0b7e648e7a0c 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1148,11 +1148,12 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
>  static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
>  				      u64 *its_cmd)
>  {
> -	struct vgic_dist *dist = &kvm->arch.vgic;
>  	u32 target1_addr = its_cmd_get_target_addr(its_cmd);
>  	u32 target2_addr = its_cmd_mask_field(its_cmd, 3, 16, 32);
>  	struct kvm_vcpu *vcpu1, *vcpu2;
>  	struct vgic_irq *irq;
> +	u32 *intids;
> +	int irq_count, i;
>  
>  	if (target1_addr >= atomic_read(&kvm->online_vcpus) ||
>  	    target2_addr >= atomic_read(&kvm->online_vcpus))
> @@ -1164,19 +1165,19 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
>  	vcpu1 = kvm_get_vcpu(kvm, target1_addr);
>  	vcpu2 = kvm_get_vcpu(kvm, target2_addr);
>  
> -	spin_lock(&dist->lpi_list_lock);
> +	irq_count = vgic_copy_lpi_list(vcpu1, &intids);
> +	if (irq_count < 0)
nit: <=?

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

> +		return irq_count;
>  
> -	list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) {
> -		spin_lock(&irq->irq_lock);
> +	for (i = 0; i < irq_count; i++) {
> +		irq = vgic_get_irq(kvm, NULL, intids[i]);
>  
> -		if (irq->target_vcpu == vcpu1)
> -			irq->target_vcpu = vcpu2;
> +		update_affinity(irq, vcpu2);
>  
> -		spin_unlock(&irq->irq_lock);
> +		vgic_put_irq(kvm, irq);
>  	}
>  
> -	spin_unlock(&dist->lpi_list_lock);
> -
> +	kfree(intids);
>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:23     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:23 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Since when updating the properties one LPI at a time, there is no
Since we update the properties one LPI at a time, ... ?
> need to perform an INV each time we read one. Instead, we rely
> on the final VINVALL that gets sent to the ITS to do the work.
The commit message is not crystal clear for me.

I understand in case of vgic_its_cmd_handle_invall you want to avoid
doing an invalidation for each physical irq but rather do an
its_invall_vpe at the end. So you add a new  @needs_inv arg to
update_lpi_config to tell whether the invalidation should be done or not.

Besides

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 2e77c7c83942..eb72eb027060 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -38,7 +38,7 @@ static int vgic_its_save_tables_v0(struct vgic_its *its);
>  static int vgic_its_restore_tables_v0(struct vgic_its *its);
>  static int vgic_its_commit_v0(struct vgic_its *its);
>  static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> -			     struct kvm_vcpu *filter_vcpu);
> +			     struct kvm_vcpu *filter_vcpu, bool needs_inv);
>  
>  /*
>   * Creates a new (reference to a) struct vgic_irq for a given LPI.
> @@ -106,7 +106,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
>  	 * However we only have those structs for mapped IRQs, so we read in
>  	 * the respective config data from memory here upon mapping the LPI.
>  	 */
> -	ret = update_lpi_config(kvm, irq, NULL);
> +	ret = update_lpi_config(kvm, irq, NULL, false);
>  	if (ret)
>  		return ERR_PTR(ret);
>  
> @@ -273,7 +273,7 @@ static struct its_collection *find_collection(struct vgic_its *its, int coll_id)
>   * VCPU. Unconditionally applies if filter_vcpu is NULL.
>   */
>  static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> -			     struct kvm_vcpu *filter_vcpu)
> +			     struct kvm_vcpu *filter_vcpu, bool needs_inv)
>  {
>  	u64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);
>  	u8 prop;
> @@ -297,7 +297,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  	}
>  
>  	if (irq->hw)
> -		return its_prop_update_vlpi(irq->host_irq, prop, true);
> +		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
>  
>  	return 0;
>  }
> @@ -1096,7 +1096,7 @@ static int vgic_its_cmd_handle_inv(struct kvm *kvm, struct vgic_its *its,
>  	if (!ite)
>  		return E_ITS_INV_UNMAPPED_INTERRUPT;
>  
> -	return update_lpi_config(kvm, ite->irq, NULL);
> +	return update_lpi_config(kvm, ite->irq, NULL, true);
>  }
>  
>  /*
> @@ -1131,12 +1131,15 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
>  		irq = vgic_get_irq(kvm, NULL, intids[i]);
>  		if (!irq)
>  			continue;
> -		update_lpi_config(kvm, irq, vcpu);
> +		update_lpi_config(kvm, irq, vcpu, false);
>  		vgic_put_irq(kvm, irq);
>  	}
>  
>  	kfree(intids);
>  
> +	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.its_vm)
> +		its_invall_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe);
> +
>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-11-07 21:23     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:23 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Since when updating the properties one LPI at a time, there is no
Since we update the properties one LPI at a time, ... ?
> need to perform an INV each time we read one. Instead, we rely
> on the final VINVALL that gets sent to the ITS to do the work.
The commit message is not crystal clear for me.

I understand in case of vgic_its_cmd_handle_invall you want to avoid
doing an invalidation for each physical irq but rather do an
its_invall_vpe at the end. So you add a new  @needs_inv arg to
update_lpi_config to tell whether the invalidation should be done or not.

Besides

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 2e77c7c83942..eb72eb027060 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -38,7 +38,7 @@ static int vgic_its_save_tables_v0(struct vgic_its *its);
>  static int vgic_its_restore_tables_v0(struct vgic_its *its);
>  static int vgic_its_commit_v0(struct vgic_its *its);
>  static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> -			     struct kvm_vcpu *filter_vcpu);
> +			     struct kvm_vcpu *filter_vcpu, bool needs_inv);
>  
>  /*
>   * Creates a new (reference to a) struct vgic_irq for a given LPI.
> @@ -106,7 +106,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
>  	 * However we only have those structs for mapped IRQs, so we read in
>  	 * the respective config data from memory here upon mapping the LPI.
>  	 */
> -	ret = update_lpi_config(kvm, irq, NULL);
> +	ret = update_lpi_config(kvm, irq, NULL, false);
>  	if (ret)
>  		return ERR_PTR(ret);
>  
> @@ -273,7 +273,7 @@ static struct its_collection *find_collection(struct vgic_its *its, int coll_id)
>   * VCPU. Unconditionally applies if filter_vcpu is NULL.
>   */
>  static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> -			     struct kvm_vcpu *filter_vcpu)
> +			     struct kvm_vcpu *filter_vcpu, bool needs_inv)
>  {
>  	u64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);
>  	u8 prop;
> @@ -297,7 +297,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  	}
>  
>  	if (irq->hw)
> -		return its_prop_update_vlpi(irq->host_irq, prop, true);
> +		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
>  
>  	return 0;
>  }
> @@ -1096,7 +1096,7 @@ static int vgic_its_cmd_handle_inv(struct kvm *kvm, struct vgic_its *its,
>  	if (!ite)
>  		return E_ITS_INV_UNMAPPED_INTERRUPT;
>  
> -	return update_lpi_config(kvm, ite->irq, NULL);
> +	return update_lpi_config(kvm, ite->irq, NULL, true);
>  }
>  
>  /*
> @@ -1131,12 +1131,15 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
>  		irq = vgic_get_irq(kvm, NULL, intids[i]);
>  		if (!irq)
>  			continue;
> -		update_lpi_config(kvm, irq, vcpu);
> +		update_lpi_config(kvm, irq, vcpu, false);
>  		vgic_put_irq(kvm, irq);
>  	}
>  
>  	kfree(intids);
>  
> +	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.its_vm)
> +		its_invall_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe);
> +
>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-11-07 21:23     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:23 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Since when updating the properties one LPI at a time, there is no
Since we update the properties one LPI at a time, ... ?
> need to perform an INV each time we read one. Instead, we rely
> on the final VINVALL that gets sent to the ITS to do the work.
The commit message is not crystal clear for me.

I understand in case of vgic_its_cmd_handle_invall you want to avoid
doing an invalidation for each physical irq but rather do an
its_invall_vpe at the end. So you add a new  @needs_inv arg to
update_lpi_config to tell whether the invalidation should be done or not.

Besides

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 2e77c7c83942..eb72eb027060 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -38,7 +38,7 @@ static int vgic_its_save_tables_v0(struct vgic_its *its);
>  static int vgic_its_restore_tables_v0(struct vgic_its *its);
>  static int vgic_its_commit_v0(struct vgic_its *its);
>  static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> -			     struct kvm_vcpu *filter_vcpu);
> +			     struct kvm_vcpu *filter_vcpu, bool needs_inv);
>  
>  /*
>   * Creates a new (reference to a) struct vgic_irq for a given LPI.
> @@ -106,7 +106,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
>  	 * However we only have those structs for mapped IRQs, so we read in
>  	 * the respective config data from memory here upon mapping the LPI.
>  	 */
> -	ret = update_lpi_config(kvm, irq, NULL);
> +	ret = update_lpi_config(kvm, irq, NULL, false);
>  	if (ret)
>  		return ERR_PTR(ret);
>  
> @@ -273,7 +273,7 @@ static struct its_collection *find_collection(struct vgic_its *its, int coll_id)
>   * VCPU. Unconditionally applies if filter_vcpu is NULL.
>   */
>  static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> -			     struct kvm_vcpu *filter_vcpu)
> +			     struct kvm_vcpu *filter_vcpu, bool needs_inv)
>  {
>  	u64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);
>  	u8 prop;
> @@ -297,7 +297,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  	}
>  
>  	if (irq->hw)
> -		return its_prop_update_vlpi(irq->host_irq, prop, true);
> +		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
>  
>  	return 0;
>  }
> @@ -1096,7 +1096,7 @@ static int vgic_its_cmd_handle_inv(struct kvm *kvm, struct vgic_its *its,
>  	if (!ite)
>  		return E_ITS_INV_UNMAPPED_INTERRUPT;
>  
> -	return update_lpi_config(kvm, ite->irq, NULL);
> +	return update_lpi_config(kvm, ite->irq, NULL, true);
>  }
>  
>  /*
> @@ -1131,12 +1131,15 @@ static int vgic_its_cmd_handle_invall(struct kvm *kvm, struct vgic_its *its,
>  		irq = vgic_get_irq(kvm, NULL, intids[i]);
>  		if (!irq)
>  			continue;
> -		update_lpi_config(kvm, irq, vcpu);
> +		update_lpi_config(kvm, irq, vcpu, false);
>  		vgic_put_irq(kvm, irq);
>  	}
>  
>  	kfree(intids);
>  
> +	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.its_vm)
> +		its_invall_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe);
> +
>  	return 0;
>  }
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:28     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:28 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Upon updating a property, we propagate it all the way to the physical
> ITS, and ask for an INV command to be executed there.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 0b7e648e7a0c..2e77c7c83942 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  		spin_unlock(&irq->irq_lock);
>  	}
>  
> +	if (irq->hw)
> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
> +
>  	return 0;
>  }
I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
needed in hw mode?

Thanks

Eric

>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-07 21:28     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:28 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Upon updating a property, we propagate it all the way to the physical
> ITS, and ask for an INV command to be executed there.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 0b7e648e7a0c..2e77c7c83942 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  		spin_unlock(&irq->irq_lock);
>  	}
>  
> +	if (irq->hw)
> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
> +
>  	return 0;
>  }
I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
needed in hw mode?

Thanks

Eric

>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-07 21:28     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:28 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Upon updating a property, we propagate it all the way to the physical
> ITS, and ask for an INV command to be executed there.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index 0b7e648e7a0c..2e77c7c83942 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  		spin_unlock(&irq->irq_lock);
>  	}
>  
> +	if (irq->hw)
> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
> +
>  	return 0;
>  }
I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
needed in hw mode?

Thanks

Eric

>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:38     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:38 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When a vPE exits, the pending_last flag is set when there are
> pending VLPIs stored in the pending table. Similarily, we set
> this flag when a doorbell interrupt fires, as it indicates the
> same condition.
This is actually done in the next patch.

Besides:
Reviewed-by: Eric Auger <eric.auger@redhat.com>


Thanks

Eric


> 
> Let's update kvm_vgic_vcpu_pending_irq() to account for that
> flag as well, making a vcpu runnable when set.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index 9d557efd1f7d..0691a2250949 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -769,6 +769,9 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
>  	if (!vcpu->kvm->arch.vgic.enabled)
>  		return false;
>  
> +	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last)
> +		return true;
> +
>  	spin_lock(&vgic_cpu->ap_list_lock);
>  
>  	list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
@ 2017-11-07 21:38     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:38 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When a vPE exits, the pending_last flag is set when there are
> pending VLPIs stored in the pending table. Similarily, we set
> this flag when a doorbell interrupt fires, as it indicates the
> same condition.
This is actually done in the next patch.

Besides:
Reviewed-by: Eric Auger <eric.auger@redhat.com>


Thanks

Eric


> 
> Let's update kvm_vgic_vcpu_pending_irq() to account for that
> flag as well, making a vcpu runnable when set.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index 9d557efd1f7d..0691a2250949 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -769,6 +769,9 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
>  	if (!vcpu->kvm->arch.vgic.enabled)
>  		return false;
>  
> +	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last)
> +		return true;
> +
>  	spin_lock(&vgic_cpu->ap_list_lock);
>  
>  	list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint
@ 2017-11-07 21:38     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:38 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When a vPE exits, the pending_last flag is set when there are
> pending VLPIs stored in the pending table. Similarily, we set
> this flag when a doorbell interrupt fires, as it indicates the
> same condition.
This is actually done in the next patch.

Besides:
Reviewed-by: Eric Auger <eric.auger@redhat.com>


Thanks

Eric


> 
> Let's update kvm_vgic_vcpu_pending_irq() to account for that
> flag as well, making a vcpu runnable when set.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index 9d557efd1f7d..0691a2250949 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -769,6 +769,9 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
>  	if (!vcpu->kvm->arch.vgic.enabled)
>  		return false;
>  
> +	if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last)
> +		return true;
> +
>  	spin_lock(&vgic_cpu->ap_list_lock);
>  
>  	list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:43     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:43 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When a vPE is not running, a VLPI being made pending results in a
> doorbell interrupt being delivered. Let's handle this interrupt
> and update the pending_last flag that indicates that VLPIs are
> pending. The corresponding vcpu is also kicked into action.
> 
> Special care is taken to prevent the doorbell from being enabled
> at request time (this is controlled separately), and to make
> the disabling on the interrupt non-lazy.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 48 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index 01a2889b7b7c..ea898af1a7a9 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -16,12 +16,24 @@
>   */
>  
>  #include <linux/interrupt.h>
> +#include <linux/irq.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kvm_host.h>
>  #include <linux/irqchip/arm-gic-v3.h>
>  
>  #include "vgic.h"
>  
> +static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
> +{
> +	struct kvm_vcpu *vcpu = info;
> +
> +	vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last = true;
> +	kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
> +	kvm_vcpu_kick(vcpu);
> +
> +	return IRQ_HANDLED;
> +}
> +
>  /**
>   * vgic_v4_init - Initialize the GICv4 data structures
>   * @kvm:	Pointer to the VM being initialized
> @@ -61,6 +73,33 @@ int vgic_v4_init(struct kvm *kvm)
>  		return ret;
>  	}
>  
> +	kvm_for_each_vcpu(i, vcpu, kvm) {
> +		int irq = dist->its_vm.vpes[i]->irq;
> +
> +		/*
> +		 * Don't automatically enable the doorbell, as we're
> +		 * flipping it back and forth when the vcpu gets
> +		 * blocked. Also disable the lazy disabling, as the
> +		 * doorbell could kick us out of the guest too
> +		 * early...
> +		 */
> +		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		ret = request_irq(irq, vgic_v4_doorbell_handler,
> +				  0, "vcpu", vcpu);
> +		if (ret) {
> +			kvm_err("failed to allocate vcpu IRQ%d\n", irq);
> +			/*
> +			 * Trick: adjust the number of vpes so we know
> +			 * how many to nuke on teardown...
> +			 */
> +			dist->its_vm.nr_vpes = i;
> +			break;
> +		}
> +	}
> +
> +	if (ret)
> +		vgic_v4_teardown(kvm);
> +
>  	return ret;
>  }
>  
> @@ -73,10 +112,19 @@ int vgic_v4_init(struct kvm *kvm)
>  void vgic_v4_teardown(struct kvm *kvm)
>  {
>  	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +	int i;
>  
>  	if (!its_vm->vpes)
>  		return;
>  
> +	for (i = 0; i < its_vm->nr_vpes; i++) {
> +		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
> +		int irq = its_vm->vpes[i]->irq;
> +
> +		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		free_irq(irq, vcpu);
> +	}
> +
>  	its_free_vcpu_irqs(its_vm);
>  	kfree(its_vm->vpes);
>  	its_vm->nr_vpes = 0;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling
@ 2017-11-07 21:43     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:43 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When a vPE is not running, a VLPI being made pending results in a
> doorbell interrupt being delivered. Let's handle this interrupt
> and update the pending_last flag that indicates that VLPIs are
> pending. The corresponding vcpu is also kicked into action.
> 
> Special care is taken to prevent the doorbell from being enabled
> at request time (this is controlled separately), and to make
> the disabling on the interrupt non-lazy.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 48 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index 01a2889b7b7c..ea898af1a7a9 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -16,12 +16,24 @@
>   */
>  
>  #include <linux/interrupt.h>
> +#include <linux/irq.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kvm_host.h>
>  #include <linux/irqchip/arm-gic-v3.h>
>  
>  #include "vgic.h"
>  
> +static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
> +{
> +	struct kvm_vcpu *vcpu = info;
> +
> +	vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last = true;
> +	kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
> +	kvm_vcpu_kick(vcpu);
> +
> +	return IRQ_HANDLED;
> +}
> +
>  /**
>   * vgic_v4_init - Initialize the GICv4 data structures
>   * @kvm:	Pointer to the VM being initialized
> @@ -61,6 +73,33 @@ int vgic_v4_init(struct kvm *kvm)
>  		return ret;
>  	}
>  
> +	kvm_for_each_vcpu(i, vcpu, kvm) {
> +		int irq = dist->its_vm.vpes[i]->irq;
> +
> +		/*
> +		 * Don't automatically enable the doorbell, as we're
> +		 * flipping it back and forth when the vcpu gets
> +		 * blocked. Also disable the lazy disabling, as the
> +		 * doorbell could kick us out of the guest too
> +		 * early...
> +		 */
> +		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		ret = request_irq(irq, vgic_v4_doorbell_handler,
> +				  0, "vcpu", vcpu);
> +		if (ret) {
> +			kvm_err("failed to allocate vcpu IRQ%d\n", irq);
> +			/*
> +			 * Trick: adjust the number of vpes so we know
> +			 * how many to nuke on teardown...
> +			 */
> +			dist->its_vm.nr_vpes = i;
> +			break;
> +		}
> +	}
> +
> +	if (ret)
> +		vgic_v4_teardown(kvm);
> +
>  	return ret;
>  }
>  
> @@ -73,10 +112,19 @@ int vgic_v4_init(struct kvm *kvm)
>  void vgic_v4_teardown(struct kvm *kvm)
>  {
>  	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +	int i;
>  
>  	if (!its_vm->vpes)
>  		return;
>  
> +	for (i = 0; i < its_vm->nr_vpes; i++) {
> +		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
> +		int irq = its_vm->vpes[i]->irq;
> +
> +		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		free_irq(irq, vcpu);
> +	}
> +
>  	its_free_vcpu_irqs(its_vm);
>  	kfree(its_vm->vpes);
>  	its_vm->nr_vpes = 0;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling
@ 2017-11-07 21:43     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:43 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> When a vPE is not running, a VLPI being made pending results in a
> doorbell interrupt being delivered. Let's handle this interrupt
> and update the pending_last flag that indicates that VLPIs are
> pending. The corresponding vcpu is also kicked into action.
> 
> Special care is taken to prevent the doorbell from being enabled
> at request time (this is controlled separately), and to make
> the disabling on the interrupt non-lazy.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 48 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index 01a2889b7b7c..ea898af1a7a9 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -16,12 +16,24 @@
>   */
>  
>  #include <linux/interrupt.h>
> +#include <linux/irq.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kvm_host.h>
>  #include <linux/irqchip/arm-gic-v3.h>
>  
>  #include "vgic.h"
>  
> +static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
> +{
> +	struct kvm_vcpu *vcpu = info;
> +
> +	vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last = true;
> +	kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
> +	kvm_vcpu_kick(vcpu);
> +
> +	return IRQ_HANDLED;
> +}
> +
>  /**
>   * vgic_v4_init - Initialize the GICv4 data structures
>   * @kvm:	Pointer to the VM being initialized
> @@ -61,6 +73,33 @@ int vgic_v4_init(struct kvm *kvm)
>  		return ret;
>  	}
>  
> +	kvm_for_each_vcpu(i, vcpu, kvm) {
> +		int irq = dist->its_vm.vpes[i]->irq;
> +
> +		/*
> +		 * Don't automatically enable the doorbell, as we're
> +		 * flipping it back and forth when the vcpu gets
> +		 * blocked. Also disable the lazy disabling, as the
> +		 * doorbell could kick us out of the guest too
> +		 * early...
> +		 */
> +		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		ret = request_irq(irq, vgic_v4_doorbell_handler,
> +				  0, "vcpu", vcpu);
> +		if (ret) {
> +			kvm_err("failed to allocate vcpu IRQ%d\n", irq);
> +			/*
> +			 * Trick: adjust the number of vpes so we know
> +			 * how many to nuke on teardown...
> +			 */
> +			dist->its_vm.nr_vpes = i;
> +			break;
> +		}
> +	}
> +
> +	if (ret)
> +		vgic_v4_teardown(kvm);
> +
>  	return ret;
>  }
>  
> @@ -73,10 +112,19 @@ int vgic_v4_init(struct kvm *kvm)
>  void vgic_v4_teardown(struct kvm *kvm)
>  {
>  	struct its_vm *its_vm = &kvm->arch.vgic.its_vm;
> +	int i;
>  
>  	if (!its_vm->vpes)
>  		return;
>  
> +	for (i = 0; i < its_vm->nr_vpes; i++) {
> +		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
> +		int irq = its_vm->vpes[i]->irq;
> +
> +		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		free_irq(irq, vcpu);
> +	}
> +
>  	its_free_vcpu_irqs(its_vm);
>  	kfree(its_vm->vpes);
>  	its_vm->nr_vpes = 0;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:45     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:45 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The doorbell interrupt is only useful if the vcpu is blocked on WFI.
> In all other cases, recieving a doorbell interrupt is just a waste
> of cycles.
> 
> So let's only enable the doorbell if a vcpu is getting blocked,
> and disable it when it is unblocked. This is very similar to
> what we're doing for the background timer.
> 
> Reviewed-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  include/kvm/arm_vgic.h      |  3 +++
>  virt/kvm/arm/arm.c          |  2 ++
>  virt/kvm/arm/vgic/vgic-v4.c | 18 ++++++++++++++++++
>  3 files changed, 23 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 2f750c770bf2..8c896540a72c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -381,4 +381,7 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>  int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>  				 struct kvm_kernel_irq_routing_entry *irq_entry);
>  
> +void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu);
> +void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu);
> +
>  #endif /* __KVM_ARM_VGIC_H */
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 8388c1cc23f6..092a4ec03e57 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -316,11 +316,13 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
>  void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
>  {
>  	kvm_timer_schedule(vcpu);
> +	kvm_vgic_v4_enable_doorbell(vcpu);
>  }
>  
>  void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
>  {
>  	kvm_timer_unschedule(vcpu);
> +	kvm_vgic_v4_disable_doorbell(vcpu);
>  }
>  
>  int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index ea898af1a7a9..1a2f2fcdfa67 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -237,3 +237,21 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
>  	mutex_unlock(&its->its_lock);
>  	return ret;
>  }
> +
> +void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu)
> +{
> +	if (vgic_supports_direct_msis(vcpu->kvm)) {
> +		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +		if (irq)
> +			enable_irq(irq);
> +	}
> +}
> +
> +void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu)
> +{
> +	if (vgic_supports_direct_msis(vcpu->kvm)) {
> +		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +		if (irq)
> +			disable_irq(irq);
> +	}
> +}
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source
@ 2017-11-07 21:45     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:45 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The doorbell interrupt is only useful if the vcpu is blocked on WFI.
> In all other cases, recieving a doorbell interrupt is just a waste
> of cycles.
> 
> So let's only enable the doorbell if a vcpu is getting blocked,
> and disable it when it is unblocked. This is very similar to
> what we're doing for the background timer.
> 
> Reviewed-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  include/kvm/arm_vgic.h      |  3 +++
>  virt/kvm/arm/arm.c          |  2 ++
>  virt/kvm/arm/vgic/vgic-v4.c | 18 ++++++++++++++++++
>  3 files changed, 23 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 2f750c770bf2..8c896540a72c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -381,4 +381,7 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>  int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>  				 struct kvm_kernel_irq_routing_entry *irq_entry);
>  
> +void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu);
> +void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu);
> +
>  #endif /* __KVM_ARM_VGIC_H */
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 8388c1cc23f6..092a4ec03e57 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -316,11 +316,13 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
>  void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
>  {
>  	kvm_timer_schedule(vcpu);
> +	kvm_vgic_v4_enable_doorbell(vcpu);
>  }
>  
>  void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
>  {
>  	kvm_timer_unschedule(vcpu);
> +	kvm_vgic_v4_disable_doorbell(vcpu);
>  }
>  
>  int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index ea898af1a7a9..1a2f2fcdfa67 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -237,3 +237,21 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
>  	mutex_unlock(&its->its_lock);
>  	return ret;
>  }
> +
> +void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu)
> +{
> +	if (vgic_supports_direct_msis(vcpu->kvm)) {
> +		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +		if (irq)
> +			enable_irq(irq);
> +	}
> +}
> +
> +void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu)
> +{
> +	if (vgic_supports_direct_msis(vcpu->kvm)) {
> +		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +		if (irq)
> +			disable_irq(irq);
> +	}
> +}
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source
@ 2017-11-07 21:45     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:45 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The doorbell interrupt is only useful if the vcpu is blocked on WFI.
> In all other cases, recieving a doorbell interrupt is just a waste
> of cycles.
> 
> So let's only enable the doorbell if a vcpu is getting blocked,
> and disable it when it is unblocked. This is very similar to
> what we're doing for the background timer.
> 
> Reviewed-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  include/kvm/arm_vgic.h      |  3 +++
>  virt/kvm/arm/arm.c          |  2 ++
>  virt/kvm/arm/vgic/vgic-v4.c | 18 ++++++++++++++++++
>  3 files changed, 23 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 2f750c770bf2..8c896540a72c 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -381,4 +381,7 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>  int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>  				 struct kvm_kernel_irq_routing_entry *irq_entry);
>  
> +void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu);
> +void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu);
> +
>  #endif /* __KVM_ARM_VGIC_H */
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 8388c1cc23f6..092a4ec03e57 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -316,11 +316,13 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
>  void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
>  {
>  	kvm_timer_schedule(vcpu);
> +	kvm_vgic_v4_enable_doorbell(vcpu);
>  }
>  
>  void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
>  {
>  	kvm_timer_unschedule(vcpu);
> +	kvm_vgic_v4_disable_doorbell(vcpu);
>  }
>  
>  int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index ea898af1a7a9..1a2f2fcdfa67 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -237,3 +237,21 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int virq,
>  	mutex_unlock(&its->its_lock);
>  	return ret;
>  }
> +
> +void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu)
> +{
> +	if (vgic_supports_direct_msis(vcpu->kvm)) {
> +		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +		if (irq)
> +			enable_irq(irq);
> +	}
> +}
> +
> +void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu)
> +{
> +	if (vgic_supports_direct_msis(vcpu->kvm)) {
> +		int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +		if (irq)
> +			disable_irq(irq);
> +	}
> +}
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 21:54     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:54 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The redistributor needs to be told which vPE is about to be run,
> and tells us whether there is any pending VLPI on exit.
> 
> Let's add the scheduling calls to the vgic flush/sync functions,
> allowing the VLPIs to be delivered to the guest.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.c    |  4 ++++
>  virt/kvm/arm/vgic/vgic.h    |  2 ++
>  3 files changed, 45 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index 1a2f2fcdfa67..d7fe610bb1f5 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
>  	its_vm->vpes = NULL;
>  }
>  
> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
> +{
> +	if (!vgic_supports_direct_msis(vcpu->kvm))
> +		return 0;
> +
> +	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
> +}
> +
> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
> +{
> +	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +	int err;
> +
> +	if (!vgic_supports_direct_msis(vcpu->kvm))
> +		return 0;
> +
> +	/*
> +	 * Before making the VPE resident, make sure the redistributor
> +	 * corresponding to our current CPU expects us here. See the
> +	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
> +	 * turns into a VMOVP command at the ITS level.
> +	 */
I don't get the above comment. Don't you set the affinity of the
doorbell irq below?

Thanks

Eric
> +	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
> +	if (err)
> +		return err;
> +
> +	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
> +	if (err)
> +		return err;
> +
> +	/*
> +	 * Now that the VPE is resident, let's get rid of a potential
> +	 * doorbell interrupt that would still be pending.
> +	 */
> +	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
> +
> +	return err;
> +}
> +
>  static struct vgic_its *vgic_get_its(struct kvm *kvm,
>  				     struct kvm_kernel_irq_routing_entry *irq_entry)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index 0691a2250949..71ca0ff1b35a 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>  {
>  	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>  
> +	WARN_ON(vgic_v4_sync_hwstate(vcpu));
> +
>  	/* An empty ap_list_head implies used_lrs == 0 */
>  	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
>  		return;
> @@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>  /* Flush our emulation state into the GIC hardware before entering the guest. */
>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
>  {
> +	WARN_ON(vgic_v4_flush_hwstate(vcpu));
> +
>  	/*
>  	 * If there are no virtual interrupts active or pending for this
>  	 * VCPU, then there is no work to do and we can bail out without
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index c4105f613f57..9cafb61b79af 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  bool vgic_supports_direct_msis(struct kvm *kvm);
>  int vgic_v4_init(struct kvm *kvm);
>  void vgic_v4_teardown(struct kvm *kvm);
> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
@ 2017-11-07 21:54     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:54 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The redistributor needs to be told which vPE is about to be run,
> and tells us whether there is any pending VLPI on exit.
> 
> Let's add the scheduling calls to the vgic flush/sync functions,
> allowing the VLPIs to be delivered to the guest.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.c    |  4 ++++
>  virt/kvm/arm/vgic/vgic.h    |  2 ++
>  3 files changed, 45 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index 1a2f2fcdfa67..d7fe610bb1f5 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
>  	its_vm->vpes = NULL;
>  }
>  
> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
> +{
> +	if (!vgic_supports_direct_msis(vcpu->kvm))
> +		return 0;
> +
> +	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
> +}
> +
> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
> +{
> +	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +	int err;
> +
> +	if (!vgic_supports_direct_msis(vcpu->kvm))
> +		return 0;
> +
> +	/*
> +	 * Before making the VPE resident, make sure the redistributor
> +	 * corresponding to our current CPU expects us here. See the
> +	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
> +	 * turns into a VMOVP command at the ITS level.
> +	 */
I don't get the above comment. Don't you set the affinity of the
doorbell irq below?

Thanks

Eric
> +	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
> +	if (err)
> +		return err;
> +
> +	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
> +	if (err)
> +		return err;
> +
> +	/*
> +	 * Now that the VPE is resident, let's get rid of a potential
> +	 * doorbell interrupt that would still be pending.
> +	 */
> +	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
> +
> +	return err;
> +}
> +
>  static struct vgic_its *vgic_get_its(struct kvm *kvm,
>  				     struct kvm_kernel_irq_routing_entry *irq_entry)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index 0691a2250949..71ca0ff1b35a 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>  {
>  	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>  
> +	WARN_ON(vgic_v4_sync_hwstate(vcpu));
> +
>  	/* An empty ap_list_head implies used_lrs == 0 */
>  	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
>  		return;
> @@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>  /* Flush our emulation state into the GIC hardware before entering the guest. */
>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
>  {
> +	WARN_ON(vgic_v4_flush_hwstate(vcpu));
> +
>  	/*
>  	 * If there are no virtual interrupts active or pending for this
>  	 * VCPU, then there is no work to do and we can bail out without
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index c4105f613f57..9cafb61b79af 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  bool vgic_supports_direct_msis(struct kvm *kvm);
>  int vgic_v4_init(struct kvm *kvm);
>  void vgic_v4_teardown(struct kvm *kvm);
> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
@ 2017-11-07 21:54     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 21:54 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> The redistributor needs to be told which vPE is about to be run,
> and tells us whether there is any pending VLPI on exit.
> 
> Let's add the scheduling calls to the vgic flush/sync functions,
> allowing the VLPIs to be delivered to the guest.
> 
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.c    |  4 ++++
>  virt/kvm/arm/vgic/vgic.h    |  2 ++
>  3 files changed, 45 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index 1a2f2fcdfa67..d7fe610bb1f5 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
>  	its_vm->vpes = NULL;
>  }
>  
> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
> +{
> +	if (!vgic_supports_direct_msis(vcpu->kvm))
> +		return 0;
> +
> +	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
> +}
> +
> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
> +{
> +	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
> +	int err;
> +
> +	if (!vgic_supports_direct_msis(vcpu->kvm))
> +		return 0;
> +
> +	/*
> +	 * Before making the VPE resident, make sure the redistributor
> +	 * corresponding to our current CPU expects us here. See the
> +	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
> +	 * turns into a VMOVP command at the ITS level.
> +	 */
I don't get the above comment. Don't you set the affinity of the
doorbell irq below?

Thanks

Eric
> +	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
> +	if (err)
> +		return err;
> +
> +	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
> +	if (err)
> +		return err;
> +
> +	/*
> +	 * Now that the VPE is resident, let's get rid of a potential
> +	 * doorbell interrupt that would still be pending.
> +	 */
> +	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
> +
> +	return err;
> +}
> +
>  static struct vgic_its *vgic_get_its(struct kvm *kvm,
>  				     struct kvm_kernel_irq_routing_entry *irq_entry)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index 0691a2250949..71ca0ff1b35a 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>  {
>  	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>  
> +	WARN_ON(vgic_v4_sync_hwstate(vcpu));
> +
>  	/* An empty ap_list_head implies used_lrs == 0 */
>  	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
>  		return;
> @@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>  /* Flush our emulation state into the GIC hardware before entering the guest. */
>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
>  {
> +	WARN_ON(vgic_v4_flush_hwstate(vcpu));
> +
>  	/*
>  	 * If there are no virtual interrupts active or pending for this
>  	 * VCPU, then there is no work to do and we can bail out without
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index c4105f613f57..9cafb61b79af 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>  bool vgic_supports_direct_msis(struct kvm *kvm);
>  int vgic_v4_init(struct kvm *kvm);
>  void vgic_v4_teardown(struct kvm *kvm);
> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
>  
>  #endif
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
  2017-11-07 21:54     ` Auger Eric
  (?)
@ 2017-11-07 22:14       ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:14 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 07/11/2017 22:54, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> The redistributor needs to be told which vPE is about to be run,
>> and tells us whether there is any pending VLPI on exit.
>>
>> Let's add the scheduling calls to the vgic flush/sync functions,
>> allowing the VLPIs to be delivered to the guest.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
>>  virt/kvm/arm/vgic/vgic.c    |  4 ++++
>>  virt/kvm/arm/vgic/vgic.h    |  2 ++
>>  3 files changed, 45 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index 1a2f2fcdfa67..d7fe610bb1f5 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
>>  	its_vm->vpes = NULL;
>>  }
>>  
>> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
>> +{
>> +	if (!vgic_supports_direct_msis(vcpu->kvm))
>> +		return 0;
>> +
>> +	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
>> +}
>> +
>> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
>> +{
>> +	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
>> +	int err;
>> +
>> +	if (!vgic_supports_direct_msis(vcpu->kvm))
>> +		return 0;
>> +
>> +	/*
>> +	 * Before making the VPE resident, make sure the redistributor
>> +	 * corresponding to our current CPU expects us here. See the
>> +	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
>> +	 * turns into a VMOVP command at the ITS level.
>> +	 */
> I don't get the above comment. Don't you set the affinity of the
> doorbell irq below?
Forget that one. I now recall the trick and see its_vpe_set_affinity()

Eric
> 
> Thanks
> 
> Eric
>> +	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
>> +	if (err)
>> +		return err;
>> +
>> +	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
>> +	if (err)
>> +		return err;
>> +
>> +	/*
>> +	 * Now that the VPE is resident, let's get rid of a potential
>> +	 * doorbell interrupt that would still be pending.
>> +	 */
>> +	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
>> +
>> +	return err;
>> +}
>> +
>>  static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>  				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>  {
>> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
>> index 0691a2250949..71ca0ff1b35a 100644
>> --- a/virt/kvm/arm/vgic/vgic.c
>> +++ b/virt/kvm/arm/vgic/vgic.c
>> @@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>>  {
>>  	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>>  
>> +	WARN_ON(vgic_v4_sync_hwstate(vcpu));
>> +
>>  	/* An empty ap_list_head implies used_lrs == 0 */
>>  	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
>>  		return;
>> @@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>>  /* Flush our emulation state into the GIC hardware before entering the guest. */
>>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
>>  {
>> +	WARN_ON(vgic_v4_flush_hwstate(vcpu));
>> +
>>  	/*
>>  	 * If there are no virtual interrupts active or pending for this
>>  	 * VCPU, then there is no work to do and we can bail out without
>> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
>> index c4105f613f57..9cafb61b79af 100644
>> --- a/virt/kvm/arm/vgic/vgic.h
>> +++ b/virt/kvm/arm/vgic/vgic.h
>> @@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>>  bool vgic_supports_direct_msis(struct kvm *kvm);
>>  int vgic_v4_init(struct kvm *kvm);
>>  void vgic_v4_teardown(struct kvm *kvm);
>> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
>> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
>>  
>>  #endif
>>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
@ 2017-11-07 22:14       ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:14 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 07/11/2017 22:54, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> The redistributor needs to be told which vPE is about to be run,
>> and tells us whether there is any pending VLPI on exit.
>>
>> Let's add the scheduling calls to the vgic flush/sync functions,
>> allowing the VLPIs to be delivered to the guest.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
>>  virt/kvm/arm/vgic/vgic.c    |  4 ++++
>>  virt/kvm/arm/vgic/vgic.h    |  2 ++
>>  3 files changed, 45 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index 1a2f2fcdfa67..d7fe610bb1f5 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
>>  	its_vm->vpes = NULL;
>>  }
>>  
>> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
>> +{
>> +	if (!vgic_supports_direct_msis(vcpu->kvm))
>> +		return 0;
>> +
>> +	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
>> +}
>> +
>> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
>> +{
>> +	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
>> +	int err;
>> +
>> +	if (!vgic_supports_direct_msis(vcpu->kvm))
>> +		return 0;
>> +
>> +	/*
>> +	 * Before making the VPE resident, make sure the redistributor
>> +	 * corresponding to our current CPU expects us here. See the
>> +	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
>> +	 * turns into a VMOVP command at the ITS level.
>> +	 */
> I don't get the above comment. Don't you set the affinity of the
> doorbell irq below?
Forget that one. I now recall the trick and see its_vpe_set_affinity()

Eric
> 
> Thanks
> 
> Eric
>> +	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
>> +	if (err)
>> +		return err;
>> +
>> +	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
>> +	if (err)
>> +		return err;
>> +
>> +	/*
>> +	 * Now that the VPE is resident, let's get rid of a potential
>> +	 * doorbell interrupt that would still be pending.
>> +	 */
>> +	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
>> +
>> +	return err;
>> +}
>> +
>>  static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>  				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>  {
>> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
>> index 0691a2250949..71ca0ff1b35a 100644
>> --- a/virt/kvm/arm/vgic/vgic.c
>> +++ b/virt/kvm/arm/vgic/vgic.c
>> @@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>>  {
>>  	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>>  
>> +	WARN_ON(vgic_v4_sync_hwstate(vcpu));
>> +
>>  	/* An empty ap_list_head implies used_lrs == 0 */
>>  	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
>>  		return;
>> @@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>>  /* Flush our emulation state into the GIC hardware before entering the guest. */
>>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
>>  {
>> +	WARN_ON(vgic_v4_flush_hwstate(vcpu));
>> +
>>  	/*
>>  	 * If there are no virtual interrupts active or pending for this
>>  	 * VCPU, then there is no work to do and we can bail out without
>> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
>> index c4105f613f57..9cafb61b79af 100644
>> --- a/virt/kvm/arm/vgic/vgic.h
>> +++ b/virt/kvm/arm/vgic/vgic.h
>> @@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>>  bool vgic_supports_direct_msis(struct kvm *kvm);
>>  int vgic_v4_init(struct kvm *kvm);
>>  void vgic_v4_teardown(struct kvm *kvm);
>> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
>> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
>>  
>>  #endif
>>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync
@ 2017-11-07 22:14       ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:14 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 07/11/2017 22:54, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> The redistributor needs to be told which vPE is about to be run,
>> and tells us whether there is any pending VLPI on exit.
>>
>> Let's add the scheduling calls to the vgic flush/sync functions,
>> allowing the VLPIs to be delivered to the guest.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-v4.c | 39 +++++++++++++++++++++++++++++++++++++++
>>  virt/kvm/arm/vgic/vgic.c    |  4 ++++
>>  virt/kvm/arm/vgic/vgic.h    |  2 ++
>>  3 files changed, 45 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index 1a2f2fcdfa67..d7fe610bb1f5 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -131,6 +131,45 @@ void vgic_v4_teardown(struct kvm *kvm)
>>  	its_vm->vpes = NULL;
>>  }
>>  
>> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu)
>> +{
>> +	if (!vgic_supports_direct_msis(vcpu->kvm))
>> +		return 0;
>> +
>> +	return its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, false);
>> +}
>> +
>> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu)
>> +{
>> +	int irq = vcpu->arch.vgic_cpu.vgic_v3.its_vpe.irq;
>> +	int err;
>> +
>> +	if (!vgic_supports_direct_msis(vcpu->kvm))
>> +		return 0;
>> +
>> +	/*
>> +	 * Before making the VPE resident, make sure the redistributor
>> +	 * corresponding to our current CPU expects us here. See the
>> +	 * doc in drivers/irqchip/irq-gic-v4.c to understand how this
>> +	 * turns into a VMOVP command at the ITS level.
>> +	 */
> I don't get the above comment. Don't you set the affinity of the
> doorbell irq below?
Forget that one. I now recall the trick and see its_vpe_set_affinity()

Eric
> 
> Thanks
> 
> Eric
>> +	err = irq_set_affinity(irq, cpumask_of(smp_processor_id()));
>> +	if (err)
>> +		return err;
>> +
>> +	err = its_schedule_vpe(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe, true);
>> +	if (err)
>> +		return err;
>> +
>> +	/*
>> +	 * Now that the VPE is resident, let's get rid of a potential
>> +	 * doorbell interrupt that would still be pending.
>> +	 */
>> +	err = irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, false);
>> +
>> +	return err;
>> +}
>> +
>>  static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>  				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>  {
>> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
>> index 0691a2250949..71ca0ff1b35a 100644
>> --- a/virt/kvm/arm/vgic/vgic.c
>> +++ b/virt/kvm/arm/vgic/vgic.c
>> @@ -709,6 +709,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>>  {
>>  	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>>  
>> +	WARN_ON(vgic_v4_sync_hwstate(vcpu));
>> +
>>  	/* An empty ap_list_head implies used_lrs == 0 */
>>  	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
>>  		return;
>> @@ -721,6 +723,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
>>  /* Flush our emulation state into the GIC hardware before entering the guest. */
>>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
>>  {
>> +	WARN_ON(vgic_v4_flush_hwstate(vcpu));
>> +
>>  	/*
>>  	 * If there are no virtual interrupts active or pending for this
>>  	 * VCPU, then there is no work to do and we can bail out without
>> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
>> index c4105f613f57..9cafb61b79af 100644
>> --- a/virt/kvm/arm/vgic/vgic.h
>> +++ b/virt/kvm/arm/vgic/vgic.h
>> @@ -243,5 +243,7 @@ struct vgic_its *vgic_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
>>  bool vgic_supports_direct_msis(struct kvm *kvm);
>>  int vgic_v4_init(struct kvm *kvm);
>>  void vgic_v4_teardown(struct kvm *kvm);
>> +int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
>> +int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
>>  
>>  #endif
>>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-07 22:17     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:17 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> We so far allocate the doorbell interrupts without taking any
> special measure regarding the affinity of these interrupts. We
> simply move them around as required when the vcpu gets scheduled
> on a different CPU.
> 
> But that's counting without userspace (and the evil irqbalance) that
> can try and move the VPE interrupt around, causing the ITS code
> to emit VMOVP commands and remap the doorbell to another redistributor.
> Worse, this can happen while the vcpu is running, causing all kind
> of trouble if the VPE is already resident, and we end-up in UNPRED
> territory.
> 
> So let's take a definitive action and prevent userspace from messing
> with us. This is just a matter of adding IRQ_NO_BALANCING to the
> set of flags we already have, letting the kernel in sole control
> of the affinity.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d7fe610bb1f5..d10e18eabd3b 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,8 @@
>  
>  #include "vgic.h"
>  
> +#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
> +
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
>  {
>  	struct kvm_vcpu *vcpu = info;
> @@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
>  		 * doorbell could kick us out of the guest too
>  		 * early...
>  		 */
> -		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_set_status_flags(irq, DB_IRQ_FLAGS);
>  		ret = request_irq(irq, vgic_v4_doorbell_handler,
>  				  0, "vcpu", vcpu);
>  		if (ret) {
> @@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
>  		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
>  		int irq = its_vm->vpes[i]->irq;
>  
> -		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
>  		free_irq(irq, vcpu);
>  	}
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
@ 2017-11-07 22:17     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:17 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> We so far allocate the doorbell interrupts without taking any
> special measure regarding the affinity of these interrupts. We
> simply move them around as required when the vcpu gets scheduled
> on a different CPU.
> 
> But that's counting without userspace (and the evil irqbalance) that
> can try and move the VPE interrupt around, causing the ITS code
> to emit VMOVP commands and remap the doorbell to another redistributor.
> Worse, this can happen while the vcpu is running, causing all kind
> of trouble if the VPE is already resident, and we end-up in UNPRED
> territory.
> 
> So let's take a definitive action and prevent userspace from messing
> with us. This is just a matter of adding IRQ_NO_BALANCING to the
> set of flags we already have, letting the kernel in sole control
> of the affinity.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d7fe610bb1f5..d10e18eabd3b 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,8 @@
>  
>  #include "vgic.h"
>  
> +#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
> +
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
>  {
>  	struct kvm_vcpu *vcpu = info;
> @@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
>  		 * doorbell could kick us out of the guest too
>  		 * early...
>  		 */
> -		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_set_status_flags(irq, DB_IRQ_FLAGS);
>  		ret = request_irq(irq, vgic_v4_doorbell_handler,
>  				  0, "vcpu", vcpu);
>  		if (ret) {
> @@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
>  		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
>  		int irq = its_vm->vpes[i]->irq;
>  
> -		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
>  		free_irq(irq, vcpu);
>  	}
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity
@ 2017-11-07 22:17     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:17 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 27/10/2017 16:28, Marc Zyngier wrote:
> We so far allocate the doorbell interrupts without taking any
> special measure regarding the affinity of these interrupts. We
> simply move them around as required when the vcpu gets scheduled
> on a different CPU.
> 
> But that's counting without userspace (and the evil irqbalance) that
> can try and move the VPE interrupt around, causing the ITS code
> to emit VMOVP commands and remap the doorbell to another redistributor.
> Worse, this can happen while the vcpu is running, causing all kind
> of trouble if the VPE is already resident, and we end-up in UNPRED
> territory.
> 
> So let's take a definitive action and prevent userspace from messing
> with us. This is just a matter of adding IRQ_NO_BALANCING to the
> set of flags we already have, letting the kernel in sole control
> of the affinity.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d7fe610bb1f5..d10e18eabd3b 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,8 @@
>  
>  #include "vgic.h"
>  
> +#define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
> +
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
>  {
>  	struct kvm_vcpu *vcpu = info;
> @@ -83,7 +85,7 @@ int vgic_v4_init(struct kvm *kvm)
>  		 * doorbell could kick us out of the guest too
>  		 * early...
>  		 */
> -		irq_set_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_set_status_flags(irq, DB_IRQ_FLAGS);
>  		ret = request_irq(irq, vgic_v4_doorbell_handler,
>  				  0, "vcpu", vcpu);
>  		if (ret) {
> @@ -121,7 +123,7 @@ void vgic_v4_teardown(struct kvm *kvm)
>  		struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, i);
>  		int irq = its_vm->vpes[i]->irq;
>  
> -		irq_clear_status_flags(irq, IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY);
> +		irq_clear_status_flags(irq, DB_IRQ_FLAGS);
>  		free_irq(irq, vcpu);
>  	}
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  2017-11-07 16:34           ` Marc Zyngier
  (?)
@ 2017-11-07 22:24             ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:24 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi

On 07/11/2017 17:34, Marc Zyngier wrote:
> On 07/11/17 16:12, Auger Eric wrote:
>> Hi Marc,
>>
>> On 07/11/2017 16:38, Marc Zyngier wrote:
>>> On 07/11/17 15:24, Auger Eric wrote:
>>>> Hi Marc,
>>>>
>>>> Hi Marc,
>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>>> work, as it doesn't give any guarantee that the pending state
>>>>> is written into the pending table.
>>>>
>>>> I don't understand where does the limitation exactly come from. Can't we
>>>> use the GICR_VPENDBASER table data?
>>> You can't. None of the tables that are written by either the ITS or the
>>> redistributors are architected. All you know is that there is one bit
>>> per vLPI, but that's it (you don't even know which one is which).
>> Oh I thought the redistributor config and pending tables were fully
>> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
>> table.
> 
> The property table is definitely architected. It is a lot less clear for
> the pending table. The main issue is that you cannot really know when
> the various bits have actually been flushed all the way from the
> redistributor caches to memory to be introspected. Yes, it sucks.
Oh OK the INV only guarantees the caches are consistent with the LPI
config table. Maybe you could clarify the commit message with those details.

So
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 22:24             ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:24 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi

On 07/11/2017 17:34, Marc Zyngier wrote:
> On 07/11/17 16:12, Auger Eric wrote:
>> Hi Marc,
>>
>> On 07/11/2017 16:38, Marc Zyngier wrote:
>>> On 07/11/17 15:24, Auger Eric wrote:
>>>> Hi Marc,
>>>>
>>>> Hi Marc,
>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>>> work, as it doesn't give any guarantee that the pending state
>>>>> is written into the pending table.
>>>>
>>>> I don't understand where does the limitation exactly come from. Can't we
>>>> use the GICR_VPENDBASER table data?
>>> You can't. None of the tables that are written by either the ITS or the
>>> redistributors are architected. All you know is that there is one bit
>>> per vLPI, but that's it (you don't even know which one is which).
>> Oh I thought the redistributor config and pending tables were fully
>> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
>> table.
> 
> The property table is definitely architected. It is a lot less clear for
> the pending table. The main issue is that you cannot really know when
> the various bits have actually been flushed all the way from the
> redistributor caches to memory to be introspected. Yes, it sucks.
Oh OK the INV only guarantees the caches are consistent with the LPI
config table. Maybe you could clarify the commit message with those details.

So
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-07 22:24             ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-07 22:24 UTC (permalink / raw)
  To: linux-arm-kernel

Hi

On 07/11/2017 17:34, Marc Zyngier wrote:
> On 07/11/17 16:12, Auger Eric wrote:
>> Hi Marc,
>>
>> On 07/11/2017 16:38, Marc Zyngier wrote:
>>> On 07/11/17 15:24, Auger Eric wrote:
>>>> Hi Marc,
>>>>
>>>> Hi Marc,
>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>>> work, as it doesn't give any guarantee that the pending state
>>>>> is written into the pending table.
>>>>
>>>> I don't understand where does the limitation exactly come from. Can't we
>>>> use the GICR_VPENDBASER table data?
>>> You can't. None of the tables that are written by either the ITS or the
>>> redistributors are architected. All you know is that there is one bit
>>> per vLPI, but that's it (you don't even know which one is which).
>> Oh I thought the redistributor config and pending tables were fully
>> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
>> table.
> 
> The property table is definitely architected. It is a lot less clear for
> the pending table. The main issue is that you cannot really know when
> the various bits have actually been flushed all the way from the
> redistributor caches to memory to be introspected. Yes, it sucks.
Oh OK the INV only guarantees the caches are consistent with the LPI
config table. Maybe you could clarify the commit message with those details.

So
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-08  8:44     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  8:44 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> All it takes is the has_v4 flag to be set in gic_kvm_info
> as well as "kvm-arm.vgic_v4_enable=1" being passed on the
> command line for GICv4 to be enabled in KVM.

What did you motivate your choice of having an enable option instead of
a disable option?

Thanks

Eric
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt |  4 ++++
>  virt/kvm/arm/vgic/vgic-v3.c                     | 14 ++++++++++++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 05496622b4ef..93c8fff399eb 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1874,6 +1874,10 @@
>  			[KVM,ARM] Trap guest accesses to GICv3 common
>  			system registers
>  
> +	kvm-arm.vgic_v4_enable=
> +			[KVM,ARM] Allow use of GICv4 for direct injection of
> +			LPIs.
> +
>  	kvm-intel.ept=	[KVM,Intel] Disable extended page tables
>  			(virtualized MMU) support on capable Intel chips.
>  			Default is 1 (enabled)
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 96ea597db0e7..405733678c2f 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -24,6 +24,7 @@
>  static bool group0_trap;
>  static bool group1_trap;
>  static bool common_trap;
> +static bool gicv4_enable;
>  
>  void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
>  {
> @@ -459,6 +460,12 @@ static int __init early_common_trap_cfg(char *buf)
>  }
>  early_param("kvm-arm.vgic_v3_common_trap", early_common_trap_cfg);
>  
> +static int __init early_gicv4_enable(char *buf)
> +{
> +	return strtobool(buf, &gicv4_enable);
> +}
> +early_param("kvm-arm.vgic_v4_enable", early_gicv4_enable);
> +
>  /**
>   * vgic_v3_probe - probe for a GICv3 compatible interrupt controller in DT
>   * @node:	pointer to the DT node
> @@ -478,6 +485,13 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
>  	kvm_vgic_global_state.can_emulate_gicv2 = false;
>  	kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>  
> +	/* GICv4 support? */
> +	if (info->has_v4) {
> +		kvm_vgic_global_state.has_gicv4 = gicv4_enable;
> +		kvm_info("GICv4 support %sabled\n",
> +			 gicv4_enable ? "en" : "dis");
> +	}
> +
>  	if (!info->vcpu.start) {
>  		kvm_info("GICv3: no GICV resource entry\n");
>  		kvm_vgic_global_state.vcpu_base = 0;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
@ 2017-11-08  8:44     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  8:44 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> All it takes is the has_v4 flag to be set in gic_kvm_info
> as well as "kvm-arm.vgic_v4_enable=1" being passed on the
> command line for GICv4 to be enabled in KVM.

What did you motivate your choice of having an enable option instead of
a disable option?

Thanks

Eric
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt |  4 ++++
>  virt/kvm/arm/vgic/vgic-v3.c                     | 14 ++++++++++++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 05496622b4ef..93c8fff399eb 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1874,6 +1874,10 @@
>  			[KVM,ARM] Trap guest accesses to GICv3 common
>  			system registers
>  
> +	kvm-arm.vgic_v4_enable=
> +			[KVM,ARM] Allow use of GICv4 for direct injection of
> +			LPIs.
> +
>  	kvm-intel.ept=	[KVM,Intel] Disable extended page tables
>  			(virtualized MMU) support on capable Intel chips.
>  			Default is 1 (enabled)
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 96ea597db0e7..405733678c2f 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -24,6 +24,7 @@
>  static bool group0_trap;
>  static bool group1_trap;
>  static bool common_trap;
> +static bool gicv4_enable;
>  
>  void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
>  {
> @@ -459,6 +460,12 @@ static int __init early_common_trap_cfg(char *buf)
>  }
>  early_param("kvm-arm.vgic_v3_common_trap", early_common_trap_cfg);
>  
> +static int __init early_gicv4_enable(char *buf)
> +{
> +	return strtobool(buf, &gicv4_enable);
> +}
> +early_param("kvm-arm.vgic_v4_enable", early_gicv4_enable);
> +
>  /**
>   * vgic_v3_probe - probe for a GICv3 compatible interrupt controller in DT
>   * @node:	pointer to the DT node
> @@ -478,6 +485,13 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
>  	kvm_vgic_global_state.can_emulate_gicv2 = false;
>  	kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>  
> +	/* GICv4 support? */
> +	if (info->has_v4) {
> +		kvm_vgic_global_state.has_gicv4 = gicv4_enable;
> +		kvm_info("GICv4 support %sabled\n",
> +			 gicv4_enable ? "en" : "dis");
> +	}
> +
>  	if (!info->vcpu.start) {
>  		kvm_info("GICv3: no GICV resource entry\n");
>  		kvm_vgic_global_state.vcpu_base = 0;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
@ 2017-11-08  8:44     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  8:44 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> All it takes is the has_v4 flag to be set in gic_kvm_info
> as well as "kvm-arm.vgic_v4_enable=1" being passed on the
> command line for GICv4 to be enabled in KVM.

What did you motivate your choice of having an enable option instead of
a disable option?

Thanks

Eric
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt |  4 ++++
>  virt/kvm/arm/vgic/vgic-v3.c                     | 14 ++++++++++++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 05496622b4ef..93c8fff399eb 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1874,6 +1874,10 @@
>  			[KVM,ARM] Trap guest accesses to GICv3 common
>  			system registers
>  
> +	kvm-arm.vgic_v4_enable=
> +			[KVM,ARM] Allow use of GICv4 for direct injection of
> +			LPIs.
> +
>  	kvm-intel.ept=	[KVM,Intel] Disable extended page tables
>  			(virtualized MMU) support on capable Intel chips.
>  			Default is 1 (enabled)
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 96ea597db0e7..405733678c2f 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -24,6 +24,7 @@
>  static bool group0_trap;
>  static bool group1_trap;
>  static bool common_trap;
> +static bool gicv4_enable;
>  
>  void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
>  {
> @@ -459,6 +460,12 @@ static int __init early_common_trap_cfg(char *buf)
>  }
>  early_param("kvm-arm.vgic_v3_common_trap", early_common_trap_cfg);
>  
> +static int __init early_gicv4_enable(char *buf)
> +{
> +	return strtobool(buf, &gicv4_enable);
> +}
> +early_param("kvm-arm.vgic_v4_enable", early_gicv4_enable);
> +
>  /**
>   * vgic_v3_probe - probe for a GICv3 compatible interrupt controller in DT
>   * @node:	pointer to the DT node
> @@ -478,6 +485,13 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
>  	kvm_vgic_global_state.can_emulate_gicv2 = false;
>  	kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>  
> +	/* GICv4 support? */
> +	if (info->has_v4) {
> +		kvm_vgic_global_state.has_gicv4 = gicv4_enable;
> +		kvm_info("GICv4 support %sabled\n",
> +			 gicv4_enable ? "en" : "dis");
> +	}
> +
>  	if (!info->vcpu.start) {
>  		kvm_info("GICv3: no GICV resource entry\n");
>  		kvm_vgic_global_state.vcpu_base = 0;
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-08  8:46     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  8:46 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> In order for VLPIs to be delivered to the guest, we must make
> sure that the 
virtual
cpuif is always enabled, irrespective of the
> presence of virtual interrupt in the LRs.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> ---
>  virt/kvm/arm/hyp/vgic-v3-sr.c | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> index 91728faa13fd..f5c3d6d7019e 100644
> --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> @@ -258,7 +258,8 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>  			cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
>  		}
>  	} else {
> -		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> +		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
> +		    cpu_if->its_vpe.its_vm)
>  			write_gicreg(0, ICH_HCR_EL2);
>  
>  		cpu_if->vgic_elrsr = 0xffff;
> @@ -337,9 +338,11 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
>  		/*
>  		 * If we need to trap system registers, we must write
>  		 * ICH_HCR_EL2 anyway, even if no interrupts are being
> -		 * injected,
> +		 * injected. Same thing if GICv4 is used, as VLPI
> +		 * delivery is gated by ICH_HCR_EL2.En.
>  		 */
> -		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> +		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
> +		    cpu_if->its_vpe.its_vm)
>  			write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
>  	}
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
@ 2017-11-08  8:46     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  8:46 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> In order for VLPIs to be delivered to the guest, we must make
> sure that the 
virtual
cpuif is always enabled, irrespective of the
> presence of virtual interrupt in the LRs.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> ---
>  virt/kvm/arm/hyp/vgic-v3-sr.c | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> index 91728faa13fd..f5c3d6d7019e 100644
> --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> @@ -258,7 +258,8 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>  			cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
>  		}
>  	} else {
> -		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> +		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
> +		    cpu_if->its_vpe.its_vm)
>  			write_gicreg(0, ICH_HCR_EL2);
>  
>  		cpu_if->vgic_elrsr = 0xffff;
> @@ -337,9 +338,11 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
>  		/*
>  		 * If we need to trap system registers, we must write
>  		 * ICH_HCR_EL2 anyway, even if no interrupts are being
> -		 * injected,
> +		 * injected. Same thing if GICv4 is used, as VLPI
> +		 * delivery is gated by ICH_HCR_EL2.En.
>  		 */
> -		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> +		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
> +		    cpu_if->its_vpe.its_vm)
>  			write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
>  	}
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered
@ 2017-11-08  8:46     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  8:46 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,
On 27/10/2017 16:28, Marc Zyngier wrote:
> In order for VLPIs to be delivered to the guest, we must make
> sure that the 
virtual
cpuif is always enabled, irrespective of the
> presence of virtual interrupt in the LRs.
> 
> Acked-by: Christoffer Dall <cdall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> ---
>  virt/kvm/arm/hyp/vgic-v3-sr.c | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> index 91728faa13fd..f5c3d6d7019e 100644
> --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> @@ -258,7 +258,8 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>  			cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
>  		}
>  	} else {
> -		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> +		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
> +		    cpu_if->its_vpe.its_vm)
>  			write_gicreg(0, ICH_HCR_EL2);
>  
>  		cpu_if->vgic_elrsr = 0xffff;
> @@ -337,9 +338,11 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
>  		/*
>  		 * If we need to trap system registers, we must write
>  		 * ICH_HCR_EL2 anyway, even if no interrupts are being
> -		 * injected,
> +		 * injected. Same thing if GICv4 is used, as VLPI
> +		 * delivery is gated by ICH_HCR_EL2.En.
>  		 */
> -		if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> +		if (static_branch_unlikely(&vgic_v3_cpuif_trap) ||
> +		    cpu_if->its_vpe.its_vm)
>  			write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
>  	}
>  
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
  2017-10-27 14:28   ` Marc Zyngier
  (?)
@ 2017-11-08  9:13     ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  9:13 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Yet another braindump so I can free some cells...
> 
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 67 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d10e18eabd3b..e367d65a0ebe 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,73 @@
>  
>  #include "vgic.h"
>  
> +/*
> + * How KVM uses GICv4 (insert rude comments here):
> + *
> + * The vgic-v4 layer acts as a bridge between several entities:
> + * - The GICv4 ITS representation offered by the ITS driver
> + * - VFIO, which is in charge of the PCI endpoint
> + * - The virtual ITS, which is the only thing the guest sees
> + *
> + * The configuration of VLPIs is triggered by a callback from VFIO,
> + * instructing KVM that a PCI device has been configured to deliver
> + * MSIs to a vITS.
We actually have a negotiation protocol between VFIO PCI (irq bypass
producer) and KVM irqfd (IRQ bypass consumer). When both recognize they
are tied together, handling an MSI tunneling, they initiate the
forwarding setup.
> + *
> + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
> + * and this is used to find the corresponding vITS data structures
> + * (ITS instance, device, event and irq) using a process that is
> + * extremely similar to the injection of an MSI.

Is it correct to say we replace the following injection chain:

pEventID|										
(pITS)	|-> pLPIID -> VFIO PCI IRQ handler -> KVM irqfd ...
pDevID	|

		vEventID|
... inject	(vITS)	| -> vLPIID
		vDevID	|

by

pEventID|
(pITS)	| -> vLPIID
pDevID	|

Thanks

Eric
> + *
> + * At this stage, we can link the guest's view of an LPI (uniquely
> + * identified by the routing entry) and the host irq, using the GICv4
> + * driver mapping operation. Should the mapping succeed, we've then
> + * successfully upgraded the guest's LPI to a VLPI. We can then start
> + * with updating GICv4's view of the property table and generating an
> + * INValidation in order to kickstart the delivery of this VLPI to the
> + * guest directly, without software intervention. Well, almost.
> + *
> + * When the PCI endpoint is deconfigured, this operation is reversed
> + * with VFIO calling kvm_vgic_v4_unset_forwarding().
> + *
> + * Once the VLPI has been mapped, it needs to follow any change the
> + * guest performs on its LPI through the vITS. For that, a number of
> + * command handlers have hooks to communicate these changes to the HW:
> + * - Any invalidation triggers a call to its_prop_update_vlpi()
> + * - The INT command results in a irq_set_irqchip_state(), which
> + *   generates an INT on the corresponding VLPI.
> + * - The CLEAR command results in a irq_set_irqchip_state(), which
> + *   generates an CLEAR on the corresponding VLPI.
> + * - DISCARD translates into an unmap, similar to a call to
> + *   kvm_vgic_v4_unset_forwarding().
> + * - MOVI is translated by an update of the existing mapping, changing
> + *   the target vcpu, resulting in a VMOVI being generated.
> + * - MOVALL is translated by a string of mapping updates (similar to
> + *   the handling of MOVI). MOVALL is horrible.
> + *
> + * Note that a DISCARD/MAPTI sequence emitted from the guest without
> + * reprogramming the PCI endpoint after MAPTI does not result in a
> + * VLPI being mapped, as there is no callback from VFIO (the guest
> + * will get the interrupt via the normal SW injection). Fixing this is
> + * not trivial, and requires some horrible messing with the VFIO
> + * internals. Not fun. Don't do that.
> + *
> + * Then there is the scheduling. Each time a vcpu is about to run on a
> + * physical CPU, KVM must tell the corresponding redistributor about
> + * it. And if we've migrated our vcpu from one CPU to another, we must
> + * tell the ITS (so that the messages reach the right redistributor).
> + * This is done in two steps: first issue a irq_set_affinity() on the
> + * irq corresponding to the vcpu, then call its_schedule_vpe(). You
> + * must be in a non-preemptible context. On exit, another call to
> + * its_schedule_vpe() tells the redistributor that we're done with the
> + * vcpu.
> + *
> + * Finally, the doorbell handling: Each vcpu is allocated an interrupt
> + * which will fire each time a VLPI is made pending whilst the vcpu is
> + * not running. Each time the vcpu gets blocked, the doorbell
> + * interrupt gets enabled. When the vcpu is unblocked (for whatever
> + * reason), the doorbell interrupt is disabled.
> + */
> +
>  #define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
>  
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
@ 2017-11-08  9:13     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  9:13 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Yet another braindump so I can free some cells...
> 
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 67 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d10e18eabd3b..e367d65a0ebe 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,73 @@
>  
>  #include "vgic.h"
>  
> +/*
> + * How KVM uses GICv4 (insert rude comments here):
> + *
> + * The vgic-v4 layer acts as a bridge between several entities:
> + * - The GICv4 ITS representation offered by the ITS driver
> + * - VFIO, which is in charge of the PCI endpoint
> + * - The virtual ITS, which is the only thing the guest sees
> + *
> + * The configuration of VLPIs is triggered by a callback from VFIO,
> + * instructing KVM that a PCI device has been configured to deliver
> + * MSIs to a vITS.
We actually have a negotiation protocol between VFIO PCI (irq bypass
producer) and KVM irqfd (IRQ bypass consumer). When both recognize they
are tied together, handling an MSI tunneling, they initiate the
forwarding setup.
> + *
> + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
> + * and this is used to find the corresponding vITS data structures
> + * (ITS instance, device, event and irq) using a process that is
> + * extremely similar to the injection of an MSI.

Is it correct to say we replace the following injection chain:

pEventID|										
(pITS)	|-> pLPIID -> VFIO PCI IRQ handler -> KVM irqfd ...
pDevID	|

		vEventID|
... inject	(vITS)	| -> vLPIID
		vDevID	|

by

pEventID|
(pITS)	| -> vLPIID
pDevID	|

Thanks

Eric
> + *
> + * At this stage, we can link the guest's view of an LPI (uniquely
> + * identified by the routing entry) and the host irq, using the GICv4
> + * driver mapping operation. Should the mapping succeed, we've then
> + * successfully upgraded the guest's LPI to a VLPI. We can then start
> + * with updating GICv4's view of the property table and generating an
> + * INValidation in order to kickstart the delivery of this VLPI to the
> + * guest directly, without software intervention. Well, almost.
> + *
> + * When the PCI endpoint is deconfigured, this operation is reversed
> + * with VFIO calling kvm_vgic_v4_unset_forwarding().
> + *
> + * Once the VLPI has been mapped, it needs to follow any change the
> + * guest performs on its LPI through the vITS. For that, a number of
> + * command handlers have hooks to communicate these changes to the HW:
> + * - Any invalidation triggers a call to its_prop_update_vlpi()
> + * - The INT command results in a irq_set_irqchip_state(), which
> + *   generates an INT on the corresponding VLPI.
> + * - The CLEAR command results in a irq_set_irqchip_state(), which
> + *   generates an CLEAR on the corresponding VLPI.
> + * - DISCARD translates into an unmap, similar to a call to
> + *   kvm_vgic_v4_unset_forwarding().
> + * - MOVI is translated by an update of the existing mapping, changing
> + *   the target vcpu, resulting in a VMOVI being generated.
> + * - MOVALL is translated by a string of mapping updates (similar to
> + *   the handling of MOVI). MOVALL is horrible.
> + *
> + * Note that a DISCARD/MAPTI sequence emitted from the guest without
> + * reprogramming the PCI endpoint after MAPTI does not result in a
> + * VLPI being mapped, as there is no callback from VFIO (the guest
> + * will get the interrupt via the normal SW injection). Fixing this is
> + * not trivial, and requires some horrible messing with the VFIO
> + * internals. Not fun. Don't do that.
> + *
> + * Then there is the scheduling. Each time a vcpu is about to run on a
> + * physical CPU, KVM must tell the corresponding redistributor about
> + * it. And if we've migrated our vcpu from one CPU to another, we must
> + * tell the ITS (so that the messages reach the right redistributor).
> + * This is done in two steps: first issue a irq_set_affinity() on the
> + * irq corresponding to the vcpu, then call its_schedule_vpe(). You
> + * must be in a non-preemptible context. On exit, another call to
> + * its_schedule_vpe() tells the redistributor that we're done with the
> + * vcpu.
> + *
> + * Finally, the doorbell handling: Each vcpu is allocated an interrupt
> + * which will fire each time a VLPI is made pending whilst the vcpu is
> + * not running. Each time the vcpu gets blocked, the doorbell
> + * interrupt gets enabled. When the vcpu is unblocked (for whatever
> + * reason), the doorbell interrupt is disabled.
> + */
> +
>  #define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
>  
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
@ 2017-11-08  9:13     ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  9:13 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 27/10/2017 16:28, Marc Zyngier wrote:
> Yet another braindump so I can free some cells...
> 
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 67 insertions(+)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index d10e18eabd3b..e367d65a0ebe 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -23,6 +23,73 @@
>  
>  #include "vgic.h"
>  
> +/*
> + * How KVM uses GICv4 (insert rude comments here):
> + *
> + * The vgic-v4 layer acts as a bridge between several entities:
> + * - The GICv4 ITS representation offered by the ITS driver
> + * - VFIO, which is in charge of the PCI endpoint
> + * - The virtual ITS, which is the only thing the guest sees
> + *
> + * The configuration of VLPIs is triggered by a callback from VFIO,
> + * instructing KVM that a PCI device has been configured to deliver
> + * MSIs to a vITS.
We actually have a negotiation protocol between VFIO PCI (irq bypass
producer) and KVM irqfd (IRQ bypass consumer). When both recognize they
are tied together, handling an MSI tunneling, they initiate the
forwarding setup.
> + *
> + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
> + * and this is used to find the corresponding vITS data structures
> + * (ITS instance, device, event and irq) using a process that is
> + * extremely similar to the injection of an MSI.

Is it correct to say we replace the following injection chain:

pEventID|										
(pITS)	|-> pLPIID -> VFIO PCI IRQ handler -> KVM irqfd ...
pDevID	|

		vEventID|
... inject	(vITS)	| -> vLPIID
		vDevID	|

by

pEventID|
(pITS)	| -> vLPIID
pDevID	|

Thanks

Eric
> + *
> + * At this stage, we can link the guest's view of an LPI (uniquely
> + * identified by the routing entry) and the host irq, using the GICv4
> + * driver mapping operation. Should the mapping succeed, we've then
> + * successfully upgraded the guest's LPI to a VLPI. We can then start
> + * with updating GICv4's view of the property table and generating an
> + * INValidation in order to kickstart the delivery of this VLPI to the
> + * guest directly, without software intervention. Well, almost.
> + *
> + * When the PCI endpoint is deconfigured, this operation is reversed
> + * with VFIO calling kvm_vgic_v4_unset_forwarding().
> + *
> + * Once the VLPI has been mapped, it needs to follow any change the
> + * guest performs on its LPI through the vITS. For that, a number of
> + * command handlers have hooks to communicate these changes to the HW:
> + * - Any invalidation triggers a call to its_prop_update_vlpi()
> + * - The INT command results in a irq_set_irqchip_state(), which
> + *   generates an INT on the corresponding VLPI.
> + * - The CLEAR command results in a irq_set_irqchip_state(), which
> + *   generates an CLEAR on the corresponding VLPI.
> + * - DISCARD translates into an unmap, similar to a call to
> + *   kvm_vgic_v4_unset_forwarding().
> + * - MOVI is translated by an update of the existing mapping, changing
> + *   the target vcpu, resulting in a VMOVI being generated.
> + * - MOVALL is translated by a string of mapping updates (similar to
> + *   the handling of MOVI). MOVALL is horrible.
> + *
> + * Note that a DISCARD/MAPTI sequence emitted from the guest without
> + * reprogramming the PCI endpoint after MAPTI does not result in a
> + * VLPI being mapped, as there is no callback from VFIO (the guest
> + * will get the interrupt via the normal SW injection). Fixing this is
> + * not trivial, and requires some horrible messing with the VFIO
> + * internals. Not fun. Don't do that.
> + *
> + * Then there is the scheduling. Each time a vcpu is about to run on a
> + * physical CPU, KVM must tell the corresponding redistributor about
> + * it. And if we've migrated our vcpu from one CPU to another, we must
> + * tell the ITS (so that the messages reach the right redistributor).
> + * This is done in two steps: first issue a irq_set_affinity() on the
> + * irq corresponding to the vcpu, then call its_schedule_vpe(). You
> + * must be in a non-preemptible context. On exit, another call to
> + * its_schedule_vpe() tells the redistributor that we're done with the
> + * vcpu.
> + *
> + * Finally, the doorbell handling: Each vcpu is allocated an interrupt
> + * which will fire each time a VLPI is made pending whilst the vcpu is
> + * not running. Each time the vcpu gets blocked, the doorbell
> + * interrupt gets enabled. When the vcpu is unblocked (for whatever
> + * reason), the doorbell interrupt is disabled.
> + */
> +
>  #define DB_IRQ_FLAGS	(IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING)
>  
>  static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
  2017-11-07 22:24             ` Auger Eric
  (?)
@ 2017-11-08  9:35               ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  9:35 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi,

On 07/11/2017 23:24, Auger Eric wrote:
> Hi
> 
> On 07/11/2017 17:34, Marc Zyngier wrote:
>> On 07/11/17 16:12, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 07/11/2017 16:38, Marc Zyngier wrote:
>>>> On 07/11/17 15:24, Auger Eric wrote:
>>>>> Hi Marc,
>>>>>
>>>>> Hi Marc,
>>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>>>> work, as it doesn't give any guarantee that the pending state
>>>>>> is written into the pending table.
>>>>>
>>>>> I don't understand where does the limitation exactly come from. Can't we
>>>>> use the GICR_VPENDBASER table data?
>>>> You can't. None of the tables that are written by either the ITS or the
>>>> redistributors are architected. All you know is that there is one bit
>>>> per vLPI, but that's it (you don't even know which one is which).
>>> Oh I thought the redistributor config and pending tables were fully
>>> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
>>> table.
>>
>> The property table is definitely architected. It is a lot less clear for
>> the pending table. The main issue is that you cannot really know when
>> the various bits have actually been flushed all the way from the
>> redistributor caches to memory to be introspected. Yes, it sucks.
> Oh OK the INV only guarantees the caches are consistent with the LPI
> config table. Maybe you could clarify the commit message with those details.
> 
> So
> Reviewed-by: Eric Auger <eric.auger@redhat.com>

Reading the spec further, looks setting the Valid bit of GICR_VPENDBASER
is supposed  to force the redistributor to retrieve pending interrupts
from the the VCPU I/F and ensure the VPT in memory is correct.

Anyway that's not a big deal at the moment as you pointed out...

Thanks

Eric
> 
> Thanks
> 
> Eric
> 
>>
>> Thanks,
>>
>> 	M.
>>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-08  9:35               ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  9:35 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

Hi,

On 07/11/2017 23:24, Auger Eric wrote:
> Hi
> 
> On 07/11/2017 17:34, Marc Zyngier wrote:
>> On 07/11/17 16:12, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 07/11/2017 16:38, Marc Zyngier wrote:
>>>> On 07/11/17 15:24, Auger Eric wrote:
>>>>> Hi Marc,
>>>>>
>>>>> Hi Marc,
>>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>>>> work, as it doesn't give any guarantee that the pending state
>>>>>> is written into the pending table.
>>>>>
>>>>> I don't understand where does the limitation exactly come from. Can't we
>>>>> use the GICR_VPENDBASER table data?
>>>> You can't. None of the tables that are written by either the ITS or the
>>>> redistributors are architected. All you know is that there is one bit
>>>> per vLPI, but that's it (you don't even know which one is which).
>>> Oh I thought the redistributor config and pending tables were fully
>>> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
>>> table.
>>
>> The property table is definitely architected. It is a lot less clear for
>> the pending table. The main issue is that you cannot really know when
>> the various bits have actually been flushed all the way from the
>> redistributor caches to memory to be introspected. Yes, it sucks.
> Oh OK the INV only guarantees the caches are consistent with the LPI
> config table. Maybe you could clarify the commit message with those details.
> 
> So
> Reviewed-by: Eric Auger <eric.auger@redhat.com>

Reading the spec further, looks setting the Valid bit of GICR_VPENDBASER
is supposed  to force the redistributor to retrieve pending interrupts
from the the VCPU I/F and ensure the VPT in memory is correct.

Anyway that's not a big deal at the moment as you pointed out...

Thanks

Eric
> 
> Thanks
> 
> Eric
> 
>>
>> Thanks,
>>
>> 	M.
>>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved
@ 2017-11-08  9:35               ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08  9:35 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 07/11/2017 23:24, Auger Eric wrote:
> Hi
> 
> On 07/11/2017 17:34, Marc Zyngier wrote:
>> On 07/11/17 16:12, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 07/11/2017 16:38, Marc Zyngier wrote:
>>>> On 07/11/17 15:24, Auger Eric wrote:
>>>>> Hi Marc,
>>>>>
>>>>> Hi Marc,
>>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>>> The GICv4 architecture doesn't make it easy for save/restore to
>>>>>> work, as it doesn't give any guarantee that the pending state
>>>>>> is written into the pending table.
>>>>>
>>>>> I don't understand where does the limitation exactly come from. Can't we
>>>>> use the GICR_VPENDBASER table data?
>>>> You can't. None of the tables that are written by either the ITS or the
>>>> redistributors are architected. All you know is that there is one bit
>>>> per vLPI, but that's it (you don't even know which one is which).
>>> Oh I thought the redistributor config and pending tables were fully
>>> specified (6.1.2 and 6.1.3 of the spec), except the 1kB of the pending
>>> table.
>>
>> The property table is definitely architected. It is a lot less clear for
>> the pending table. The main issue is that you cannot really know when
>> the various bits have actually been flushed all the way from the
>> redistributor caches to memory to be introspected. Yes, it sucks.
> Oh OK the INV only guarantees the caches are consistent with the LPI
> config table. Maybe you could clarify the commit message with those details.
> 
> So
> Reviewed-by: Eric Auger <eric.auger@redhat.com>

Reading the spec further, looks setting the Valid bit of GICR_VPENDBASER
is supposed  to force the redistributor to retrieve pending interrupts
from the the VCPU I/F and ensure the VPT in memory is correct.

Anyway that's not a big deal at the moment as you pointed out...

Thanks

Eric
> 
> Thanks
> 
> Eric
> 
>>
>> Thanks,
>>
>> 	M.
>>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-11-07 15:59         ` Auger Eric
  (?)
@ 2017-11-08 11:30           ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:30 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

On 07/11/17 15:59, Auger Eric wrote:
> Hi,
> 
> On 07/11/2017 15:42, Marc Zyngier wrote:
>> Hi Eric,
>>
>> On 07/11/17 13:06, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Let's use the irq bypass mechanism introduced for platform device
>>>> interrupts
>>> nit: I would remove "introduced for platform device interrupts"
>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>
>>>>
>>>  and establish our LPI->VLPI mapping.
>>>>
>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>> --- a/include/kvm/arm_vgic.h
>>>> +++ b/include/kvm/arm_vgic.h
>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>  
>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>  
>>>> +struct kvm_kernel_irq_routing_entry;
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>> --- a/virt/kvm/arm/arm.c
>>>> +++ b/virt/kvm/arm/arm.c
>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return 0;
>>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>> +					  &irqfd->irq_entry);
>>>>  }
>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  				      struct irq_bypass_producer *prod)
>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return;
>>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>> +				     &irqfd->irq_entry);
>>>>  }
>>>>  
>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>> @@ -18,6 +18,7 @@
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/irqdomain.h>
>>>>  #include <linux/kvm_host.h>
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>  
>>>>  #include "vgic.h"
>>>>  
>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>  	its_vm->nr_vpes = 0;
>>>>  	its_vm->vpes = NULL;
>>>>  }
>>>> +
>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>>> +		.address_lo	= irq_entry->msi.address_lo,
>>>> +		.address_hi	= irq_entry->msi.address_hi,
>>>> +		.data		= irq_entry->msi.data,
>>>> +		.flags		= irq_entry->msi.flags,
>>>> +		.devid		= irq_entry->msi.devid,
>>>> +	};
>>>> +
>>>> +	/*
>>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>>> +	 * translation for any of our vITSs.
>>>> +	 */
>>> I don't understand the relevance of the above comment.
>>
>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>> refcounted, and we don't deal with LPIs here.
>>
>>>> +	return vgic_msi_to_its(kvm, &msi);
>>>> +}
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
>>> @virq is the host linux irq. virq naming is a bit confusing to me.
>>
>> There is plenty of irq-related code that uses this naming. In this
>> context, we tend to use irq for the vgic view, hwirq for the irqchip
>> view. How would you call this one?
> OK
>>
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct vgic_its *its;
>>>> +	struct vgic_irq *irq;
>>>> +	struct its_vlpi_map map;
>>>> +	int ret;
>>>> +
>>> Don't you need to check that the linux irq (virq) is an LPI? You may
>>> encounter some VFIO "producers" for irq that are not LPIs, typically if
>>> we eventually upstream SPI forwarding.
>>
>> That's indeed a concern. The issue is that we don't really have a way to
>> check this, other than following the irq_data pointers and check that
>> the hwirq is within a certain range. And even that doesn't guarantee
>> anything.
> OK. But somehow this means the userspace can setup forwarding between an
> SPI and a vLPI, right?

Not really, or at least not as long as we only support PCI. It would
have to be that although you have a GICv4 system, you end-up with an MSI
that is assigned to a GICv2m on the physical side. This cannot happen,
by definition.

So we're actually left with the platform case. An easy way to filter
things would be to look at the routing entry flags, and see if we have a
DEVID there. That would guarantee us that this is an LPI.

[...]

>>>> +	map = (struct its_vlpi_map) {
>>>> +		.vm		= &kvm->arch.vgic.its_vm,
>>>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>>>> +		.vintid		= irq->intid,
>>>> +		.properties	= ((irq->priority & 0xfc) |
>>>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>>>> +				   LPI_PROP_GROUP1),
>>> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
>>> and this setting:
>>>
>>> * @properties: Priority and enable bits (as written in the prop table)
>>
>> Which inconsistency?
> I was confused by LPI_PROP_GROUP1 which was not documented in the
> comment. But looking more carefully in the spec it corresponds to [1] =
> RES1.

Yes, this is a leftover from previous revision of the architecture. That
bit used to be the Group configuration, with only group-1 being valid...
It got turned into a RES1 at a later time.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-08 11:30           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:30 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

On 07/11/17 15:59, Auger Eric wrote:
> Hi,
> 
> On 07/11/2017 15:42, Marc Zyngier wrote:
>> Hi Eric,
>>
>> On 07/11/17 13:06, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Let's use the irq bypass mechanism introduced for platform device
>>>> interrupts
>>> nit: I would remove "introduced for platform device interrupts"
>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>
>>>>
>>>  and establish our LPI->VLPI mapping.
>>>>
>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>> --- a/include/kvm/arm_vgic.h
>>>> +++ b/include/kvm/arm_vgic.h
>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>  
>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>  
>>>> +struct kvm_kernel_irq_routing_entry;
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>> --- a/virt/kvm/arm/arm.c
>>>> +++ b/virt/kvm/arm/arm.c
>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return 0;
>>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>> +					  &irqfd->irq_entry);
>>>>  }
>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  				      struct irq_bypass_producer *prod)
>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return;
>>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>> +				     &irqfd->irq_entry);
>>>>  }
>>>>  
>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>> @@ -18,6 +18,7 @@
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/irqdomain.h>
>>>>  #include <linux/kvm_host.h>
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>  
>>>>  #include "vgic.h"
>>>>  
>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>  	its_vm->nr_vpes = 0;
>>>>  	its_vm->vpes = NULL;
>>>>  }
>>>> +
>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>>> +		.address_lo	= irq_entry->msi.address_lo,
>>>> +		.address_hi	= irq_entry->msi.address_hi,
>>>> +		.data		= irq_entry->msi.data,
>>>> +		.flags		= irq_entry->msi.flags,
>>>> +		.devid		= irq_entry->msi.devid,
>>>> +	};
>>>> +
>>>> +	/*
>>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>>> +	 * translation for any of our vITSs.
>>>> +	 */
>>> I don't understand the relevance of the above comment.
>>
>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>> refcounted, and we don't deal with LPIs here.
>>
>>>> +	return vgic_msi_to_its(kvm, &msi);
>>>> +}
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
>>> @virq is the host linux irq. virq naming is a bit confusing to me.
>>
>> There is plenty of irq-related code that uses this naming. In this
>> context, we tend to use irq for the vgic view, hwirq for the irqchip
>> view. How would you call this one?
> OK
>>
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct vgic_its *its;
>>>> +	struct vgic_irq *irq;
>>>> +	struct its_vlpi_map map;
>>>> +	int ret;
>>>> +
>>> Don't you need to check that the linux irq (virq) is an LPI? You may
>>> encounter some VFIO "producers" for irq that are not LPIs, typically if
>>> we eventually upstream SPI forwarding.
>>
>> That's indeed a concern. The issue is that we don't really have a way to
>> check this, other than following the irq_data pointers and check that
>> the hwirq is within a certain range. And even that doesn't guarantee
>> anything.
> OK. But somehow this means the userspace can setup forwarding between an
> SPI and a vLPI, right?

Not really, or at least not as long as we only support PCI. It would
have to be that although you have a GICv4 system, you end-up with an MSI
that is assigned to a GICv2m on the physical side. This cannot happen,
by definition.

So we're actually left with the platform case. An easy way to filter
things would be to look at the routing entry flags, and see if we have a
DEVID there. That would guarantee us that this is an LPI.

[...]

>>>> +	map = (struct its_vlpi_map) {
>>>> +		.vm		= &kvm->arch.vgic.its_vm,
>>>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>>>> +		.vintid		= irq->intid,
>>>> +		.properties	= ((irq->priority & 0xfc) |
>>>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>>>> +				   LPI_PROP_GROUP1),
>>> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
>>> and this setting:
>>>
>>> * @properties: Priority and enable bits (as written in the prop table)
>>
>> Which inconsistency?
> I was confused by LPI_PROP_GROUP1 which was not documented in the
> comment. But looking more carefully in the spec it corresponds to [1] =
> RES1.

Yes, this is a leftover from previous revision of the architecture. That
bit used to be the Group configuration, with only group-1 being valid...
It got turned into a RES1 at a later time.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-08 11:30           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:30 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/11/17 15:59, Auger Eric wrote:
> Hi,
> 
> On 07/11/2017 15:42, Marc Zyngier wrote:
>> Hi Eric,
>>
>> On 07/11/17 13:06, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Let's use the irq bypass mechanism introduced for platform device
>>>> interrupts
>>> nit: I would remove "introduced for platform device interrupts"
>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>
>>>>
>>>  and establish our LPI->VLPI mapping.
>>>>
>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>> --- a/include/kvm/arm_vgic.h
>>>> +++ b/include/kvm/arm_vgic.h
>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>  
>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>  
>>>> +struct kvm_kernel_irq_routing_entry;
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>> --- a/virt/kvm/arm/arm.c
>>>> +++ b/virt/kvm/arm/arm.c
>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return 0;
>>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>> +					  &irqfd->irq_entry);
>>>>  }
>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  				      struct irq_bypass_producer *prod)
>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return;
>>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>> +				     &irqfd->irq_entry);
>>>>  }
>>>>  
>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>> @@ -18,6 +18,7 @@
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/irqdomain.h>
>>>>  #include <linux/kvm_host.h>
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>  
>>>>  #include "vgic.h"
>>>>  
>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>  	its_vm->nr_vpes = 0;
>>>>  	its_vm->vpes = NULL;
>>>>  }
>>>> +
>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>>> +		.address_lo	= irq_entry->msi.address_lo,
>>>> +		.address_hi	= irq_entry->msi.address_hi,
>>>> +		.data		= irq_entry->msi.data,
>>>> +		.flags		= irq_entry->msi.flags,
>>>> +		.devid		= irq_entry->msi.devid,
>>>> +	};
>>>> +
>>>> +	/*
>>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>>> +	 * translation for any of our vITSs.
>>>> +	 */
>>> I don't understand the relevance of the above comment.
>>
>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>> refcounted, and we don't deal with LPIs here.
>>
>>>> +	return vgic_msi_to_its(kvm, &msi);
>>>> +}
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int virq,
>>> @virq is the host linux irq. virq naming is a bit confusing to me.
>>
>> There is plenty of irq-related code that uses this naming. In this
>> context, we tend to use irq for the vgic view, hwirq for the irqchip
>> view. How would you call this one?
> OK
>>
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct vgic_its *its;
>>>> +	struct vgic_irq *irq;
>>>> +	struct its_vlpi_map map;
>>>> +	int ret;
>>>> +
>>> Don't you need to check that the linux irq (virq) is an LPI? You may
>>> encounter some VFIO "producers" for irq that are not LPIs, typically if
>>> we eventually upstream SPI forwarding.
>>
>> That's indeed a concern. The issue is that we don't really have a way to
>> check this, other than following the irq_data pointers and check that
>> the hwirq is within a certain range. And even that doesn't guarantee
>> anything.
> OK. But somehow this means the userspace can setup forwarding between an
> SPI and a vLPI, right?

Not really, or at least not as long as we only support PCI. It would
have to be that although you have a GICv4 system, you end-up with an MSI
that is assigned to a GICv2m on the physical side. This cannot happen,
by definition.

So we're actually left with the platform case. An easy way to filter
things would be to look at the routing entry flags, and see if we have a
DEVID there. That would guarantee us that this is an LPI.

[...]

>>>> +	map = (struct its_vlpi_map) {
>>>> +		.vm		= &kvm->arch.vgic.its_vm,
>>>> +		.vpe		= &irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe,
>>>> +		.vintid		= irq->intid,
>>>> +		.properties	= ((irq->priority & 0xfc) |
>>>> +				   (irq->enabled ? LPI_PROP_ENABLED : 0) |
>>>> +				   LPI_PROP_GROUP1),
>>> there is an inconsistency between the comment in irqchip/arm-gic-v4.h
>>> and this setting:
>>>
>>> * @properties: Priority and enable bits (as written in the prop table)
>>
>> Which inconsistency?
> I was confused by LPI_PROP_GROUP1 which was not documented in the
> comment. But looking more carefully in the spec it corresponds to [1] =
> RES1.

Yes, this is a leftover from previous revision of the architecture. That
bit used to be the Group configuration, with only group-1 being valid...
It got turned into a RES1 at a later time.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
  2017-11-07 20:15     ` Auger Eric
  (?)
@ 2017-11-08 11:40       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:40 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

On 07/11/17 20:15, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> If the guest issues an INT command targetting a VLPI, let's
>> call into the irq_set_irqchip_state() helper to make it pending
>> on the physical side.
>>
>> This works just as well if userspace decides to inject an interrupt
>> using the normal userspace API...
> There is also another path:
> KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
> vgic_its_inject_msi / vgic_its_trigger_msi

Isn't this path covered by this very patch?

> I wonder whether we shouldn't prevent the userspace from messing up with
> the host irq pending state?

What do we gain from that limitation? Here, we're just making sure
things will work correctly, and we're not preventing userspace from
doing something silly (the guest will only see spurious interrupts anyway).

Thanks,

	M.

> Thanks
> 
> Eric
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index 89768d2b6a91..b2a678d131d0 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>>  	if (err)
>>  		return err;
>>  
>> +	if (irq->hw)
>> +		return irq_set_irqchip_state(irq->host_irq,
>> +					     IRQCHIP_STATE_PENDING, true);
>> +
>>  	spin_lock(&irq->irq_lock);
>>  	irq->pending_latch = true;
>>  	vgic_queue_irq_unlock(kvm, irq);
>>


-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-11-08 11:40       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:40 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

On 07/11/17 20:15, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> If the guest issues an INT command targetting a VLPI, let's
>> call into the irq_set_irqchip_state() helper to make it pending
>> on the physical side.
>>
>> This works just as well if userspace decides to inject an interrupt
>> using the normal userspace API...
> There is also another path:
> KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
> vgic_its_inject_msi / vgic_its_trigger_msi

Isn't this path covered by this very patch?

> I wonder whether we shouldn't prevent the userspace from messing up with
> the host irq pending state?

What do we gain from that limitation? Here, we're just making sure
things will work correctly, and we're not preventing userspace from
doing something silly (the guest will only see spurious interrupts anyway).

Thanks,

	M.

> Thanks
> 
> Eric
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index 89768d2b6a91..b2a678d131d0 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>>  	if (err)
>>  		return err;
>>  
>> +	if (irq->hw)
>> +		return irq_set_irqchip_state(irq->host_irq,
>> +					     IRQCHIP_STATE_PENDING, true);
>> +
>>  	spin_lock(&irq->irq_lock);
>>  	irq->pending_latch = true;
>>  	vgic_queue_irq_unlock(kvm, irq);
>>


-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-11-08 11:40       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:40 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/11/17 20:15, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> If the guest issues an INT command targetting a VLPI, let's
>> call into the irq_set_irqchip_state() helper to make it pending
>> on the physical side.
>>
>> This works just as well if userspace decides to inject an interrupt
>> using the normal userspace API...
> There is also another path:
> KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
> vgic_its_inject_msi / vgic_its_trigger_msi

Isn't this path covered by this very patch?

> I wonder whether we shouldn't prevent the userspace from messing up with
> the host irq pending state?

What do we gain from that limitation? Here, we're just making sure
things will work correctly, and we're not preventing userspace from
doing something silly (the guest will only see spurious interrupts anyway).

Thanks,

	M.

> Thanks
> 
> Eric
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index 89768d2b6a91..b2a678d131d0 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>>  	if (err)
>>  		return err;
>>  
>> +	if (irq->hw)
>> +		return irq_set_irqchip_state(irq->host_irq,
>> +					     IRQCHIP_STATE_PENDING, true);
>> +
>>  	spin_lock(&irq->irq_lock);
>>  	irq->pending_latch = true;
>>  	vgic_queue_irq_unlock(kvm, irq);
>>


-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
  2017-11-07 20:28     ` Auger Eric
  (?)
@ 2017-11-08 11:52       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:52 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

On 07/11/17 20:28, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> When freeing an LPI (on a DISCARD command, for example), we need
>> to unmap the VLPI down to the physical ITS level.
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index b2a678d131d0..c9b1c0967426 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>>  	list_del(&ite->ite_list);
>>  
>>  	/* This put matches the get in vgic_add_lpi. */
>> -	if (ite->irq)
>> +	if (ite->irq) {
>> +		if (ite->irq->hw)
>> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
>> +
>>  		vgic_put_irq(kvm, ite->irq);
> You could have put the its_unmap_vlpi() directly in vgic_put_irq which
> is meant to decr the ref/release the LPI irq.

Do you see this as an issue?

I'm trying hard to keep the GICv4 changes local to the ITS code, and not
spread it everywhere in the vgic, but I'll happily change it if you spot
something that seems wrong.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-11-08 11:52       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:52 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

On 07/11/17 20:28, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> When freeing an LPI (on a DISCARD command, for example), we need
>> to unmap the VLPI down to the physical ITS level.
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index b2a678d131d0..c9b1c0967426 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>>  	list_del(&ite->ite_list);
>>  
>>  	/* This put matches the get in vgic_add_lpi. */
>> -	if (ite->irq)
>> +	if (ite->irq) {
>> +		if (ite->irq->hw)
>> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
>> +
>>  		vgic_put_irq(kvm, ite->irq);
> You could have put the its_unmap_vlpi() directly in vgic_put_irq which
> is meant to decr the ref/release the LPI irq.

Do you see this as an issue?

I'm trying hard to keep the GICv4 changes local to the ITS code, and not
spread it everywhere in the vgic, but I'll happily change it if you spot
something that seems wrong.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-11-08 11:52       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 11:52 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/11/17 20:28, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> When freeing an LPI (on a DISCARD command, for example), we need
>> to unmap the VLPI down to the physical ITS level.
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index b2a678d131d0..c9b1c0967426 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>>  	list_del(&ite->ite_list);
>>  
>>  	/* This put matches the get in vgic_add_lpi. */
>> -	if (ite->irq)
>> +	if (ite->irq) {
>> +		if (ite->irq->hw)
>> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
>> +
>>  		vgic_put_irq(kvm, ite->irq);
> You could have put the its_unmap_vlpi() directly in vgic_put_irq which
> is meant to decr the ref/release the LPI irq.

Do you see this as an issue?

I'm trying hard to keep the GICv4 changes local to the ITS code, and not
spread it everywhere in the vgic, but I'll happily change it if you spot
something that seems wrong.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
  2017-11-07 21:01     ` Auger Eric
  (?)
@ 2017-11-08 12:05       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 12:05 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On 07/11/17 21:01, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> When the guest issues an affinity change, we need to tell the physical
>> ITS that we're now targetting a new vcpu.  This is done by extracting
>> the current mapping, updating the target, and reapplying the mapping.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
>>  1 file changed, 15 insertions(+), 1 deletion(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index c9b1c0967426..42ffb9084bb7 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>>  
>>  static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
>>  {
>> +	int ret = 0;
>> +
>>  	spin_lock(&irq->irq_lock);
>>  	irq->target_vcpu = vcpu;
>>  	spin_unlock(&irq->irq_lock);
>>  
>> -	return 0;
>> +	if (irq->hw) {
>> +		struct its_vlpi_map map;
>> +
>> +		ret = its_get_vlpi(irq->host_irq, &map);
>> +		if (ret)
>> +			return ret;
>> +
>> +		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
>> +
>> +		ret = its_map_vlpi(irq->host_irq, &map);
> Do we check somewhere the virtual LPI ID does not exceed the capacity of
> the VPT. I fail to find that in the irq-gic-v3-its.c driver.

The worse case is that we end-up with a VMAPVI that silently fails, and
the guest doesn't gets its interrupts. The real issue here is that we
don't expose the size of the VPT allocation to the guest. I'll see how
to address this.

> Also in case the hw part fails, should we still have the irq->target_cpu
> set to the new value. In other words shouldn't we first do the hw block
> and upon success set the target_vcpu?

Sure. But given that we'll never use target_vcpu for anything either (we
keep the vlpi mapped), I'm not sure what this gets us...

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
@ 2017-11-08 12:05       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 12:05 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On 07/11/17 21:01, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> When the guest issues an affinity change, we need to tell the physical
>> ITS that we're now targetting a new vcpu.  This is done by extracting
>> the current mapping, updating the target, and reapplying the mapping.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
>>  1 file changed, 15 insertions(+), 1 deletion(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index c9b1c0967426..42ffb9084bb7 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>>  
>>  static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
>>  {
>> +	int ret = 0;
>> +
>>  	spin_lock(&irq->irq_lock);
>>  	irq->target_vcpu = vcpu;
>>  	spin_unlock(&irq->irq_lock);
>>  
>> -	return 0;
>> +	if (irq->hw) {
>> +		struct its_vlpi_map map;
>> +
>> +		ret = its_get_vlpi(irq->host_irq, &map);
>> +		if (ret)
>> +			return ret;
>> +
>> +		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
>> +
>> +		ret = its_map_vlpi(irq->host_irq, &map);
> Do we check somewhere the virtual LPI ID does not exceed the capacity of
> the VPT. I fail to find that in the irq-gic-v3-its.c driver.

The worse case is that we end-up with a VMAPVI that silently fails, and
the guest doesn't gets its interrupts. The real issue here is that we
don't expose the size of the VPT allocation to the guest. I'll see how
to address this.

> Also in case the hw part fails, should we still have the irq->target_cpu
> set to the new value. In other words shouldn't we first do the hw block
> and upon success set the target_vcpu?

Sure. But given that we'll never use target_vcpu for anything either (we
keep the vlpi mapped), I'm not sure what this gets us...

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS
@ 2017-11-08 12:05       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 12:05 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/11/17 21:01, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> When the guest issues an affinity change, we need to tell the physical
>> ITS that we're now targetting a new vcpu.  This is done by extracting
>> the current mapping, updating the target, and reapplying the mapping.
>>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 16 +++++++++++++++-
>>  1 file changed, 15 insertions(+), 1 deletion(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index c9b1c0967426..42ffb9084bb7 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -337,11 +337,25 @@ static int vgic_copy_lpi_list(struct kvm_vcpu *vcpu, u32 **intid_ptr)
>>  
>>  static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
>>  {
>> +	int ret = 0;
>> +
>>  	spin_lock(&irq->irq_lock);
>>  	irq->target_vcpu = vcpu;
>>  	spin_unlock(&irq->irq_lock);
>>  
>> -	return 0;
>> +	if (irq->hw) {
>> +		struct its_vlpi_map map;
>> +
>> +		ret = its_get_vlpi(irq->host_irq, &map);
>> +		if (ret)
>> +			return ret;
>> +
>> +		map.vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
>> +
>> +		ret = its_map_vlpi(irq->host_irq, &map);
> Do we check somewhere the virtual LPI ID does not exceed the capacity of
> the VPT. I fail to find that in the irq-gic-v3-its.c driver.

The worse case is that we end-up with a VMAPVI that silently fails, and
the guest doesn't gets its interrupts. The real issue here is that we
don't expose the size of the VPT allocation to the guest. I'll see how
to address this.

> Also in case the hw part fails, should we still have the irq->target_cpu
> set to the new value. In other words shouldn't we first do the hw block
> and upon success set the target_vcpu?

Sure. But given that we'll never use target_vcpu for anything either (we
keep the vlpi mapped), I'm not sure what this gets us...

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
  2017-11-08 11:52       ` Marc Zyngier
  (?)
@ 2017-11-08 14:14         ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08 14:14 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Hi,

On 08/11/2017 12:52, Marc Zyngier wrote:
> On 07/11/17 20:28, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> When freeing an LPI (on a DISCARD command, for example), we need
>>> to unmap the VLPI down to the physical ITS level.
>>>
>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index b2a678d131d0..c9b1c0967426 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>>>  	list_del(&ite->ite_list);
>>>  
>>>  	/* This put matches the get in vgic_add_lpi. */
>>> -	if (ite->irq)
>>> +	if (ite->irq) {
>>> +		if (ite->irq->hw)
>>> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
>>> +
>>>  		vgic_put_irq(kvm, ite->irq);
>> You could have put the its_unmap_vlpi() directly in vgic_put_irq which
>> is meant to decr the ref/release the LPI irq.
> 
> Do you see this as an issue?

not really. I just thought it would be nicer in vgic_put_irq but that's
a detail.

Thanks

Eric
> 
> I'm trying hard to keep the GICv4 changes local to the ITS code, and not
> spread it everywhere in the vgic, but I'll happily change it if you spot
> something that seems wrong.
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-11-08 14:14         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08 14:14 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara, Christoffer Dall

Hi,

On 08/11/2017 12:52, Marc Zyngier wrote:
> On 07/11/17 20:28, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> When freeing an LPI (on a DISCARD command, for example), we need
>>> to unmap the VLPI down to the physical ITS level.
>>>
>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index b2a678d131d0..c9b1c0967426 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>>>  	list_del(&ite->ite_list);
>>>  
>>>  	/* This put matches the get in vgic_add_lpi. */
>>> -	if (ite->irq)
>>> +	if (ite->irq) {
>>> +		if (ite->irq->hw)
>>> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
>>> +
>>>  		vgic_put_irq(kvm, ite->irq);
>> You could have put the its_unmap_vlpi() directly in vgic_put_irq which
>> is meant to decr the ref/release the LPI irq.
> 
> Do you see this as an issue?

not really. I just thought it would be nicer in vgic_put_irq but that's
a detail.

Thanks

Eric
> 
> I'm trying hard to keep the GICv4 changes local to the ITS code, and not
> spread it everywhere in the vgic, but I'll happily change it if you spot
> something that seems wrong.
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI
@ 2017-11-08 14:14         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08 14:14 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On 08/11/2017 12:52, Marc Zyngier wrote:
> On 07/11/17 20:28, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> When freeing an LPI (on a DISCARD command, for example), we need
>>> to unmap the VLPI down to the physical ITS level.
>>>
>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  virt/kvm/arm/vgic/vgic-its.c | 6 +++++-
>>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index b2a678d131d0..c9b1c0967426 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -628,8 +628,12 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
>>>  	list_del(&ite->ite_list);
>>>  
>>>  	/* This put matches the get in vgic_add_lpi. */
>>> -	if (ite->irq)
>>> +	if (ite->irq) {
>>> +		if (ite->irq->hw)
>>> +			WARN_ON(its_unmap_vlpi(ite->irq->host_irq));
>>> +
>>>  		vgic_put_irq(kvm, ite->irq);
>> You could have put the its_unmap_vlpi() directly in vgic_put_irq which
>> is meant to decr the ref/release the LPI irq.
> 
> Do you see this as an issue?

not really. I just thought it would be nicer in vgic_put_irq but that's
a detail.

Thanks

Eric
> 
> I'm trying hard to keep the GICv4 changes local to the ITS code, and not
> spread it everywhere in the vgic, but I'll happily change it if you spot
> something that seems wrong.
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
  2017-11-08 11:40       ` Marc Zyngier
  (?)
@ 2017-11-08 14:14         ` Auger Eric
  -1 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08 14:14 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi Marc,

On 08/11/2017 12:40, Marc Zyngier wrote:
> On 07/11/17 20:15, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> If the guest issues an INT command targetting a VLPI, let's
>>> call into the irq_set_irqchip_state() helper to make it pending
>>> on the physical side.
>>>
>>> This works just as well if userspace decides to inject an interrupt
>>> using the normal userspace API...
>> There is also another path:
>> KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
>> vgic_its_inject_msi / vgic_its_trigger_msi
> 
> Isn't this path covered by this very patch?

I should have read the last sentence of the commit msg to see you
haven't ignored it ;-)
> 
>> I wonder whether we shouldn't prevent the userspace from messing up with
>> the host irq pending state?
> 
> What do we gain from that limitation? Here, we're just making sure
> things will work correctly, and we're not preventing userspace from
> doing something silly (the guest will only see spurious interrupts anyway).

OK. Just wanted to make sure this was not an issue.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 
>> Thanks
>>
>> Eric
>>>
>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>>>  1 file changed, 4 insertions(+)
>>>
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index 89768d2b6a91..b2a678d131d0 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>>>  	if (err)
>>>  		return err;
>>>  
>>> +	if (irq->hw)
>>> +		return irq_set_irqchip_state(irq->host_irq,
>>> +					     IRQCHIP_STATE_PENDING, true);
>>> +
>>>  	spin_lock(&irq->irq_lock);
>>>  	irq->pending_latch = true;
>>>  	vgic_queue_irq_unlock(kvm, irq);
>>>
> 
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-11-08 14:14         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08 14:14 UTC (permalink / raw)
  To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

Hi Marc,

On 08/11/2017 12:40, Marc Zyngier wrote:
> On 07/11/17 20:15, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> If the guest issues an INT command targetting a VLPI, let's
>>> call into the irq_set_irqchip_state() helper to make it pending
>>> on the physical side.
>>>
>>> This works just as well if userspace decides to inject an interrupt
>>> using the normal userspace API...
>> There is also another path:
>> KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
>> vgic_its_inject_msi / vgic_its_trigger_msi
> 
> Isn't this path covered by this very patch?

I should have read the last sentence of the commit msg to see you
haven't ignored it ;-)
> 
>> I wonder whether we shouldn't prevent the userspace from messing up with
>> the host irq pending state?
> 
> What do we gain from that limitation? Here, we're just making sure
> things will work correctly, and we're not preventing userspace from
> doing something silly (the guest will only see spurious interrupts anyway).

OK. Just wanted to make sure this was not an issue.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 
>> Thanks
>>
>> Eric
>>>
>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>>>  1 file changed, 4 insertions(+)
>>>
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index 89768d2b6a91..b2a678d131d0 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>>>  	if (err)
>>>  		return err;
>>>  
>>> +	if (irq->hw)
>>> +		return irq_set_irqchip_state(irq->host_irq,
>>> +					     IRQCHIP_STATE_PENDING, true);
>>> +
>>>  	spin_lock(&irq->irq_lock);
>>>  	irq->pending_latch = true;
>>>  	vgic_queue_irq_unlock(kvm, irq);
>>>
> 
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI
@ 2017-11-08 14:14         ` Auger Eric
  0 siblings, 0 replies; 234+ messages in thread
From: Auger Eric @ 2017-11-08 14:14 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 08/11/2017 12:40, Marc Zyngier wrote:
> On 07/11/17 20:15, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> If the guest issues an INT command targetting a VLPI, let's
>>> call into the irq_set_irqchip_state() helper to make it pending
>>> on the physical side.
>>>
>>> This works just as well if userspace decides to inject an interrupt
>>> using the normal userspace API...
>> There is also another path:
>> KVM_SIGNAL_MSI ioctl / kvm_send_userspace_msi / kvm_set_msi /
>> vgic_its_inject_msi / vgic_its_trigger_msi
> 
> Isn't this path covered by this very patch?

I should have read the last sentence of the commit msg to see you
haven't ignored it ;-)
> 
>> I wonder whether we shouldn't prevent the userspace from messing up with
>> the host irq pending state?
> 
> What do we gain from that limitation? Here, we're just making sure
> things will work correctly, and we're not preventing userspace from
> doing something silly (the guest will only see spurious interrupts anyway).

OK. Just wanted to make sure this was not an issue.

Thanks

Eric
> 
> Thanks,
> 
> 	M.
> 
>> Thanks
>>
>> Eric
>>>
>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  virt/kvm/arm/vgic/vgic-its.c | 4 ++++
>>>  1 file changed, 4 insertions(+)
>>>
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index 89768d2b6a91..b2a678d131d0 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -578,6 +578,10 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,
>>>  	if (err)
>>>  		return err;
>>>  
>>> +	if (irq->hw)
>>> +		return irq_set_irqchip_state(irq->host_irq,
>>> +					     IRQCHIP_STATE_PENDING, true);
>>> +
>>>  	spin_lock(&irq->irq_lock);
>>>  	irq->pending_latch = true;
>>>  	vgic_queue_irq_unlock(kvm, irq);
>>>
> 
> 

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  2017-11-07 21:28     ` Auger Eric
  (?)
@ 2017-11-08 15:08       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:08 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

On 07/11/17 21:28, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Upon updating a property, we propagate it all the way to the physical
>> ITS, and ask for an INV command to be executed there.
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index 0b7e648e7a0c..2e77c7c83942 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>>  		spin_unlock(&irq->irq_lock);
>>  	}
>>  
>> +	if (irq->hw)
>> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
>> +
>>  	return 0;
>>  }
> I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
> needed in hw mode?

It's not. I guess we could bypass this altogether and take a short cut
after having updated the priority and enabled fields.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-08 15:08       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:08 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

On 07/11/17 21:28, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Upon updating a property, we propagate it all the way to the physical
>> ITS, and ask for an INV command to be executed there.
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index 0b7e648e7a0c..2e77c7c83942 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>>  		spin_unlock(&irq->irq_lock);
>>  	}
>>  
>> +	if (irq->hw)
>> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
>> +
>>  	return 0;
>>  }
> I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
> needed in hw mode?

It's not. I guess we could bypass this altogether and take a short cut
after having updated the priority and enabled fields.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-08 15:08       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:08 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/11/17 21:28, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Upon updating a property, we propagate it all the way to the physical
>> ITS, and ask for an INV command to be executed there.
>>
>> Acked-by: Christoffer Dall <cdall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>> index 0b7e648e7a0c..2e77c7c83942 100644
>> --- a/virt/kvm/arm/vgic/vgic-its.c
>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>>  		spin_unlock(&irq->irq_lock);
>>  	}
>>  
>> +	if (irq->hw)
>> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
>> +
>>  	return 0;
>>  }
> I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
> needed in hw mode?

It's not. I guess we could bypass this altogether and take a short cut
after having updated the priority and enabled fields.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
  2017-11-08  8:44     ` Auger Eric
  (?)
@ 2017-11-08 15:14       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:14 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Christoffer Dall, Andre Przywara,
	Shameerali Kolothum Thodi, Christoffer Dall, Shanker Donthineni

On 08/11/17 08:44, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> All it takes is the has_v4 flag to be set in gic_kvm_info
>> as well as "kvm-arm.vgic_v4_enable=1" being passed on the
>> command line for GICv4 to be enabled in KVM.
> 
> What did you motivate your choice of having an enable option instead of
> a disable option?

Currently, SW MSI injection works really well. It's been validated,
plenty of people are using it. On the other hand, GICv4 is new and
probably buggy. I only know of 2 other people having tried it. I'm also
not sure it gives the best possible performance in the general use case
(oversubscribed host with freely migrating vcpus).

Given that, I'm not keen on making it the default. But if you can report
that it is better and more reliable than SW injection, I'll switch the
default the other way around! ;-)

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
@ 2017-11-08 15:14       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:14 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Christoffer Dall, Andre Przywara

On 08/11/17 08:44, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> All it takes is the has_v4 flag to be set in gic_kvm_info
>> as well as "kvm-arm.vgic_v4_enable=1" being passed on the
>> command line for GICv4 to be enabled in KVM.
> 
> What did you motivate your choice of having an enable option instead of
> a disable option?

Currently, SW MSI injection works really well. It's been validated,
plenty of people are using it. On the other hand, GICv4 is new and
probably buggy. I only know of 2 other people having tried it. I'm also
not sure it gives the best possible performance in the general use case
(oversubscribed host with freely migrating vcpus).

Given that, I'm not keen on making it the default. But if you can report
that it is better and more reliable than SW injection, I'll switch the
default the other way around! ;-)

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support
@ 2017-11-08 15:14       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:14 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/11/17 08:44, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> All it takes is the has_v4 flag to be set in gic_kvm_info
>> as well as "kvm-arm.vgic_v4_enable=1" being passed on the
>> command line for GICv4 to be enabled in KVM.
> 
> What did you motivate your choice of having an enable option instead of
> a disable option?

Currently, SW MSI injection works really well. It's been validated,
plenty of people are using it. On the other hand, GICv4 is new and
probably buggy. I only know of 2 other people having tried it. I'm also
not sure it gives the best possible performance in the general use case
(oversubscribed host with freely migrating vcpus).

Given that, I'm not keen on making it the default. But if you can report
that it is better and more reliable than SW injection, I'll switch the
default the other way around! ;-)

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
  2017-11-08  9:13     ` Auger Eric
  (?)
@ 2017-11-08 15:19       ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:19 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel
  Cc: Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On 08/11/17 09:13, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Yet another braindump so I can free some cells...
>>
>> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 67 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index d10e18eabd3b..e367d65a0ebe 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -23,6 +23,73 @@
>>  
>>  #include "vgic.h"
>>  
>> +/*
>> + * How KVM uses GICv4 (insert rude comments here):
>> + *
>> + * The vgic-v4 layer acts as a bridge between several entities:
>> + * - The GICv4 ITS representation offered by the ITS driver
>> + * - VFIO, which is in charge of the PCI endpoint
>> + * - The virtual ITS, which is the only thing the guest sees
>> + *
>> + * The configuration of VLPIs is triggered by a callback from VFIO,
>> + * instructing KVM that a PCI device has been configured to deliver
>> + * MSIs to a vITS.
> We actually have a negotiation protocol between VFIO PCI (irq bypass
> producer) and KVM irqfd (IRQ bypass consumer). When both recognize they
> are tied together, handling an MSI tunneling, they initiate the
> forwarding setup.
>> + *
>> + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
>> + * and this is used to find the corresponding vITS data structures
>> + * (ITS instance, device, event and irq) using a process that is
>> + * extremely similar to the injection of an MSI.
> 
> Is it correct to say we replace the following injection chain:
> 
> pEventID|										
> (pITS)	|-> pLPIID -> VFIO PCI IRQ handler -> KVM irqfd ...
> pDevID	|
> 
> 		vEventID|
> ... inject	(vITS)	| -> vLPIID
> 		vDevID	|
> 
> by
> 
> pEventID|
> (pITS)	| -> vLPIID
> pDevID	|

That's indeed a pretty accurate description of what happens, with a side
messaging (doorbells).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
@ 2017-11-08 15:19       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:19 UTC (permalink / raw)
  To: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel; +Cc: Andre Przywara

On 08/11/17 09:13, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Yet another braindump so I can free some cells...
>>
>> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 67 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index d10e18eabd3b..e367d65a0ebe 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -23,6 +23,73 @@
>>  
>>  #include "vgic.h"
>>  
>> +/*
>> + * How KVM uses GICv4 (insert rude comments here):
>> + *
>> + * The vgic-v4 layer acts as a bridge between several entities:
>> + * - The GICv4 ITS representation offered by the ITS driver
>> + * - VFIO, which is in charge of the PCI endpoint
>> + * - The virtual ITS, which is the only thing the guest sees
>> + *
>> + * The configuration of VLPIs is triggered by a callback from VFIO,
>> + * instructing KVM that a PCI device has been configured to deliver
>> + * MSIs to a vITS.
> We actually have a negotiation protocol between VFIO PCI (irq bypass
> producer) and KVM irqfd (IRQ bypass consumer). When both recognize they
> are tied together, handling an MSI tunneling, they initiate the
> forwarding setup.
>> + *
>> + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
>> + * and this is used to find the corresponding vITS data structures
>> + * (ITS instance, device, event and irq) using a process that is
>> + * extremely similar to the injection of an MSI.
> 
> Is it correct to say we replace the following injection chain:
> 
> pEventID|										
> (pITS)	|-> pLPIID -> VFIO PCI IRQ handler -> KVM irqfd ...
> pDevID	|
> 
> 		vEventID|
> ... inject	(vITS)	| -> vLPIID
> 		vDevID	|
> 
> by
> 
> pEventID|
> (pITS)	| -> vLPIID
> pDevID	|

That's indeed a pretty accurate description of what happens, with a side
messaging (doorbells).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations
@ 2017-11-08 15:19       ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-08 15:19 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/11/17 09:13, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
>> Yet another braindump so I can free some cells...
>>
>> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 67 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>> index d10e18eabd3b..e367d65a0ebe 100644
>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>> @@ -23,6 +23,73 @@
>>  
>>  #include "vgic.h"
>>  
>> +/*
>> + * How KVM uses GICv4 (insert rude comments here):
>> + *
>> + * The vgic-v4 layer acts as a bridge between several entities:
>> + * - The GICv4 ITS representation offered by the ITS driver
>> + * - VFIO, which is in charge of the PCI endpoint
>> + * - The virtual ITS, which is the only thing the guest sees
>> + *
>> + * The configuration of VLPIs is triggered by a callback from VFIO,
>> + * instructing KVM that a PCI device has been configured to deliver
>> + * MSIs to a vITS.
> We actually have a negotiation protocol between VFIO PCI (irq bypass
> producer) and KVM irqfd (IRQ bypass consumer). When both recognize they
> are tied together, handling an MSI tunneling, they initiate the
> forwarding setup.
>> + *
>> + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry,
>> + * and this is used to find the corresponding vITS data structures
>> + * (ITS instance, device, event and irq) using a process that is
>> + * extremely similar to the injection of an MSI.
> 
> Is it correct to say we replace the following injection chain:
> 
> pEventID|										
> (pITS)	|-> pLPIID -> VFIO PCI IRQ handler -> KVM irqfd ...
> pDevID	|
> 
> 		vEventID|
> ... inject	(vITS)	| -> vLPIID
> 		vDevID	|
> 
> by
> 
> pEventID|
> (pITS)	| -> vLPIID
> pDevID	|

That's indeed a pretty accurate description of what happens, with a side
messaging (doorbells).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  2017-11-07 13:08     ` Auger Eric
  (?)
@ 2017-11-10  8:20       ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:20 UTC (permalink / raw)
  To: Auger Eric
  Cc: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

On Tue, Nov 07, 2017 at 02:08:23PM +0100, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
> > In order to control the GICv4 view of virtual CPUs, we rely
> > on an irqdomain allocated for that purpose. Let's add a couple
> > of helpers to that effect.
> > 
> > At the same time, the vgic data structures gain new fields to
> > track all this... erm... wonderful stuff.
> > 
> > The way we hook into the vgic init is slightly convoluted. We
> > need the vgic to be initialized (in order to guarantee that
> > the number of vcpus is now fixed), and we must have a vITS
> > (otherwise this is all very pointless). So we end-up calling
> > the init from both vgic_init and vgic_its_create.
> > 
> > Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> > Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> > ---
> >  arch/arm/kvm/Makefile         |  1 +
> >  arch/arm64/kvm/Makefile       |  1 +
> >  include/kvm/arm_vgic.h        | 19 ++++++++++
> >  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
> >  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
> >  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
> >  virt/kvm/arm/vgic/vgic.h      |  2 ++
> >  7 files changed, 123 insertions(+)
> >  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> > 
> > diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> > index d9beee652d36..0a1dd2cdb928 100644
> > --- a/arch/arm/kvm/Makefile
> > +++ b/arch/arm/kvm/Makefile
> > @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
> >  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
> >  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> >  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> > +obj-y += $(KVM)/arm/vgic/vgic-v4.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> > diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> > index 5d9810086c25..c30fd388ef80 100644
> > --- a/arch/arm64/kvm/Makefile
> > +++ b/arch/arm64/kvm/Makefile
> > @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> > +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> > diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> > index ba9fb450aa1b..7eeb6c2a2f9c 100644
> > --- a/include/kvm/arm_vgic.h
> > +++ b/include/kvm/arm_vgic.h
> > @@ -26,6 +26,8 @@
> >  #include <linux/list.h>
> >  #include <linux/jump_label.h>
> >  
> > +#include <linux/irqchip/arm-gic-v4.h>
> > +
> >  #define VGIC_V3_MAX_CPUS	255
> >  #define VGIC_V2_MAX_CPUS	8
> >  #define VGIC_NR_IRQS_LEGACY     256
> > @@ -236,6 +238,15 @@ struct vgic_dist {
> >  
> >  	/* used by vgic-debug */
> >  	struct vgic_state_iter *iter;
> > +
> > +	/*
> > +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> > +	 * array, the property table pointer as well as allocation
> > +	 * data. This essentially ties the Linux IRQ core and ITS
> > +	 * together, and avoids leaking KVM's data structures anywhere
> > +	 * else.
> > +	 */
> > +	struct its_vm		its_vm;
> >  };
> >  
> >  struct vgic_v2_cpu_if {
> > @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
> >  	u32		vgic_ap0r[4];
> >  	u32		vgic_ap1r[4];
> >  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> > +
> > +	/*
> > +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> > +	 * pending table pointer, the its_vm pointer and a few other
> > +	 * HW specific things. As for the its_vm structure, this is
> > +	 * linking the Linux IRQ subsystem and the ITS together.
> > +	 */
> > +	struct its_vpe	its_vpe;
> >  };
> >  
> >  struct vgic_cpu {
> > diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> > index 5801261f3add..40be908da238 100644
> > --- a/virt/kvm/arm/vgic/vgic-init.c
> > +++ b/virt/kvm/arm/vgic/vgic-init.c
> > @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
> >  	if (ret)
> >  		goto out;
> >  
> > +	if (vgic_supports_direct_msis(kvm)) {
> > +		ret = vgic_v4_init(kvm);
> > +		if (ret)
> > +			goto out;
> > +	}
> > +
> >  	kvm_for_each_vcpu(i, vcpu, kvm)
> >  		kvm_vgic_vcpu_enable(vcpu);
> >  
> > @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
> >  
> >  	kfree(dist->spis);
> >  	dist->nr_spis = 0;
> > +
> > +	if (vgic_supports_direct_msis(kvm))
> > +		vgic_v4_teardown(kvm);
> >  }
> >  
> >  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> > diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> > index 8ee03f1e89fc..89768d2b6a91 100644
> > --- a/virt/kvm/arm/vgic/vgic-its.c
> > +++ b/virt/kvm/arm/vgic/vgic-its.c
> > @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
> >  	if (!its)
> >  		return -ENOMEM;
> >  
> > +	if (vgic_initialized(dev->kvm)) {
> Don't we need to test vgic_supports_direct_msis() on this path too?
> 

Seems to me that we should, otherwise creating an ITS after the VGIC has
been initialized would fail on non-GICv4 compatible systems, right?

How about this patch as a follow-up to the series:

commit 48ec1662d0f10d6468907cdc7e12c46ca1ef497c (HEAD -> next-gicv4)
Author: Christoffer Dall <christoffer.dall@linaro.org>
Date:   Fri Nov 10 09:16:23 2017 +0100

    KVM: arm/arm64: Fix GICv4 ITS initialization issues
    
    We should only try to initialize GICv4 data structures on a GICv4
    capable system.  Move the vgic_supports_direct_msis() check inito
    vgic_v4_init() so that any KVM VGIC initialization path does not fail
    on non-GICv4 systems.
    
    Also be slightly more strict in the checking of the return value in
    vgic_its_create, and only error out on negative return values from the
    vgic_v4_init() function.  This is important because the kvm device code
    only treats negative values as errors and only cleans up in this case.
    Errornously treating a positive return value as an error from the
    vgic_v4_init() function can lead to NULL pointer dereferences, as has
    recently been observed.
    
    Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>

diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 40be908da238..62310122ee78 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -285,11 +285,9 @@ int vgic_init(struct kvm *kvm)
 	if (ret)
 		goto out;
 
-	if (vgic_supports_direct_msis(kvm)) {
-		ret = vgic_v4_init(kvm);
-		if (ret)
-			goto out;
-	}
+	ret = vgic_v4_init(kvm);
+	if (ret)
+		goto out;
 
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_enable(vcpu);
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index b8c1b724ba3e..c93ecd4a903b 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1673,7 +1673,7 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
 
 	if (vgic_initialized(dev->kvm)) {
 		int ret = vgic_v4_init(dev->kvm);
-		if (ret) {
+		if (ret < 0) {
 			kfree(its);
 			return ret;
 		}
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index e367d65a0ebe..bb7e31fcee35 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -118,6 +118,9 @@ int vgic_v4_init(struct kvm *kvm)
 	struct kvm_vcpu *vcpu;
 	int i, nr_vcpus, ret;
 
+	if (!vgic_supports_direct_msis(kvm))
+		return 0; /* Nothing to see here... move along. */
+
 	if (dist->its_vm.vpes)
 		return 0;
 

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-10  8:20       ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:20 UTC (permalink / raw)
  To: Auger Eric
  Cc: kvm, Marc Zyngier, Andre Przywara, linux-kernel,
	linux-arm-kernel, kvmarm

On Tue, Nov 07, 2017 at 02:08:23PM +0100, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
> > In order to control the GICv4 view of virtual CPUs, we rely
> > on an irqdomain allocated for that purpose. Let's add a couple
> > of helpers to that effect.
> > 
> > At the same time, the vgic data structures gain new fields to
> > track all this... erm... wonderful stuff.
> > 
> > The way we hook into the vgic init is slightly convoluted. We
> > need the vgic to be initialized (in order to guarantee that
> > the number of vcpus is now fixed), and we must have a vITS
> > (otherwise this is all very pointless). So we end-up calling
> > the init from both vgic_init and vgic_its_create.
> > 
> > Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> > Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> > ---
> >  arch/arm/kvm/Makefile         |  1 +
> >  arch/arm64/kvm/Makefile       |  1 +
> >  include/kvm/arm_vgic.h        | 19 ++++++++++
> >  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
> >  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
> >  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
> >  virt/kvm/arm/vgic/vgic.h      |  2 ++
> >  7 files changed, 123 insertions(+)
> >  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> > 
> > diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> > index d9beee652d36..0a1dd2cdb928 100644
> > --- a/arch/arm/kvm/Makefile
> > +++ b/arch/arm/kvm/Makefile
> > @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
> >  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
> >  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> >  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> > +obj-y += $(KVM)/arm/vgic/vgic-v4.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> > diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> > index 5d9810086c25..c30fd388ef80 100644
> > --- a/arch/arm64/kvm/Makefile
> > +++ b/arch/arm64/kvm/Makefile
> > @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> > +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> > diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> > index ba9fb450aa1b..7eeb6c2a2f9c 100644
> > --- a/include/kvm/arm_vgic.h
> > +++ b/include/kvm/arm_vgic.h
> > @@ -26,6 +26,8 @@
> >  #include <linux/list.h>
> >  #include <linux/jump_label.h>
> >  
> > +#include <linux/irqchip/arm-gic-v4.h>
> > +
> >  #define VGIC_V3_MAX_CPUS	255
> >  #define VGIC_V2_MAX_CPUS	8
> >  #define VGIC_NR_IRQS_LEGACY     256
> > @@ -236,6 +238,15 @@ struct vgic_dist {
> >  
> >  	/* used by vgic-debug */
> >  	struct vgic_state_iter *iter;
> > +
> > +	/*
> > +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> > +	 * array, the property table pointer as well as allocation
> > +	 * data. This essentially ties the Linux IRQ core and ITS
> > +	 * together, and avoids leaking KVM's data structures anywhere
> > +	 * else.
> > +	 */
> > +	struct its_vm		its_vm;
> >  };
> >  
> >  struct vgic_v2_cpu_if {
> > @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
> >  	u32		vgic_ap0r[4];
> >  	u32		vgic_ap1r[4];
> >  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> > +
> > +	/*
> > +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> > +	 * pending table pointer, the its_vm pointer and a few other
> > +	 * HW specific things. As for the its_vm structure, this is
> > +	 * linking the Linux IRQ subsystem and the ITS together.
> > +	 */
> > +	struct its_vpe	its_vpe;
> >  };
> >  
> >  struct vgic_cpu {
> > diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> > index 5801261f3add..40be908da238 100644
> > --- a/virt/kvm/arm/vgic/vgic-init.c
> > +++ b/virt/kvm/arm/vgic/vgic-init.c
> > @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
> >  	if (ret)
> >  		goto out;
> >  
> > +	if (vgic_supports_direct_msis(kvm)) {
> > +		ret = vgic_v4_init(kvm);
> > +		if (ret)
> > +			goto out;
> > +	}
> > +
> >  	kvm_for_each_vcpu(i, vcpu, kvm)
> >  		kvm_vgic_vcpu_enable(vcpu);
> >  
> > @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
> >  
> >  	kfree(dist->spis);
> >  	dist->nr_spis = 0;
> > +
> > +	if (vgic_supports_direct_msis(kvm))
> > +		vgic_v4_teardown(kvm);
> >  }
> >  
> >  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> > diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> > index 8ee03f1e89fc..89768d2b6a91 100644
> > --- a/virt/kvm/arm/vgic/vgic-its.c
> > +++ b/virt/kvm/arm/vgic/vgic-its.c
> > @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
> >  	if (!its)
> >  		return -ENOMEM;
> >  
> > +	if (vgic_initialized(dev->kvm)) {
> Don't we need to test vgic_supports_direct_msis() on this path too?
> 

Seems to me that we should, otherwise creating an ITS after the VGIC has
been initialized would fail on non-GICv4 compatible systems, right?

How about this patch as a follow-up to the series:

commit 48ec1662d0f10d6468907cdc7e12c46ca1ef497c (HEAD -> next-gicv4)
Author: Christoffer Dall <christoffer.dall@linaro.org>
Date:   Fri Nov 10 09:16:23 2017 +0100

    KVM: arm/arm64: Fix GICv4 ITS initialization issues
    
    We should only try to initialize GICv4 data structures on a GICv4
    capable system.  Move the vgic_supports_direct_msis() check inito
    vgic_v4_init() so that any KVM VGIC initialization path does not fail
    on non-GICv4 systems.
    
    Also be slightly more strict in the checking of the return value in
    vgic_its_create, and only error out on negative return values from the
    vgic_v4_init() function.  This is important because the kvm device code
    only treats negative values as errors and only cleans up in this case.
    Errornously treating a positive return value as an error from the
    vgic_v4_init() function can lead to NULL pointer dereferences, as has
    recently been observed.
    
    Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>

diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 40be908da238..62310122ee78 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -285,11 +285,9 @@ int vgic_init(struct kvm *kvm)
 	if (ret)
 		goto out;
 
-	if (vgic_supports_direct_msis(kvm)) {
-		ret = vgic_v4_init(kvm);
-		if (ret)
-			goto out;
-	}
+	ret = vgic_v4_init(kvm);
+	if (ret)
+		goto out;
 
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_enable(vcpu);
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index b8c1b724ba3e..c93ecd4a903b 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1673,7 +1673,7 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
 
 	if (vgic_initialized(dev->kvm)) {
 		int ret = vgic_v4_init(dev->kvm);
-		if (ret) {
+		if (ret < 0) {
 			kfree(its);
 			return ret;
 		}
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index e367d65a0ebe..bb7e31fcee35 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -118,6 +118,9 @@ int vgic_v4_init(struct kvm *kvm)
 	struct kvm_vcpu *vcpu;
 	int i, nr_vcpus, ret;
 
+	if (!vgic_supports_direct_msis(kvm))
+		return 0; /* Nothing to see here... move along. */
+
 	if (dist->its_vm.vpes)
 		return 0;

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-10  8:20       ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:20 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Nov 07, 2017 at 02:08:23PM +0100, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
> > In order to control the GICv4 view of virtual CPUs, we rely
> > on an irqdomain allocated for that purpose. Let's add a couple
> > of helpers to that effect.
> > 
> > At the same time, the vgic data structures gain new fields to
> > track all this... erm... wonderful stuff.
> > 
> > The way we hook into the vgic init is slightly convoluted. We
> > need the vgic to be initialized (in order to guarantee that
> > the number of vcpus is now fixed), and we must have a vITS
> > (otherwise this is all very pointless). So we end-up calling
> > the init from both vgic_init and vgic_its_create.
> > 
> > Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> > Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> > ---
> >  arch/arm/kvm/Makefile         |  1 +
> >  arch/arm64/kvm/Makefile       |  1 +
> >  include/kvm/arm_vgic.h        | 19 ++++++++++
> >  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
> >  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
> >  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
> >  virt/kvm/arm/vgic/vgic.h      |  2 ++
> >  7 files changed, 123 insertions(+)
> >  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
> > 
> > diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> > index d9beee652d36..0a1dd2cdb928 100644
> > --- a/arch/arm/kvm/Makefile
> > +++ b/arch/arm/kvm/Makefile
> > @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
> >  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
> >  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> >  obj-y += $(KVM)/arm/vgic/vgic-v3.o
> > +obj-y += $(KVM)/arm/vgic/vgic-v4.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> > diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> > index 5d9810086c25..c30fd388ef80 100644
> > --- a/arch/arm64/kvm/Makefile
> > +++ b/arch/arm64/kvm/Makefile
> > @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
> > +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> > diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> > index ba9fb450aa1b..7eeb6c2a2f9c 100644
> > --- a/include/kvm/arm_vgic.h
> > +++ b/include/kvm/arm_vgic.h
> > @@ -26,6 +26,8 @@
> >  #include <linux/list.h>
> >  #include <linux/jump_label.h>
> >  
> > +#include <linux/irqchip/arm-gic-v4.h>
> > +
> >  #define VGIC_V3_MAX_CPUS	255
> >  #define VGIC_V2_MAX_CPUS	8
> >  #define VGIC_NR_IRQS_LEGACY     256
> > @@ -236,6 +238,15 @@ struct vgic_dist {
> >  
> >  	/* used by vgic-debug */
> >  	struct vgic_state_iter *iter;
> > +
> > +	/*
> > +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
> > +	 * array, the property table pointer as well as allocation
> > +	 * data. This essentially ties the Linux IRQ core and ITS
> > +	 * together, and avoids leaking KVM's data structures anywhere
> > +	 * else.
> > +	 */
> > +	struct its_vm		its_vm;
> >  };
> >  
> >  struct vgic_v2_cpu_if {
> > @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
> >  	u32		vgic_ap0r[4];
> >  	u32		vgic_ap1r[4];
> >  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> > +
> > +	/*
> > +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
> > +	 * pending table pointer, the its_vm pointer and a few other
> > +	 * HW specific things. As for the its_vm structure, this is
> > +	 * linking the Linux IRQ subsystem and the ITS together.
> > +	 */
> > +	struct its_vpe	its_vpe;
> >  };
> >  
> >  struct vgic_cpu {
> > diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> > index 5801261f3add..40be908da238 100644
> > --- a/virt/kvm/arm/vgic/vgic-init.c
> > +++ b/virt/kvm/arm/vgic/vgic-init.c
> > @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
> >  	if (ret)
> >  		goto out;
> >  
> > +	if (vgic_supports_direct_msis(kvm)) {
> > +		ret = vgic_v4_init(kvm);
> > +		if (ret)
> > +			goto out;
> > +	}
> > +
> >  	kvm_for_each_vcpu(i, vcpu, kvm)
> >  		kvm_vgic_vcpu_enable(vcpu);
> >  
> > @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
> >  
> >  	kfree(dist->spis);
> >  	dist->nr_spis = 0;
> > +
> > +	if (vgic_supports_direct_msis(kvm))
> > +		vgic_v4_teardown(kvm);
> >  }
> >  
> >  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
> > diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> > index 8ee03f1e89fc..89768d2b6a91 100644
> > --- a/virt/kvm/arm/vgic/vgic-its.c
> > +++ b/virt/kvm/arm/vgic/vgic-its.c
> > @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
> >  	if (!its)
> >  		return -ENOMEM;
> >  
> > +	if (vgic_initialized(dev->kvm)) {
> Don't we need to test vgic_supports_direct_msis() on this path too?
> 

Seems to me that we should, otherwise creating an ITS after the VGIC has
been initialized would fail on non-GICv4 compatible systems, right?

How about this patch as a follow-up to the series:

commit 48ec1662d0f10d6468907cdc7e12c46ca1ef497c (HEAD -> next-gicv4)
Author: Christoffer Dall <christoffer.dall@linaro.org>
Date:   Fri Nov 10 09:16:23 2017 +0100

    KVM: arm/arm64: Fix GICv4 ITS initialization issues
    
    We should only try to initialize GICv4 data structures on a GICv4
    capable system.  Move the vgic_supports_direct_msis() check inito
    vgic_v4_init() so that any KVM VGIC initialization path does not fail
    on non-GICv4 systems.
    
    Also be slightly more strict in the checking of the return value in
    vgic_its_create, and only error out on negative return values from the
    vgic_v4_init() function.  This is important because the kvm device code
    only treats negative values as errors and only cleans up in this case.
    Errornously treating a positive return value as an error from the
    vgic_v4_init() function can lead to NULL pointer dereferences, as has
    recently been observed.
    
    Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>

diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 40be908da238..62310122ee78 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -285,11 +285,9 @@ int vgic_init(struct kvm *kvm)
 	if (ret)
 		goto out;
 
-	if (vgic_supports_direct_msis(kvm)) {
-		ret = vgic_v4_init(kvm);
-		if (ret)
-			goto out;
-	}
+	ret = vgic_v4_init(kvm);
+	if (ret)
+		goto out;
 
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_enable(vcpu);
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index b8c1b724ba3e..c93ecd4a903b 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -1673,7 +1673,7 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
 
 	if (vgic_initialized(dev->kvm)) {
 		int ret = vgic_v4_init(dev->kvm);
-		if (ret) {
+		if (ret < 0) {
 			kfree(its);
 			return ret;
 		}
diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
index e367d65a0ebe..bb7e31fcee35 100644
--- a/virt/kvm/arm/vgic/vgic-v4.c
+++ b/virt/kvm/arm/vgic/vgic-v4.c
@@ -118,6 +118,9 @@ int vgic_v4_init(struct kvm *kvm)
 	struct kvm_vcpu *vcpu;
 	int i, nr_vcpus, ret;
 
+	if (!vgic_supports_direct_msis(kvm))
+		return 0; /* Nothing to see here... move along. */
+
 	if (dist->its_vm.vpes)
 		return 0;
 

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-11-07 14:42       ` Marc Zyngier
  (?)
@ 2017-11-10  8:28         ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:28 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

Hi Eric and Marc,

On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
> Hi Eric,
> 
> On 07/11/17 13:06, Auger Eric wrote:
> > Hi Marc,
> > 
> > On 27/10/2017 16:28, Marc Zyngier wrote:
> >> Let's use the irq bypass mechanism introduced for platform device
> >> interrupts
> > nit: I would remove "introduced for platform device interrupts"
> > as this is not upstream yet. x86 posted interrupts also use it.
> > 
> >>
> >  and establish our LPI->VLPI mapping.

I have tweaked the commit message.

> >>
> >> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >>  include/kvm/arm_vgic.h      |   8 ++++
> >>  virt/kvm/arm/arm.c          |   6 ++-
> >>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
> >>  3 files changed, 120 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index 7eeb6c2a2f9c..2f750c770bf2 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
> >>  
> >>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
> >>  
> >> +struct kvm_kernel_irq_routing_entry;
> >> +
> >> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
> >> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
> >> +
> >> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
> >> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
> >> +
> >>  #endif /* __KVM_ARM_VGIC_H */
> >> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> >> index 5d5218ecd547..8388c1cc23f6 100644
> >> --- a/virt/kvm/arm/arm.c
> >> +++ b/virt/kvm/arm/arm.c
> >> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> >>  	struct kvm_kernel_irqfd *irqfd =
> >>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
> >>  
> >> -	return 0;
> >> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
> >> +					  &irqfd->irq_entry);
> >>  }
> >>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> >>  				      struct irq_bypass_producer *prod)
> >> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> >>  	struct kvm_kernel_irqfd *irqfd =
> >>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
> >>  
> >> -	return;
> >> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
> >> +				     &irqfd->irq_entry);
> >>  }
> >>  
> >>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> >> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> >> index c794f0cef09e..01a2889b7b7c 100644
> >> --- a/virt/kvm/arm/vgic/vgic-v4.c
> >> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> >> @@ -18,6 +18,7 @@
> >>  #include <linux/interrupt.h>
> >>  #include <linux/irqdomain.h>
> >>  #include <linux/kvm_host.h>
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >>  
> >>  #include "vgic.h"
> >>  
> >> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
> >>  	its_vm->nr_vpes = 0;
> >>  	its_vm->vpes = NULL;
> >>  }
> >> +
> >> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
> >> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
> >> +{
> >> +	struct kvm_msi msi  = (struct kvm_msi) {
> >> +		.address_lo	= irq_entry->msi.address_lo,
> >> +		.address_hi	= irq_entry->msi.address_hi,
> >> +		.data		= irq_entry->msi.data,
> >> +		.flags		= irq_entry->msi.flags,
> >> +		.devid		= irq_entry->msi.devid,
> >> +	};
> >> +
> >> +	/*
> >> +	 * Get a reference on the LPI. If NULL, this is not a valid
> >> +	 * translation for any of our vITSs.
> >> +	 */
> > I don't understand the relevance of the above comment.
> 
> Hmmm. The first part looks like an outdated leftover, as the ITS is not
> refcounted, and we don't deal with LPIs here.
> 

I simply removed this comment.

[...]

I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
thing on its_map_vlpi() failures, which Marc can fix post -rc1.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-10  8:28         ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:28 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: kvm, Andre Przywara, linux-kernel, linux-arm-kernel, kvmarm

Hi Eric and Marc,

On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
> Hi Eric,
> 
> On 07/11/17 13:06, Auger Eric wrote:
> > Hi Marc,
> > 
> > On 27/10/2017 16:28, Marc Zyngier wrote:
> >> Let's use the irq bypass mechanism introduced for platform device
> >> interrupts
> > nit: I would remove "introduced for platform device interrupts"
> > as this is not upstream yet. x86 posted interrupts also use it.
> > 
> >>
> >  and establish our LPI->VLPI mapping.

I have tweaked the commit message.

> >>
> >> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >>  include/kvm/arm_vgic.h      |   8 ++++
> >>  virt/kvm/arm/arm.c          |   6 ++-
> >>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
> >>  3 files changed, 120 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index 7eeb6c2a2f9c..2f750c770bf2 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
> >>  
> >>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
> >>  
> >> +struct kvm_kernel_irq_routing_entry;
> >> +
> >> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
> >> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
> >> +
> >> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
> >> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
> >> +
> >>  #endif /* __KVM_ARM_VGIC_H */
> >> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> >> index 5d5218ecd547..8388c1cc23f6 100644
> >> --- a/virt/kvm/arm/arm.c
> >> +++ b/virt/kvm/arm/arm.c
> >> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> >>  	struct kvm_kernel_irqfd *irqfd =
> >>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
> >>  
> >> -	return 0;
> >> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
> >> +					  &irqfd->irq_entry);
> >>  }
> >>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> >>  				      struct irq_bypass_producer *prod)
> >> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> >>  	struct kvm_kernel_irqfd *irqfd =
> >>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
> >>  
> >> -	return;
> >> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
> >> +				     &irqfd->irq_entry);
> >>  }
> >>  
> >>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> >> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> >> index c794f0cef09e..01a2889b7b7c 100644
> >> --- a/virt/kvm/arm/vgic/vgic-v4.c
> >> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> >> @@ -18,6 +18,7 @@
> >>  #include <linux/interrupt.h>
> >>  #include <linux/irqdomain.h>
> >>  #include <linux/kvm_host.h>
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >>  
> >>  #include "vgic.h"
> >>  
> >> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
> >>  	its_vm->nr_vpes = 0;
> >>  	its_vm->vpes = NULL;
> >>  }
> >> +
> >> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
> >> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
> >> +{
> >> +	struct kvm_msi msi  = (struct kvm_msi) {
> >> +		.address_lo	= irq_entry->msi.address_lo,
> >> +		.address_hi	= irq_entry->msi.address_hi,
> >> +		.data		= irq_entry->msi.data,
> >> +		.flags		= irq_entry->msi.flags,
> >> +		.devid		= irq_entry->msi.devid,
> >> +	};
> >> +
> >> +	/*
> >> +	 * Get a reference on the LPI. If NULL, this is not a valid
> >> +	 * translation for any of our vITSs.
> >> +	 */
> > I don't understand the relevance of the above comment.
> 
> Hmmm. The first part looks like an outdated leftover, as the ITS is not
> refcounted, and we don't deal with LPIs here.
> 

I simply removed this comment.

[...]

I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
thing on its_map_vlpi() failures, which Marc can fix post -rc1.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-10  8:28         ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:28 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Eric and Marc,

On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
> Hi Eric,
> 
> On 07/11/17 13:06, Auger Eric wrote:
> > Hi Marc,
> > 
> > On 27/10/2017 16:28, Marc Zyngier wrote:
> >> Let's use the irq bypass mechanism introduced for platform device
> >> interrupts
> > nit: I would remove "introduced for platform device interrupts"
> > as this is not upstream yet. x86 posted interrupts also use it.
> > 
> >>
> >  and establish our LPI->VLPI mapping.

I have tweaked the commit message.

> >>
> >> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >>  include/kvm/arm_vgic.h      |   8 ++++
> >>  virt/kvm/arm/arm.c          |   6 ++-
> >>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
> >>  3 files changed, 120 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index 7eeb6c2a2f9c..2f750c770bf2 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
> >>  
> >>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
> >>  
> >> +struct kvm_kernel_irq_routing_entry;
> >> +
> >> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
> >> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
> >> +
> >> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
> >> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
> >> +
> >>  #endif /* __KVM_ARM_VGIC_H */
> >> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> >> index 5d5218ecd547..8388c1cc23f6 100644
> >> --- a/virt/kvm/arm/arm.c
> >> +++ b/virt/kvm/arm/arm.c
> >> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> >>  	struct kvm_kernel_irqfd *irqfd =
> >>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
> >>  
> >> -	return 0;
> >> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
> >> +					  &irqfd->irq_entry);
> >>  }
> >>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> >>  				      struct irq_bypass_producer *prod)
> >> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
> >>  	struct kvm_kernel_irqfd *irqfd =
> >>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
> >>  
> >> -	return;
> >> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
> >> +				     &irqfd->irq_entry);
> >>  }
> >>  
> >>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
> >> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> >> index c794f0cef09e..01a2889b7b7c 100644
> >> --- a/virt/kvm/arm/vgic/vgic-v4.c
> >> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> >> @@ -18,6 +18,7 @@
> >>  #include <linux/interrupt.h>
> >>  #include <linux/irqdomain.h>
> >>  #include <linux/kvm_host.h>
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >>  
> >>  #include "vgic.h"
> >>  
> >> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
> >>  	its_vm->nr_vpes = 0;
> >>  	its_vm->vpes = NULL;
> >>  }
> >> +
> >> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
> >> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
> >> +{
> >> +	struct kvm_msi msi  = (struct kvm_msi) {
> >> +		.address_lo	= irq_entry->msi.address_lo,
> >> +		.address_hi	= irq_entry->msi.address_hi,
> >> +		.data		= irq_entry->msi.data,
> >> +		.flags		= irq_entry->msi.flags,
> >> +		.devid		= irq_entry->msi.devid,
> >> +	};
> >> +
> >> +	/*
> >> +	 * Get a reference on the LPI. If NULL, this is not a valid
> >> +	 * translation for any of our vITSs.
> >> +	 */
> > I don't understand the relevance of the above comment.
> 
> Hmmm. The first part looks like an outdated leftover, as the ITS is not
> refcounted, and we don't deal with LPIs here.
> 

I simply removed this comment.

[...]

I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
thing on its_map_vlpi() failures, which Marc can fix post -rc1.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  2017-11-08 15:08       ` Marc Zyngier
  (?)
@ 2017-11-10  8:37         ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:37 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On Wed, Nov 08, 2017 at 03:08:36PM +0000, Marc Zyngier wrote:
> On 07/11/17 21:28, Auger Eric wrote:
> > Hi Marc,
> > 
> > On 27/10/2017 16:28, Marc Zyngier wrote:
> >> Upon updating a property, we propagate it all the way to the physical
> >> ITS, and ask for an INV command to be executed there.
> >>
> >> Acked-by: Christoffer Dall <cdall@linaro.org>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
> >>  1 file changed, 3 insertions(+)
> >>
> >> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> >> index 0b7e648e7a0c..2e77c7c83942 100644
> >> --- a/virt/kvm/arm/vgic/vgic-its.c
> >> +++ b/virt/kvm/arm/vgic/vgic-its.c
> >> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> >>  		spin_unlock(&irq->irq_lock);
> >>  	}
> >>  
> >> +	if (irq->hw)
> >> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
> >> +
> >>  	return 0;
> >>  }
> > I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
> > needed in hw mode?
> 
> It's not. I guess we could bypass this altogether and take a short cut
> after having updated the priority and enabled fields.
> 

I can apply this on top of the series as well if you're happy with it:

commit b54fba93b1330803a59ca75f3a5102e22cadc871 (HEAD -> next-gicv4)
Author: Christoffer Dall <christoffer.dall@linaro.org>
Date:   Fri Nov 10 09:34:54 2017 +0100

    KVM: arm/arm64: Don't queue VLPIs on INV/INVALL
    
    Since VLPIs are injected directly by the hardware there's no need to
    mark these as pending in software and queue them on the AP list.
    
    Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index c93ecd4a903b..a3754ec719c4 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -292,11 +292,14 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 		irq->priority = LPI_PROP_PRIORITY(prop);
 		irq->enabled = LPI_PROP_ENABLE_BIT(prop);
 
-		vgic_queue_irq_unlock(kvm, irq, flags);
-	} else {
-		spin_unlock_irqrestore(&irq->irq_lock, flags);
+		if (!irq->hw) {
+			vgic_queue_irq_unlock(kvm, irq, flags);
+			return 0;
+		}
 	}
 
+	spin_unlock_irqrestore(&irq->irq_lock, flags);
+
 	if (irq->hw)
 		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
 


Thanks,
-Christoffer

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-10  8:37         ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:37 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On Wed, Nov 08, 2017 at 03:08:36PM +0000, Marc Zyngier wrote:
> On 07/11/17 21:28, Auger Eric wrote:
> > Hi Marc,
> > 
> > On 27/10/2017 16:28, Marc Zyngier wrote:
> >> Upon updating a property, we propagate it all the way to the physical
> >> ITS, and ask for an INV command to be executed there.
> >>
> >> Acked-by: Christoffer Dall <cdall@linaro.org>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
> >>  1 file changed, 3 insertions(+)
> >>
> >> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> >> index 0b7e648e7a0c..2e77c7c83942 100644
> >> --- a/virt/kvm/arm/vgic/vgic-its.c
> >> +++ b/virt/kvm/arm/vgic/vgic-its.c
> >> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> >>  		spin_unlock(&irq->irq_lock);
> >>  	}
> >>  
> >> +	if (irq->hw)
> >> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
> >> +
> >>  	return 0;
> >>  }
> > I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
> > needed in hw mode?
> 
> It's not. I guess we could bypass this altogether and take a short cut
> after having updated the priority and enabled fields.
> 

I can apply this on top of the series as well if you're happy with it:

commit b54fba93b1330803a59ca75f3a5102e22cadc871 (HEAD -> next-gicv4)
Author: Christoffer Dall <christoffer.dall@linaro.org>
Date:   Fri Nov 10 09:34:54 2017 +0100

    KVM: arm/arm64: Don't queue VLPIs on INV/INVALL
    
    Since VLPIs are injected directly by the hardware there's no need to
    mark these as pending in software and queue them on the AP list.
    
    Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index c93ecd4a903b..a3754ec719c4 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -292,11 +292,14 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 		irq->priority = LPI_PROP_PRIORITY(prop);
 		irq->enabled = LPI_PROP_ENABLE_BIT(prop);
 
-		vgic_queue_irq_unlock(kvm, irq, flags);
-	} else {
-		spin_unlock_irqrestore(&irq->irq_lock, flags);
+		if (!irq->hw) {
+			vgic_queue_irq_unlock(kvm, irq, flags);
+			return 0;
+		}
 	}
 
+	spin_unlock_irqrestore(&irq->irq_lock, flags);
+
 	if (irq->hw)
 		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
 


Thanks,
-Christoffer

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-10  8:37         ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:37 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Nov 08, 2017 at 03:08:36PM +0000, Marc Zyngier wrote:
> On 07/11/17 21:28, Auger Eric wrote:
> > Hi Marc,
> > 
> > On 27/10/2017 16:28, Marc Zyngier wrote:
> >> Upon updating a property, we propagate it all the way to the physical
> >> ITS, and ask for an INV command to be executed there.
> >>
> >> Acked-by: Christoffer Dall <cdall@linaro.org>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
> >>  1 file changed, 3 insertions(+)
> >>
> >> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> >> index 0b7e648e7a0c..2e77c7c83942 100644
> >> --- a/virt/kvm/arm/vgic/vgic-its.c
> >> +++ b/virt/kvm/arm/vgic/vgic-its.c
> >> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
> >>  		spin_unlock(&irq->irq_lock);
> >>  	}
> >>  
> >> +	if (irq->hw)
> >> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
> >> +
> >>  	return 0;
> >>  }
> > I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
> > needed in hw mode?
> 
> It's not. I guess we could bypass this altogether and take a short cut
> after having updated the priority and enabled fields.
> 

I can apply this on top of the series as well if you're happy with it:

commit b54fba93b1330803a59ca75f3a5102e22cadc871 (HEAD -> next-gicv4)
Author: Christoffer Dall <christoffer.dall@linaro.org>
Date:   Fri Nov 10 09:34:54 2017 +0100

    KVM: arm/arm64: Don't queue VLPIs on INV/INVALL
    
    Since VLPIs are injected directly by the hardware there's no need to
    mark these as pending in software and queue them on the AP list.
    
    Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>

diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index c93ecd4a903b..a3754ec719c4 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -292,11 +292,14 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 		irq->priority = LPI_PROP_PRIORITY(prop);
 		irq->enabled = LPI_PROP_ENABLE_BIT(prop);
 
-		vgic_queue_irq_unlock(kvm, irq, flags);
-	} else {
-		spin_unlock_irqrestore(&irq->irq_lock, flags);
+		if (!irq->hw) {
+			vgic_queue_irq_unlock(kvm, irq, flags);
+			return 0;
+		}
 	}
 
+	spin_unlock_irqrestore(&irq->irq_lock, flags);
+
 	if (irq->hw)
 		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
 


Thanks,
-Christoffer

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
  2017-11-07 21:23     ` Auger Eric
  (?)
@ 2017-11-10  8:41       ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:41 UTC (permalink / raw)
  To: Auger Eric
  Cc: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On Tue, Nov 07, 2017 at 10:23:25PM +0100, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
> > Since when updating the properties one LPI at a time, there is no
> Since we update the properties one LPI at a time, ... ?
> > need to perform an INV each time we read one. Instead, we rely
> > on the final VINVALL that gets sent to the ITS to do the work.
> The commit message is not crystal clear for me.
> 
> I understand in case of vgic_its_cmd_handle_invall you want to avoid
> doing an invalidation for each physical irq but rather do an
> its_invall_vpe at the end. So you add a new  @needs_inv arg to
> update_lpi_config to tell whether the invalidation should be done or not.

I've reworded it to:

  There is no need to perform an INV for each interrupt when updating
  multiple interrupts.  Instead, we can rely on the final VINVALL that
  gets sent to the ITS to do the work for all of them.


Shout quickly if you have any objections.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-11-10  8:41       ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:41 UTC (permalink / raw)
  To: Auger Eric
  Cc: Marc Zyngier, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On Tue, Nov 07, 2017 at 10:23:25PM +0100, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
> > Since when updating the properties one LPI at a time, there is no
> Since we update the properties one LPI at a time, ... ?
> > need to perform an INV each time we read one. Instead, we rely
> > on the final VINVALL that gets sent to the ITS to do the work.
> The commit message is not crystal clear for me.
> 
> I understand in case of vgic_its_cmd_handle_invall you want to avoid
> doing an invalidation for each physical irq but rather do an
> its_invall_vpe at the end. So you add a new  @needs_inv arg to
> update_lpi_config to tell whether the invalidation should be done or not.

I've reworded it to:

  There is no need to perform an INV for each interrupt when updating
  multiple interrupts.  Instead, we can rely on the final VINVALL that
  gets sent to the ITS to do the work for all of them.


Shout quickly if you have any objections.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-11-10  8:41       ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  8:41 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Nov 07, 2017 at 10:23:25PM +0100, Auger Eric wrote:
> Hi Marc,
> 
> On 27/10/2017 16:28, Marc Zyngier wrote:
> > Since when updating the properties one LPI at a time, there is no
> Since we update the properties one LPI at a time, ... ?
> > need to perform an INV each time we read one. Instead, we rely
> > on the final VINVALL that gets sent to the ITS to do the work.
> The commit message is not crystal clear for me.
> 
> I understand in case of vgic_its_cmd_handle_invall you want to avoid
> doing an invalidation for each physical irq but rather do an
> its_invall_vpe at the end. So you add a new  @needs_inv arg to
> update_lpi_config to tell whether the invalidation should be done or not.

I've reworded it to:

  There is no need to perform an INV for each interrupt when updating
  multiple interrupts.  Instead, we can rely on the final VINVALL that
  gets sent to the ITS to do the work for all of them.


Shout quickly if you have any objections.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
  2017-11-10  8:20       ` Christoffer Dall
  (?)
@ 2017-11-10  8:55         ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:55 UTC (permalink / raw)
  To: Christoffer Dall, Auger Eric
  Cc: linux-arm-kernel, kvmarm, kvm, linux-kernel, Christoffer Dall,
	Shanker Donthineni, Mark Rutland, Shameerali Kolothum Thodi,
	Andre Przywara

On 10/11/17 08:20, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 02:08:23PM +0100, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> In order to control the GICv4 view of virtual CPUs, we rely
>>> on an irqdomain allocated for that purpose. Let's add a couple
>>> of helpers to that effect.
>>>
>>> At the same time, the vgic data structures gain new fields to
>>> track all this... erm... wonderful stuff.
>>>
>>> The way we hook into the vgic init is slightly convoluted. We
>>> need the vgic to be initialized (in order to guarantee that
>>> the number of vcpus is now fixed), and we must have a vITS
>>> (otherwise this is all very pointless). So we end-up calling
>>> the init from both vgic_init and vgic_its_create.
>>>
>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  arch/arm/kvm/Makefile         |  1 +
>>>  arch/arm64/kvm/Makefile       |  1 +
>>>  include/kvm/arm_vgic.h        | 19 ++++++++++
>>>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>>>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>>>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>>>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>>>  7 files changed, 123 insertions(+)
>>>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
>>>
>>> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
>>> index d9beee652d36..0a1dd2cdb928 100644
>>> --- a/arch/arm/kvm/Makefile
>>> +++ b/arch/arm/kvm/Makefile
>>> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
>>> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>>> index 5d9810086c25..c30fd388ef80 100644
>>> --- a/arch/arm64/kvm/Makefile
>>> +++ b/arch/arm64/kvm/Makefile
>>> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
>>> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>> index ba9fb450aa1b..7eeb6c2a2f9c 100644
>>> --- a/include/kvm/arm_vgic.h
>>> +++ b/include/kvm/arm_vgic.h
>>> @@ -26,6 +26,8 @@
>>>  #include <linux/list.h>
>>>  #include <linux/jump_label.h>
>>>  
>>> +#include <linux/irqchip/arm-gic-v4.h>
>>> +
>>>  #define VGIC_V3_MAX_CPUS	255
>>>  #define VGIC_V2_MAX_CPUS	8
>>>  #define VGIC_NR_IRQS_LEGACY     256
>>> @@ -236,6 +238,15 @@ struct vgic_dist {
>>>  
>>>  	/* used by vgic-debug */
>>>  	struct vgic_state_iter *iter;
>>> +
>>> +	/*
>>> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
>>> +	 * array, the property table pointer as well as allocation
>>> +	 * data. This essentially ties the Linux IRQ core and ITS
>>> +	 * together, and avoids leaking KVM's data structures anywhere
>>> +	 * else.
>>> +	 */
>>> +	struct its_vm		its_vm;
>>>  };
>>>  
>>>  struct vgic_v2_cpu_if {
>>> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>>>  	u32		vgic_ap0r[4];
>>>  	u32		vgic_ap1r[4];
>>>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
>>> +
>>> +	/*
>>> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
>>> +	 * pending table pointer, the its_vm pointer and a few other
>>> +	 * HW specific things. As for the its_vm structure, this is
>>> +	 * linking the Linux IRQ subsystem and the ITS together.
>>> +	 */
>>> +	struct its_vpe	its_vpe;
>>>  };
>>>  
>>>  struct vgic_cpu {
>>> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
>>> index 5801261f3add..40be908da238 100644
>>> --- a/virt/kvm/arm/vgic/vgic-init.c
>>> +++ b/virt/kvm/arm/vgic/vgic-init.c
>>> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>>>  	if (ret)
>>>  		goto out;
>>>  
>>> +	if (vgic_supports_direct_msis(kvm)) {
>>> +		ret = vgic_v4_init(kvm);
>>> +		if (ret)
>>> +			goto out;
>>> +	}
>>> +
>>>  	kvm_for_each_vcpu(i, vcpu, kvm)
>>>  		kvm_vgic_vcpu_enable(vcpu);
>>>  
>>> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>>>  
>>>  	kfree(dist->spis);
>>>  	dist->nr_spis = 0;
>>> +
>>> +	if (vgic_supports_direct_msis(kvm))
>>> +		vgic_v4_teardown(kvm);
>>>  }
>>>  
>>>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index 8ee03f1e89fc..89768d2b6a91 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>>>  	if (!its)
>>>  		return -ENOMEM;
>>>  
>>> +	if (vgic_initialized(dev->kvm)) {
>> Don't we need to test vgic_supports_direct_msis() on this path too?
>>
> 
> Seems to me that we should, otherwise creating an ITS after the VGIC has
> been initialized would fail on non-GICv4 compatible systems, right?
> 
> How about this patch as a follow-up to the series:
> 
> commit 48ec1662d0f10d6468907cdc7e12c46ca1ef497c (HEAD -> next-gicv4)
> Author: Christoffer Dall <christoffer.dall@linaro.org>
> Date:   Fri Nov 10 09:16:23 2017 +0100
> 
>     KVM: arm/arm64: Fix GICv4 ITS initialization issues
>     
>     We should only try to initialize GICv4 data structures on a GICv4
>     capable system.  Move the vgic_supports_direct_msis() check inito
>     vgic_v4_init() so that any KVM VGIC initialization path does not fail
>     on non-GICv4 systems.
>     
>     Also be slightly more strict in the checking of the return value in
>     vgic_its_create, and only error out on negative return values from the
>     vgic_v4_init() function.  This is important because the kvm device code
>     only treats negative values as errors and only cleans up in this case.
>     Errornously treating a positive return value as an error from the
>     vgic_v4_init() function can lead to NULL pointer dereferences, as has
>     recently been observed.
>     
>     Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 40be908da238..62310122ee78 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,11 +285,9 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> -	if (vgic_supports_direct_msis(kvm)) {
> -		ret = vgic_v4_init(kvm);
> -		if (ret)
> -			goto out;
> -	}
> +	ret = vgic_v4_init(kvm);
> +	if (ret)
> +		goto out;
>  
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index b8c1b724ba3e..c93ecd4a903b 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1673,7 +1673,7 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  
>  	if (vgic_initialized(dev->kvm)) {
>  		int ret = vgic_v4_init(dev->kvm);
> -		if (ret) {
> +		if (ret < 0) {
>  			kfree(its);
>  			return ret;
>  		}
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index e367d65a0ebe..bb7e31fcee35 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -118,6 +118,9 @@ int vgic_v4_init(struct kvm *kvm)
>  	struct kvm_vcpu *vcpu;
>  	int i, nr_vcpus, ret;
>  
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0; /* Nothing to see here... move along. */
> +
>  	if (dist->its_vm.vpes)
>  		return 0;
>  
> 

Yup, this seems like the right thing to do.

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-10  8:55         ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:55 UTC (permalink / raw)
  To: Christoffer Dall, Auger Eric
  Cc: linux-arm-kernel, kvmarm, kvm, linux-kernel, Christoffer Dall,
	Shanker Donthineni, Mark Rutland, Shameerali Kolothum Thodi,
	Andre Przywara

On 10/11/17 08:20, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 02:08:23PM +0100, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> In order to control the GICv4 view of virtual CPUs, we rely
>>> on an irqdomain allocated for that purpose. Let's add a couple
>>> of helpers to that effect.
>>>
>>> At the same time, the vgic data structures gain new fields to
>>> track all this... erm... wonderful stuff.
>>>
>>> The way we hook into the vgic init is slightly convoluted. We
>>> need the vgic to be initialized (in order to guarantee that
>>> the number of vcpus is now fixed), and we must have a vITS
>>> (otherwise this is all very pointless). So we end-up calling
>>> the init from both vgic_init and vgic_its_create.
>>>
>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  arch/arm/kvm/Makefile         |  1 +
>>>  arch/arm64/kvm/Makefile       |  1 +
>>>  include/kvm/arm_vgic.h        | 19 ++++++++++
>>>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>>>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>>>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>>>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>>>  7 files changed, 123 insertions(+)
>>>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
>>>
>>> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
>>> index d9beee652d36..0a1dd2cdb928 100644
>>> --- a/arch/arm/kvm/Makefile
>>> +++ b/arch/arm/kvm/Makefile
>>> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
>>> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>>> index 5d9810086c25..c30fd388ef80 100644
>>> --- a/arch/arm64/kvm/Makefile
>>> +++ b/arch/arm64/kvm/Makefile
>>> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
>>> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>> index ba9fb450aa1b..7eeb6c2a2f9c 100644
>>> --- a/include/kvm/arm_vgic.h
>>> +++ b/include/kvm/arm_vgic.h
>>> @@ -26,6 +26,8 @@
>>>  #include <linux/list.h>
>>>  #include <linux/jump_label.h>
>>>  
>>> +#include <linux/irqchip/arm-gic-v4.h>
>>> +
>>>  #define VGIC_V3_MAX_CPUS	255
>>>  #define VGIC_V2_MAX_CPUS	8
>>>  #define VGIC_NR_IRQS_LEGACY     256
>>> @@ -236,6 +238,15 @@ struct vgic_dist {
>>>  
>>>  	/* used by vgic-debug */
>>>  	struct vgic_state_iter *iter;
>>> +
>>> +	/*
>>> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
>>> +	 * array, the property table pointer as well as allocation
>>> +	 * data. This essentially ties the Linux IRQ core and ITS
>>> +	 * together, and avoids leaking KVM's data structures anywhere
>>> +	 * else.
>>> +	 */
>>> +	struct its_vm		its_vm;
>>>  };
>>>  
>>>  struct vgic_v2_cpu_if {
>>> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>>>  	u32		vgic_ap0r[4];
>>>  	u32		vgic_ap1r[4];
>>>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
>>> +
>>> +	/*
>>> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
>>> +	 * pending table pointer, the its_vm pointer and a few other
>>> +	 * HW specific things. As for the its_vm structure, this is
>>> +	 * linking the Linux IRQ subsystem and the ITS together.
>>> +	 */
>>> +	struct its_vpe	its_vpe;
>>>  };
>>>  
>>>  struct vgic_cpu {
>>> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
>>> index 5801261f3add..40be908da238 100644
>>> --- a/virt/kvm/arm/vgic/vgic-init.c
>>> +++ b/virt/kvm/arm/vgic/vgic-init.c
>>> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>>>  	if (ret)
>>>  		goto out;
>>>  
>>> +	if (vgic_supports_direct_msis(kvm)) {
>>> +		ret = vgic_v4_init(kvm);
>>> +		if (ret)
>>> +			goto out;
>>> +	}
>>> +
>>>  	kvm_for_each_vcpu(i, vcpu, kvm)
>>>  		kvm_vgic_vcpu_enable(vcpu);
>>>  
>>> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>>>  
>>>  	kfree(dist->spis);
>>>  	dist->nr_spis = 0;
>>> +
>>> +	if (vgic_supports_direct_msis(kvm))
>>> +		vgic_v4_teardown(kvm);
>>>  }
>>>  
>>>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index 8ee03f1e89fc..89768d2b6a91 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>>>  	if (!its)
>>>  		return -ENOMEM;
>>>  
>>> +	if (vgic_initialized(dev->kvm)) {
>> Don't we need to test vgic_supports_direct_msis() on this path too?
>>
> 
> Seems to me that we should, otherwise creating an ITS after the VGIC has
> been initialized would fail on non-GICv4 compatible systems, right?
> 
> How about this patch as a follow-up to the series:
> 
> commit 48ec1662d0f10d6468907cdc7e12c46ca1ef497c (HEAD -> next-gicv4)
> Author: Christoffer Dall <christoffer.dall@linaro.org>
> Date:   Fri Nov 10 09:16:23 2017 +0100
> 
>     KVM: arm/arm64: Fix GICv4 ITS initialization issues
>     
>     We should only try to initialize GICv4 data structures on a GICv4
>     capable system.  Move the vgic_supports_direct_msis() check inito
>     vgic_v4_init() so that any KVM VGIC initialization path does not fail
>     on non-GICv4 systems.
>     
>     Also be slightly more strict in the checking of the return value in
>     vgic_its_create, and only error out on negative return values from the
>     vgic_v4_init() function.  This is important because the kvm device code
>     only treats negative values as errors and only cleans up in this case.
>     Errornously treating a positive return value as an error from the
>     vgic_v4_init() function can lead to NULL pointer dereferences, as has
>     recently been observed.
>     
>     Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 40be908da238..62310122ee78 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,11 +285,9 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> -	if (vgic_supports_direct_msis(kvm)) {
> -		ret = vgic_v4_init(kvm);
> -		if (ret)
> -			goto out;
> -	}
> +	ret = vgic_v4_init(kvm);
> +	if (ret)
> +		goto out;
>  
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index b8c1b724ba3e..c93ecd4a903b 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1673,7 +1673,7 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  
>  	if (vgic_initialized(dev->kvm)) {
>  		int ret = vgic_v4_init(dev->kvm);
> -		if (ret) {
> +		if (ret < 0) {
>  			kfree(its);
>  			return ret;
>  		}
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index e367d65a0ebe..bb7e31fcee35 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -118,6 +118,9 @@ int vgic_v4_init(struct kvm *kvm)
>  	struct kvm_vcpu *vcpu;
>  	int i, nr_vcpus, ret;
>  
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0; /* Nothing to see here... move along. */
> +
>  	if (dist->its_vm.vpes)
>  		return 0;
>  
> 

Yup, this seems like the right thing to do.

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain
@ 2017-11-10  8:55         ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:55 UTC (permalink / raw)
  To: linux-arm-kernel

On 10/11/17 08:20, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 02:08:23PM +0100, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> In order to control the GICv4 view of virtual CPUs, we rely
>>> on an irqdomain allocated for that purpose. Let's add a couple
>>> of helpers to that effect.
>>>
>>> At the same time, the vgic data structures gain new fields to
>>> track all this... erm... wonderful stuff.
>>>
>>> The way we hook into the vgic init is slightly convoluted. We
>>> need the vgic to be initialized (in order to guarantee that
>>> the number of vcpus is now fixed), and we must have a vITS
>>> (otherwise this is all very pointless). So we end-up calling
>>> the init from both vgic_init and vgic_its_create.
>>>
>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>>  arch/arm/kvm/Makefile         |  1 +
>>>  arch/arm64/kvm/Makefile       |  1 +
>>>  include/kvm/arm_vgic.h        | 19 ++++++++++
>>>  virt/kvm/arm/vgic/vgic-init.c |  9 +++++
>>>  virt/kvm/arm/vgic/vgic-its.c  |  8 +++++
>>>  virt/kvm/arm/vgic/vgic-v4.c   | 83 +++++++++++++++++++++++++++++++++++++++++++
>>>  virt/kvm/arm/vgic/vgic.h      |  2 ++
>>>  7 files changed, 123 insertions(+)
>>>  create mode 100644 virt/kvm/arm/vgic/vgic-v4.c
>>>
>>> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
>>> index d9beee652d36..0a1dd2cdb928 100644
>>> --- a/arch/arm/kvm/Makefile
>>> +++ b/arch/arm/kvm/Makefile
>>> @@ -31,6 +31,7 @@ obj-y += $(KVM)/arm/vgic/vgic-init.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-v3.o
>>> +obj-y += $(KVM)/arm/vgic/vgic-v4.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>>> index 5d9810086c25..c30fd388ef80 100644
>>> --- a/arch/arm64/kvm/Makefile
>>> +++ b/arch/arm64/kvm/Makefile
>>> @@ -26,6 +26,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-init.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-irqfd.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v2.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v3.o
>>> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-v4.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o
>>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>> index ba9fb450aa1b..7eeb6c2a2f9c 100644
>>> --- a/include/kvm/arm_vgic.h
>>> +++ b/include/kvm/arm_vgic.h
>>> @@ -26,6 +26,8 @@
>>>  #include <linux/list.h>
>>>  #include <linux/jump_label.h>
>>>  
>>> +#include <linux/irqchip/arm-gic-v4.h>
>>> +
>>>  #define VGIC_V3_MAX_CPUS	255
>>>  #define VGIC_V2_MAX_CPUS	8
>>>  #define VGIC_NR_IRQS_LEGACY     256
>>> @@ -236,6 +238,15 @@ struct vgic_dist {
>>>  
>>>  	/* used by vgic-debug */
>>>  	struct vgic_state_iter *iter;
>>> +
>>> +	/*
>>> +	 * GICv4 ITS per-VM data, containing the IRQ domain, the VPE
>>> +	 * array, the property table pointer as well as allocation
>>> +	 * data. This essentially ties the Linux IRQ core and ITS
>>> +	 * together, and avoids leaking KVM's data structures anywhere
>>> +	 * else.
>>> +	 */
>>> +	struct its_vm		its_vm;
>>>  };
>>>  
>>>  struct vgic_v2_cpu_if {
>>> @@ -254,6 +265,14 @@ struct vgic_v3_cpu_if {
>>>  	u32		vgic_ap0r[4];
>>>  	u32		vgic_ap1r[4];
>>>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
>>> +
>>> +	/*
>>> +	 * GICv4 ITS per-VPE data, containing the doorbell IRQ, the
>>> +	 * pending table pointer, the its_vm pointer and a few other
>>> +	 * HW specific things. As for the its_vm structure, this is
>>> +	 * linking the Linux IRQ subsystem and the ITS together.
>>> +	 */
>>> +	struct its_vpe	its_vpe;
>>>  };
>>>  
>>>  struct vgic_cpu {
>>> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
>>> index 5801261f3add..40be908da238 100644
>>> --- a/virt/kvm/arm/vgic/vgic-init.c
>>> +++ b/virt/kvm/arm/vgic/vgic-init.c
>>> @@ -285,6 +285,12 @@ int vgic_init(struct kvm *kvm)
>>>  	if (ret)
>>>  		goto out;
>>>  
>>> +	if (vgic_supports_direct_msis(kvm)) {
>>> +		ret = vgic_v4_init(kvm);
>>> +		if (ret)
>>> +			goto out;
>>> +	}
>>> +
>>>  	kvm_for_each_vcpu(i, vcpu, kvm)
>>>  		kvm_vgic_vcpu_enable(vcpu);
>>>  
>>> @@ -320,6 +326,9 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
>>>  
>>>  	kfree(dist->spis);
>>>  	dist->nr_spis = 0;
>>> +
>>> +	if (vgic_supports_direct_msis(kvm))
>>> +		vgic_v4_teardown(kvm);
>>>  }
>>>  
>>>  void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>> index 8ee03f1e89fc..89768d2b6a91 100644
>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>> @@ -1603,6 +1603,14 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>>>  	if (!its)
>>>  		return -ENOMEM;
>>>  
>>> +	if (vgic_initialized(dev->kvm)) {
>> Don't we need to test vgic_supports_direct_msis() on this path too?
>>
> 
> Seems to me that we should, otherwise creating an ITS after the VGIC has
> been initialized would fail on non-GICv4 compatible systems, right?
> 
> How about this patch as a follow-up to the series:
> 
> commit 48ec1662d0f10d6468907cdc7e12c46ca1ef497c (HEAD -> next-gicv4)
> Author: Christoffer Dall <christoffer.dall@linaro.org>
> Date:   Fri Nov 10 09:16:23 2017 +0100
> 
>     KVM: arm/arm64: Fix GICv4 ITS initialization issues
>     
>     We should only try to initialize GICv4 data structures on a GICv4
>     capable system.  Move the vgic_supports_direct_msis() check inito
>     vgic_v4_init() so that any KVM VGIC initialization path does not fail
>     on non-GICv4 systems.
>     
>     Also be slightly more strict in the checking of the return value in
>     vgic_its_create, and only error out on negative return values from the
>     vgic_v4_init() function.  This is important because the kvm device code
>     only treats negative values as errors and only cleans up in this case.
>     Errornously treating a positive return value as an error from the
>     vgic_v4_init() function can lead to NULL pointer dereferences, as has
>     recently been observed.
>     
>     Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 40be908da238..62310122ee78 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -285,11 +285,9 @@ int vgic_init(struct kvm *kvm)
>  	if (ret)
>  		goto out;
>  
> -	if (vgic_supports_direct_msis(kvm)) {
> -		ret = vgic_v4_init(kvm);
> -		if (ret)
> -			goto out;
> -	}
> +	ret = vgic_v4_init(kvm);
> +	if (ret)
> +		goto out;
>  
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_enable(vcpu);
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index b8c1b724ba3e..c93ecd4a903b 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -1673,7 +1673,7 @@ static int vgic_its_create(struct kvm_device *dev, u32 type)
>  
>  	if (vgic_initialized(dev->kvm)) {
>  		int ret = vgic_v4_init(dev->kvm);
> -		if (ret) {
> +		if (ret < 0) {
>  			kfree(its);
>  			return ret;
>  		}
> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
> index e367d65a0ebe..bb7e31fcee35 100644
> --- a/virt/kvm/arm/vgic/vgic-v4.c
> +++ b/virt/kvm/arm/vgic/vgic-v4.c
> @@ -118,6 +118,9 @@ int vgic_v4_init(struct kvm *kvm)
>  	struct kvm_vcpu *vcpu;
>  	int i, nr_vcpus, ret;
>  
> +	if (!vgic_supports_direct_msis(kvm))
> +		return 0; /* Nothing to see here... move along. */
> +
>  	if (dist->its_vm.vpes)
>  		return 0;
>  
> 

Yup, this seems like the right thing to do.

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
  2017-11-10  8:41       ` Christoffer Dall
  (?)
@ 2017-11-10  8:56         ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:56 UTC (permalink / raw)
  To: Christoffer Dall, Auger Eric
  Cc: linux-arm-kernel, kvmarm, kvm, linux-kernel, Mark Rutland,
	Andre Przywara, Shameerali Kolothum Thodi, Christoffer Dall,
	Shanker Donthineni

On 10/11/17 08:41, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 10:23:25PM +0100, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> Since when updating the properties one LPI at a time, there is no
>> Since we update the properties one LPI at a time, ... ?
>>> need to perform an INV each time we read one. Instead, we rely
>>> on the final VINVALL that gets sent to the ITS to do the work.
>> The commit message is not crystal clear for me.
>>
>> I understand in case of vgic_its_cmd_handle_invall you want to avoid
>> doing an invalidation for each physical irq but rather do an
>> its_invall_vpe at the end. So you add a new  @needs_inv arg to
>> update_lpi_config to tell whether the invalidation should be done or not.
> 
> I've reworded it to:
> 
>   There is no need to perform an INV for each interrupt when updating
>   multiple interrupts.  Instead, we can rely on the final VINVALL that
>   gets sent to the ITS to do the work for all of them.
> 
> 
> Shout quickly if you have any objections.

Works for me.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-11-10  8:56         ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:56 UTC (permalink / raw)
  To: Christoffer Dall, Auger Eric
  Cc: kvm, Andre Przywara, linux-kernel, kvmarm, linux-arm-kernel

On 10/11/17 08:41, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 10:23:25PM +0100, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> Since when updating the properties one LPI at a time, there is no
>> Since we update the properties one LPI at a time, ... ?
>>> need to perform an INV each time we read one. Instead, we rely
>>> on the final VINVALL that gets sent to the ITS to do the work.
>> The commit message is not crystal clear for me.
>>
>> I understand in case of vgic_its_cmd_handle_invall you want to avoid
>> doing an invalidation for each physical irq but rather do an
>> its_invall_vpe at the end. So you add a new  @needs_inv arg to
>> update_lpi_config to tell whether the invalidation should be done or not.
> 
> I've reworded it to:
> 
>   There is no need to perform an INV for each interrupt when updating
>   multiple interrupts.  Instead, we can rely on the final VINVALL that
>   gets sent to the ITS to do the work for all of them.
> 
> 
> Shout quickly if you have any objections.

Works for me.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE
@ 2017-11-10  8:56         ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:56 UTC (permalink / raw)
  To: linux-arm-kernel

On 10/11/17 08:41, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 10:23:25PM +0100, Auger Eric wrote:
>> Hi Marc,
>>
>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>> Since when updating the properties one LPI at a time, there is no
>> Since we update the properties one LPI at a time, ... ?
>>> need to perform an INV each time we read one. Instead, we rely
>>> on the final VINVALL that gets sent to the ITS to do the work.
>> The commit message is not crystal clear for me.
>>
>> I understand in case of vgic_its_cmd_handle_invall you want to avoid
>> doing an invalidation for each physical irq but rather do an
>> its_invall_vpe at the end. So you add a new  @needs_inv arg to
>> update_lpi_config to tell whether the invalidation should be done or not.
> 
> I've reworded it to:
> 
>   There is no need to perform an INV for each interrupt when updating
>   multiple interrupts.  Instead, we can rely on the final VINVALL that
>   gets sent to the ITS to do the work for all of them.
> 
> 
> Shout quickly if you have any objections.

Works for me.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
  2017-11-10  8:37         ` Christoffer Dall
  (?)
@ 2017-11-10  8:58           ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:58 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Mark Rutland, Andre Przywara, Shameerali Kolothum Thodi,
	Christoffer Dall, Shanker Donthineni

On 10/11/17 08:37, Christoffer Dall wrote:
> On Wed, Nov 08, 2017 at 03:08:36PM +0000, Marc Zyngier wrote:
>> On 07/11/17 21:28, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Upon updating a property, we propagate it all the way to the physical
>>>> ITS, and ask for an INV command to be executed there.
>>>>
>>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>>>>  1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>>> index 0b7e648e7a0c..2e77c7c83942 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>>> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>>>>  		spin_unlock(&irq->irq_lock);
>>>>  	}
>>>>  
>>>> +	if (irq->hw)
>>>> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
>>>> +
>>>>  	return 0;
>>>>  }
>>> I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
>>> needed in hw mode?
>>
>> It's not. I guess we could bypass this altogether and take a short cut
>> after having updated the priority and enabled fields.
>>
> 
> I can apply this on top of the series as well if you're happy with it:
> 
> commit b54fba93b1330803a59ca75f3a5102e22cadc871 (HEAD -> next-gicv4)
> Author: Christoffer Dall <christoffer.dall@linaro.org>
> Date:   Fri Nov 10 09:34:54 2017 +0100
> 
>     KVM: arm/arm64: Don't queue VLPIs on INV/INVALL
>     
>     Since VLPIs are injected directly by the hardware there's no need to
>     mark these as pending in software and queue them on the AP list.
>     
>     Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index c93ecd4a903b..a3754ec719c4 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -292,11 +292,14 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  		irq->priority = LPI_PROP_PRIORITY(prop);
>  		irq->enabled = LPI_PROP_ENABLE_BIT(prop);
>  
> -		vgic_queue_irq_unlock(kvm, irq, flags);
> -	} else {
> -		spin_unlock_irqrestore(&irq->irq_lock, flags);
> +		if (!irq->hw) {
> +			vgic_queue_irq_unlock(kvm, irq, flags);
> +			return 0;
> +		}
>  	}
>  
> +	spin_unlock_irqrestore(&irq->irq_lock, flags);
> +
>  	if (irq->hw)
>  		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
>  

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-10  8:58           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:58 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: kvm, Andre Przywara, linux-kernel, kvmarm, linux-arm-kernel

On 10/11/17 08:37, Christoffer Dall wrote:
> On Wed, Nov 08, 2017 at 03:08:36PM +0000, Marc Zyngier wrote:
>> On 07/11/17 21:28, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Upon updating a property, we propagate it all the way to the physical
>>>> ITS, and ask for an INV command to be executed there.
>>>>
>>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>>>>  1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>>> index 0b7e648e7a0c..2e77c7c83942 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>>> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>>>>  		spin_unlock(&irq->irq_lock);
>>>>  	}
>>>>  
>>>> +	if (irq->hw)
>>>> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
>>>> +
>>>>  	return 0;
>>>>  }
>>> I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
>>> needed in hw mode?
>>
>> It's not. I guess we could bypass this altogether and take a short cut
>> after having updated the priority and enabled fields.
>>
> 
> I can apply this on top of the series as well if you're happy with it:
> 
> commit b54fba93b1330803a59ca75f3a5102e22cadc871 (HEAD -> next-gicv4)
> Author: Christoffer Dall <christoffer.dall@linaro.org>
> Date:   Fri Nov 10 09:34:54 2017 +0100
> 
>     KVM: arm/arm64: Don't queue VLPIs on INV/INVALL
>     
>     Since VLPIs are injected directly by the hardware there's no need to
>     mark these as pending in software and queue them on the AP list.
>     
>     Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index c93ecd4a903b..a3754ec719c4 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -292,11 +292,14 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  		irq->priority = LPI_PROP_PRIORITY(prop);
>  		irq->enabled = LPI_PROP_ENABLE_BIT(prop);
>  
> -		vgic_queue_irq_unlock(kvm, irq, flags);
> -	} else {
> -		spin_unlock_irqrestore(&irq->irq_lock, flags);
> +		if (!irq->hw) {
> +			vgic_queue_irq_unlock(kvm, irq, flags);
> +			return 0;
> +		}
>  	}
>  
> +	spin_unlock_irqrestore(&irq->irq_lock, flags);
> +
>  	if (irq->hw)
>  		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
>  

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs
@ 2017-11-10  8:58           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  8:58 UTC (permalink / raw)
  To: linux-arm-kernel

On 10/11/17 08:37, Christoffer Dall wrote:
> On Wed, Nov 08, 2017 at 03:08:36PM +0000, Marc Zyngier wrote:
>> On 07/11/17 21:28, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Upon updating a property, we propagate it all the way to the physical
>>>> ITS, and ask for an INV command to be executed there.
>>>>
>>>> Acked-by: Christoffer Dall <cdall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  virt/kvm/arm/vgic/vgic-its.c | 3 +++
>>>>  1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
>>>> index 0b7e648e7a0c..2e77c7c83942 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-its.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-its.c
>>>> @@ -296,6 +296,9 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>>>>  		spin_unlock(&irq->irq_lock);
>>>>  	}
>>>>  
>>>> +	if (irq->hw)
>>>> +		return its_prop_update_vlpi(irq->host_irq, prop, true);
>>>> +
>>>>  	return 0;
>>>>  }
>>> I am confused by the vgic_queue_irq_unlock() on the "hw" path. Why is it
>>> needed in hw mode?
>>
>> It's not. I guess we could bypass this altogether and take a short cut
>> after having updated the priority and enabled fields.
>>
> 
> I can apply this on top of the series as well if you're happy with it:
> 
> commit b54fba93b1330803a59ca75f3a5102e22cadc871 (HEAD -> next-gicv4)
> Author: Christoffer Dall <christoffer.dall@linaro.org>
> Date:   Fri Nov 10 09:34:54 2017 +0100
> 
>     KVM: arm/arm64: Don't queue VLPIs on INV/INVALL
>     
>     Since VLPIs are injected directly by the hardware there's no need to
>     mark these as pending in software and queue them on the AP list.
>     
>     Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
> index c93ecd4a903b..a3754ec719c4 100644
> --- a/virt/kvm/arm/vgic/vgic-its.c
> +++ b/virt/kvm/arm/vgic/vgic-its.c
> @@ -292,11 +292,14 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
>  		irq->priority = LPI_PROP_PRIORITY(prop);
>  		irq->enabled = LPI_PROP_ENABLE_BIT(prop);
>  
> -		vgic_queue_irq_unlock(kvm, irq, flags);
> -	} else {
> -		spin_unlock_irqrestore(&irq->irq_lock, flags);
> +		if (!irq->hw) {
> +			vgic_queue_irq_unlock(kvm, irq, flags);
> +			return 0;
> +		}
>  	}
>  
> +	spin_unlock_irqrestore(&irq->irq_lock, flags);
> +
>  	if (irq->hw)
>  		return its_prop_update_vlpi(irq->host_irq, prop, needs_inv);
>  

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-11-10  8:28         ` Christoffer Dall
  (?)
@ 2017-11-10  9:05           ` Marc Zyngier
  -1 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  9:05 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Christoffer Dall, Shanker Donthineni, Mark Rutland,
	Shameerali Kolothum Thodi, Andre Przywara

On 10/11/17 08:28, Christoffer Dall wrote:
> Hi Eric and Marc,
> 
> On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
>> Hi Eric,
>>
>> On 07/11/17 13:06, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Let's use the irq bypass mechanism introduced for platform device
>>>> interrupts
>>> nit: I would remove "introduced for platform device interrupts"
>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>
>>>>
>>>  and establish our LPI->VLPI mapping.
> 
> I have tweaked the commit message.
> 
>>>>
>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>> --- a/include/kvm/arm_vgic.h
>>>> +++ b/include/kvm/arm_vgic.h
>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>  
>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>  
>>>> +struct kvm_kernel_irq_routing_entry;
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>> --- a/virt/kvm/arm/arm.c
>>>> +++ b/virt/kvm/arm/arm.c
>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return 0;
>>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>> +					  &irqfd->irq_entry);
>>>>  }
>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  				      struct irq_bypass_producer *prod)
>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return;
>>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>> +				     &irqfd->irq_entry);
>>>>  }
>>>>  
>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>> @@ -18,6 +18,7 @@
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/irqdomain.h>
>>>>  #include <linux/kvm_host.h>
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>  
>>>>  #include "vgic.h"
>>>>  
>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>  	its_vm->nr_vpes = 0;
>>>>  	its_vm->vpes = NULL;
>>>>  }
>>>> +
>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>>> +		.address_lo	= irq_entry->msi.address_lo,
>>>> +		.address_hi	= irq_entry->msi.address_hi,
>>>> +		.data		= irq_entry->msi.data,
>>>> +		.flags		= irq_entry->msi.flags,
>>>> +		.devid		= irq_entry->msi.devid,
>>>> +	};
>>>> +
>>>> +	/*
>>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>>> +	 * translation for any of our vITSs.
>>>> +	 */
>>> I don't understand the relevance of the above comment.
>>
>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>> refcounted, and we don't deal with LPIs here.
>>
> 
> I simply removed this comment.
> 
> [...]
> 
> I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
> thing on its_map_vlpi() failures, which Marc can fix post -rc1.

Here's what I've queued on the irqchip side:

>From 9c0733704b99c89773416af3a412332b0e8bd2a4 Mon Sep 17 00:00:00 2001
From: Marc Zyngier <marc.zyngier@arm.com>
Date: Fri, 10 Nov 2017 09:00:41 +0000
Subject: [PATCH] irqchip/gic-v4: Clear IRQ_DISABLE_UNLAZY again if mapping
 fails

Should the call to irq_set_vcpu_affinity() fail at map time,
we should restore the normal lazy-disable behaviour instead
of staying with the eager disable that GICv4 requires.

Reported-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v4.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
index cd0bcc3b7e33..dba9d67cb9c1 100644
--- a/drivers/irqchip/irq-gic-v4.c
+++ b/drivers/irqchip/irq-gic-v4.c
@@ -177,6 +177,7 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
 			.map      = map,
 		},
 	};
+	int ret;
 
 	/*
 	 * The host will never see that interrupt firing again, so it
@@ -184,7 +185,11 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
 	 */
 	irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
 
-	return irq_set_vcpu_affinity(irq, &info);
+	ret = irq_set_vcpu_affinity(irq, &info);
+	if (ret)
+		irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
+
+	return ret;
 }
 
 int its_get_vlpi(int irq, struct its_vlpi_map *map)
-- 
2.14.2


Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-10  9:05           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  9:05 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: kvm, Andre Przywara, linux-kernel, linux-arm-kernel, kvmarm

On 10/11/17 08:28, Christoffer Dall wrote:
> Hi Eric and Marc,
> 
> On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
>> Hi Eric,
>>
>> On 07/11/17 13:06, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Let's use the irq bypass mechanism introduced for platform device
>>>> interrupts
>>> nit: I would remove "introduced for platform device interrupts"
>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>
>>>>
>>>  and establish our LPI->VLPI mapping.
> 
> I have tweaked the commit message.
> 
>>>>
>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>> --- a/include/kvm/arm_vgic.h
>>>> +++ b/include/kvm/arm_vgic.h
>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>  
>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>  
>>>> +struct kvm_kernel_irq_routing_entry;
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>> --- a/virt/kvm/arm/arm.c
>>>> +++ b/virt/kvm/arm/arm.c
>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return 0;
>>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>> +					  &irqfd->irq_entry);
>>>>  }
>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  				      struct irq_bypass_producer *prod)
>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return;
>>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>> +				     &irqfd->irq_entry);
>>>>  }
>>>>  
>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>> @@ -18,6 +18,7 @@
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/irqdomain.h>
>>>>  #include <linux/kvm_host.h>
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>  
>>>>  #include "vgic.h"
>>>>  
>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>  	its_vm->nr_vpes = 0;
>>>>  	its_vm->vpes = NULL;
>>>>  }
>>>> +
>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>>> +		.address_lo	= irq_entry->msi.address_lo,
>>>> +		.address_hi	= irq_entry->msi.address_hi,
>>>> +		.data		= irq_entry->msi.data,
>>>> +		.flags		= irq_entry->msi.flags,
>>>> +		.devid		= irq_entry->msi.devid,
>>>> +	};
>>>> +
>>>> +	/*
>>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>>> +	 * translation for any of our vITSs.
>>>> +	 */
>>> I don't understand the relevance of the above comment.
>>
>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>> refcounted, and we don't deal with LPIs here.
>>
> 
> I simply removed this comment.
> 
> [...]
> 
> I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
> thing on its_map_vlpi() failures, which Marc can fix post -rc1.

Here's what I've queued on the irqchip side:

>From 9c0733704b99c89773416af3a412332b0e8bd2a4 Mon Sep 17 00:00:00 2001
From: Marc Zyngier <marc.zyngier@arm.com>
Date: Fri, 10 Nov 2017 09:00:41 +0000
Subject: [PATCH] irqchip/gic-v4: Clear IRQ_DISABLE_UNLAZY again if mapping
 fails

Should the call to irq_set_vcpu_affinity() fail at map time,
we should restore the normal lazy-disable behaviour instead
of staying with the eager disable that GICv4 requires.

Reported-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v4.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
index cd0bcc3b7e33..dba9d67cb9c1 100644
--- a/drivers/irqchip/irq-gic-v4.c
+++ b/drivers/irqchip/irq-gic-v4.c
@@ -177,6 +177,7 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
 			.map      = map,
 		},
 	};
+	int ret;
 
 	/*
 	 * The host will never see that interrupt firing again, so it
@@ -184,7 +185,11 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
 	 */
 	irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
 
-	return irq_set_vcpu_affinity(irq, &info);
+	ret = irq_set_vcpu_affinity(irq, &info);
+	if (ret)
+		irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
+
+	return ret;
 }
 
 int its_get_vlpi(int irq, struct its_vlpi_map *map)
-- 
2.14.2


Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-10  9:05           ` Marc Zyngier
  0 siblings, 0 replies; 234+ messages in thread
From: Marc Zyngier @ 2017-11-10  9:05 UTC (permalink / raw)
  To: linux-arm-kernel

On 10/11/17 08:28, Christoffer Dall wrote:
> Hi Eric and Marc,
> 
> On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
>> Hi Eric,
>>
>> On 07/11/17 13:06, Auger Eric wrote:
>>> Hi Marc,
>>>
>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>> Let's use the irq bypass mechanism introduced for platform device
>>>> interrupts
>>> nit: I would remove "introduced for platform device interrupts"
>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>
>>>>
>>>  and establish our LPI->VLPI mapping.
> 
> I have tweaked the commit message.
> 
>>>>
>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>> --- a/include/kvm/arm_vgic.h
>>>> +++ b/include/kvm/arm_vgic.h
>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>  
>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>  
>>>> +struct kvm_kernel_irq_routing_entry;
>>>> +
>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>> +			       struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>> +				 struct kvm_kernel_irq_routing_entry *irq_entry);
>>>> +
>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>> --- a/virt/kvm/arm/arm.c
>>>> +++ b/virt/kvm/arm/arm.c
>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return 0;
>>>> +	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>> +					  &irqfd->irq_entry);
>>>>  }
>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  				      struct irq_bypass_producer *prod)
>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>  	struct kvm_kernel_irqfd *irqfd =
>>>>  		container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>  
>>>> -	return;
>>>> +	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>> +				     &irqfd->irq_entry);
>>>>  }
>>>>  
>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>> @@ -18,6 +18,7 @@
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/irqdomain.h>
>>>>  #include <linux/kvm_host.h>
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>  
>>>>  #include "vgic.h"
>>>>  
>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>  	its_vm->nr_vpes = 0;
>>>>  	its_vm->vpes = NULL;
>>>>  }
>>>> +
>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>> +				     struct kvm_kernel_irq_routing_entry *irq_entry)
>>>> +{
>>>> +	struct kvm_msi msi  = (struct kvm_msi) {
>>>> +		.address_lo	= irq_entry->msi.address_lo,
>>>> +		.address_hi	= irq_entry->msi.address_hi,
>>>> +		.data		= irq_entry->msi.data,
>>>> +		.flags		= irq_entry->msi.flags,
>>>> +		.devid		= irq_entry->msi.devid,
>>>> +	};
>>>> +
>>>> +	/*
>>>> +	 * Get a reference on the LPI. If NULL, this is not a valid
>>>> +	 * translation for any of our vITSs.
>>>> +	 */
>>> I don't understand the relevance of the above comment.
>>
>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>> refcounted, and we don't deal with LPIs here.
>>
> 
> I simply removed this comment.
> 
> [...]
> 
> I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
> thing on its_map_vlpi() failures, which Marc can fix post -rc1.

Here's what I've queued on the irqchip side:

>From 9c0733704b99c89773416af3a412332b0e8bd2a4 Mon Sep 17 00:00:00 2001
From: Marc Zyngier <marc.zyngier@arm.com>
Date: Fri, 10 Nov 2017 09:00:41 +0000
Subject: [PATCH] irqchip/gic-v4: Clear IRQ_DISABLE_UNLAZY again if mapping
 fails

Should the call to irq_set_vcpu_affinity() fail at map time,
we should restore the normal lazy-disable behaviour instead
of staying with the eager disable that GICv4 requires.

Reported-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v4.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
index cd0bcc3b7e33..dba9d67cb9c1 100644
--- a/drivers/irqchip/irq-gic-v4.c
+++ b/drivers/irqchip/irq-gic-v4.c
@@ -177,6 +177,7 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
 			.map      = map,
 		},
 	};
+	int ret;
 
 	/*
 	 * The host will never see that interrupt firing again, so it
@@ -184,7 +185,11 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
 	 */
 	irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
 
-	return irq_set_vcpu_affinity(irq, &info);
+	ret = irq_set_vcpu_affinity(irq, &info);
+	if (ret)
+		irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
+
+	return ret;
 }
 
 int its_get_vlpi(int irq, struct its_vlpi_map *map)
-- 
2.14.2


Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply related	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
  2017-11-10  9:05           ` Marc Zyngier
  (?)
@ 2017-11-10  9:41             ` Christoffer Dall
  -1 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  9:41 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Auger Eric, linux-arm-kernel, kvmarm, kvm, linux-kernel,
	Shanker Donthineni, Mark Rutland, Shameerali Kolothum Thodi,
	Andre Przywara

On Fri, Nov 10, 2017 at 10:05 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 10/11/17 08:28, Christoffer Dall wrote:
>> Hi Eric and Marc,
>>
>> On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
>>> Hi Eric,
>>>
>>> On 07/11/17 13:06, Auger Eric wrote:
>>>> Hi Marc,
>>>>
>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>> Let's use the irq bypass mechanism introduced for platform device
>>>>> interrupts
>>>> nit: I would remove "introduced for platform device interrupts"
>>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>>
>>>>>
>>>>  and establish our LPI->VLPI mapping.
>>
>> I have tweaked the commit message.
>>
>>>>>
>>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>>> ---
>>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>>> --- a/include/kvm/arm_vgic.h
>>>>> +++ b/include/kvm/arm_vgic.h
>>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>>
>>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>>
>>>>> +struct kvm_kernel_irq_routing_entry;
>>>>> +
>>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>>> +                         struct kvm_kernel_irq_routing_entry *irq_entry);
>>>>> +
>>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>>> +                           struct kvm_kernel_irq_routing_entry *irq_entry);
>>>>> +
>>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>>> --- a/virt/kvm/arm/arm.c
>>>>> +++ b/virt/kvm/arm/arm.c
>>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>>    struct kvm_kernel_irqfd *irqfd =
>>>>>            container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>>
>>>>> -  return 0;
>>>>> +  return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>>> +                                    &irqfd->irq_entry);
>>>>>  }
>>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>>                                  struct irq_bypass_producer *prod)
>>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>>    struct kvm_kernel_irqfd *irqfd =
>>>>>            container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>>
>>>>> -  return;
>>>>> +  kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>>> +                               &irqfd->irq_entry);
>>>>>  }
>>>>>
>>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>>> @@ -18,6 +18,7 @@
>>>>>  #include <linux/interrupt.h>
>>>>>  #include <linux/irqdomain.h>
>>>>>  #include <linux/kvm_host.h>
>>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>>
>>>>>  #include "vgic.h"
>>>>>
>>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>>    its_vm->nr_vpes = 0;
>>>>>    its_vm->vpes = NULL;
>>>>>  }
>>>>> +
>>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>>> +                               struct kvm_kernel_irq_routing_entry *irq_entry)
>>>>> +{
>>>>> +  struct kvm_msi msi  = (struct kvm_msi) {
>>>>> +          .address_lo     = irq_entry->msi.address_lo,
>>>>> +          .address_hi     = irq_entry->msi.address_hi,
>>>>> +          .data           = irq_entry->msi.data,
>>>>> +          .flags          = irq_entry->msi.flags,
>>>>> +          .devid          = irq_entry->msi.devid,
>>>>> +  };
>>>>> +
>>>>> +  /*
>>>>> +   * Get a reference on the LPI. If NULL, this is not a valid
>>>>> +   * translation for any of our vITSs.
>>>>> +   */
>>>> I don't understand the relevance of the above comment.
>>>
>>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>>> refcounted, and we don't deal with LPIs here.
>>>
>>
>> I simply removed this comment.
>>
>> [...]
>>
>> I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
>> thing on its_map_vlpi() failures, which Marc can fix post -rc1.
>
> Here's what I've queued on the irqchip side:
>
> From 9c0733704b99c89773416af3a412332b0e8bd2a4 Mon Sep 17 00:00:00 2001
> From: Marc Zyngier <marc.zyngier@arm.com>
> Date: Fri, 10 Nov 2017 09:00:41 +0000
> Subject: [PATCH] irqchip/gic-v4: Clear IRQ_DISABLE_UNLAZY again if mapping
>  fails
>
> Should the call to irq_set_vcpu_affinity() fail at map time,
> we should restore the normal lazy-disable behaviour instead
> of staying with the eager disable that GICv4 requires.
>
> Reported-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/irqchip/irq-gic-v4.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
> index cd0bcc3b7e33..dba9d67cb9c1 100644
> --- a/drivers/irqchip/irq-gic-v4.c
> +++ b/drivers/irqchip/irq-gic-v4.c
> @@ -177,6 +177,7 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
>                         .map      = map,
>                 },
>         };
> +       int ret;
>
>         /*
>          * The host will never see that interrupt firing again, so it
> @@ -184,7 +185,11 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
>          */
>         irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
>
> -       return irq_set_vcpu_affinity(irq, &info);
> +       ret = irq_set_vcpu_affinity(irq, &info);
> +       if (ret)
> +               irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
> +
> +       return ret;
>  }
>
>  int its_get_vlpi(int irq, struct its_vlpi_map *map)
> --
> 2.14.2
>
>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* Re: [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-10  9:41             ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  9:41 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: kvm, Andre Przywara, linux-kernel, linux-arm-kernel, kvmarm

On Fri, Nov 10, 2017 at 10:05 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 10/11/17 08:28, Christoffer Dall wrote:
>> Hi Eric and Marc,
>>
>> On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
>>> Hi Eric,
>>>
>>> On 07/11/17 13:06, Auger Eric wrote:
>>>> Hi Marc,
>>>>
>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>> Let's use the irq bypass mechanism introduced for platform device
>>>>> interrupts
>>>> nit: I would remove "introduced for platform device interrupts"
>>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>>
>>>>>
>>>>  and establish our LPI->VLPI mapping.
>>
>> I have tweaked the commit message.
>>
>>>>>
>>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>>> ---
>>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>>> --- a/include/kvm/arm_vgic.h
>>>>> +++ b/include/kvm/arm_vgic.h
>>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>>
>>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>>
>>>>> +struct kvm_kernel_irq_routing_entry;
>>>>> +
>>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>>> +                         struct kvm_kernel_irq_routing_entry *irq_entry);
>>>>> +
>>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>>> +                           struct kvm_kernel_irq_routing_entry *irq_entry);
>>>>> +
>>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>>> --- a/virt/kvm/arm/arm.c
>>>>> +++ b/virt/kvm/arm/arm.c
>>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>>    struct kvm_kernel_irqfd *irqfd =
>>>>>            container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>>
>>>>> -  return 0;
>>>>> +  return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>>> +                                    &irqfd->irq_entry);
>>>>>  }
>>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>>                                  struct irq_bypass_producer *prod)
>>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>>    struct kvm_kernel_irqfd *irqfd =
>>>>>            container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>>
>>>>> -  return;
>>>>> +  kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>>> +                               &irqfd->irq_entry);
>>>>>  }
>>>>>
>>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>>> @@ -18,6 +18,7 @@
>>>>>  #include <linux/interrupt.h>
>>>>>  #include <linux/irqdomain.h>
>>>>>  #include <linux/kvm_host.h>
>>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>>
>>>>>  #include "vgic.h"
>>>>>
>>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>>    its_vm->nr_vpes = 0;
>>>>>    its_vm->vpes = NULL;
>>>>>  }
>>>>> +
>>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>>> +                               struct kvm_kernel_irq_routing_entry *irq_entry)
>>>>> +{
>>>>> +  struct kvm_msi msi  = (struct kvm_msi) {
>>>>> +          .address_lo     = irq_entry->msi.address_lo,
>>>>> +          .address_hi     = irq_entry->msi.address_hi,
>>>>> +          .data           = irq_entry->msi.data,
>>>>> +          .flags          = irq_entry->msi.flags,
>>>>> +          .devid          = irq_entry->msi.devid,
>>>>> +  };
>>>>> +
>>>>> +  /*
>>>>> +   * Get a reference on the LPI. If NULL, this is not a valid
>>>>> +   * translation for any of our vITSs.
>>>>> +   */
>>>> I don't understand the relevance of the above comment.
>>>
>>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>>> refcounted, and we don't deal with LPIs here.
>>>
>>
>> I simply removed this comment.
>>
>> [...]
>>
>> I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
>> thing on its_map_vlpi() failures, which Marc can fix post -rc1.
>
> Here's what I've queued on the irqchip side:
>
> From 9c0733704b99c89773416af3a412332b0e8bd2a4 Mon Sep 17 00:00:00 2001
> From: Marc Zyngier <marc.zyngier@arm.com>
> Date: Fri, 10 Nov 2017 09:00:41 +0000
> Subject: [PATCH] irqchip/gic-v4: Clear IRQ_DISABLE_UNLAZY again if mapping
>  fails
>
> Should the call to irq_set_vcpu_affinity() fail at map time,
> we should restore the normal lazy-disable behaviour instead
> of staying with the eager disable that GICv4 requires.
>
> Reported-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/irqchip/irq-gic-v4.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
> index cd0bcc3b7e33..dba9d67cb9c1 100644
> --- a/drivers/irqchip/irq-gic-v4.c
> +++ b/drivers/irqchip/irq-gic-v4.c
> @@ -177,6 +177,7 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
>                         .map      = map,
>                 },
>         };
> +       int ret;
>
>         /*
>          * The host will never see that interrupt firing again, so it
> @@ -184,7 +185,11 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
>          */
>         irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
>
> -       return irq_set_vcpu_affinity(irq, &info);
> +       ret = irq_set_vcpu_affinity(irq, &info);
> +       if (ret)
> +               irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
> +
> +       return ret;
>  }
>
>  int its_get_vlpi(int irq, struct its_vlpi_map *map)
> --
> 2.14.2
>
>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 234+ messages in thread
* [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass
@ 2017-11-10  9:41             ` Christoffer Dall
  0 siblings, 0 replies; 234+ messages in thread
From: Christoffer Dall @ 2017-11-10  9:41 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Nov 10, 2017 at 10:05 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 10/11/17 08:28, Christoffer Dall wrote:
>> Hi Eric and Marc,
>>
>> On Tue, Nov 07, 2017 at 02:42:44PM +0000, Marc Zyngier wrote:
>>> Hi Eric,
>>>
>>> On 07/11/17 13:06, Auger Eric wrote:
>>>> Hi Marc,
>>>>
>>>> On 27/10/2017 16:28, Marc Zyngier wrote:
>>>>> Let's use the irq bypass mechanism introduced for platform device
>>>>> interrupts
>>>> nit: I would remove "introduced for platform device interrupts"
>>>> as this is not upstream yet. x86 posted interrupts also use it.
>>>>
>>>>>
>>>>  and establish our LPI->VLPI mapping.
>>
>> I have tweaked the commit message.
>>
>>>>>
>>>>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>>> ---
>>>>>  include/kvm/arm_vgic.h      |   8 ++++
>>>>>  virt/kvm/arm/arm.c          |   6 ++-
>>>>>  virt/kvm/arm/vgic/vgic-v4.c | 108 ++++++++++++++++++++++++++++++++++++++++++++
>>>>>  3 files changed, 120 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>>> index 7eeb6c2a2f9c..2f750c770bf2 100644
>>>>> --- a/include/kvm/arm_vgic.h
>>>>> +++ b/include/kvm/arm_vgic.h
>>>>> @@ -373,4 +373,12 @@ int kvm_vgic_setup_default_irq_routing(struct kvm *kvm);
>>>>>
>>>>>  int kvm_vgic_set_owner(struct kvm_vcpu *vcpu, unsigned int intid, void *owner);
>>>>>
>>>>> +struct kvm_kernel_irq_routing_entry;
>>>>> +
>>>>> +int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
>>>>> +                         struct kvm_kernel_irq_routing_entry *irq_entry);
>>>>> +
>>>>> +int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
>>>>> +                           struct kvm_kernel_irq_routing_entry *irq_entry);
>>>>> +
>>>>>  #endif /* __KVM_ARM_VGIC_H */
>>>>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>>>>> index 5d5218ecd547..8388c1cc23f6 100644
>>>>> --- a/virt/kvm/arm/arm.c
>>>>> +++ b/virt/kvm/arm/arm.c
>>>>> @@ -1462,7 +1462,8 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
>>>>>    struct kvm_kernel_irqfd *irqfd =
>>>>>            container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>>
>>>>> -  return 0;
>>>>> +  return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
>>>>> +                                    &irqfd->irq_entry);
>>>>>  }
>>>>>  void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>>                                  struct irq_bypass_producer *prod)
>>>>> @@ -1470,7 +1471,8 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
>>>>>    struct kvm_kernel_irqfd *irqfd =
>>>>>            container_of(cons, struct kvm_kernel_irqfd, consumer);
>>>>>
>>>>> -  return;
>>>>> +  kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq,
>>>>> +                               &irqfd->irq_entry);
>>>>>  }
>>>>>
>>>>>  void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
>>>>> diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c
>>>>> index c794f0cef09e..01a2889b7b7c 100644
>>>>> --- a/virt/kvm/arm/vgic/vgic-v4.c
>>>>> +++ b/virt/kvm/arm/vgic/vgic-v4.c
>>>>> @@ -18,6 +18,7 @@
>>>>>  #include <linux/interrupt.h>
>>>>>  #include <linux/irqdomain.h>
>>>>>  #include <linux/kvm_host.h>
>>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>>
>>>>>  #include "vgic.h"
>>>>>
>>>>> @@ -81,3 +82,110 @@ void vgic_v4_teardown(struct kvm *kvm)
>>>>>    its_vm->nr_vpes = 0;
>>>>>    its_vm->vpes = NULL;
>>>>>  }
>>>>> +
>>>>> +static struct vgic_its *vgic_get_its(struct kvm *kvm,
>>>>> +                               struct kvm_kernel_irq_routing_entry *irq_entry)
>>>>> +{
>>>>> +  struct kvm_msi msi  = (struct kvm_msi) {
>>>>> +          .address_lo     = irq_entry->msi.address_lo,
>>>>> +          .address_hi     = irq_entry->msi.address_hi,
>>>>> +          .data           = irq_entry->msi.data,
>>>>> +          .flags          = irq_entry->msi.flags,
>>>>> +          .devid          = irq_entry->msi.devid,
>>>>> +  };
>>>>> +
>>>>> +  /*
>>>>> +   * Get a reference on the LPI. If NULL, this is not a valid
>>>>> +   * translation for any of our vITSs.
>>>>> +   */
>>>> I don't understand the relevance of the above comment.
>>>
>>> Hmmm. The first part looks like an outdated leftover, as the ITS is not
>>> refcounted, and we don't deal with LPIs here.
>>>
>>
>> I simply removed this comment.
>>
>> [...]
>>
>> I think the only thing left to fix on this patch is the IRQ_DISABLE_LAZY
>> thing on its_map_vlpi() failures, which Marc can fix post -rc1.
>
> Here's what I've queued on the irqchip side:
>
> From 9c0733704b99c89773416af3a412332b0e8bd2a4 Mon Sep 17 00:00:00 2001
> From: Marc Zyngier <marc.zyngier@arm.com>
> Date: Fri, 10 Nov 2017 09:00:41 +0000
> Subject: [PATCH] irqchip/gic-v4: Clear IRQ_DISABLE_UNLAZY again if mapping
>  fails
>
> Should the call to irq_set_vcpu_affinity() fail at map time,
> we should restore the normal lazy-disable behaviour instead
> of staying with the eager disable that GICv4 requires.
>
> Reported-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/irqchip/irq-gic-v4.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
> index cd0bcc3b7e33..dba9d67cb9c1 100644
> --- a/drivers/irqchip/irq-gic-v4.c
> +++ b/drivers/irqchip/irq-gic-v4.c
> @@ -177,6 +177,7 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
>                         .map      = map,
>                 },
>         };
> +       int ret;
>
>         /*
>          * The host will never see that interrupt firing again, so it
> @@ -184,7 +185,11 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map)
>          */
>         irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
>
> -       return irq_set_vcpu_affinity(irq, &info);
> +       ret = irq_set_vcpu_affinity(irq, &info);
> +       if (ret)
> +               irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
> +
> +       return ret;
>  }
>
>  int its_get_vlpi(int irq, struct its_vlpi_map *map)
> --
> 2.14.2
>
>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 234+ messages in thread
end of thread, other threads:[~2017-11-10  9:41 UTC | newest]

Thread overview: 234+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-10-27 14:28 [PATCH v5 00/26] KVM/ARM: Add support for GICv4 Marc Zyngier
2017-10-27 14:28 ` Marc Zyngier
2017-10-27 14:28 ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 01/26] irqchip/gic-v3-its: Setup VLPI properties at map time Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-30  6:46   ` Christoffer Dall
2017-10-30  6:46     ` Christoffer Dall
2017-10-30  6:46     ` Christoffer Dall
2017-10-27 14:28 ` [PATCH v5 02/26] KVM: arm/arm64: register irq bypass consumer on ARM/ARM64 Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-30  6:47   ` Christoffer Dall
2017-10-30  6:47     ` Christoffer Dall
2017-10-30  6:47     ` Christoffer Dall
2017-10-27 14:28 ` [PATCH v5 03/26] KVM: arm/arm64: vgic: restructure kvm_vgic_(un)map_phys_irq Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 04/26] KVM: arm: Select ARM_GIC_V3 and ARM_GIC_V3_ITS Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 05/26] KVM: arm/arm64: vgic: Move kvm_vgic_destroy call around Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 06/26] KVM: arm/arm64: vITS: Add MSI translation helpers Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 20:34   ` Auger Eric
2017-11-07 20:34     ` Auger Eric
2017-11-07 20:34     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 07/26] KVM: arm/arm64: vITS: Add a helper to update the affinity of an LPI Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 13:44   ` Auger Eric
2017-11-07 13:44     ` Auger Eric
2017-11-07 13:44     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 08/26] KVM: arm/arm64: GICv4: Add property field and per-VM predicate Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 20:30   ` Auger Eric
2017-11-07 20:30     ` Auger Eric
2017-11-07 20:30     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 09/26] KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 13:08   ` Auger Eric
2017-11-07 13:08     ` Auger Eric
2017-11-07 13:08     ` Auger Eric
2017-11-10  8:20     ` Christoffer Dall
2017-11-10  8:20       ` Christoffer Dall
2017-11-10  8:20       ` Christoffer Dall
2017-11-10  8:55       ` Marc Zyngier
2017-11-10  8:55         ` Marc Zyngier
2017-11-10  8:55         ` Marc Zyngier
2017-11-07 13:09   ` Auger Eric
2017-11-07 13:09     ` Auger Eric
2017-11-07 13:09     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 10/26] KVM: arm/arm64: GICv4: Wire mapping/unmapping of VLPIs in VFIO irq bypass Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 13:06   ` Auger Eric
2017-11-07 13:06     ` Auger Eric
2017-11-07 13:06     ` Auger Eric
2017-11-07 14:42     ` Marc Zyngier
2017-11-07 14:42       ` Marc Zyngier
2017-11-07 14:42       ` Marc Zyngier
2017-11-07 15:59       ` Auger Eric
2017-11-07 15:59         ` Auger Eric
2017-11-07 15:59         ` Auger Eric
2017-11-08 11:30         ` Marc Zyngier
2017-11-08 11:30           ` Marc Zyngier
2017-11-08 11:30           ` Marc Zyngier
2017-11-10  8:28       ` Christoffer Dall
2017-11-10  8:28         ` Christoffer Dall
2017-11-10  8:28         ` Christoffer Dall
2017-11-10  9:05         ` Marc Zyngier
2017-11-10  9:05           ` Marc Zyngier
2017-11-10  9:05           ` Marc Zyngier
2017-11-10  9:41           ` Christoffer Dall
2017-11-10  9:41             ` Christoffer Dall
2017-11-10  9:41             ` Christoffer Dall
2017-10-27 14:28 ` [PATCH v5 11/26] KVM: arm/arm64: GICv4: Handle INT command applied to a VLPI Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 20:15   ` Auger Eric
2017-11-07 20:15     ` Auger Eric
2017-11-07 20:15     ` Auger Eric
2017-11-08 11:40     ` Marc Zyngier
2017-11-08 11:40       ` Marc Zyngier
2017-11-08 11:40       ` Marc Zyngier
2017-11-08 14:14       ` Auger Eric
2017-11-08 14:14         ` Auger Eric
2017-11-08 14:14         ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 12/26] KVM: arm/arm64: GICv4: Unmap VLPI when freeing an LPI Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 20:28   ` Auger Eric
2017-11-07 20:28     ` Auger Eric
2017-11-07 20:28     ` Auger Eric
2017-11-08 11:52     ` Marc Zyngier
2017-11-08 11:52       ` Marc Zyngier
2017-11-08 11:52       ` Marc Zyngier
2017-11-08 14:14       ` Auger Eric
2017-11-08 14:14         ` Auger Eric
2017-11-08 14:14         ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 13/26] KVM: arm/arm64: GICv4: Propagate affinity changes to the physical ITS Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:01   ` Auger Eric
2017-11-07 21:01     ` Auger Eric
2017-11-07 21:01     ` Auger Eric
2017-11-08 12:05     ` Marc Zyngier
2017-11-08 12:05       ` Marc Zyngier
2017-11-08 12:05       ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 14/26] KVM: arm/arm64: GICv4: Handle CLEAR applied to a VLPI Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:04   ` Auger Eric
2017-11-07 21:04     ` Auger Eric
2017-11-07 21:04     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 15/26] KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:06   ` Auger Eric
2017-11-07 21:06     ` Auger Eric
2017-11-07 21:06     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 16/26] KVM: arm/arm64: GICv4: Propagate property updates to VLPIs Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:28   ` Auger Eric
2017-11-07 21:28     ` Auger Eric
2017-11-07 21:28     ` Auger Eric
2017-11-08 15:08     ` Marc Zyngier
2017-11-08 15:08       ` Marc Zyngier
2017-11-08 15:08       ` Marc Zyngier
2017-11-10  8:37       ` Christoffer Dall
2017-11-10  8:37         ` Christoffer Dall
2017-11-10  8:37         ` Christoffer Dall
2017-11-10  8:58         ` Marc Zyngier
2017-11-10  8:58           ` Marc Zyngier
2017-11-10  8:58           ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 17/26] KVM: arm/arm64: GICv4: Handle INVALL applied to a vPE Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:23   ` Auger Eric
2017-11-07 21:23     ` Auger Eric
2017-11-07 21:23     ` Auger Eric
2017-11-10  8:41     ` Christoffer Dall
2017-11-10  8:41       ` Christoffer Dall
2017-11-10  8:41       ` Christoffer Dall
2017-11-10  8:56       ` Marc Zyngier
2017-11-10  8:56         ` Marc Zyngier
2017-11-10  8:56         ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 18/26] KVM: arm/arm64: GICv4: Use pending_last as a scheduling hint Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:38   ` Auger Eric
2017-11-07 21:38     ` Auger Eric
2017-11-07 21:38     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 19/26] KVM: arm/arm64: GICv4: Add doorbell interrupt handling Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:43   ` Auger Eric
2017-11-07 21:43     ` Auger Eric
2017-11-07 21:43     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 20/26] KVM: arm/arm64: GICv4: Use the doorbell interrupt as an unblocking source Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:45   ` Auger Eric
2017-11-07 21:45     ` Auger Eric
2017-11-07 21:45     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 21/26] KVM: arm/arm64: GICv4: Hook vPE scheduling into vgic flush/sync Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 21:54   ` Auger Eric
2017-11-07 21:54     ` Auger Eric
2017-11-07 21:54     ` Auger Eric
2017-11-07 22:14     ` Auger Eric
2017-11-07 22:14       ` Auger Eric
2017-11-07 22:14       ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 22/26] KVM: arm/arm64: GICv4: Enable virtual cpuif if VLPIs can be delivered Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-08  8:46   ` Auger Eric
2017-11-08  8:46     ` Auger Eric
2017-11-08  8:46     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 23/26] KVM: arm/arm64: GICv4: Prevent a VM using GICv4 from being saved Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-07 15:24   ` Auger Eric
2017-11-07 15:24     ` Auger Eric
2017-11-07 15:24     ` Auger Eric
2017-11-07 15:38     ` Marc Zyngier
2017-11-07 15:38       ` Marc Zyngier
2017-11-07 15:38       ` Marc Zyngier
2017-11-07 16:12       ` Auger Eric
2017-11-07 16:12         ` Auger Eric
2017-11-07 16:12         ` Auger Eric
2017-11-07 16:34         ` Marc Zyngier
2017-11-07 16:34           ` Marc Zyngier
2017-11-07 16:34           ` Marc Zyngier
2017-11-07 22:24           ` Auger Eric
2017-11-07 22:24             ` Auger Eric
2017-11-07 22:24             ` Auger Eric
2017-11-08  9:35             ` Auger Eric
2017-11-08  9:35               ` Auger Eric
2017-11-08  9:35               ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 24/26] KVM: arm/arm64: GICv4: Prevent userspace from changing doorbell affinity Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-30  6:51   ` Christoffer Dall
2017-10-30  6:51     ` Christoffer Dall
2017-10-30  6:51     ` Christoffer Dall
2017-11-07 22:17   ` Auger Eric
2017-11-07 22:17     ` Auger Eric
2017-11-07 22:17     ` Auger Eric
2017-10-27 14:28 ` [PATCH v5 25/26] KVM: arm/arm64: GICv4: Enable VLPI support Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-08  8:44   ` Auger Eric
2017-11-08  8:44     ` Auger Eric
2017-11-08  8:44     ` Auger Eric
2017-11-08 15:14     ` Marc Zyngier
2017-11-08 15:14       ` Marc Zyngier
2017-11-08 15:14       ` Marc Zyngier
2017-10-27 14:28 ` [PATCH v5 26/26] KVM: arm/arm64: GICv4: Theory of operations Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-10-27 14:28   ` Marc Zyngier
2017-11-08  9:13   ` Auger Eric
2017-11-08  9:13     ` Auger Eric
2017-11-08  9:13     ` Auger Eric
2017-11-08 15:19     ` Marc Zyngier
2017-11-08 15:19       ` Marc Zyngier
2017-11-08 15:19       ` Marc Zyngier

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.