All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code
@ 2018-03-09 15:11 Andre Przywara
  2018-03-09 15:11 ` [PATCH 01/17] ARM: vGICv3: clarify on GUEST_GICV3_RDIST_REGIONS symbol Andre Przywara
                   ` (17 more replies)
  0 siblings, 18 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

To get away from that scary xx/57 number in the new vGIC series, these
are the first few patches split off. They prepare the existing Xen code
to better observe the split between the vGIC emulation and the physical
GIC driver.
This affects the first 20 patches from the new vGIC series. Most of them
have already been ACKed. Exceptions are:
- Patch 01/57 has been merged already.
- Patch 04/57 has been dropped, the one #define left has been moved into
  patch 03/17 here. I dropped Julien's ACK because of that.
- Patch 15/57 has been dropped, we don't need it anymore in the new vGIC.
- Patches 07, 08, 09 and 14/17 were changed according to the review comments on
  patches 09, 10, 11 and 17/57, respectively.

Every patch has been compile tested on arm and arm64, also the end result
has been briefly tested with two guests on a Juno(arm64) and Midway(arm)
platform.

A branch with those patches is available at the new-vgic/prep branch at:
git://linux-arm.org/xen-ap.git
http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/vgic-new/prep

Cheers,
Andre

Andre Przywara (17):
  ARM: vGICv3: clarify on GUEST_GICV3_RDIST_REGIONS symbol
  ARM: GICv3: use hardware GICv3 redistributor values for Dom0
  ARM: vGICv3: always use architected redist stride
  ARM: vGICv3: remove rdist_stride from VGIC structure
  ARM: VGIC: rename gic_inject() and gic_clear_lrs()
  ARM: VGIC: Move gic_remove_from_lr_pending() prototype
  ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific
  ARM: VGIC: rename gic_event_needs_delivery()
  ARM: VGIC: change to level-IRQ compatible IRQ injection interface
  ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist
  ARM: VGIC: reorder prototypes in vgic.h
  ARM: VGIC: Introduce gic_get_nr_lrs()
  ARM: GICv3: rename HYP interface definitions to use ICH_ prefix
  ARM: Implement vcpu_kick()
  ARM: GICv2: introduce gicv2_poke_irq()
  ARM: GICv3: poke_irq: make RWP optional
  ARM: GICv2: fix GICH_V2_LR definitions

 xen/arch/arm/domain.c             |  24 +++----
 xen/arch/arm/gic-v2.c             |  20 +++---
 xen/arch/arm/gic-v3-lpi.c         |   2 +-
 xen/arch/arm/gic-v3.c             |  82 ++++++++++-----------
 xen/arch/arm/gic-vgic.c           |  30 +++++---
 xen/arch/arm/irq.c                |   2 +-
 xen/arch/arm/time.c               |   2 +-
 xen/arch/arm/traps.c              |   4 +-
 xen/arch/arm/vgic-v3.c            |  40 +++++------
 xen/arch/arm/vgic.c               |  64 ++++++++++-------
 xen/arch/arm/vpl011.c             |   2 +-
 xen/arch/arm/vtimer.c             |   4 +-
 xen/include/asm-arm/domain.h      |  93 +++---------------------
 xen/include/asm-arm/event.h       |   2 +-
 xen/include/asm-arm/gic.h         |  14 ++--
 xen/include/asm-arm/gic_v3_defs.h |  54 +++++++-------
 xen/include/asm-arm/vgic.h        | 146 +++++++++++++++++++++++++++++++-------
 xen/include/public/arch-arm.h     |   1 -
 18 files changed, 315 insertions(+), 271 deletions(-)

-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 01/17] ARM: vGICv3: clarify on GUEST_GICV3_RDIST_REGIONS symbol
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 02/17] ARM: GICv3: use hardware GICv3 redistributor values for Dom0 Andre Przywara
                   ` (16 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

Normally there is only one GICv3 redistributor region, and we use
that for DomU guests using a GICv3.
Explain the background in a comment and why we need to keep the number
of hardware regions for Dom0.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/vgic-v3.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 2ad8a6be62..d5b34a7d0f 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1632,6 +1632,16 @@ static int vgic_v3_vcpu_init(struct vcpu *v)
 
 static inline unsigned int vgic_v3_rdist_count(struct domain *d)
 {
+    /*
+     * Normally there is only one GICv3 redistributor region.
+     * The GICv3 DT binding provisions for multiple regions, since there are
+     * platforms out there which need those (multi-socket systems).
+     * For Dom0 we have to live with the MMIO layout the hardware provides,
+     * so we have to copy the multiple regions - as the first region may not
+     * provide enough space to hold all redistributors we need.
+     * However DomU get a constructed memory map, so we can go with
+     * the architected single redistributor region.
+     */
     return is_hardware_domain(d) ? vgic_v3_hw.nr_rdist_regions :
                GUEST_GICV3_RDIST_REGIONS;
 }
@@ -1692,7 +1702,7 @@ static int vgic_v3_domain_init(struct domain *d)
     {
         d->arch.vgic.dbase = GUEST_GICV3_GICD_BASE;
 
-        /* XXX: Only one Re-distributor region mapped for the guest */
+        /* A single Re-distributor region is mapped for the guest. */
         BUILD_BUG_ON(GUEST_GICV3_RDIST_REGIONS != 1);
 
         d->arch.vgic.rdist_stride = GUEST_GICV3_RDIST_STRIDE;
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 02/17] ARM: GICv3: use hardware GICv3 redistributor values for Dom0
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
  2018-03-09 15:11 ` [PATCH 01/17] ARM: vGICv3: clarify on GUEST_GICV3_RDIST_REGIONS symbol Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 03/17] ARM: vGICv3: always use architected redist stride Andre Przywara
                   ` (15 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

The code to generate the DT node or MADT table for Dom0 reaches into the
domain's vGIC structure to learn the number of redistributor regions and
their base addresses.
Since those values are copied from the hardware, we can as well use
those hardware values directly when setting up the hardware domain.

This avoids the hardware GIC code to reference vGIC data structures.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/gic-v3.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 25c30bb9ea..b1f8a86409 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1162,13 +1162,11 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
     if ( res )
         return res;
 
-    res = fdt_property_cell(fdt, "redistributor-stride",
-                            d->arch.vgic.rdist_stride);
+    res = fdt_property_cell(fdt, "redistributor-stride", gicv3.rdist_stride);
     if ( res )
         return res;
 
-    res = fdt_property_cell(fdt, "#redistributor-regions",
-                            d->arch.vgic.nr_regions);
+    res = fdt_property_cell(fdt, "#redistributor-regions", gicv3.rdist_count);
     if ( res )
         return res;
 
@@ -1178,7 +1176,7 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
      * CPU interface and virtual cpu interfaces accessesed as System registers
      * So cells are created only for Distributor and rdist regions
      */
-    new_len = new_len * (d->arch.vgic.nr_regions + 1);
+    new_len = new_len * (gicv3.rdist_count + 1);
 
     hw_reg = dt_get_property(gic, "reg", &len);
     if ( !hw_reg )
@@ -1406,13 +1404,13 @@ static int gicv3_make_hwdom_madt(const struct domain *d, u32 offset)
 
     /* Add Generic Redistributor */
     size = sizeof(struct acpi_madt_generic_redistributor);
-    for ( i = 0; i < d->arch.vgic.nr_regions; i++ )
+    for ( i = 0; i < gicv3.rdist_count; i++ )
     {
         gicr = (struct acpi_madt_generic_redistributor *)(base_ptr + table_len);
         gicr->header.type = ACPI_MADT_TYPE_GENERIC_REDISTRIBUTOR;
         gicr->header.length = size;
-        gicr->base_address = d->arch.vgic.rdist_regions[i].base;
-        gicr->length = d->arch.vgic.rdist_regions[i].size;
+        gicr->base_address = gicv3.rdist_regions[i].base;
+        gicr->length = gicv3.rdist_regions[i].size;
         table_len += size;
     }
 
@@ -1425,8 +1423,7 @@ static unsigned long gicv3_get_hwdom_extra_madt_size(const struct domain *d)
 {
     unsigned long size;
 
-    size = sizeof(struct acpi_madt_generic_redistributor)
-           * d->arch.vgic.nr_regions;
+    size = sizeof(struct acpi_madt_generic_redistributor) * gicv3.rdist_count;
 
     size += sizeof(struct acpi_madt_generic_translator)
             * vgic_v3_its_count(d);
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 03/17] ARM: vGICv3: always use architected redist stride
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
  2018-03-09 15:11 ` [PATCH 01/17] ARM: vGICv3: clarify on GUEST_GICV3_RDIST_REGIONS symbol Andre Przywara
  2018-03-09 15:11 ` [PATCH 02/17] ARM: GICv3: use hardware GICv3 redistributor values for Dom0 Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-12 11:08   ` Julien Grall
  2018-03-09 15:11 ` [PATCH 04/17] ARM: vGICv3: remove rdist_stride from VGIC structure Andre Przywara
                   ` (14 subsequent siblings)
  17 siblings, 1 reply; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

The redistributor-stride property in a GICv3 DT node is only there to
cover broken platforms where this value deviates from the architected one.
Since we emulate the GICv3 distributor even for Dom0, we don't need to
copy the broken behaviour. All the special handling for Dom0s using
GICv3 is just for using the hardware's memory map, which is unaffected
by the redistributor stride - it can never be smaller than the
architected two pages.
Remove the redistributor-stride property from Dom0's DT node and also
remove the code that tried to reuse the hardware value for Dom0's GICv3
emulation.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog:
- merge in GICV3_GICR_SIZE definition

 xen/arch/arm/gic-v3.c             |  4 ----
 xen/arch/arm/vgic-v3.c            | 14 ++++++--------
 xen/include/asm-arm/gic_v3_defs.h |  5 +++++
 3 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index b1f8a86409..047af691b1 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1162,10 +1162,6 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
     if ( res )
         return res;
 
-    res = fdt_property_cell(fdt, "redistributor-stride", gicv3.rdist_stride);
-    if ( res )
-        return res;
-
     res = fdt_property_cell(fdt, "#redistributor-regions", gicv3.rdist_count);
     if ( res )
         return res;
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index d5b34a7d0f..56cc38ffcc 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1024,10 +1024,9 @@ static struct vcpu *get_vcpu_from_rdist(struct domain *d,
     paddr_t gpa, uint32_t *offset)
 {
     struct vcpu *v;
-    uint32_t stride = d->arch.vgic.rdist_stride;
     unsigned int vcpu_id;
 
-    vcpu_id = region->first_cpu + ((gpa - region->base) / stride);
+    vcpu_id = region->first_cpu + ((gpa - region->base) / GICV3_GICR_SIZE);
     if ( unlikely(vcpu_id >= d->max_vcpus) )
         return NULL;
 
@@ -1586,7 +1585,6 @@ static int vgic_v3_vcpu_init(struct vcpu *v)
 
     /* Convenient alias */
     struct domain *d = v->domain;
-    uint32_t rdist_stride = d->arch.vgic.rdist_stride;
 
     /*
      * Find the region where the re-distributor lives. For this purpose,
@@ -1602,11 +1600,11 @@ static int vgic_v3_vcpu_init(struct vcpu *v)
 
     /* Get the base address of the redistributor */
     rdist_base = region->base;
-    rdist_base += (v->vcpu_id - region->first_cpu) * rdist_stride;
+    rdist_base += (v->vcpu_id - region->first_cpu) * GICV3_GICR_SIZE;
 
     /* Check if a valid region was found for the re-distributor */
     if ( (rdist_base < region->base) ||
-         ((rdist_base + rdist_stride) > (region->base + region->size)) )
+         ((rdist_base + GICV3_GICR_SIZE) > (region->base + region->size)) )
     {
         dprintk(XENLOG_ERR,
                 "d%u: Unable to find a re-distributor for VCPU %u\n",
@@ -1622,7 +1620,7 @@ static int vgic_v3_vcpu_init(struct vcpu *v)
      * VGIC_V3_RDIST_LAST flags.
      * Note that we are assuming max_vcpus will never change.
      */
-    last_cpu = (region->size / rdist_stride) + region->first_cpu - 1;
+    last_cpu = (region->size / GICV3_GICR_SIZE) + region->first_cpu - 1;
 
     if ( v->vcpu_id == last_cpu || (v->vcpu_id == (d->max_vcpus - 1)) )
         v->arch.vgic.flags |= VGIC_V3_RDIST_LAST;
@@ -1693,7 +1691,7 @@ static int vgic_v3_domain_init(struct domain *d)
             /* Set the first CPU handled by this region */
             d->arch.vgic.rdist_regions[i].first_cpu = first_cpu;
 
-            first_cpu += size / d->arch.vgic.rdist_stride;
+            first_cpu += size / GICV3_GICR_SIZE;
         }
 
         d->arch.vgic.intid_bits = vgic_v3_hw.intid_bits;
@@ -1708,7 +1706,7 @@ static int vgic_v3_domain_init(struct domain *d)
         d->arch.vgic.rdist_stride = GUEST_GICV3_RDIST_STRIDE;
 
         /* The first redistributor should contain enough space for all CPUs */
-        BUILD_BUG_ON((GUEST_GICV3_GICR0_SIZE / GUEST_GICV3_RDIST_STRIDE) < MAX_VIRT_CPUS);
+        BUILD_BUG_ON((GUEST_GICV3_GICR0_SIZE / GICV3_GICR_SIZE) < MAX_VIRT_CPUS);
         d->arch.vgic.rdist_regions[0].base = GUEST_GICV3_GICR0_BASE;
         d->arch.vgic.rdist_regions[0].size = GUEST_GICV3_GICR0_SIZE;
         d->arch.vgic.rdist_regions[0].first_cpu = 0;
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 65c9dc47cf..bb34d17eca 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -18,6 +18,8 @@
 #ifndef __ASM_ARM_GIC_V3_DEFS_H__
 #define __ASM_ARM_GIC_V3_DEFS_H__
 
+#include <xen/sizes.h>
+
 /*
  * Additional registers defined in GIC v3.
  * Common GICD registers are defined in gic.h
@@ -68,6 +70,9 @@
 #define GICV3_GICD_IIDR_VAL          0x34c
 #define GICV3_GICR_IIDR_VAL          GICV3_GICD_IIDR_VAL
 
+/* Two pages for the RD_base and SGI_base register frame. */
+#define GICV3_GICR_SIZE              (2 * SZ_64K)
+
 #define GICR_CTLR                    (0x0000)
 #define GICR_IIDR                    (0x0004)
 #define GICR_TYPER                   (0x0008)
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 04/17] ARM: vGICv3: remove rdist_stride from VGIC structure
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (2 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 03/17] ARM: vGICv3: always use architected redist stride Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 05/17] ARM: VGIC: rename gic_inject() and gic_clear_lrs() Andre Przywara
                   ` (13 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

The last patch removed the usage of the hardware's redistributor-stride
value from our (Dom0) GICv3 emulation. This means we no longer need to
store this value in the VGIC data structure.
Remove that variable and every code snippet that handled that, instead
simply always use the architected value.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
 xen/arch/arm/gic-v3.c         |  3 +--
 xen/arch/arm/vgic-v3.c        | 14 --------------
 xen/include/asm-arm/domain.h  |  1 -
 xen/include/asm-arm/vgic.h    |  1 -
 xen/include/public/arch-arm.h |  1 -
 5 files changed, 1 insertion(+), 19 deletions(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 047af691b1..4acdd0ad91 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1680,8 +1680,7 @@ static int __init gicv3_init(void)
     reg = readl_relaxed(GICD + GICD_TYPER);
     intid_bits = GICD_TYPE_ID_BITS(reg);
 
-    vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions,
-                     gicv3.rdist_stride, intid_bits);
+    vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions, intid_bits);
     gicv3_init_v2();
 
     spin_lock_init(&gicv3.lock);
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 56cc38ffcc..4b42739a52 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -58,21 +58,18 @@ static struct {
     /* Re-distributor regions */
     unsigned int nr_rdist_regions;
     const struct rdist_region *regions;
-    uint32_t rdist_stride; /* Re-distributor stride */
     unsigned int intid_bits;  /* Number of interrupt ID bits */
 } vgic_v3_hw;
 
 void vgic_v3_setup_hw(paddr_t dbase,
                       unsigned int nr_rdist_regions,
                       const struct rdist_region *regions,
-                      uint32_t rdist_stride,
                       unsigned int intid_bits)
 {
     vgic_v3_hw.enabled = true;
     vgic_v3_hw.dbase = dbase;
     vgic_v3_hw.nr_rdist_regions = nr_rdist_regions;
     vgic_v3_hw.regions = regions;
-    vgic_v3_hw.rdist_stride = rdist_stride;
     vgic_v3_hw.intid_bits = intid_bits;
 }
 
@@ -1672,15 +1669,6 @@ static int vgic_v3_domain_init(struct domain *d)
 
         d->arch.vgic.dbase = vgic_v3_hw.dbase;
 
-        d->arch.vgic.rdist_stride = vgic_v3_hw.rdist_stride;
-        /*
-         * If the stride is not set, the default stride for GICv3 is 2 * 64K:
-         *     - first 64k page for Control and Physical LPIs
-         *     - second 64k page for Control and Generation of SGIs
-         */
-        if ( !d->arch.vgic.rdist_stride )
-            d->arch.vgic.rdist_stride = 2 * SZ_64K;
-
         for ( i = 0; i < vgic_v3_hw.nr_rdist_regions; i++ )
         {
             paddr_t size = vgic_v3_hw.regions[i].size;
@@ -1703,8 +1691,6 @@ static int vgic_v3_domain_init(struct domain *d)
         /* A single Re-distributor region is mapped for the guest. */
         BUILD_BUG_ON(GUEST_GICV3_RDIST_REGIONS != 1);
 
-        d->arch.vgic.rdist_stride = GUEST_GICV3_RDIST_STRIDE;
-
         /* The first redistributor should contain enough space for all CPUs */
         BUILD_BUG_ON((GUEST_GICV3_GICR0_SIZE / GICV3_GICR_SIZE) < MAX_VIRT_CPUS);
         d->arch.vgic.rdist_regions[0].base = GUEST_GICV3_GICR0_BASE;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 0dd8c954e2..aee247a037 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -105,7 +105,6 @@ struct arch_domain
             unsigned int first_cpu;         /* First CPU handled */
         } *rdist_regions;
         int nr_regions;                     /* Number of rdist regions */
-        uint32_t rdist_stride;              /* Re-Distributor stride */
         unsigned long int nr_lpis;
         uint64_t rdist_propbase;
         struct rb_root its_devices;         /* Devices mapped to an ITS */
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 6ea9f140a7..d61b54867b 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -261,7 +261,6 @@ struct rdist_region;
 void vgic_v3_setup_hw(paddr_t dbase,
                       unsigned int nr_rdist_regions,
                       const struct rdist_region *regions,
-                      uint32_t rdist_stride,
                       unsigned int intid_bits);
 #endif
 
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 05fd11ca38..eb424e8286 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -401,7 +401,6 @@ typedef uint64_t xen_callback_t;
 #define GUEST_GICV3_GICD_BASE      xen_mk_ullong(0x03001000)
 #define GUEST_GICV3_GICD_SIZE      xen_mk_ullong(0x00010000)
 
-#define GUEST_GICV3_RDIST_STRIDE   xen_mk_ullong(0x00020000)
 #define GUEST_GICV3_RDIST_REGIONS  1
 
 #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 05/17] ARM: VGIC: rename gic_inject() and gic_clear_lrs()
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (3 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 04/17] ARM: vGICv3: remove rdist_stride from VGIC structure Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 06/17] ARM: VGIC: Move gic_remove_from_lr_pending() prototype Andre Przywara
                   ` (12 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

The two central functions to synchronise our emulated VGIC state with
the GIC hardware (the LRs, really), are named somewhat confusingly.
Rename them from gic_inject() to vgic_sync_to_lrs() and from
gic_clear_lrs() to vgic_sync_from_lrs(), to make the code more readable.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/gic-vgic.c   | 4 ++--
 xen/arch/arm/traps.c      | 4 ++--
 xen/include/asm-arm/gic.h | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index d273863556..c0fe38fd37 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -247,7 +247,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
     }
 }
 
-void gic_clear_lrs(struct vcpu *v)
+void vgic_sync_from_lrs(struct vcpu *v)
 {
     int i = 0;
     unsigned long flags;
@@ -377,7 +377,7 @@ out:
     return rc;
 }
 
-void gic_inject(void)
+void vgic_sync_to_lrs(void)
 {
     ASSERT(!local_irq_is_enabled());
 
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 1cba7e584d..7411bff7a7 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2024,7 +2024,7 @@ static void enter_hypervisor_head(struct cpu_user_regs *regs)
         if ( current->arch.hcr_el2 & HCR_VA )
             current->arch.hcr_el2 = READ_SYSREG(HCR_EL2);
 
-        gic_clear_lrs(current);
+        vgic_sync_from_lrs(current);
     }
 }
 
@@ -2234,7 +2234,7 @@ void leave_hypervisor_tail(void)
     {
         local_irq_disable();
         if (!softirq_pending(smp_processor_id())) {
-            gic_inject();
+            vgic_sync_to_lrs();
 
             /*
              * If the SErrors handle option is "DIVERSE", we have to prevent
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 497f195bc1..e2ae4254ed 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -237,7 +237,7 @@ extern int gic_route_irq_to_guest(struct domain *, unsigned int virq,
 int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
                               struct irq_desc *desc);
 
-extern void gic_inject(void);
+extern void vgic_sync_to_lrs(void);
 extern void gic_clear_pending_irqs(struct vcpu *v);
 extern int gic_events_need_delivery(void);
 
@@ -295,7 +295,7 @@ extern unsigned int gic_number_lines(void);
 /* IRQ translation function for the device tree */
 int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
                   unsigned int *out_hwirq, unsigned int *out_type);
-void gic_clear_lrs(struct vcpu *v);
+void vgic_sync_from_lrs(struct vcpu *v);
 
 struct gic_info {
     /* GIC version */
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 06/17] ARM: VGIC: Move gic_remove_from_lr_pending() prototype
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (4 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 05/17] ARM: VGIC: rename gic_inject() and gic_clear_lrs() Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 07/17] ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific Andre Przywara
                   ` (11 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

The prototype for gic_remove_from_lr_pending() is the last function in
gic.h which references a VGIC data structure.
Move it over to vgic.h, so that we can remove the inclusion of vgic.h
from gic.h. We add it to asm/domain.h instead, where it is actually
needed.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/include/asm-arm/domain.h | 1 +
 xen/include/asm-arm/gic.h    | 2 --
 xen/include/asm-arm/vgic.h   | 1 +
 3 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index aee247a037..c6aa5cf389 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -8,6 +8,7 @@
 #include <asm/vfp.h>
 #include <asm/mmio.h>
 #include <asm/gic.h>
+#include <asm/vgic.h>
 #include <public/hvm/params.h>
 #include <xen/serial.h>
 #include <xen/rbtree.h>
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index e2ae4254ed..3b2d0217a6 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -156,7 +156,6 @@
 #ifndef __ASSEMBLY__
 #include <xen/device_tree.h>
 #include <xen/irq.h>
-#include <asm-arm/vgic.h>
 
 #define DT_COMPAT_GIC_CORTEX_A15 "arm,cortex-a15-gic"
 
@@ -245,7 +244,6 @@ extern void init_maintenance_interrupt(void);
 extern void gic_raise_guest_irq(struct vcpu *v, unsigned int irq,
         unsigned int priority);
 extern void gic_raise_inflight_irq(struct vcpu *v, unsigned int virtual_irq);
-extern void gic_remove_from_lr_pending(struct vcpu *v, struct pending_irq *p);
 
 /* Accept an interrupt from the GIC and dispatch its handler */
 extern void gic_interrupt(struct cpu_user_regs *regs, int is_fiq);
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index d61b54867b..d03298e12c 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -205,6 +205,7 @@ extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq);
 extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
 extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
 extern void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p);
+extern void gic_remove_from_lr_pending(struct vcpu *v, struct pending_irq *p);
 extern void vgic_clear_pending_irqs(struct vcpu *v);
 extern void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq);
 extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 07/17] ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (5 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 06/17] ARM: VGIC: Move gic_remove_from_lr_pending() prototype Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-12 11:09   ` Julien Grall
  2018-03-09 15:11 ` [PATCH 08/17] ARM: VGIC: rename gic_event_needs_delivery() Andre Przywara
                   ` (10 subsequent siblings)
  17 siblings, 1 reply; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

domain_max_vcpus(), which is used by generic Xen code, returns the
maximum number of VCPUs for a domain, which on ARM is mostly limited by
the VGIC model emulated (a (v)GICv2 can only handle 8 CPUs).
Our current implementation lives in arch/arm/domain.c, but reaches into
VGIC internal data structures.
Move the actual functionality into vgic.c, and provide a shim in
domain.h, to keep this VGIC internal.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog:
- rename helper function and wrap in domain.h

 xen/arch/arm/domain.c        | 14 --------------
 xen/arch/arm/vgic.c          | 14 ++++++++++++++
 xen/include/asm-arm/domain.h |  6 +++++-
 xen/include/asm-arm/vgic.h   |  2 ++
 4 files changed, 21 insertions(+), 15 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 8de4c0a343..6b902fa30f 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -967,20 +967,6 @@ void vcpu_block_unless_event_pending(struct vcpu *v)
         vcpu_unblock(current);
 }
 
-unsigned int domain_max_vcpus(const struct domain *d)
-{
-    /*
-     * Since evtchn_init would call domain_max_vcpus for poll_mask
-     * allocation when the vgic_ops haven't been initialised yet,
-     * we return MAX_VIRT_CPUS if d->arch.vgic.handler is null.
-     */
-    if ( !d->arch.vgic.handler )
-        return MAX_VIRT_CPUS;
-    else
-        return min_t(unsigned int, MAX_VIRT_CPUS,
-                     d->arch.vgic.handler->max_vcpus);
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 34269bcf27..fa00c21a69 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -665,6 +665,20 @@ void vgic_free_virq(struct domain *d, unsigned int virq)
     clear_bit(virq, d->arch.vgic.allocated_irqs);
 }
 
+unsigned int vgic_max_vcpus(const struct domain *d)
+{
+    /*
+     * Since evtchn_init would call domain_max_vcpus for poll_mask
+     * allocation when the vgic_ops haven't been initialised yet,
+     * we return MAX_VIRT_CPUS if d->arch.vgic.handler is null.
+     */
+    if ( !d->arch.vgic.handler )
+        return MAX_VIRT_CPUS;
+    else
+        return min_t(unsigned int, MAX_VIRT_CPUS,
+                     d->arch.vgic.handler->max_vcpus);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index c6aa5cf389..e730e07fcf 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -289,7 +289,11 @@ void vcpu_show_execution_state(struct vcpu *);
 void vcpu_show_registers(const struct vcpu *);
 void vcpu_switch_to_aarch64_mode(struct vcpu *);
 
-unsigned int domain_max_vcpus(const struct domain *);
+/* On ARM, the number of VCPUs is limited by the type of GIC emulated. */
+static inline unsigned int domain_max_vcpus(const struct domain *d)
+{
+    return vgic_max_vcpus(d);
+}
 
 /*
  * Due to the restriction of GICv3, the number of vCPUs in AFF0 is
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index d03298e12c..afb4776ad4 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -254,6 +254,8 @@ static inline int vgic_allocate_spi(struct domain *d)
 
 extern void vgic_free_virq(struct domain *d, unsigned int virq);
 
+unsigned int vgic_max_vcpus(const struct domain *d);
+
 void vgic_v2_setup_hw(paddr_t dbase, paddr_t cbase, paddr_t csize,
                       paddr_t vbase, uint32_t aliased_offset);
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 08/17] ARM: VGIC: rename gic_event_needs_delivery()
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (6 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 07/17] ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-12 11:10   ` Julien Grall
  2018-03-09 15:11 ` [PATCH 09/17] ARM: VGIC: change to level-IRQ compatible IRQ injection interface Andre Przywara
                   ` (9 subsequent siblings)
  17 siblings, 1 reply; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

gic_event_needs_delivery() is not named very intuitively, especially
the gic_ prefix is somewhat misleading.
Rename it to vgic_pending_irq(), which makes it clear that this relates
to the virtual GIC and is about interrupts.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog:
- Add vcpu parameter
- Rename to vgic_vcpu_pending_irq()

 xen/arch/arm/gic-vgic.c     | 16 ++++++++++++++--
 xen/include/asm-arm/event.h |  2 +-
 xen/include/asm-arm/gic.h   |  2 +-
 3 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index c0fe38fd37..f4c98bffd1 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -339,9 +339,18 @@ void gic_clear_pending_irqs(struct vcpu *v)
         gic_remove_from_lr_pending(v, p);
 }
 
-int gic_events_need_delivery(void)
+/**
+ * vgic_vcpu_pending_irq() - determine if interrupts need to be injected
+ * @vcpu: The vCPU on which to check for interrupts.
+ *
+ * Checks whether there is an interrupt on the given VCPU which needs
+ * handling in the guest. This requires at least one IRQ to be pending
+ * and enabled.
+ *
+ * Returns: 1 if the guest should run to handle interrupts, 0 otherwise.
+ */
+int vgic_vcpu_pending_irq(struct vcpu *v)
 {
-    struct vcpu *v = current;
     struct pending_irq *p;
     unsigned long flags;
     const unsigned long apr = gic_hw_ops->read_apr(0);
@@ -349,6 +358,9 @@ int gic_events_need_delivery(void)
     int active_priority;
     int rc = 0;
 
+    /* We rely on reading the VMCR, which is only accessible locally. */
+    ASSERT(v == current);
+
     mask_priority = gic_hw_ops->read_vmcr_priority();
     active_priority = find_next_bit(&apr, 32, 0);
 
diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
index e8c2a6cb44..c7a415ef57 100644
--- a/xen/include/asm-arm/event.h
+++ b/xen/include/asm-arm/event.h
@@ -24,7 +24,7 @@ static inline int local_events_need_delivery_nomask(void)
      * interrupts disabled so this shouldn't be a problem in the general
      * case.
      */
-    if ( gic_events_need_delivery() )
+    if ( vgic_vcpu_pending_irq(current) )
         return 1;
 
     if ( !vcpu_info(current, evtchn_upcall_pending) )
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 3b2d0217a6..ff0b22451b 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -238,7 +238,7 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
 
 extern void vgic_sync_to_lrs(void);
 extern void gic_clear_pending_irqs(struct vcpu *v);
-extern int gic_events_need_delivery(void);
+extern int vgic_vcpu_pending_irq(struct vcpu *v);
 
 extern void init_maintenance_interrupt(void);
 extern void gic_raise_guest_irq(struct vcpu *v, unsigned int irq,
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 09/17] ARM: VGIC: change to level-IRQ compatible IRQ injection interface
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (7 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 08/17] ARM: VGIC: rename gic_event_needs_delivery() Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-12 11:29   ` Julien Grall
  2018-03-09 15:11 ` [PATCH 10/17] ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist Andre Przywara
                   ` (8 subsequent siblings)
  17 siblings, 1 reply; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

At the moment vgic_vcpu_inject_irq() is the interface for Xen internal
code and virtual devices to inject IRQs into a guest. This interface has
two shortcomings:
1) It requires a VCPU pointer, which we may not know (and don't need!)
for shared interrupts. A second function (vgic_vcpu_inject_spi()), was
there to work around this issue.
2) This interface only really supports edge triggered IRQs, which is
what the Xen VGIC emulates only anyway. However this needs to and will
change, so we need to add the desired level (high or low) to the
interface.
This replaces the existing injection call (taking a VCPU and an IRQ
parameter) with a new one, taking domain, VCPU, IRQ and level parameters.
The VCPU can be NULL in case we don't know and don't care.
We change all call sites to use this new interface. This still doesn't
give us the missing level IRQ handling, but at least prepares the callers
to do the right thing later automatically.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog:
- keep function as returning void

 xen/arch/arm/domain.c      |  4 ++--
 xen/arch/arm/gic-v3-lpi.c  |  2 +-
 xen/arch/arm/irq.c         |  2 +-
 xen/arch/arm/time.c        |  2 +-
 xen/arch/arm/vgic.c        | 39 +++++++++++++++++++++++----------------
 xen/arch/arm/vpl011.c      |  2 +-
 xen/arch/arm/vtimer.c      |  4 ++--
 xen/include/asm-arm/vgic.h |  4 ++--
 8 files changed, 33 insertions(+), 26 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 6b902fa30f..bc10f412ba 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -951,14 +951,14 @@ void vcpu_mark_events_pending(struct vcpu *v)
     if ( already_pending )
         return;
 
-    vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
+    vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
 }
 
 /* The ARM spec declares that even if local irqs are masked in
  * the CPSR register, an irq should wake up a cpu from WFI anyway.
  * For this reason we need to check for irqs that need delivery,
  * ignoring the CPSR register, *after* calling SCHEDOP_block to
- * avoid races with vgic_vcpu_inject_irq.
+ * avoid races with vgic_inject_irq.
  */
 void vcpu_block_unless_event_pending(struct vcpu *v)
 {
diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
index 84582157b8..efd5cd62fb 100644
--- a/xen/arch/arm/gic-v3-lpi.c
+++ b/xen/arch/arm/gic-v3-lpi.c
@@ -153,7 +153,7 @@ void vgic_vcpu_inject_lpi(struct domain *d, unsigned int virq)
     if ( vcpu_id >= d->max_vcpus )
           return;
 
-    vgic_vcpu_inject_irq(d->vcpu[vcpu_id], virq);
+    vgic_inject_irq(d, d->vcpu[vcpu_id], virq, true);
 }
 
 /*
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 29af10e82c..aa4e832cae 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -225,7 +225,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
          * The irq cannot be a PPI, we only support delivery of SPIs to
          * guests.
 	 */
-        vgic_vcpu_inject_spi(info->d, info->virq);
+        vgic_inject_irq(info->d, NULL, info->virq, true);
         goto out_no_end;
     }
 
diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c
index 36f640f0c1..c11fcfeadd 100644
--- a/xen/arch/arm/time.c
+++ b/xen/arch/arm/time.c
@@ -260,7 +260,7 @@ static void vtimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
 
     current->arch.virt_timer.ctl = READ_SYSREG32(CNTV_CTL_EL0);
     WRITE_SYSREG32(current->arch.virt_timer.ctl | CNTx_CTL_MASK, CNTV_CTL_EL0);
-    vgic_vcpu_inject_irq(current, current->arch.virt_timer.irq);
+    vgic_inject_irq(current->domain, current, current->arch.virt_timer.irq, true);
 }
 
 /*
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index fa00c21a69..eb09d9ca54 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -291,7 +291,7 @@ bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq)
         vgic_remove_irq_from_queues(old, p);
         irq_set_affinity(p->desc, cpumask_of(new->processor));
         spin_unlock_irqrestore(&old->arch.vgic.lock, flags);
-        vgic_vcpu_inject_irq(new, irq);
+        vgic_inject_irq(new->domain, new, irq, true);
         return true;
     }
     /* if the IRQ is in a GICH_LR register, set GIC_IRQ_GUEST_MIGRATING
@@ -450,7 +450,7 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode,
                         sgir, target->list);
                 continue;
             }
-            vgic_vcpu_inject_irq(d->vcpu[vcpuid], virq);
+            vgic_inject_irq(d, d->vcpu[vcpuid], virq, true);
         }
         break;
     case SGI_TARGET_OTHERS:
@@ -459,12 +459,12 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode,
         {
             if ( i != current->vcpu_id && d->vcpu[i] != NULL &&
                  is_vcpu_online(d->vcpu[i]) )
-                vgic_vcpu_inject_irq(d->vcpu[i], virq);
+                vgic_inject_irq(d, d->vcpu[i], virq, true);
         }
         break;
     case SGI_TARGET_SELF:
         perfc_incr(vgic_sgi_self);
-        vgic_vcpu_inject_irq(d->vcpu[current->vcpu_id], virq);
+        vgic_inject_irq(d, current, virq, true);
         break;
     default:
         gprintk(XENLOG_WARNING,
@@ -524,13 +524,29 @@ void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p)
     gic_remove_from_lr_pending(v, p);
 }
 
-void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
+void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
+                     bool level)
 {
     uint8_t priority;
     struct pending_irq *iter, *n;
     unsigned long flags;
     bool running;
 
+    /*
+     * For edge triggered interrupts we always ignore a "falling edge".
+     * For level triggered interrupts we shouldn't, but do anyways.
+     */
+    if ( !level )
+        return;
+
+    if ( !v )
+    {
+        /* The IRQ needs to be an SPI if no vCPU is specified. */
+        ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
+
+        v = vgic_get_target_vcpu(d->vcpu[0], virq);
+    };
+
     spin_lock_irqsave(&v->arch.vgic.lock, flags);
 
     n = irq_to_pending(v, virq);
@@ -582,22 +598,13 @@ out:
         perfc_incr(vgic_cross_cpu_intr_inject);
         smp_send_event_check_mask(cpumask_of(v->processor));
     }
-}
-
-void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
-{
-    struct vcpu *v;
-
-    /* the IRQ needs to be an SPI */
-    ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
 
-    v = vgic_get_target_vcpu(d->vcpu[0], virq);
-    vgic_vcpu_inject_irq(v, virq);
+    return;
 }
 
 void arch_evtchn_inject(struct vcpu *v)
 {
-    vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
+    vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
 }
 
 bool vgic_evtchn_irq_pending(struct vcpu *v)
diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index 7788c2fc32..5dcf4bec18 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -68,7 +68,7 @@ static void vpl011_update_interrupt_status(struct domain *d)
      * status bit has been set since the last time.
      */
     if ( uartmis & ~vpl011->shadow_uartmis )
-        vgic_vcpu_inject_spi(d, GUEST_VPL011_SPI);
+        vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, true);
 
     vpl011->shadow_uartmis = uartmis;
 }
diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
index f52a723a5f..8164f6c7f1 100644
--- a/xen/arch/arm/vtimer.c
+++ b/xen/arch/arm/vtimer.c
@@ -46,7 +46,7 @@ static void phys_timer_expired(void *data)
     if ( !(t->ctl & CNTx_CTL_MASK) )
     {
         perfc_incr(vtimer_phys_inject);
-        vgic_vcpu_inject_irq(t->v, t->irq);
+        vgic_inject_irq(t->v->domain, t->v, t->irq, true);
     }
     else
         perfc_incr(vtimer_phys_masked);
@@ -56,7 +56,7 @@ static void virt_timer_expired(void *data)
 {
     struct vtimer *t = data;
     t->ctl |= CNTx_CTL_MASK;
-    vgic_vcpu_inject_irq(t->v, t->irq);
+    vgic_inject_irq(t->v->domain, t->v, t->irq, true);
     perfc_incr(vtimer_virt_inject);
 }
 
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index afb4776ad4..8af6d816c9 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -202,8 +202,8 @@ extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
 extern void domain_vgic_free(struct domain *d);
 extern int vcpu_vgic_init(struct vcpu *v);
 extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq);
-extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
-extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
+extern void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
+                            bool level);
 extern void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p);
 extern void gic_remove_from_lr_pending(struct vcpu *v, struct pending_irq *p);
 extern void vgic_clear_pending_irqs(struct vcpu *v);
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 10/17] ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (8 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 09/17] ARM: VGIC: change to level-IRQ compatible IRQ injection interface Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 11/17] ARM: VGIC: reorder prototypes in vgic.h Andre Przywara
                   ` (7 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

Currently we describe the VGIC specific fields in a structure
*embedded* in struct arch_domain and struct arch_vcpu. These members
there are however related to the current VGIC implementation, and will
be substantially different in the future.
To allow coexistence of two implementations, move the definition of these
embedded structures into vgic.h, and just use the opaque type in the arch
specific structures.
This allows easy switching between different implementations later.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/include/asm-arm/domain.h | 85 +-----------------------------------------
 xen/include/asm-arm/vgic.h   | 88 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 90 insertions(+), 83 deletions(-)

diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index e730e07fcf..7ba6528a74 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -71,57 +71,7 @@ struct arch_domain
         uint64_t offset;
     } virt_timer_base;
 
-    struct {
-        /* Version of the vGIC */
-        enum gic_version version;
-        /* GIC HW version specific vGIC driver handler */
-        const struct vgic_ops *handler;
-        /*
-         * Covers access to other members of this struct _except_ for
-         * shared_irqs where each member contains its own locking.
-         *
-         * If both class of lock is required then this lock must be
-         * taken first. If multiple rank locks are required (including
-         * the per-vcpu private_irqs rank) then they must be taken in
-         * rank order.
-         */
-        spinlock_t lock;
-        uint32_t ctlr;
-        int nr_spis; /* Number of SPIs */
-        unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
-        struct vgic_irq_rank *shared_irqs;
-        /*
-         * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
-         * struct arch_vcpu.
-         */
-        struct pending_irq *pending_irqs;
-        /* Base address for guest GIC */
-        paddr_t dbase; /* Distributor base address */
-#ifdef CONFIG_HAS_GICV3
-        /* GIC V3 addressing */
-        /* List of contiguous occupied by the redistributors */
-        struct vgic_rdist_region {
-            paddr_t base;                   /* Base address */
-            paddr_t size;                   /* Size */
-            unsigned int first_cpu;         /* First CPU handled */
-        } *rdist_regions;
-        int nr_regions;                     /* Number of rdist regions */
-        unsigned long int nr_lpis;
-        uint64_t rdist_propbase;
-        struct rb_root its_devices;         /* Devices mapped to an ITS */
-        spinlock_t its_devices_lock;        /* Protects the its_devices tree */
-        struct radix_tree_root pend_lpi_tree; /* Stores struct pending_irq's */
-        rwlock_t pend_lpi_tree_lock;        /* Protects the pend_lpi_tree */
-        struct list_head vits_list;         /* List of virtual ITSes */
-        unsigned int intid_bits;
-        /*
-         * TODO: if there are more bool's being added below, consider
-         * a flags variable instead.
-         */
-        bool rdists_enabled;                /* Is any redistributor enabled? */
-        bool has_its;
-#endif
-    } vgic;
+    struct vgic_dist vgic;
 
     struct vuart {
 #define VUART_BUF_SIZE 128
@@ -244,38 +194,7 @@ struct arch_vcpu
     union gic_state_data gic;
     uint64_t lr_mask;
 
-    struct {
-        /*
-         * SGIs and PPIs are per-VCPU, SPIs are domain global and in
-         * struct arch_domain.
-         */
-        struct pending_irq pending_irqs[32];
-        struct vgic_irq_rank *private_irqs;
-
-        /* This list is ordered by IRQ priority and it is used to keep
-         * track of the IRQs that the VGIC injected into the guest.
-         * Depending on the availability of LR registers, the IRQs might
-         * actually be in an LR, and therefore injected into the guest,
-         * or queued in gic.lr_pending.
-         * As soon as an IRQ is EOI'd by the guest and removed from the
-         * corresponding LR it is also removed from this list. */
-        struct list_head inflight_irqs;
-        /* lr_pending is used to queue IRQs (struct pending_irq) that the
-         * vgic tried to inject in the guest (calling gic_set_guest_irq) but
-         * no LRs were available at the time.
-         * As soon as an LR is freed we remove the first IRQ from this
-         * list and write it to the LR register.
-         * lr_pending is a subset of vgic.inflight_irqs. */
-        struct list_head lr_pending;
-        spinlock_t lock;
-
-        /* GICv3: redistributor base and flags for this vCPU */
-        paddr_t rdist_base;
-        uint64_t rdist_pendbase;
-#define VGIC_V3_RDIST_LAST      (1 << 0)        /* last vCPU of the rdist */
-#define VGIC_V3_LPIS_ENABLED    (1 << 1)
-        uint8_t flags;
-    } vgic;
+    struct vgic_cpu vgic;
 
     /* Timer registers  */
     uint32_t cntkctl;
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 8af6d816c9..d6f550ff44 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -19,6 +19,9 @@
 #define __ASM_ARM_VGIC_H__
 
 #include <xen/bitops.h>
+#include <xen/radix-tree.h>
+#include <xen/rbtree.h>
+#include <asm/gic.h>
 #include <asm/mmio.h>
 #include <asm/vreg.h>
 
@@ -123,6 +126,91 @@ struct vgic_irq_rank {
     uint8_t vcpu[32];
 };
 
+struct vgic_dist {
+    /* Version of the vGIC */
+    enum gic_version version;
+    /* GIC HW version specific vGIC driver handler */
+    const struct vgic_ops *handler;
+    /*
+     * Covers access to other members of this struct _except_ for
+     * shared_irqs where each member contains its own locking.
+     *
+     * If both class of lock is required then this lock must be
+     * taken first. If multiple rank locks are required (including
+     * the per-vcpu private_irqs rank) then they must be taken in
+     * rank order.
+     */
+    spinlock_t lock;
+    uint32_t ctlr;
+    int nr_spis; /* Number of SPIs */
+    unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
+    struct vgic_irq_rank *shared_irqs;
+    /*
+     * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
+     * struct arch_vcpu.
+     */
+    struct pending_irq *pending_irqs;
+    /* Base address for guest GIC */
+    paddr_t dbase; /* Distributor base address */
+#ifdef CONFIG_HAS_GICV3
+    /* GIC V3 addressing */
+    /* List of contiguous occupied by the redistributors */
+    struct vgic_rdist_region {
+        paddr_t base;                   /* Base address */
+        paddr_t size;                   /* Size */
+        unsigned int first_cpu;         /* First CPU handled */
+    } *rdist_regions;
+    int nr_regions;                     /* Number of rdist regions */
+    unsigned long int nr_lpis;
+    uint64_t rdist_propbase;
+    struct rb_root its_devices;         /* Devices mapped to an ITS */
+    spinlock_t its_devices_lock;        /* Protects the its_devices tree */
+    struct radix_tree_root pend_lpi_tree; /* Stores struct pending_irq's */
+    rwlock_t pend_lpi_tree_lock;        /* Protects the pend_lpi_tree */
+    struct list_head vits_list;         /* List of virtual ITSes */
+    unsigned int intid_bits;
+    /*
+     * TODO: if there are more bool's being added below, consider
+     * a flags variable instead.
+     */
+    bool rdists_enabled;                /* Is any redistributor enabled? */
+    bool has_its;
+#endif
+};
+
+struct vgic_cpu {
+    /*
+     * SGIs and PPIs are per-VCPU, SPIs are domain global and in
+     * struct arch_domain.
+     */
+    struct pending_irq pending_irqs[32];
+    struct vgic_irq_rank *private_irqs;
+
+    /* This list is ordered by IRQ priority and it is used to keep
+     * track of the IRQs that the VGIC injected into the guest.
+     * Depending on the availability of LR registers, the IRQs might
+     * actually be in an LR, and therefore injected into the guest,
+     * or queued in gic.lr_pending.
+     * As soon as an IRQ is EOI'd by the guest and removed from the
+     * corresponding LR it is also removed from this list. */
+    struct list_head inflight_irqs;
+    /* lr_pending is used to queue IRQs (struct pending_irq) that the
+     * vgic tried to inject in the guest (calling gic_set_guest_irq) but
+     * no LRs were available at the time.
+     * As soon as an LR is freed we remove the first IRQ from this
+     * list and write it to the LR register.
+     * lr_pending is a subset of vgic.inflight_irqs. */
+    struct list_head lr_pending;
+    spinlock_t lock;
+
+    /* GICv3: redistributor base and flags for this vCPU */
+    paddr_t rdist_base;
+    uint64_t rdist_pendbase;
+#define VGIC_V3_RDIST_LAST      (1 << 0)        /* last vCPU of the rdist */
+#define VGIC_V3_LPIS_ENABLED    (1 << 1)
+    uint8_t flags;
+};
+
 struct sgi_target {
     uint8_t aff1;
     uint16_t list;
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 11/17] ARM: VGIC: reorder prototypes in vgic.h
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (9 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 10/17] ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 12/17] ARM: VGIC: Introduce gic_get_nr_lrs() Andre Przywara
                   ` (6 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

Currently vgic.h both contains prototypes used by Xen arch code outside
of the actual VGIC (for instance vgic_vcpu_inject_irq()), and prototypes
for functions used by the VGIC internally.
Group them to later allow an easy split with one #ifdef.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/include/asm-arm/vgic.h | 54 +++++++++++++++++++++++++---------------------
 1 file changed, 30 insertions(+), 24 deletions(-)

diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index d6f550ff44..0787ba9549 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -279,49 +279,34 @@ enum gic_sgi_mode;
  */
 #define REG_RANK_INDEX(b, n, s) ((((n) >> s) & ((b)-1)) % 32)
 
-/*
- * In the moment vgic_num_irqs() just covers SPIs and the private IRQs,
- * as it's mostly used for allocating the pending_irq and irq_desc array,
- * in which LPIs don't participate.
- */
-#define vgic_num_irqs(d)        ((d)->arch.vgic.nr_spis + 32)
 
-extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
-extern void domain_vgic_free(struct domain *d);
-extern int vcpu_vgic_init(struct vcpu *v);
 extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq);
-extern void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
-                            bool level);
 extern void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p);
 extern void gic_remove_from_lr_pending(struct vcpu *v, struct pending_irq *p);
-extern void vgic_clear_pending_irqs(struct vcpu *v);
 extern void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq);
 extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
 extern struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq);
 extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, int b, int n, int s);
 extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
-extern bool vgic_emulate(struct cpu_user_regs *regs, union hsr hsr);
 extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
 int vgic_v2_init(struct domain *d, int *mmio_count);
 int vgic_v3_init(struct domain *d, int *mmio_count);
 
-bool vgic_evtchn_irq_pending(struct vcpu *v);
-struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
-                                      unsigned int virq);
-int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
-                        struct irq_desc *desc, bool connect);
-
-extern int domain_vgic_register(struct domain *d, int *mmio_count);
-extern int vcpu_vgic_free(struct vcpu *v);
 extern bool vgic_to_sgi(struct vcpu *v, register_t sgir,
                         enum gic_sgi_mode irqmode, int virq,
                         const struct sgi_target *target);
 extern bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq);
 
-/* Reserve a specific guest vIRQ */
-extern bool vgic_reserve_virq(struct domain *d, unsigned int virq);
+/*** Common VGIC functions used by Xen arch code ****/
+
+/*
+ * In the moment vgic_num_irqs() just covers SPIs and the private IRQs,
+ * as it's mostly used for allocating the pending_irq and irq_desc array,
+ * in which LPIs don't participate.
+ */
+#define vgic_num_irqs(d)        ((d)->arch.vgic.nr_spis + 32)
 
 /*
  * Allocate a guest VIRQ
@@ -329,6 +314,9 @@ extern bool vgic_reserve_virq(struct domain *d, unsigned int virq);
  *  - spi == 1 => allocate an SPI
  */
 extern int vgic_allocate_virq(struct domain *d, bool spi);
+/* Reserve a specific guest vIRQ */
+extern bool vgic_reserve_virq(struct domain *d, unsigned int virq);
+extern void vgic_free_virq(struct domain *d, unsigned int virq);
 
 static inline int vgic_allocate_ppi(struct domain *d)
 {
@@ -340,7 +328,25 @@ static inline int vgic_allocate_spi(struct domain *d)
     return vgic_allocate_virq(d, true /* spi */);
 }
 
-extern void vgic_free_virq(struct domain *d, unsigned int virq);
+struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
+                                      unsigned int virq);
+int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
+                        struct irq_desc *desc, bool connect);
+
+bool vgic_evtchn_irq_pending(struct vcpu *v);
+
+int domain_vgic_register(struct domain *d, int *mmio_count);
+int domain_vgic_init(struct domain *d, unsigned int nr_spis);
+void domain_vgic_free(struct domain *d);
+int vcpu_vgic_init(struct vcpu *vcpu);
+int vcpu_vgic_free(struct vcpu *vcpu);
+
+void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
+                     bool level);
+
+extern void vgic_clear_pending_irqs(struct vcpu *v);
+
+extern bool vgic_emulate(struct cpu_user_regs *regs, union hsr hsr);
 
 unsigned int vgic_max_vcpus(const struct domain *d);
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 12/17] ARM: VGIC: Introduce gic_get_nr_lrs()
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (10 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 11/17] ARM: VGIC: reorder prototypes in vgic.h Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 13/17] ARM: GICv3: rename HYP interface definitions to use ICH_ prefix Andre Przywara
                   ` (5 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

So far the number of list registers (LRs) a GIC implements is only
needed in the hardware facing side of the VGIC code (gic-vgic.c).
The new VGIC will need this information in more and multiple places, so
export a function that returns the number.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/gic-vgic.c   | 10 +++++-----
 xen/include/asm-arm/gic.h |  6 ++++++
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index f4c98bffd1..61f093db50 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -25,7 +25,7 @@
 #include <asm/gic.h>
 #include <asm/vgic.h>
 
-#define lr_all_full() (this_cpu(lr_mask) == ((1 << gic_hw_ops->info->nr_lrs) - 1))
+#define lr_all_full() (this_cpu(lr_mask) == ((1 << gic_get_nr_lrs()) - 1))
 
 #undef GIC_DEBUG
 
@@ -110,7 +110,7 @@ static unsigned int gic_find_unused_lr(struct vcpu *v,
                                        struct pending_irq *p,
                                        unsigned int lr)
 {
-    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
+    unsigned int nr_lrs = gic_get_nr_lrs();
     unsigned long *lr_mask = (unsigned long *) &this_cpu(lr_mask);
     struct gic_lr lr_val;
 
@@ -137,7 +137,7 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
         unsigned int priority)
 {
     int i;
-    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
+    unsigned int nr_lrs = gic_get_nr_lrs();
     struct pending_irq *p = irq_to_pending(v, virtual_irq);
 
     ASSERT(spin_is_locked(&v->arch.vgic.lock));
@@ -251,7 +251,7 @@ void vgic_sync_from_lrs(struct vcpu *v)
 {
     int i = 0;
     unsigned long flags;
-    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
+    unsigned int nr_lrs = gic_get_nr_lrs();
 
     /* The idle domain has no LRs to be cleared. Since gic_restore_state
      * doesn't write any LR registers for the idle domain they could be
@@ -278,7 +278,7 @@ static void gic_restore_pending_irqs(struct vcpu *v)
     struct pending_irq *p, *t, *p_r;
     struct list_head *inflight_r;
     unsigned long flags;
-    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
+    unsigned int nr_lrs = gic_get_nr_lrs();
     int lrs = nr_lrs;
 
     spin_lock_irqsave(&v->arch.vgic.lock, flags);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index ff0b22451b..49cb94f792 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -374,6 +374,12 @@ struct gic_hw_operations {
 };
 
 extern const struct gic_hw_operations *gic_hw_ops;
+
+static inline unsigned int gic_get_nr_lrs(void)
+{
+    return gic_hw_ops->info->nr_lrs;
+}
+
 void register_gic_ops(const struct gic_hw_operations *ops);
 int gic_make_hwdom_dt_node(const struct domain *d,
                            const struct dt_device_node *gic,
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 13/17] ARM: GICv3: rename HYP interface definitions to use ICH_ prefix
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (11 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 12/17] ARM: VGIC: Introduce gic_get_nr_lrs() Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 14/17] ARM: Implement vcpu_kick() Andre Przywara
                   ` (4 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

On a GICv3 in non-compat mode the hypervisor interface is always
accessed via system registers. Those register names have a "ICH_" prefix
in the manual, to differentiate them from the MMIO registers. Also those
registers are mostly 64-bit (compared to the 32-bit GICv2 registers) and
use different bit assignments.
To make this obvious and to avoid clashes with double definitions using
the same names for actually different bits, lets change all GICv3
hypervisor interface registers to use the "ICH_" prefix from the manual.
This renames the definitions in gic_v3_defs.h and their usage in gic-v3.c
and is needed to allow co-existence of the GICv2 and GICv3 definitions
in the same file.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/gic-v3.c             | 48 +++++++++++++++++++-------------------
 xen/include/asm-arm/gic_v3_defs.h | 49 +++++++++++++++++++--------------------
 2 files changed, 48 insertions(+), 49 deletions(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 4acdd0ad91..8b41704cf1 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -828,14 +828,14 @@ static void gicv3_hyp_init(void)
     uint32_t vtr;
 
     vtr = READ_SYSREG32(ICH_VTR_EL2);
-    gicv3_info.nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
-    gicv3.nr_priorities = ((vtr >> GICH_VTR_PRIBITS_SHIFT) &
-                          GICH_VTR_PRIBITS_MASK) + 1;
+    gicv3_info.nr_lrs  = (vtr & ICH_VTR_NRLRGS) + 1;
+    gicv3.nr_priorities = ((vtr >> ICH_VTR_PRIBITS_SHIFT) &
+                          ICH_VTR_PRIBITS_MASK) + 1;
 
     if ( !((gicv3.nr_priorities > 4) && (gicv3.nr_priorities < 8)) )
         panic("GICv3: Invalid number of priority bits\n");
 
-    WRITE_SYSREG32(GICH_VMCR_EOI | GICH_VMCR_VENG1, ICH_VMCR_EL2);
+    WRITE_SYSREG32(ICH_VMCR_EOI | ICH_VMCR_VENG1, ICH_VMCR_EL2);
     WRITE_SYSREG32(GICH_HCR_EN, ICH_HCR_EL2);
 }
 
@@ -974,21 +974,21 @@ static void gicv3_update_lr(int lr, unsigned int virq, uint8_t priority,
     BUG_ON(lr >= gicv3_info.nr_lrs);
     BUG_ON(lr < 0);
 
-    val =  (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT);
+    val =  (((uint64_t)state & 0x3) << ICH_LR_STATE_SHIFT);
 
     /*
      * When the guest is GICv3, all guest IRQs are Group 1, as Group0
      * would result in a FIQ in the guest, which it wouldn't expect
      */
     if ( current->domain->arch.vgic.version == GIC_V3 )
-        val |= GICH_LR_GRP1;
+        val |= ICH_LR_GRP1;
 
-    val |= (uint64_t)priority << GICH_LR_PRIORITY_SHIFT;
-    val |= ((uint64_t)virq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
+    val |= (uint64_t)priority << ICH_LR_PRIORITY_SHIFT;
+    val |= ((uint64_t)virq & ICH_LR_VIRTUAL_MASK) << ICH_LR_VIRTUAL_SHIFT;
 
    if ( hw_irq != INVALID_IRQ )
-       val |= GICH_LR_HW | (((uint64_t)hw_irq & GICH_LR_PHYSICAL_MASK)
-                           << GICH_LR_PHYSICAL_SHIFT);
+       val |= ICH_LR_HW | (((uint64_t)hw_irq & ICH_LR_PHYSICAL_MASK)
+                           << ICH_LR_PHYSICAL_SHIFT);
 
     gicv3_ich_write_lr(lr, val);
 }
@@ -1004,25 +1004,25 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
 
     lrv = gicv3_ich_read_lr(lr);
 
-    lr_reg->pirq = (lrv >> GICH_LR_PHYSICAL_SHIFT) & GICH_LR_PHYSICAL_MASK;
-    lr_reg->virq = (lrv >> GICH_LR_VIRTUAL_SHIFT) & GICH_LR_VIRTUAL_MASK;
+    lr_reg->pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
+    lr_reg->virq = (lrv >> ICH_LR_VIRTUAL_SHIFT) & ICH_LR_VIRTUAL_MASK;
 
-    lr_reg->priority  = (lrv >> GICH_LR_PRIORITY_SHIFT) & GICH_LR_PRIORITY_MASK;
-    lr_reg->state     = (lrv >> GICH_LR_STATE_SHIFT) & GICH_LR_STATE_MASK;
-    lr_reg->hw_status = (lrv >> GICH_LR_HW_SHIFT) & GICH_LR_HW_MASK;
-    lr_reg->grp       = (lrv >> GICH_LR_GRP_SHIFT) & GICH_LR_GRP_MASK;
+    lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
+    lr_reg->state     = (lrv >> ICH_LR_STATE_SHIFT) & ICH_LR_STATE_MASK;
+    lr_reg->hw_status = (lrv >> ICH_LR_HW_SHIFT) & ICH_LR_HW_MASK;
+    lr_reg->grp       = (lrv >> ICH_LR_GRP_SHIFT) & ICH_LR_GRP_MASK;
 }
 
 static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
 {
     uint64_t lrv = 0;
 
-    lrv = ( ((u64)(lr->pirq & GICH_LR_PHYSICAL_MASK) << GICH_LR_PHYSICAL_SHIFT)|
-        ((u64)(lr->virq & GICH_LR_VIRTUAL_MASK)  << GICH_LR_VIRTUAL_SHIFT) |
-        ((u64)(lr->priority & GICH_LR_PRIORITY_MASK) << GICH_LR_PRIORITY_SHIFT)|
-        ((u64)(lr->state & GICH_LR_STATE_MASK) << GICH_LR_STATE_SHIFT) |
-        ((u64)(lr->hw_status & GICH_LR_HW_MASK) << GICH_LR_HW_SHIFT)  |
-        ((u64)(lr->grp & GICH_LR_GRP_MASK) << GICH_LR_GRP_SHIFT) );
+    lrv = ( ((u64)(lr->pirq & ICH_LR_PHYSICAL_MASK) << ICH_LR_PHYSICAL_SHIFT)|
+        ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
+        ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT)|
+        ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) |
+        ((u64)(lr->hw_status & ICH_LR_HW_MASK) << ICH_LR_HW_SHIFT)  |
+        ((u64)(lr->grp & ICH_LR_GRP_MASK) << ICH_LR_GRP_SHIFT) );
 
     gicv3_ich_write_lr(lr_reg, lrv);
 }
@@ -1041,8 +1041,8 @@ static void gicv3_hcr_status(uint32_t flag, bool status)
 
 static unsigned int gicv3_read_vmcr_priority(void)
 {
-   return ((READ_SYSREG32(ICH_VMCR_EL2) >> GICH_VMCR_PRIORITY_SHIFT) &
-            GICH_VMCR_PRIORITY_MASK);
+   return ((READ_SYSREG32(ICH_VMCR_EL2) >> ICH_VMCR_PRIORITY_SHIFT) &
+            ICH_VMCR_PRIORITY_MASK);
 }
 
 /* Only support reading GRP1 APRn registers */
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index bb34d17eca..ccb72cf0f1 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -160,31 +160,30 @@
 #define LPI_PROP_RES1                (1 << 1)
 #define LPI_PROP_ENABLED             (1 << 0)
 
-#define GICH_VMCR_EOI                (1 << 9)
-#define GICH_VMCR_VENG1              (1 << 1)
-
-#define GICH_LR_VIRTUAL_MASK         0xffff
-#define GICH_LR_VIRTUAL_SHIFT        0
-#define GICH_LR_PHYSICAL_MASK        0x3ff
-#define GICH_LR_PHYSICAL_SHIFT       32
-#define GICH_LR_STATE_MASK           0x3
-#define GICH_LR_STATE_SHIFT          62
-#define GICH_LR_PRIORITY_MASK        0xff
-#define GICH_LR_PRIORITY_SHIFT       48
-#define GICH_LR_HW_MASK              0x1
-#define GICH_LR_HW_SHIFT             61
-#define GICH_LR_GRP_MASK             0x1
-#define GICH_LR_GRP_SHIFT            60
-#define GICH_LR_MAINTENANCE_IRQ      (1UL<<41)
-#define GICH_LR_GRP1                 (1UL<<60)
-#define GICH_LR_HW                   (1UL<<61)
-
-#define GICH_VTR_NRLRGS              0x3f
-#define GICH_VTR_PRIBITS_MASK        0x7
-#define GICH_VTR_PRIBITS_SHIFT       29
-
-#define GICH_VMCR_PRIORITY_MASK      0xff
-#define GICH_VMCR_PRIORITY_SHIFT     24
+#define ICH_VMCR_EOI                 (1 << 9)
+#define ICH_VMCR_VENG1               (1 << 1)
+#define ICH_VMCR_PRIORITY_MASK       0xff
+#define ICH_VMCR_PRIORITY_SHIFT      24
+
+#define ICH_LR_VIRTUAL_MASK          0xffff
+#define ICH_LR_VIRTUAL_SHIFT         0
+#define ICH_LR_PHYSICAL_MASK         0x3ff
+#define ICH_LR_PHYSICAL_SHIFT        32
+#define ICH_LR_STATE_MASK            0x3
+#define ICH_LR_STATE_SHIFT           62
+#define ICH_LR_PRIORITY_MASK         0xff
+#define ICH_LR_PRIORITY_SHIFT        48
+#define ICH_LR_HW_MASK               0x1
+#define ICH_LR_HW_SHIFT              61
+#define ICH_LR_GRP_MASK              0x1
+#define ICH_LR_GRP_SHIFT             60
+#define ICH_LR_MAINTENANCE_IRQ       (1UL<<41)
+#define ICH_LR_GRP1                  (1UL<<60)
+#define ICH_LR_HW                    (1UL<<61)
+
+#define ICH_VTR_NRLRGS               0x3f
+#define ICH_VTR_PRIBITS_MASK         0x7
+#define ICH_VTR_PRIBITS_SHIFT        29
 
 #define ICH_SGI_IRQMODE_SHIFT        40
 #define ICH_SGI_IRQMODE_MASK         0x1
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 14/17] ARM: Implement vcpu_kick()
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (12 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 13/17] ARM: GICv3: rename HYP interface definitions to use ICH_ prefix Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-12 11:41   ` Julien Grall
  2018-03-09 15:11 ` [PATCH 15/17] ARM: GICv2: introduce gicv2_poke_irq() Andre Przywara
                   ` (3 subsequent siblings)
  17 siblings, 1 reply; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

If we change something in a vCPU that affects its runnability or
otherwise needs the vCPU's attention, we might need to tell the scheduler
about it.
We are using this in one place (vIRQ injection) at the moment, but will
need this at more places soon.
So let's factor out this functionality, using the already existing
vcpu_kick() prototype (used in x86 only so far), to make this available
to the rest of the Xen code.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
- Rename to vcpu_kick(), to blend in with existing (x86) prototype

 xen/arch/arm/domain.c | 12 ++++++++++++
 xen/arch/arm/vgic.c   | 11 +++--------
 2 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index bc10f412ba..650712e0f2 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -967,6 +967,18 @@ void vcpu_block_unless_event_pending(struct vcpu *v)
         vcpu_unblock(current);
 }
 
+void vcpu_kick(struct vcpu *vcpu)
+{
+    bool running = vcpu->is_running;
+
+    vcpu_unblock(vcpu);
+    if ( running && vcpu != current )
+    {
+        perfc_incr(vgic_cross_cpu_intr_inject);
+        smp_send_event_check_mask(cpumask_of(vcpu->processor));
+    }
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index eb09d9ca54..3fafdd0b66 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -26,6 +26,7 @@
 #include <xen/sched.h>
 #include <xen/perfc.h>
 
+#include <asm/event.h>
 #include <asm/current.h>
 
 #include <asm/mmio.h>
@@ -530,7 +531,6 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
     uint8_t priority;
     struct pending_irq *iter, *n;
     unsigned long flags;
-    bool running;
 
     /*
      * For edge triggered interrupts we always ignore a "falling edge".
@@ -590,14 +590,9 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
     list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
 out:
     spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+
     /* we have a new higher priority irq, inject it into the guest */
-    running = v->is_running;
-    vcpu_unblock(v);
-    if ( running && v != current )
-    {
-        perfc_incr(vgic_cross_cpu_intr_inject);
-        smp_send_event_check_mask(cpumask_of(v->processor));
-    }
+    vcpu_kick(v);
 
     return;
 }
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 15/17] ARM: GICv2: introduce gicv2_poke_irq()
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (13 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 14/17] ARM: Implement vcpu_kick() Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 16/17] ARM: GICv3: poke_irq: make RWP optional Andre Przywara
                   ` (2 subsequent siblings)
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

The GICv2 uses bitmaps spanning several MMIO registers for holding some
interrupt state. Similar to GICv3, add a poke helper functions to set a bit
for a given irq_desc in one of those bitmaps.
At the moment there is only one use in gic-v2.c, but there will be more
coming soon.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/gic-v2.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 2b271ba322..fa9afc2be8 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -235,6 +235,11 @@ static unsigned int gicv2_read_irq(void)
     return (readl_gicc(GICC_IAR) & GICC_IA_IRQ);
 }
 
+static void gicv2_poke_irq(struct irq_desc *irqd, uint32_t offset)
+{
+    writel_gicd(1U << (irqd->irq % 32), offset + (irqd->irq / 32) * 4);
+}
+
 static void gicv2_set_irq_type(struct irq_desc *desc, unsigned int type)
 {
     uint32_t cfg, actual, edgebit;
@@ -509,7 +514,6 @@ static unsigned int gicv2_read_apr(int apr_reg)
 static void gicv2_irq_enable(struct irq_desc *desc)
 {
     unsigned long flags;
-    int irq = desc->irq;
 
     ASSERT(spin_is_locked(&desc->lock));
 
@@ -517,20 +521,19 @@ static void gicv2_irq_enable(struct irq_desc *desc)
     clear_bit(_IRQ_DISABLED, &desc->status);
     dsb(sy);
     /* Enable routing */
-    writel_gicd((1u << (irq % 32)), GICD_ISENABLER + (irq / 32) * 4);
+    gicv2_poke_irq(desc, GICD_ISENABLER);
     spin_unlock_irqrestore(&gicv2.lock, flags);
 }
 
 static void gicv2_irq_disable(struct irq_desc *desc)
 {
     unsigned long flags;
-    int irq = desc->irq;
 
     ASSERT(spin_is_locked(&desc->lock));
 
     spin_lock_irqsave(&gicv2.lock, flags);
     /* Disable routing */
-    writel_gicd(1u << (irq % 32), GICD_ICENABLER + (irq / 32) * 4);
+    gicv2_poke_irq(desc, GICD_ICENABLER);
     set_bit(_IRQ_DISABLED, &desc->status);
     spin_unlock_irqrestore(&gicv2.lock, flags);
 }
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 16/17] ARM: GICv3: poke_irq: make RWP optional
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (14 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 15/17] ARM: GICv2: introduce gicv2_poke_irq() Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-09 15:11 ` [PATCH 17/17] ARM: GICv2: fix GICH_V2_LR definitions Andre Przywara
  2018-03-12 12:08 ` [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Julien Grall
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

A GICv3 hardware implementation can be implemented in several parts that
communicate with each other (think multi-socket systems).
To make sure that critical settings have arrived at all endpoints, some
bits are tracked using the RWP bit in the GICD_CTLR register, which
signals whether a register write is still in progress.
However this only applies to *some* registers, namely the bits in the
GICD_ICENABLER (disabling interrupts) and some bits in the GICD_CTLR
register (cf. Arm IHI 0069D, 8.9.4: RWP, bit[31]).
But our gicv3_poke_irq() was always polling this bit before returning,
resulting in pointless MMIO reads for many registers.
Add an option to gicv3_poke_irq() to state whether we want to wait for
this bit and use it accordingly to match the spec.
Replace a "1 << " with a "1U << " on the way to fix a potentially
undefined behaviour when the argument evaluates to 31.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/gic-v3.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 8b41704cf1..09b49a07d5 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -428,9 +428,9 @@ static void gicv3_dump_state(const struct vcpu *v)
     }
 }
 
-static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset)
+static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset, bool wait_for_rwp)
 {
-    u32 mask = 1 << (irqd->irq % 32);
+    u32 mask = 1U << (irqd->irq % 32);
     void __iomem *base;
 
     if ( irqd->irq < NR_GIC_LOCAL_IRQS )
@@ -439,17 +439,19 @@ static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset)
         base = GICD;
 
     writel_relaxed(mask, base + offset + (irqd->irq / 32) * 4);
-    gicv3_wait_for_rwp(irqd->irq);
+
+    if ( wait_for_rwp )
+        gicv3_wait_for_rwp(irqd->irq);
 }
 
 static void gicv3_unmask_irq(struct irq_desc *irqd)
 {
-    gicv3_poke_irq(irqd, GICD_ISENABLER);
+    gicv3_poke_irq(irqd, GICD_ISENABLER, false);
 }
 
 static void gicv3_mask_irq(struct irq_desc *irqd)
 {
-    gicv3_poke_irq(irqd, GICD_ICENABLER);
+    gicv3_poke_irq(irqd, GICD_ICENABLER, true);
 }
 
 static void gicv3_eoi_irq(struct irq_desc *irqd)
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 17/17] ARM: GICv2: fix GICH_V2_LR definitions
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (15 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 16/17] ARM: GICv3: poke_irq: make RWP optional Andre Przywara
@ 2018-03-09 15:11 ` Andre Przywara
  2018-03-12 12:08 ` [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Julien Grall
  17 siblings, 0 replies; 24+ messages in thread
From: Andre Przywara @ 2018-03-09 15:11 UTC (permalink / raw)
  To: Stefano Stabellini, Julien Grall; +Cc: xen-devel

The bit definition for the CPUID mask in the GICv2 LR register was
wrong, fortunately the current implementation does not use that bit.
Fix it up (it's starting at bit 10, not bit 9) and clean up some
nearby definitions on the way.
This will be used by the new VGIC shortly.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
 xen/arch/arm/gic-v2.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index fa9afc2be8..f16e17c1a3 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -57,10 +57,11 @@
 #define GICH_V2_LR_HW_MASK         0x1
 #define GICH_V2_LR_GRP_SHIFT       30
 #define GICH_V2_LR_GRP_MASK        0x1
-#define GICH_V2_LR_MAINTENANCE_IRQ (1<<19)
-#define GICH_V2_LR_GRP1            (1<<30)
-#define GICH_V2_LR_HW              (1<<31)
-#define GICH_V2_LR_CPUID_SHIFT     9
+#define GICH_V2_LR_MAINTENANCE_IRQ (1U << 19)
+#define GICH_V2_LR_GRP1            (1U << 30)
+#define GICH_V2_LR_HW              (1U << GICH_V2_LR_HW_SHIFT)
+#define GICH_V2_LR_CPUID_SHIFT     10
+#define GICH_V2_LR_CPUID_MASK      0x7
 #define GICH_V2_VTR_NRLRGS         0x3f
 
 #define GICH_V2_VMCR_PRIORITY_MASK   0x1f
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH 03/17] ARM: vGICv3: always use architected redist stride
  2018-03-09 15:11 ` [PATCH 03/17] ARM: vGICv3: always use architected redist stride Andre Przywara
@ 2018-03-12 11:08   ` Julien Grall
  0 siblings, 0 replies; 24+ messages in thread
From: Julien Grall @ 2018-03-12 11:08 UTC (permalink / raw)
  To: Andre Przywara, Stefano Stabellini; +Cc: xen-devel

Hi Andre,

On 09/03/18 15:11, Andre Przywara wrote:
> The redistributor-stride property in a GICv3 DT node is only there to
> cover broken platforms where this value deviates from the architected one.
> Since we emulate the GICv3 distributor even for Dom0, we don't need to
> copy the broken behaviour. All the special handling for Dom0s using
> GICv3 is just for using the hardware's memory map, which is unaffected
> by the redistributor stride - it can never be smaller than the
> architected two pages.
> Remove the redistributor-stride property from Dom0's DT node and also
> remove the code that tried to reuse the hardware value for Dom0's GICv3
> emulation.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog:
> - merge in GICV3_GICR_SIZE definition
> 
>   xen/arch/arm/gic-v3.c             |  4 ----
>   xen/arch/arm/vgic-v3.c            | 14 ++++++--------
>   xen/include/asm-arm/gic_v3_defs.h |  5 +++++
>   3 files changed, 11 insertions(+), 12 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index b1f8a86409..047af691b1 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1162,10 +1162,6 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
>       if ( res )
>           return res;
>   
> -    res = fdt_property_cell(fdt, "redistributor-stride", gicv3.rdist_stride);
> -    if ( res )
> -        return res;
> -
>       res = fdt_property_cell(fdt, "#redistributor-regions", gicv3.rdist_count);
>       if ( res )
>           return res;
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index d5b34a7d0f..56cc38ffcc 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1024,10 +1024,9 @@ static struct vcpu *get_vcpu_from_rdist(struct domain *d,
>       paddr_t gpa, uint32_t *offset)
>   {
>       struct vcpu *v;
> -    uint32_t stride = d->arch.vgic.rdist_stride;
>       unsigned int vcpu_id;
>   
> -    vcpu_id = region->first_cpu + ((gpa - region->base) / stride);
> +    vcpu_id = region->first_cpu + ((gpa - region->base) / GICV3_GICR_SIZE);
>       if ( unlikely(vcpu_id >= d->max_vcpus) )
>           return NULL;
>   
> @@ -1586,7 +1585,6 @@ static int vgic_v3_vcpu_init(struct vcpu *v)
>   
>       /* Convenient alias */
>       struct domain *d = v->domain;
> -    uint32_t rdist_stride = d->arch.vgic.rdist_stride;
>   
>       /*
>        * Find the region where the re-distributor lives. For this purpose,
> @@ -1602,11 +1600,11 @@ static int vgic_v3_vcpu_init(struct vcpu *v)
>   
>       /* Get the base address of the redistributor */
>       rdist_base = region->base;
> -    rdist_base += (v->vcpu_id - region->first_cpu) * rdist_stride;
> +    rdist_base += (v->vcpu_id - region->first_cpu) * GICV3_GICR_SIZE;
>   
>       /* Check if a valid region was found for the re-distributor */
>       if ( (rdist_base < region->base) ||
> -         ((rdist_base + rdist_stride) > (region->base + region->size)) )
> +         ((rdist_base + GICV3_GICR_SIZE) > (region->base + region->size)) )
>       {
>           dprintk(XENLOG_ERR,
>                   "d%u: Unable to find a re-distributor for VCPU %u\n",
> @@ -1622,7 +1620,7 @@ static int vgic_v3_vcpu_init(struct vcpu *v)
>        * VGIC_V3_RDIST_LAST flags.
>        * Note that we are assuming max_vcpus will never change.
>        */
> -    last_cpu = (region->size / rdist_stride) + region->first_cpu - 1;
> +    last_cpu = (region->size / GICV3_GICR_SIZE) + region->first_cpu - 1;
>   
>       if ( v->vcpu_id == last_cpu || (v->vcpu_id == (d->max_vcpus - 1)) )
>           v->arch.vgic.flags |= VGIC_V3_RDIST_LAST;
> @@ -1693,7 +1691,7 @@ static int vgic_v3_domain_init(struct domain *d)
>               /* Set the first CPU handled by this region */
>               d->arch.vgic.rdist_regions[i].first_cpu = first_cpu;
>   
> -            first_cpu += size / d->arch.vgic.rdist_stride;
> +            first_cpu += size / GICV3_GICR_SIZE;
>           }
>   
>           d->arch.vgic.intid_bits = vgic_v3_hw.intid_bits;
> @@ -1708,7 +1706,7 @@ static int vgic_v3_domain_init(struct domain *d)
>           d->arch.vgic.rdist_stride = GUEST_GICV3_RDIST_STRIDE;
>   
>           /* The first redistributor should contain enough space for all CPUs */
> -        BUILD_BUG_ON((GUEST_GICV3_GICR0_SIZE / GUEST_GICV3_RDIST_STRIDE) < MAX_VIRT_CPUS);
> +        BUILD_BUG_ON((GUEST_GICV3_GICR0_SIZE / GICV3_GICR_SIZE) < MAX_VIRT_CPUS);
>           d->arch.vgic.rdist_regions[0].base = GUEST_GICV3_GICR0_BASE;
>           d->arch.vgic.rdist_regions[0].size = GUEST_GICV3_GICR0_SIZE;
>           d->arch.vgic.rdist_regions[0].first_cpu = 0;
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 65c9dc47cf..bb34d17eca 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -18,6 +18,8 @@
>   #ifndef __ASM_ARM_GIC_V3_DEFS_H__
>   #define __ASM_ARM_GIC_V3_DEFS_H__
>   
> +#include <xen/sizes.h>
> +
>   /*
>    * Additional registers defined in GIC v3.
>    * Common GICD registers are defined in gic.h
> @@ -68,6 +70,9 @@
>   #define GICV3_GICD_IIDR_VAL          0x34c
>   #define GICV3_GICR_IIDR_VAL          GICV3_GICD_IIDR_VAL
>   
> +/* Two pages for the RD_base and SGI_base register frame. */
> +#define GICV3_GICR_SIZE              (2 * SZ_64K)
> +
>   #define GICR_CTLR                    (0x0000)
>   #define GICR_IIDR                    (0x0004)
>   #define GICR_TYPER                   (0x0008)
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 07/17] ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific
  2018-03-09 15:11 ` [PATCH 07/17] ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific Andre Przywara
@ 2018-03-12 11:09   ` Julien Grall
  0 siblings, 0 replies; 24+ messages in thread
From: Julien Grall @ 2018-03-12 11:09 UTC (permalink / raw)
  To: Andre Przywara, Stefano Stabellini; +Cc: xen-devel

Hi Andre,

On 09/03/18 15:11, Andre Przywara wrote:
> domain_max_vcpus(), which is used by generic Xen code, returns the
> maximum number of VCPUs for a domain, which on ARM is mostly limited by
> the VGIC model emulated (a (v)GICv2 can only handle 8 CPUs).
> Our current implementation lives in arch/arm/domain.c, but reaches into
> VGIC internal data structures.
> Move the actual functionality into vgic.c, and provide a shim in
> domain.h, to keep this VGIC internal.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog:
> - rename helper function and wrap in domain.h
> 
>   xen/arch/arm/domain.c        | 14 --------------
>   xen/arch/arm/vgic.c          | 14 ++++++++++++++
>   xen/include/asm-arm/domain.h |  6 +++++-
>   xen/include/asm-arm/vgic.h   |  2 ++
>   4 files changed, 21 insertions(+), 15 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 8de4c0a343..6b902fa30f 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -967,20 +967,6 @@ void vcpu_block_unless_event_pending(struct vcpu *v)
>           vcpu_unblock(current);
>   }
>   
> -unsigned int domain_max_vcpus(const struct domain *d)
> -{
> -    /*
> -     * Since evtchn_init would call domain_max_vcpus for poll_mask
> -     * allocation when the vgic_ops haven't been initialised yet,
> -     * we return MAX_VIRT_CPUS if d->arch.vgic.handler is null.
> -     */
> -    if ( !d->arch.vgic.handler )
> -        return MAX_VIRT_CPUS;
> -    else
> -        return min_t(unsigned int, MAX_VIRT_CPUS,
> -                     d->arch.vgic.handler->max_vcpus);
> -}
> -
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 34269bcf27..fa00c21a69 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -665,6 +665,20 @@ void vgic_free_virq(struct domain *d, unsigned int virq)
>       clear_bit(virq, d->arch.vgic.allocated_irqs);
>   }
>   
> +unsigned int vgic_max_vcpus(const struct domain *d)
> +{
> +    /*
> +     * Since evtchn_init would call domain_max_vcpus for poll_mask
> +     * allocation when the vgic_ops haven't been initialised yet,
> +     * we return MAX_VIRT_CPUS if d->arch.vgic.handler is null.
> +     */
> +    if ( !d->arch.vgic.handler )
> +        return MAX_VIRT_CPUS;
> +    else
> +        return min_t(unsigned int, MAX_VIRT_CPUS,
> +                     d->arch.vgic.handler->max_vcpus);
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c6aa5cf389..e730e07fcf 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -289,7 +289,11 @@ void vcpu_show_execution_state(struct vcpu *);
>   void vcpu_show_registers(const struct vcpu *);
>   void vcpu_switch_to_aarch64_mode(struct vcpu *);
>   
> -unsigned int domain_max_vcpus(const struct domain *);
> +/* On ARM, the number of VCPUs is limited by the type of GIC emulated. */
> +static inline unsigned int domain_max_vcpus(const struct domain *d)
> +{
> +    return vgic_max_vcpus(d);
> +}
>   
>   /*
>    * Due to the restriction of GICv3, the number of vCPUs in AFF0 is
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index d03298e12c..afb4776ad4 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -254,6 +254,8 @@ static inline int vgic_allocate_spi(struct domain *d)
>   
>   extern void vgic_free_virq(struct domain *d, unsigned int virq);
>   
> +unsigned int vgic_max_vcpus(const struct domain *d);
> +
>   void vgic_v2_setup_hw(paddr_t dbase, paddr_t cbase, paddr_t csize,
>                         paddr_t vbase, uint32_t aliased_offset);
>   
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 08/17] ARM: VGIC: rename gic_event_needs_delivery()
  2018-03-09 15:11 ` [PATCH 08/17] ARM: VGIC: rename gic_event_needs_delivery() Andre Przywara
@ 2018-03-12 11:10   ` Julien Grall
  0 siblings, 0 replies; 24+ messages in thread
From: Julien Grall @ 2018-03-12 11:10 UTC (permalink / raw)
  To: Andre Przywara, Stefano Stabellini; +Cc: xen-devel

Hi,

On 09/03/18 15:11, Andre Przywara wrote:
> gic_event_needs_delivery() is not named very intuitively, especially
> the gic_ prefix is somewhat misleading.
> Rename it to vgic_pending_irq(), which makes it clear that this relates > to the virtual GIC and is about interrupts.

It looks like the commit message hasn't been updated. Also, could you 
mention in the commit message that you added a new parameter.

Cheers,

> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> Changelog:
> - Add vcpu parameter
> - Rename to vgic_vcpu_pending_irq()
> 
>   xen/arch/arm/gic-vgic.c     | 16 ++++++++++++++--
>   xen/include/asm-arm/event.h |  2 +-
>   xen/include/asm-arm/gic.h   |  2 +-
>   3 files changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> index c0fe38fd37..f4c98bffd1 100644
> --- a/xen/arch/arm/gic-vgic.c
> +++ b/xen/arch/arm/gic-vgic.c
> @@ -339,9 +339,18 @@ void gic_clear_pending_irqs(struct vcpu *v)
>           gic_remove_from_lr_pending(v, p);
>   }
>   
> -int gic_events_need_delivery(void)
> +/**
> + * vgic_vcpu_pending_irq() - determine if interrupts need to be injected
> + * @vcpu: The vCPU on which to check for interrupts.
> + *
> + * Checks whether there is an interrupt on the given VCPU which needs
> + * handling in the guest. This requires at least one IRQ to be pending
> + * and enabled.
> + *
> + * Returns: 1 if the guest should run to handle interrupts, 0 otherwise.
> + */
> +int vgic_vcpu_pending_irq(struct vcpu *v)
>   {
> -    struct vcpu *v = current;
>       struct pending_irq *p;
>       unsigned long flags;
>       const unsigned long apr = gic_hw_ops->read_apr(0);
> @@ -349,6 +358,9 @@ int gic_events_need_delivery(void)
>       int active_priority;
>       int rc = 0;
>   
> +    /* We rely on reading the VMCR, which is only accessible locally. */
> +    ASSERT(v == current);
> +
>       mask_priority = gic_hw_ops->read_vmcr_priority();
>       active_priority = find_next_bit(&apr, 32, 0);
>   
> diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
> index e8c2a6cb44..c7a415ef57 100644
> --- a/xen/include/asm-arm/event.h
> +++ b/xen/include/asm-arm/event.h
> @@ -24,7 +24,7 @@ static inline int local_events_need_delivery_nomask(void)
>        * interrupts disabled so this shouldn't be a problem in the general
>        * case.
>        */
> -    if ( gic_events_need_delivery() )
> +    if ( vgic_vcpu_pending_irq(current) )
>           return 1;
>   
>       if ( !vcpu_info(current, evtchn_upcall_pending) )
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 3b2d0217a6..ff0b22451b 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -238,7 +238,7 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
>   
>   extern void vgic_sync_to_lrs(void);
>   extern void gic_clear_pending_irqs(struct vcpu *v);
> -extern int gic_events_need_delivery(void);
> +extern int vgic_vcpu_pending_irq(struct vcpu *v);
>   
>   extern void init_maintenance_interrupt(void);
>   extern void gic_raise_guest_irq(struct vcpu *v, unsigned int irq,
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 09/17] ARM: VGIC: change to level-IRQ compatible IRQ injection interface
  2018-03-09 15:11 ` [PATCH 09/17] ARM: VGIC: change to level-IRQ compatible IRQ injection interface Andre Przywara
@ 2018-03-12 11:29   ` Julien Grall
  0 siblings, 0 replies; 24+ messages in thread
From: Julien Grall @ 2018-03-12 11:29 UTC (permalink / raw)
  To: Andre Przywara, Stefano Stabellini; +Cc: xen-devel

Hi,

On 09/03/18 15:11, Andre Przywara wrote:
> At the moment vgic_vcpu_inject_irq() is the interface for Xen internal
> code and virtual devices to inject IRQs into a guest. This interface has
> two shortcomings:
> 1) It requires a VCPU pointer, which we may not know (and don't need!)
> for shared interrupts. A second function (vgic_vcpu_inject_spi()), was
> there to work around this issue.
> 2) This interface only really supports edge triggered IRQs, which is
> what the Xen VGIC emulates only anyway. However this needs to and will
> change, so we need to add the desired level (high or low) to the
> interface.
> This replaces the existing injection call (taking a VCPU and an IRQ
> parameter) with a new one, taking domain, VCPU, IRQ and level parameters.
> The VCPU can be NULL in case we don't know and don't care.
> We change all call sites to use this new interface. This still doesn't
> give us the missing level IRQ handling, but at least prepares the callers
> to do the right thing later automatically.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog:
> - keep function as returning void
> 
>   xen/arch/arm/domain.c      |  4 ++--
>   xen/arch/arm/gic-v3-lpi.c  |  2 +-
>   xen/arch/arm/irq.c         |  2 +-
>   xen/arch/arm/time.c        |  2 +-
>   xen/arch/arm/vgic.c        | 39 +++++++++++++++++++++++----------------
>   xen/arch/arm/vpl011.c      |  2 +-
>   xen/arch/arm/vtimer.c      |  4 ++--
>   xen/include/asm-arm/vgic.h |  4 ++--
>   8 files changed, 33 insertions(+), 26 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 6b902fa30f..bc10f412ba 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -951,14 +951,14 @@ void vcpu_mark_events_pending(struct vcpu *v)
>       if ( already_pending )
>           return;
>   
> -    vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
> +    vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
>   }
>   
>   /* The ARM spec declares that even if local irqs are masked in
>    * the CPSR register, an irq should wake up a cpu from WFI anyway.
>    * For this reason we need to check for irqs that need delivery,
>    * ignoring the CPSR register, *after* calling SCHEDOP_block to
> - * avoid races with vgic_vcpu_inject_irq.
> + * avoid races with vgic_inject_irq.
>    */
>   void vcpu_block_unless_event_pending(struct vcpu *v)
>   {
> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
> index 84582157b8..efd5cd62fb 100644
> --- a/xen/arch/arm/gic-v3-lpi.c
> +++ b/xen/arch/arm/gic-v3-lpi.c
> @@ -153,7 +153,7 @@ void vgic_vcpu_inject_lpi(struct domain *d, unsigned int virq)
>       if ( vcpu_id >= d->max_vcpus )
>             return;
>   
> -    vgic_vcpu_inject_irq(d->vcpu[vcpu_id], virq);
> +    vgic_inject_irq(d, d->vcpu[vcpu_id], virq, true);
>   }
>   
>   /*
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 29af10e82c..aa4e832cae 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -225,7 +225,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>            * The irq cannot be a PPI, we only support delivery of SPIs to
>            * guests.
>   	 */
> -        vgic_vcpu_inject_spi(info->d, info->virq);
> +        vgic_inject_irq(info->d, NULL, info->virq, true);
>           goto out_no_end;
>       }
>   
> diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c
> index 36f640f0c1..c11fcfeadd 100644
> --- a/xen/arch/arm/time.c
> +++ b/xen/arch/arm/time.c
> @@ -260,7 +260,7 @@ static void vtimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
>   
>       current->arch.virt_timer.ctl = READ_SYSREG32(CNTV_CTL_EL0);
>       WRITE_SYSREG32(current->arch.virt_timer.ctl | CNTx_CTL_MASK, CNTV_CTL_EL0);
> -    vgic_vcpu_inject_irq(current, current->arch.virt_timer.irq);
> +    vgic_inject_irq(current->domain, current, current->arch.virt_timer.irq, true);
>   }
>   
>   /*
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index fa00c21a69..eb09d9ca54 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -291,7 +291,7 @@ bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq)
>           vgic_remove_irq_from_queues(old, p);
>           irq_set_affinity(p->desc, cpumask_of(new->processor));
>           spin_unlock_irqrestore(&old->arch.vgic.lock, flags);
> -        vgic_vcpu_inject_irq(new, irq);
> +        vgic_inject_irq(new->domain, new, irq, true);
>           return true;
>       }
>       /* if the IRQ is in a GICH_LR register, set GIC_IRQ_GUEST_MIGRATING
> @@ -450,7 +450,7 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode,
>                           sgir, target->list);
>                   continue;
>               }
> -            vgic_vcpu_inject_irq(d->vcpu[vcpuid], virq);
> +            vgic_inject_irq(d, d->vcpu[vcpuid], virq, true);
>           }
>           break;
>       case SGI_TARGET_OTHERS:
> @@ -459,12 +459,12 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode,
>           {
>               if ( i != current->vcpu_id && d->vcpu[i] != NULL &&
>                    is_vcpu_online(d->vcpu[i]) )
> -                vgic_vcpu_inject_irq(d->vcpu[i], virq);
> +                vgic_inject_irq(d, d->vcpu[i], virq, true);
>           }
>           break;
>       case SGI_TARGET_SELF:
>           perfc_incr(vgic_sgi_self);
> -        vgic_vcpu_inject_irq(d->vcpu[current->vcpu_id], virq);
> +        vgic_inject_irq(d, current, virq, true);
>           break;
>       default:
>           gprintk(XENLOG_WARNING,
> @@ -524,13 +524,29 @@ void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p)
>       gic_remove_from_lr_pending(v, p);
>   }
>   
> -void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
> +void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
> +                     bool level)
>   {
>       uint8_t priority;
>       struct pending_irq *iter, *n;
>       unsigned long flags;
>       bool running;
>   
> +    /*
> +     * For edge triggered interrupts we always ignore a "falling edge".
> +     * For level triggered interrupts we shouldn't, but do anyways.
> +     */
> +    if ( !level )
> +        return;
> +
> +    if ( !v )
> +    {
> +        /* The IRQ needs to be an SPI if no vCPU is specified. */
> +        ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
> +
> +        v = vgic_get_target_vcpu(d->vcpu[0], virq);
> +    };
> +
>       spin_lock_irqsave(&v->arch.vgic.lock, flags);
>   
>       n = irq_to_pending(v, virq);
> @@ -582,22 +598,13 @@ out:
>           perfc_incr(vgic_cross_cpu_intr_inject);
>           smp_send_event_check_mask(cpumask_of(v->processor));
>       }
> -}
> -
> -void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
> -{
> -    struct vcpu *v;
> -
> -    /* the IRQ needs to be an SPI */
> -    ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
>   
> -    v = vgic_get_target_vcpu(d->vcpu[0], virq);
> -    vgic_vcpu_inject_irq(v, virq);
> +    return;
>   }
>   
>   void arch_evtchn_inject(struct vcpu *v)
>   {
> -    vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
> +    vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
>   }
>   
>   bool vgic_evtchn_irq_pending(struct vcpu *v)
> diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> index 7788c2fc32..5dcf4bec18 100644
> --- a/xen/arch/arm/vpl011.c
> +++ b/xen/arch/arm/vpl011.c
> @@ -68,7 +68,7 @@ static void vpl011_update_interrupt_status(struct domain *d)
>        * status bit has been set since the last time.
>        */
>       if ( uartmis & ~vpl011->shadow_uartmis )
> -        vgic_vcpu_inject_spi(d, GUEST_VPL011_SPI);
> +        vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, true);
>   
>       vpl011->shadow_uartmis = uartmis;
>   }
> diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
> index f52a723a5f..8164f6c7f1 100644
> --- a/xen/arch/arm/vtimer.c
> +++ b/xen/arch/arm/vtimer.c
> @@ -46,7 +46,7 @@ static void phys_timer_expired(void *data)
>       if ( !(t->ctl & CNTx_CTL_MASK) )
>       {
>           perfc_incr(vtimer_phys_inject);
> -        vgic_vcpu_inject_irq(t->v, t->irq);
> +        vgic_inject_irq(t->v->domain, t->v, t->irq, true);
>       }
>       else
>           perfc_incr(vtimer_phys_masked);
> @@ -56,7 +56,7 @@ static void virt_timer_expired(void *data)
>   {
>       struct vtimer *t = data;
>       t->ctl |= CNTx_CTL_MASK;
> -    vgic_vcpu_inject_irq(t->v, t->irq);
> +    vgic_inject_irq(t->v->domain, t->v, t->irq, true);
>       perfc_incr(vtimer_virt_inject);
>   }
>   
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index afb4776ad4..8af6d816c9 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -202,8 +202,8 @@ extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
>   extern void domain_vgic_free(struct domain *d);
>   extern int vcpu_vgic_init(struct vcpu *v);
>   extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq);
> -extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
> -extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
> +extern void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
> +                            bool level);
>   extern void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p);
>   extern void gic_remove_from_lr_pending(struct vcpu *v, struct pending_irq *p);
>   extern void vgic_clear_pending_irqs(struct vcpu *v);
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 14/17] ARM: Implement vcpu_kick()
  2018-03-09 15:11 ` [PATCH 14/17] ARM: Implement vcpu_kick() Andre Przywara
@ 2018-03-12 11:41   ` Julien Grall
  0 siblings, 0 replies; 24+ messages in thread
From: Julien Grall @ 2018-03-12 11:41 UTC (permalink / raw)
  To: Andre Przywara, Stefano Stabellini; +Cc: xen-devel

Hi Andre,

On 09/03/18 15:11, Andre Przywara wrote:
> If we change something in a vCPU that affects its runnability or
> otherwise needs the vCPU's attention, we might need to tell the scheduler
> about it.
> We are using this in one place (vIRQ injection) at the moment, but will
> need this at more places soon.
> So let's factor out this functionality, using the already existing
> vcpu_kick() prototype (used in x86 only so far), to make this available
> to the rest of the Xen code.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> - Rename to vcpu_kick(), to blend in with existing (x86) prototype
> 
>   xen/arch/arm/domain.c | 12 ++++++++++++
>   xen/arch/arm/vgic.c   | 11 +++--------
>   2 files changed, 15 insertions(+), 8 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index bc10f412ba..650712e0f2 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -967,6 +967,18 @@ void vcpu_block_unless_event_pending(struct vcpu *v)
>           vcpu_unblock(current);
>   }
>   
> +void vcpu_kick(struct vcpu *vcpu)
> +{
> +    bool running = vcpu->is_running;
> +
> +    vcpu_unblock(vcpu);
> +    if ( running && vcpu != current )
> +    {
> +        perfc_incr(vgic_cross_cpu_intr_inject);

Sorry, it looks like I forgot to mention that in the previous version. 
The perfcounter should at least be renamed to match the new function.

> +        smp_send_event_check_mask(cpumask_of(vcpu->processor));
> +    }
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index eb09d9ca54..3fafdd0b66 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -26,6 +26,7 @@
>   #include <xen/sched.h>
>   #include <xen/perfc.h>
>   
> +#include <asm/event.h>
>   #include <asm/current.h>
>   
>   #include <asm/mmio.h>
> @@ -530,7 +531,6 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>       uint8_t priority;
>       struct pending_irq *iter, *n;
>       unsigned long flags;
> -    bool running;
>   
>       /*
>        * For edge triggered interrupts we always ignore a "falling edge".
> @@ -590,14 +590,9 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>       list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
>   out:
>       spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> +
>       /* we have a new higher priority irq, inject it into the guest */
> -    running = v->is_running;
> -    vcpu_unblock(v);
> -    if ( running && v != current )
> -    {
> -        perfc_incr(vgic_cross_cpu_intr_inject);
> -        smp_send_event_check_mask(cpumask_of(v->processor));
> -    }
> +    vcpu_kick(v);
>   
>       return;
>   }
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code
  2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
                   ` (16 preceding siblings ...)
  2018-03-09 15:11 ` [PATCH 17/17] ARM: GICv2: fix GICH_V2_LR definitions Andre Przywara
@ 2018-03-12 12:08 ` Julien Grall
  17 siblings, 0 replies; 24+ messages in thread
From: Julien Grall @ 2018-03-12 12:08 UTC (permalink / raw)
  To: Andre Przywara, Stefano Stabellini; +Cc: xen-devel

Hi,

I have committed all the patches except #8 and #14 as I had some 
comments on them.

45 patches to go before finishing the vGIC :).

Thank you,

On 09/03/18 15:11, Andre Przywara wrote:
> To get away from that scary xx/57 number in the new vGIC series, these
> are the first few patches split off. They prepare the existing Xen code
> to better observe the split between the vGIC emulation and the physical
> GIC driver.
> This affects the first 20 patches from the new vGIC series. Most of them
> have already been ACKed. Exceptions are:
> - Patch 01/57 has been merged already.
> - Patch 04/57 has been dropped, the one #define left has been moved into
>    patch 03/17 here. I dropped Julien's ACK because of that.
> - Patch 15/57 has been dropped, we don't need it anymore in the new vGIC.
> - Patches 07, 08, 09 and 14/17 were changed according to the review comments on
>    patches 09, 10, 11 and 17/57, respectively.
> 
> Every patch has been compile tested on arm and arm64, also the end result
> has been briefly tested with two guests on a Juno(arm64) and Midway(arm)
> platform.
> 
> A branch with those patches is available at the new-vgic/prep branch at:
> git://linux-arm.org/xen-ap.git
> http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/vgic-new/prep
> 
> Cheers,
> Andre
> 
> Andre Przywara (17):
>    ARM: vGICv3: clarify on GUEST_GICV3_RDIST_REGIONS symbol
>    ARM: GICv3: use hardware GICv3 redistributor values for Dom0
>    ARM: vGICv3: always use architected redist stride
>    ARM: vGICv3: remove rdist_stride from VGIC structure
>    ARM: VGIC: rename gic_inject() and gic_clear_lrs()
>    ARM: VGIC: Move gic_remove_from_lr_pending() prototype
>    ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific
>    ARM: VGIC: rename gic_event_needs_delivery()
>    ARM: VGIC: change to level-IRQ compatible IRQ injection interface
>    ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist
>    ARM: VGIC: reorder prototypes in vgic.h
>    ARM: VGIC: Introduce gic_get_nr_lrs()
>    ARM: GICv3: rename HYP interface definitions to use ICH_ prefix
>    ARM: Implement vcpu_kick()
>    ARM: GICv2: introduce gicv2_poke_irq()
>    ARM: GICv3: poke_irq: make RWP optional
>    ARM: GICv2: fix GICH_V2_LR definitions
> 
>   xen/arch/arm/domain.c             |  24 +++----
>   xen/arch/arm/gic-v2.c             |  20 +++---
>   xen/arch/arm/gic-v3-lpi.c         |   2 +-
>   xen/arch/arm/gic-v3.c             |  82 ++++++++++-----------
>   xen/arch/arm/gic-vgic.c           |  30 +++++---
>   xen/arch/arm/irq.c                |   2 +-
>   xen/arch/arm/time.c               |   2 +-
>   xen/arch/arm/traps.c              |   4 +-
>   xen/arch/arm/vgic-v3.c            |  40 +++++------
>   xen/arch/arm/vgic.c               |  64 ++++++++++-------
>   xen/arch/arm/vpl011.c             |   2 +-
>   xen/arch/arm/vtimer.c             |   4 +-
>   xen/include/asm-arm/domain.h      |  93 +++---------------------
>   xen/include/asm-arm/event.h       |   2 +-
>   xen/include/asm-arm/gic.h         |  14 ++--
>   xen/include/asm-arm/gic_v3_defs.h |  54 +++++++-------
>   xen/include/asm-arm/vgic.h        | 146 +++++++++++++++++++++++++++++++-------
>   xen/include/public/arch-arm.h     |   1 -
>   18 files changed, 315 insertions(+), 271 deletions(-)
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2018-03-12 12:08 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-09 15:11 [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Andre Przywara
2018-03-09 15:11 ` [PATCH 01/17] ARM: vGICv3: clarify on GUEST_GICV3_RDIST_REGIONS symbol Andre Przywara
2018-03-09 15:11 ` [PATCH 02/17] ARM: GICv3: use hardware GICv3 redistributor values for Dom0 Andre Przywara
2018-03-09 15:11 ` [PATCH 03/17] ARM: vGICv3: always use architected redist stride Andre Przywara
2018-03-12 11:08   ` Julien Grall
2018-03-09 15:11 ` [PATCH 04/17] ARM: vGICv3: remove rdist_stride from VGIC structure Andre Przywara
2018-03-09 15:11 ` [PATCH 05/17] ARM: VGIC: rename gic_inject() and gic_clear_lrs() Andre Przywara
2018-03-09 15:11 ` [PATCH 06/17] ARM: VGIC: Move gic_remove_from_lr_pending() prototype Andre Przywara
2018-03-09 15:11 ` [PATCH 07/17] ARM: VGIC: Adjust domain_max_vcpus() to be VGIC specific Andre Przywara
2018-03-12 11:09   ` Julien Grall
2018-03-09 15:11 ` [PATCH 08/17] ARM: VGIC: rename gic_event_needs_delivery() Andre Przywara
2018-03-12 11:10   ` Julien Grall
2018-03-09 15:11 ` [PATCH 09/17] ARM: VGIC: change to level-IRQ compatible IRQ injection interface Andre Przywara
2018-03-12 11:29   ` Julien Grall
2018-03-09 15:11 ` [PATCH 10/17] ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist Andre Przywara
2018-03-09 15:11 ` [PATCH 11/17] ARM: VGIC: reorder prototypes in vgic.h Andre Przywara
2018-03-09 15:11 ` [PATCH 12/17] ARM: VGIC: Introduce gic_get_nr_lrs() Andre Przywara
2018-03-09 15:11 ` [PATCH 13/17] ARM: GICv3: rename HYP interface definitions to use ICH_ prefix Andre Przywara
2018-03-09 15:11 ` [PATCH 14/17] ARM: Implement vcpu_kick() Andre Przywara
2018-03-12 11:41   ` Julien Grall
2018-03-09 15:11 ` [PATCH 15/17] ARM: GICv2: introduce gicv2_poke_irq() Andre Przywara
2018-03-09 15:11 ` [PATCH 16/17] ARM: GICv3: poke_irq: make RWP optional Andre Przywara
2018-03-09 15:11 ` [PATCH 17/17] ARM: GICv2: fix GICH_V2_LR definitions Andre Przywara
2018-03-12 12:08 ` [PATCH 00/17] ARM: vGIC: prepare for splitting the vGIC code Julien Grall

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.