All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v8 0/7] xen/arm: Add GICv3 support
@ 2014-07-23 13:41 vijay.kilari
  2014-07-23 13:41 ` [PATCH v8 1/7] xen/arm: Introduce sizes.h vijay.kilari
                   ` (7 more replies)
  0 siblings, 8 replies; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Add GICv3 support for arm64 platform.
>From v6 version, this patch series is split into v6a and v6.
All the GICv2 and VGIC code refactoring patches of v6a are merged.

The v8 version of GICv3 patches are posted here.

Tested with ARM64 simulator with multicore core
and booted Dom0 kernel.

Major changes in v8:
  - Introduced assembler aliases for GIC system registers
    to be able to compile with Linaro toolchain
  - Few PIDRn macros definitions are moved to vgic-v3.c
  - Introduced 2 more patches. Patch 1 for introducing
    sizes.h file and Patch 2 for {READ,WRITE}_SYSREG macro changes

Major changes in v7:
  - Introduced vgic callback to handle sysreg emulation
  - Coding style fixes
  - undef64_exception instead of undef_exception for sysreg error

Major changes in v6:
  - Added read_apr() to read ICH_APRn in GICv3
  - Added asserts
  - PIDR2 check for architecture value
  - Remove update_cpu_lr_mask call
  - Changed parameter ordering for gicv3_send_sgi()
  - Save and restore ICC_SRE_EL1 register
  - Computed cpumask in gicv3_send_sgi() function

Major changes in v5:
 - Introduced new patch for checking platform capability for gicv3
 - Introduced more patches for vgic clean up and code movement
 - Added synchronization barriers and clean up in GICv3 driver
 - Rebase on top of master branch +
   remotes/origin/no_maintenance_interrupts-v8 patch set
 - Code base available in github
   git clone https://github.com/vijaykilari/Xen-GICv3.git
 - Fixed comments and coding style

Major changes in v4:
 - Changed io handlers to take mmio address and size as
   parameters
 - ioremap is used instead of fixmap to map GICv2 address
   space. Removed /4 in GICv2 register definitions
 - vGIC driver now uses register size to calculate IRQ rank
 - GICv2 LR register definitions are declared locally in GICv2 driver
 - GICR & GICD common register handling in vgic-v3 driver are segregated
   in one common function
 - irq_hw_controller definition is managed in GICv2 and GICv3 drivers
 - Made irq_ops const
 - GIC DT node is updated in respective drivers
 - Comments and coding style fixes
 - Rebased on remotes/origin/no_maintenance_interrupts-v8 + Julien's
   patch set

Major changes in v3:
 - Moved io.h from xen/arch/arm to xen/include/asm-arm
   and renamed as mmio.h
 - Made all the io handlers as domain specific and removed
   check handlers. Instead address and size are passed
 - Cleaned up sgi handling functions in V2 gic driver and
   made generic
 - Removed unused parameters in do_sgi call
 - VGIC driver is chosen based on GIC driver initialized
 - Implemented SGI handling for GICv3 to boot multicore
 - Missing coding style fixes
 - Rebased to remotes/origin/no_maintenance_interrupts-v6
   branch of Stephano's unstable tree

Major Changes in v2:
 - Added per domain io handler
 - Use device api to initialize gic driver
 - Avoid use of void pointer to access common data
 - Moved vgic address information data from domain.h
 - Coding style

Vijaya Kumar K (7):
  xen/arm: Introduce sizes.h
  xen/arm: Stringify the register name in sysreg read write macros
  xen/arm: Add support for GIC v3
  xen/arm: Add virtual GICv3 support
  xen/arm: Update Dom0 GIC dt node with GICv3 information
  xen/arm: add SGI handling for GICv3
  xen/arm: check for GICv3 platform support

 xen/arch/arm/Makefile                 |    2 +
 xen/arch/arm/domain_build.c           |   47 +-
 xen/arch/arm/gic-v2.c                 |   51 ++
 xen/arch/arm/gic-v3.c                 | 1270 +++++++++++++++++++++++++++++++++
 xen/arch/arm/gic.c                    |    6 +
 xen/arch/arm/setup.c                  |    5 +-
 xen/arch/arm/traps.c                  |   15 +
 xen/arch/arm/vgic-v2.c                |    8 +-
 xen/arch/arm/vgic-v3.c                |  973 +++++++++++++++++++++++++
 xen/arch/arm/vgic.c                   |   15 +
 xen/include/asm-arm/arm64/processor.h |   10 +-
 xen/include/asm-arm/cpufeature.h      |    1 +
 xen/include/asm-arm/domain.h          |    8 +
 xen/include/asm-arm/gic.h             |   24 +
 xen/include/asm-arm/gic_v3_defs.h     |  165 +++++
 xen/include/asm-arm/processor.h       |   18 +-
 xen/include/asm-arm/sysregs.h         |   56 ++
 xen/include/asm-arm/vgic.h            |   16 +-
 xen/include/xen/sizes.h               |   48 ++
 19 files changed, 2680 insertions(+), 58 deletions(-)
 create mode 100644 xen/arch/arm/gic-v3.c
 create mode 100644 xen/arch/arm/vgic-v3.c
 create mode 100644 xen/include/asm-arm/gic_v3_defs.h
 create mode 100644 xen/include/xen/sizes.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 39+ messages in thread

* [PATCH v8 1/7] xen/arm: Introduce sizes.h
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
@ 2014-07-23 13:41 ` vijay.kilari
  2014-07-28 13:55   ` Ian Campbell
  2014-07-23 13:41 ` [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros vijay.kilari
                   ` (6 subsequent siblings)
  7 siblings, 1 reply; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Add sizes.h file similar to include/linux/sizes.h of
kernel code. These size macros helps in porting
kernel drivers to Xen

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/include/xen/sizes.h |   48 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/xen/include/xen/sizes.h b/xen/include/xen/sizes.h
new file mode 100644
index 0000000..f7b728d
--- /dev/null
+++ b/xen/include/xen/sizes.h
@@ -0,0 +1,48 @@
+#ifndef __XEN_SIZES_H__
+#define __XEN_SIZES_H__
+
+/*
+ * Taken from kernel code include/linux/size.h
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#define SZ_1                            0x00000001
+#define SZ_2                            0x00000002
+#define SZ_4                            0x00000004
+#define SZ_8                            0x00000008
+#define SZ_16                           0x00000010
+#define SZ_32                           0x00000020
+#define SZ_64                           0x00000040
+#define SZ_128                          0x00000080
+#define SZ_256                          0x00000100
+#define SZ_512                          0x00000200
+
+#define SZ_1K                           0x00000400
+#define SZ_2K                           0x00000800
+#define SZ_4K                           0x00001000
+#define SZ_8K                           0x00002000
+#define SZ_16K                          0x00004000
+#define SZ_32K                          0x00008000
+#define SZ_64K                          0x00010000
+#define SZ_128K                         0x00020000
+#define SZ_256K                         0x00040000
+#define SZ_512K                         0x00080000
+
+#define SZ_1M                           0x00100000
+#define SZ_2M                           0x00200000
+#define SZ_4M                           0x00400000
+#define SZ_8M                           0x00800000
+#define SZ_16M                          0x01000000
+#define SZ_32M                          0x02000000
+#define SZ_64M                          0x04000000
+#define SZ_128M                         0x08000000
+#define SZ_256M                         0x10000000
+#define SZ_512M                         0x20000000
+
+#define SZ_1G                           0x40000000
+#define SZ_2G                           0x80000000
+
+#endif /* __XEN_SIZES_H__ */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
  2014-07-23 13:41 ` [PATCH v8 1/7] xen/arm: Introduce sizes.h vijay.kilari
@ 2014-07-23 13:41 ` vijay.kilari
  2014-07-23 17:20   ` Julien Grall
  2014-07-28 13:56   ` Ian Campbell
  2014-07-23 13:41 ` [PATCH v8 3/7] xen/arm: Add support for GIC v3 vijay.kilari
                   ` (5 subsequent siblings)
  7 siblings, 2 replies; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

The register name parameter in {READ,WRITE}_SYSREG{32,64}
macros is replaced as register name in mrs and msr assembly
instruction.

If this register name is macro indirection, the register
name is not replaced with designated macro. So replace the register
name with __stringify macro, which replaces register name
if it is macro

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/include/asm-arm/arm64/processor.h |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/xen/include/asm-arm/arm64/processor.h b/xen/include/asm-arm/arm64/processor.h
index 5bf0867..3a9c0cd 100644
--- a/xen/include/asm-arm/arm64/processor.h
+++ b/xen/include/asm-arm/arm64/processor.h
@@ -1,6 +1,8 @@
 #ifndef __ASM_ARM_ARM64_PROCESSOR_H
 #define __ASM_ARM_ARM64_PROCESSOR_H
 
+#include <xen/stringify.h>
+
 #ifndef __ASSEMBLY__
 
 /* Anonymous union includes both 32- and 64-bit names (e.g., r0/x0). */
@@ -85,20 +87,20 @@ struct cpu_user_regs
 
 #define READ_SYSREG32(name) ({                          \
     uint32_t _r;                                        \
-    asm volatile("mrs  %0, "#name : "=r" (_r));         \
+    asm volatile("mrs  %0, "__stringify(name) : "=r" (_r));         \
     _r; })
 #define WRITE_SYSREG32(v, name) do {                    \
     uint32_t _r = v;                                    \
-    asm volatile("msr "#name", %0" : : "r" (_r));       \
+    asm volatile("msr "__stringify(name)", %0" : : "r" (_r));       \
 } while (0)
 
 #define WRITE_SYSREG64(v, name) do {                    \
     uint64_t _r = v;                                    \
-    asm volatile("msr "#name", %0" : : "r" (_r));       \
+    asm volatile("msr "__stringify(name)", %0" : : "r" (_r));       \
 } while (0)
 #define READ_SYSREG64(name) ({                          \
     uint64_t _r;                                        \
-    asm volatile("mrs  %0, "#name : "=r" (_r));         \
+    asm volatile("mrs  %0, "__stringify(name) : "=r" (_r));         \
     _r; })
 
 #define READ_SYSREG(name)     READ_SYSREG64(name)
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [PATCH v8 3/7] xen/arm: Add support for GIC v3
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
  2014-07-23 13:41 ` [PATCH v8 1/7] xen/arm: Introduce sizes.h vijay.kilari
  2014-07-23 13:41 ` [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros vijay.kilari
@ 2014-07-23 13:41 ` vijay.kilari
  2014-07-23 17:28   ` Julien Grall
  2014-07-23 13:41 ` [PATCH v8 4/7] xen/arm: Add virtual GICv3 support vijay.kilari
                   ` (4 subsequent siblings)
  7 siblings, 1 reply; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Add support for GIC v3 specification System register access(SRE)
is enabled to access cpu and virtual interface registers based
on kernel GICv3 driver.

This patch adds only basic v3 support.
Does not support Interrupt Translation support (ITS)

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v8: - Introduced assembler aliases for GIC system registers
    - Removed usage of lib.h for adding size macro, instead used
      sizes.h
    - Renamed PIDR macros
    - Removed check for number of cpus in cluster in gicv3_send_sgi()
      and instead check is added in gicv3_populate_rdist()
    - Added comments and fixed coding style
    - Removed nr_lrs parameter in gicv3_{restore,save}_lrs()
    - Rename GICV3 compat strings.
v7: - Fixed wrong check for irq number in gicv3_set_irq_properties.
    - Introduced panic during cpu_init on wrong priority bits.
v6: - Added read_apr() to read ICH_APRn in GICv3
    - Added asserts
    - PIDR2 check for architecture valu
    - Remove update_cpu_lr_mask call
    - Changed parameter ordering for gicv3_send_sgi()
    - Save and restore ICC_SRE_EL1 registers
---
 xen/arch/arm/Makefile             |    1 +
 xen/arch/arm/gic-v3.c             | 1185 +++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/domain.h      |    8 +
 xen/include/asm-arm/gic.h         |   20 +
 xen/include/asm-arm/gic_v3_defs.h |  158 +++++
 xen/include/asm-arm/processor.h   |   14 +
 xen/include/asm-arm/sysregs.h     |   53 ++
 7 files changed, 1439 insertions(+)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 20f59f4..1684c09 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -11,6 +11,7 @@ obj-y += domctl.o
 obj-y += sysctl.o
 obj-y += domain_build.o
 obj-y += gic.o gic-v2.o
+obj-$(CONFIG_ARM_64) += gic-v3.o
 obj-y += io.o
 obj-y += irq.o
 obj-y += kernel.o
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
new file mode 100644
index 0000000..2b801f1
--- /dev/null
+++ b/xen/arch/arm/gic-v3.c
@@ -0,0 +1,1185 @@
+/*
+ * xen/arch/arm/gic-v3.c
+ *
+ * ARM Generic Interrupt Controller support v3 version
+ * based on xen/arch/arm/gic-v2.c and kernel GICv3 driver
+ *
+ * Copyright (C) 2012,2013 - ARM Ltd
+ * Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>, Cavium Inc
+ * ported to Xen
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/config.h>
+#include <xen/lib.h>
+#include <xen/init.h>
+#include <xen/cpu.h>
+#include <xen/mm.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/errno.h>
+#include <xen/delay.h>
+#include <xen/device_tree.h>
+#include <xen/sizes.h>
+#include <asm/p2m.h>
+#include <asm/domain.h>
+#include <asm/io.h>
+#include <asm/device.h>
+#include <asm/gic.h>
+#include <asm/gic_v3_defs.h>
+
+struct rdist_region {
+    paddr_t base;
+    paddr_t size;
+    void __iomem *map_base;
+};
+
+/* Global state */
+static struct {
+    paddr_t dbase;            /* Address of distributor registers */
+    paddr_t dbase_size;
+    void __iomem *map_dbase;  /* Mapped address of distributor registers */
+    struct rdist_region *rdist_regions;
+    uint32_t  rdist_stride;
+    unsigned int rdist_count; /* Number of rdist regions count */
+    unsigned int nr_priorities;
+    spinlock_t lock;
+} gicv3;
+
+static struct gic_info gicv3_info;
+
+/* per-cpu re-distributor base */
+static DEFINE_PER_CPU(void __iomem*, rbase);
+
+#define GICD                   (gicv3.map_dbase)
+#define GICD_RDIST_BASE        (this_cpu(rbase))
+#define GICD_RDIST_SGI_BASE    (GICD_RDIST_BASE + SZ_64K)
+
+/*
+ * Saves all 16(Max) LR registers. Though number of LRs implemented
+ * is implementation specific.
+ */
+static inline void gicv3_save_lrs(struct vcpu *v)
+{
+    /* Fall through for all the cases */
+    switch ( gicv3_info.nr_lrs )
+    {
+    case 16:
+        v->arch.gic.v3.lr[15] = READ_SYSREG(ICH_LR15_EL2);
+    case 15:
+        v->arch.gic.v3.lr[14] = READ_SYSREG(ICH_LR14_EL2);
+    case 14:
+        v->arch.gic.v3.lr[13] = READ_SYSREG(ICH_LR13_EL2);
+    case 13:
+        v->arch.gic.v3.lr[12] = READ_SYSREG(ICH_LR12_EL2);
+    case 12:
+        v->arch.gic.v3.lr[11] = READ_SYSREG(ICH_LR11_EL2);
+    case 11:
+        v->arch.gic.v3.lr[10] = READ_SYSREG(ICH_LR10_EL2);
+    case 10:
+        v->arch.gic.v3.lr[9] = READ_SYSREG(ICH_LR9_EL2);
+    case 9:
+        v->arch.gic.v3.lr[8] = READ_SYSREG(ICH_LR8_EL2);
+    case 8:
+        v->arch.gic.v3.lr[7] = READ_SYSREG(ICH_LR7_EL2);
+    case 7:
+        v->arch.gic.v3.lr[6] = READ_SYSREG(ICH_LR6_EL2);
+    case 6:
+        v->arch.gic.v3.lr[5] = READ_SYSREG(ICH_LR5_EL2);
+    case 5:
+        v->arch.gic.v3.lr[4] = READ_SYSREG(ICH_LR4_EL2);
+    case 4:
+        v->arch.gic.v3.lr[3] = READ_SYSREG(ICH_LR3_EL2);
+    case 3:
+        v->arch.gic.v3.lr[2] = READ_SYSREG(ICH_LR2_EL2);
+    case 2:
+        v->arch.gic.v3.lr[1] = READ_SYSREG(ICH_LR1_EL2);
+    case 1:
+         v->arch.gic.v3.lr[0] = READ_SYSREG(ICH_LR0_EL2);
+         break;
+    default:
+         BUG();
+    }
+}
+
+/*
+ * Restores all 16(Max) LR registers. Though number of LRs implemented
+ * is implementation specific.
+ */
+static inline void gicv3_restore_lrs(const struct vcpu *v)
+{
+    /* Fall through for all the cases */
+    switch ( gicv3_info.nr_lrs )
+    {
+    case 16:
+        WRITE_SYSREG(v->arch.gic.v3.lr[15], ICH_LR15_EL2);
+    case 15:
+        WRITE_SYSREG(v->arch.gic.v3.lr[14], ICH_LR14_EL2);
+    case 14:
+        WRITE_SYSREG(v->arch.gic.v3.lr[13], ICH_LR13_EL2);
+    case 13:
+        WRITE_SYSREG(v->arch.gic.v3.lr[12], ICH_LR12_EL2);
+    case 12:
+        WRITE_SYSREG(v->arch.gic.v3.lr[11], ICH_LR11_EL2);
+    case 11:
+        WRITE_SYSREG(v->arch.gic.v3.lr[10], ICH_LR10_EL2);
+    case 10:
+        WRITE_SYSREG(v->arch.gic.v3.lr[9], ICH_LR9_EL2);
+    case 9:
+        WRITE_SYSREG(v->arch.gic.v3.lr[8], ICH_LR8_EL2);
+    case 8:
+        WRITE_SYSREG(v->arch.gic.v3.lr[7], ICH_LR7_EL2);
+    case 7:
+        WRITE_SYSREG(v->arch.gic.v3.lr[6], ICH_LR6_EL2);
+    case 6:
+        WRITE_SYSREG(v->arch.gic.v3.lr[5], ICH_LR5_EL2);
+    case 5:
+        WRITE_SYSREG(v->arch.gic.v3.lr[4], ICH_LR4_EL2);
+    case 4:
+        WRITE_SYSREG(v->arch.gic.v3.lr[3], ICH_LR3_EL2);
+    case 3:
+        WRITE_SYSREG(v->arch.gic.v3.lr[2], ICH_LR2_EL2);
+    case 2:
+        WRITE_SYSREG(v->arch.gic.v3.lr[1], ICH_LR1_EL2);
+    case 1:
+        WRITE_SYSREG(v->arch.gic.v3.lr[0], ICH_LR0_EL2);
+        break;
+    default:
+         BUG();
+    }
+}
+
+static uint64_t gicv3_ich_read_lr(int lr)
+{
+    switch ( lr )
+    {
+    case 0: return READ_SYSREG(ICH_LR0_EL2);
+    case 1: return READ_SYSREG(ICH_LR1_EL2);
+    case 2: return READ_SYSREG(ICH_LR2_EL2);
+    case 3: return READ_SYSREG(ICH_LR3_EL2);
+    case 4: return READ_SYSREG(ICH_LR4_EL2);
+    case 5: return READ_SYSREG(ICH_LR5_EL2);
+    case 6: return READ_SYSREG(ICH_LR6_EL2);
+    case 7: return READ_SYSREG(ICH_LR7_EL2);
+    case 8: return READ_SYSREG(ICH_LR8_EL2);
+    case 9: return READ_SYSREG(ICH_LR9_EL2);
+    case 10: return READ_SYSREG(ICH_LR10_EL2);
+    case 11: return READ_SYSREG(ICH_LR11_EL2);
+    case 12: return READ_SYSREG(ICH_LR12_EL2);
+    case 13: return READ_SYSREG(ICH_LR13_EL2);
+    case 14: return READ_SYSREG(ICH_LR14_EL2);
+    case 15: return READ_SYSREG(ICH_LR15_EL2);
+    default:
+        BUG();
+    }
+}
+
+static void gicv3_ich_write_lr(int lr, uint64_t val)
+{
+    switch ( lr )
+    {
+    case 0:
+        WRITE_SYSREG(val, ICH_LR0_EL2);
+        break;
+    case 1:
+        WRITE_SYSREG(val, ICH_LR1_EL2);
+        break;
+    case 2:
+        WRITE_SYSREG(val, ICH_LR2_EL2);
+        break;
+    case 3:
+        WRITE_SYSREG(val, ICH_LR3_EL2);
+        break;
+    case 4:
+        WRITE_SYSREG(val, ICH_LR4_EL2);
+        break;
+    case 5:
+        WRITE_SYSREG(val, ICH_LR5_EL2);
+        break;
+    case 6:
+        WRITE_SYSREG(val, ICH_LR6_EL2);
+        break;
+    case 7:
+        WRITE_SYSREG(val, ICH_LR7_EL2);
+        break;
+    case 8:
+        WRITE_SYSREG(val, ICH_LR8_EL2);
+        break;
+    case 9:
+        WRITE_SYSREG(val, ICH_LR9_EL2);
+        break;
+    case 10:
+        WRITE_SYSREG(val, ICH_LR10_EL2);
+        break;
+    case 11:
+        WRITE_SYSREG(val, ICH_LR11_EL2);
+        break;
+    case 12:
+        WRITE_SYSREG(val, ICH_LR12_EL2);
+        break;
+    case 13:
+        WRITE_SYSREG(val, ICH_LR13_EL2);
+        break;
+    case 14:
+        WRITE_SYSREG(val, ICH_LR14_EL2);
+        break;
+    case 15:
+        WRITE_SYSREG(val, ICH_LR15_EL2);
+        break;
+    default:
+        return;
+    }
+    isb();
+}
+
+/*
+ * System Register Enable (SRE). Enable to access CPU & Virtual
+ * interface registers as system registers in EL2
+ */
+static void gicv3_enable_sre(void)
+{
+    uint32_t val;
+
+    val = READ_SYSREG32(ICC_SRE_EL2);
+    val |= GICC_SRE_EL2_SRE | GICC_SRE_EL2_ENEL1;
+
+    WRITE_SYSREG32(val, ICC_SRE_EL2);
+    isb();
+}
+
+/* Wait for completion of a distributor change */
+static void gicv3_do_wait_for_rwp(void __iomem *base)
+{
+    uint32_t val;
+    bool_t timeout = 0;
+    s_time_t deadline = NOW() + MILLISECS(1000);
+
+    do {
+        val = readl_relaxed(base + GICD_CTLR);
+        if ( !(val & GICD_CTLR_RWP) )
+            break;
+        if ( NOW() > deadline )
+        {
+            timeout = 1;
+            break;
+        }
+        cpu_relax();
+        udelay(1);
+    } while ( 1 );
+
+    if ( timeout )
+        dprintk(XENLOG_ERR, "RWP timeout\n");
+}
+
+static void gicv3_dist_wait_for_rwp(void)
+{
+    gicv3_do_wait_for_rwp(GICD);
+}
+
+static void gicv3_redist_wait_for_rwp(void)
+{
+    gicv3_do_wait_for_rwp(GICD_RDIST_BASE);
+}
+
+static void gicv3_wait_for_rwp(int irq)
+{
+    if ( irq < NR_LOCAL_IRQS )
+         gicv3_redist_wait_for_rwp();
+    else
+         gicv3_dist_wait_for_rwp();
+}
+
+static unsigned int gicv3_get_cpu_from_mask(const cpumask_t *cpumask)
+{
+    unsigned int cpu;
+    cpumask_t possible_mask;
+
+    cpumask_and(&possible_mask, cpumask, &cpu_possible_map);
+    cpu = cpumask_any(&possible_mask);
+
+    return cpu;
+}
+
+static void restore_aprn_regs(const union gic_state_data *d)
+{
+    /* Write APRn register based on number of priorities
+       platform has implemented */
+    switch ( gicv3.nr_priorities )
+    {
+    case 7:
+        WRITE_SYSREG32(d->v3.apr0[2], ICH_AP0R2_EL2);
+        WRITE_SYSREG32(d->v3.apr1[2], ICH_AP1R2_EL2);
+        /* Fall through */
+    case 6:
+        WRITE_SYSREG32(d->v3.apr0[1], ICH_AP0R1_EL2);
+        WRITE_SYSREG32(d->v3.apr1[1], ICH_AP1R1_EL2);
+        /* Fall through */
+    case 5:
+        WRITE_SYSREG32(d->v3.apr0[0], ICH_AP0R0_EL2);
+        WRITE_SYSREG32(d->v3.apr1[0], ICH_AP1R0_EL2);
+        break;
+    default:
+        BUG();
+    }
+}
+
+static void save_aprn_regs(union gic_state_data *d)
+{
+    /* Read APRn register based on number of priorities
+       platform has implemented */
+    switch ( gicv3.nr_priorities )
+    {
+    case 7:
+        d->v3.apr0[2] = READ_SYSREG32(ICH_AP0R2_EL2);
+        d->v3.apr1[2] = READ_SYSREG32(ICH_AP1R2_EL2);
+        /* Fall through */
+    case 6:
+        d->v3.apr0[1] = READ_SYSREG32(ICH_AP0R1_EL2);
+        d->v3.apr1[1] = READ_SYSREG32(ICH_AP1R1_EL2);
+        /* Fall through */
+    case 5:
+        d->v3.apr0[0] = READ_SYSREG32(ICH_AP0R0_EL2);
+        d->v3.apr1[0] = READ_SYSREG32(ICH_AP1R0_EL2);
+        break;
+    default:
+        BUG();
+    }
+}
+
+/*
+ * As per section 4.8.17 of the GICv3 spec following
+ * registers are save and restored on guest swap
+ */
+static void gicv3_save_state(struct vcpu *v)
+{
+
+    /* No need for spinlocks here because interrupts are disabled around
+     * this call and it only accesses struct vcpu fields that cannot be
+     * accessed simultaneously by another pCPU.
+     *
+     * Make sure all stores to the GIC via the memory mapped interface
+     * are now visible to the system register interface
+     */
+    dsb(sy);
+    gicv3_save_lrs(v);
+    save_aprn_regs(&v->arch.gic);
+    v->arch.gic.v3.vmcr = READ_SYSREG32(ICH_VMCR_EL2);
+    v->arch.gic.v3.sre_el1 = READ_SYSREG32(ICC_SRE_EL1);
+}
+
+static void gicv3_restore_state(const struct vcpu *v)
+{
+    WRITE_SYSREG32(v->arch.gic.v3.sre_el1, ICC_SRE_EL1);
+    WRITE_SYSREG32(v->arch.gic.v3.vmcr, ICH_VMCR_EL2);
+    restore_aprn_regs(&v->arch.gic);
+    gicv3_restore_lrs(v);
+
+    /*
+     * Make sure all stores are visible the GIC
+     */
+    dsb(sy);
+}
+
+static void gicv3_dump_state(const struct vcpu *v)
+{
+    int i;
+
+    if ( v == current )
+    {
+        for ( i = 0; i < gicv3_info.nr_lrs; i++ )
+            printk("   HW_LR[%d]=%lx\n", i, gicv3_ich_read_lr(i));
+    }
+    else
+    {
+        for ( i = 0; i < gicv3_info.nr_lrs; i++ )
+            printk("   VCPU_LR[%d]=%lx\n", i, v->arch.gic.v3.lr[i]);
+    }
+}
+
+static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset)
+{
+    u32 mask = 1 << (irqd->irq % 32);
+    void __iomem *base;
+
+    if ( irqd->irq < NR_GIC_LOCAL_IRQS )
+        base = GICD_RDIST_SGI_BASE;
+    else
+        base = GICD;
+
+    writel_relaxed(mask, base + offset + (irqd->irq / 32) * 4);
+    gicv3_wait_for_rwp(irqd->irq);
+}
+
+static void gicv3_unmask_irq(struct irq_desc *irqd)
+{
+    gicv3_poke_irq(irqd, GICD_ISENABLER);
+}
+
+static void gicv3_mask_irq(struct irq_desc *irqd)
+{
+    gicv3_poke_irq(irqd, GICD_ICENABLER);
+}
+
+static void gicv3_eoi_irq(struct irq_desc *irqd)
+{
+    /* Lower the priority */
+    WRITE_SYSREG32(irqd->irq, ICC_EOIR1_EL1);
+    isb();
+}
+
+static void gicv3_dir_irq(struct irq_desc *irqd)
+{
+    /* Deactivate */
+    WRITE_SYSREG32(irqd->irq, ICC_DIR_EL1);
+    isb();
+}
+
+static unsigned int gicv3_read_irq(void)
+{
+    return (READ_SYSREG32(ICC_IAR1_EL1) & GICC_IA_IRQ);
+}
+
+static inline uint64_t gicv3_mpidr_to_affinity(int cpu)
+{
+     uint64_t mpidr = cpu_logical_map(cpu);
+     return (MPIDR_AFFINITY_LEVEL(mpidr, 3) << 32 |
+             MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16 |
+             MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8  |
+             MPIDR_AFFINITY_LEVEL(mpidr, 0));
+}
+
+static void gicv3_set_irq_properties(struct irq_desc *desc,
+                                     const cpumask_t *cpu_mask,
+                                     unsigned int priority)
+{
+    uint32_t cfg, edgebit;
+    uint64_t affinity;
+    void __iomem *base;
+    unsigned int cpu = gicv3_get_cpu_from_mask(cpu_mask);
+    unsigned int irq = desc->irq;
+    unsigned int type = desc->arch.type;
+
+    /* SGI's are always edge-triggered not need to call GICD_ICFGR0 */
+    ASSERT(irq >= NR_GIC_SGI);
+
+    spin_lock(&gicv3.lock);
+
+    if ( irq >= NR_GIC_LOCAL_IRQS)
+        base = GICD + GICD_ICFGR + (irq / 16) * 4;
+    else
+        base = GICD_RDIST_SGI_BASE + GICR_ICFGR1;
+
+    cfg = readl_relaxed(base);
+
+    edgebit = 2u << (2 * (irq % 16));
+    if ( type & DT_IRQ_TYPE_LEVEL_MASK )
+        cfg &= ~edgebit;
+    else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
+        cfg |= edgebit;
+
+    writel_relaxed(cfg, base);
+
+    affinity = gicv3_mpidr_to_affinity(cpu);
+    /* Make sure we don't broadcast the interrupt */
+    affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
+
+    if ( irq >= NR_GIC_LOCAL_IRQS )
+        writeq_relaxed(affinity, (GICD + GICD_IROUTER + irq * 8));
+
+    /* Set priority */
+    if ( irq < NR_GIC_LOCAL_IRQS )
+        writeb_relaxed(priority, GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irq);
+    else
+        writeb_relaxed(priority, GICD + GICD_IPRIORITYR + irq);
+
+    spin_unlock(&gicv3.lock);
+}
+
+static void __init gicv3_dist_init(void)
+{
+    uint32_t type;
+    uint32_t priority;
+    uint64_t affinity;
+    int i;
+
+    /* Disable the distributor */
+    writel_relaxed(0, GICD + GICD_CTLR);
+
+    type = readl_relaxed(GICD + GICD_TYPER);
+    gicv3_info.nr_lines = 32 * ((type & GICD_TYPE_LINES) + 1);
+
+    printk("GICv3: %d lines, (IID %8.8x).\n",
+           gicv3_info.nr_lines, readl_relaxed(GICD + GICD_IIDR));
+
+    /* Default all global IRQs to level, active low */
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 16 )
+        writel_relaxed(0, GICD + GICD_ICFGR + (i / 16) * 4);
+
+    /* Default priority for global interrupts */
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 4 )
+    {
+        priority = (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
+                    GIC_PRI_IRQ << 8 | GIC_PRI_IRQ);
+        writel_relaxed(priority, GICD + GICD_IPRIORITYR + (i / 4) * 4);
+    }
+
+    /* Disable all global interrupts */
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 32 )
+        writel_relaxed(0xffffffff, GICD + GICD_ICENABLER + (i / 32) * 4);
+
+    gicv3_dist_wait_for_rwp();
+
+    /* Turn on the distributor */
+    writel_relaxed(GICD_CTL_ENABLE | GICD_CTLR_ARE_NS |
+                GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1, GICD + GICD_CTLR);
+
+    /* Route all global IRQs to this CPU */
+    affinity = gicv3_mpidr_to_affinity(smp_processor_id());
+    /* Make sure we don't broadcast the interrupt */
+    affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
+
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i++ )
+        writeq_relaxed(affinity, GICD + GICD_IROUTER + i * 8);
+}
+
+static int gicv3_enable_redist(void)
+{
+    uint32_t val;
+    bool_t timeout = 0;
+    s_time_t deadline = NOW() + MILLISECS(1000);
+
+    /* Wake up this CPU redistributor */
+    val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
+    val &= ~GICR_WAKER_ProcessorSleep;
+    writel_relaxed(val, GICD_RDIST_BASE + GICR_WAKER);
+
+    do {
+        val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
+        if ( !(val & GICR_WAKER_ChildrenAsleep) )
+            break;
+        if ( NOW() > deadline )
+        {
+            timeout = 1;
+            break;
+        }
+        cpu_relax();
+        udelay(1);
+    } while ( timeout );
+
+    if ( timeout )
+    {
+        dprintk(XENLOG_ERR, "GICv3: Redist enable RWP timeout\n");
+        return 1;
+    }
+
+    return 0;
+}
+
+static int __init gicv3_populate_rdist(void)
+{
+    int i;
+    uint32_t aff;
+    uint32_t reg;
+    uint64_t typer;
+    uint64_t mpidr = cpu_logical_map(smp_processor_id());
+
+    /*
+     * If we ever get a cluster of more than 16 CPUs, just scream.
+     */
+    if ( (mpidr & 0xff) >= 16 )
+          dprintk(XENLOG_WARNING, "GICv3:Cluster with more than 16's cpus\n");
+
+    /*
+     * Convert affinity to a 32bit value that can be matched to GICR_TYPER
+     * bits [63:32]
+     */
+    aff = (MPIDR_AFFINITY_LEVEL(mpidr, 3) << 24 |
+           MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16 |
+           MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8 |
+           MPIDR_AFFINITY_LEVEL(mpidr, 0));
+
+    for ( i = 0; i < gicv3.rdist_count; i++ )
+    {
+        void __iomem *ptr = gicv3.rdist_regions[i].map_base;
+
+        reg = readl_relaxed(ptr + GICR_PIDR2) & GICR_PIDR2_ARCH_REV_MASK;
+        if ( (reg >> GICR_PIDR2_ARCH_REV_OFFSET) != GICR_PIDR2_ARCH_GICV3 )
+        {
+            dprintk(XENLOG_ERR,
+                    "GICv3: No redistributor present @%"PRIpaddr"\n",
+                    gicv3.rdist_regions[i].base);
+            break;
+        }
+
+        do {
+            typer = readq_relaxed(ptr + GICR_TYPER);
+
+            if ( (typer >> 32) == aff )
+            {
+                this_cpu(rbase) = ptr;
+                printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
+                        smp_processor_id(), i, ptr);
+                return 0;
+            }
+            if ( gicv3.rdist_stride )
+                ptr += gicv3.rdist_stride;
+            else
+            {
+                ptr += SZ_64K * 2;
+                if ( typer & GICR_TYPER_VLPIS )
+                    ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+            }
+
+        } while ( !(typer & GICR_TYPER_LAST) );
+    }
+
+    dprintk(XENLOG_ERR, "GICv3: CPU%d: mpidr 0x%x has no re-distributor!\n",
+            smp_processor_id(), cpu_logical_map(smp_processor_id()));
+
+    return -ENODEV;
+}
+
+static int __cpuinit gicv3_cpu_init(void)
+{
+    int i;
+    uint32_t priority;
+
+    /* Register ourselves with the rest of the world */
+    if ( gicv3_populate_rdist() )
+        return -ENODEV;
+
+    if ( gicv3_enable_redist() )
+        return -ENODEV;
+
+    /* Set priority on PPI and SGI interrupts */
+    priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
+                GIC_PRI_IPI);
+    for (i = 0; i < NR_GIC_SGI; i += 4)
+        writel_relaxed(priority,
+                GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
+
+    priority = (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 | GIC_PRI_IRQ << 8 |
+                GIC_PRI_IRQ);
+    for (i = NR_GIC_SGI; i < NR_GIC_LOCAL_IRQS; i += 4)
+        writel_relaxed(priority,
+                GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
+
+    /*
+     * Disable all PPI interrupts, ensure all SGI interrupts are
+     * enabled.
+     */
+    writel_relaxed(0xffff0000, GICD_RDIST_SGI_BASE + GICR_ICENABLER0);
+    writel_relaxed(0x0000ffff, GICD_RDIST_SGI_BASE + GICR_ISENABLER0);
+
+    gicv3_redist_wait_for_rwp();
+
+    /* Enable system registers */
+    gicv3_enable_sre();
+
+    /* No priority grouping */
+    WRITE_SYSREG32(0, ICC_BPR1_EL1);
+
+    /* Set priority mask register */
+    WRITE_SYSREG32(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+
+    /* EOI drops priority too (mode 0) */
+    WRITE_SYSREG32(GICC_CTLR_EL1_EOImode_drop, ICC_CTLR_EL1);
+
+    /* Enable Group1 interrupts */
+    WRITE_SYSREG32(1, ICC_IGRPEN1_EL1);
+
+    /* Sync at once at the end of cpu interface configuration */
+    isb();
+
+    return 0;
+}
+
+static void gicv3_cpu_disable(void)
+{
+    WRITE_SYSREG32(0, ICC_CTLR_EL1);
+    isb();
+}
+
+static void __cpuinit gicv3_hyp_init(void)
+{
+    uint32_t vtr;
+
+    vtr = READ_SYSREG32(ICH_VTR_EL2);
+    gicv3_info.nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
+    gicv3.nr_priorities = ((vtr >> GICH_VTR_PRIBITS_SHIFT) &
+                          GICH_VTR_PRIBITS_MASK) + 1;
+
+    if ( !((gicv3.nr_priorities > 4) && (gicv3.nr_priorities < 8)) )
+        panic("GICv3: Invalid number of priority bits\n");
+
+    WRITE_SYSREG32(GICH_VMCR_EOI | GICH_VMCR_VENG1, ICH_VMCR_EL2);
+    WRITE_SYSREG32(GICH_HCR_EN, ICH_HCR_EL2);
+}
+
+/* Set up the per-CPU parts of the GIC for a secondary CPU */
+static int gicv3_secondary_cpu_init(void)
+{
+    int res;
+
+    spin_lock(&gicv3.lock);
+
+    res = gicv3_cpu_init();
+    gicv3_hyp_init();
+
+    spin_unlock(&gicv3.lock);
+
+    return res;
+}
+
+static void __cpuinit gicv3_hyp_disable(void)
+{
+    uint32_t hcr;
+
+    hcr = READ_SYSREG32(ICH_HCR_EL2);
+    hcr &= ~GICH_HCR_EN;
+    WRITE_SYSREG32(hcr, ICH_HCR_EL2);
+    isb();
+}
+
+static u16 gicv3_compute_target_list(int *base_cpu, const struct cpumask *mask,
+                                     uint64_t cluster_id)
+{
+    int cpu = *base_cpu;
+    uint64_t mpidr = cpu_logical_map(cpu);
+    u16 tlist = 0;
+
+    while ( cpu < nr_cpu_ids )
+    {
+        /*
+         * Assume that each cluster does not have more than 16 CPU's.
+         * Check is made during GICv3 initialization (gicv3_populate_rdist())
+         * on mpidr value for this. So skip this check here.
+         */
+        tlist |= 1 << (mpidr & 0xf);
+
+        cpu = cpumask_next(cpu, mask);
+        if ( cpu == nr_cpu_ids )
+        {
+            cpu--;
+            goto out;
+        }
+
+        mpidr = cpu_logical_map(cpu);
+        if ( cluster_id != (mpidr & ~MPIDR_AFF0_MASK) ) {
+            cpu--;
+            goto out;
+        }
+    }
+out:
+    *base_cpu = cpu;
+
+    return tlist;
+}
+
+static void gicv3_send_sgi(enum gic_sgi sgi, enum gic_sgi_mode mode,
+                           const cpumask_t *cpumask)
+{
+    int cpu = 0;
+    uint64_t val;
+
+    for_each_cpu(cpu, cpumask)
+    {
+        /* Mask lower 8 bits. It represent cpu in affinity level 0 */
+        uint64_t cluster_id = cpu_logical_map(cpu) & ~MPIDR_AFF0_MASK;
+        u16 tlist;
+
+        /* Get targetlist for the cluster to send SGI */
+        tlist = gicv3_compute_target_list(&cpu, cpumask, cluster_id);
+
+        /*
+         * Prepare affinity path of the cluster for which SGI is generated
+         * along with SGI number
+         */
+        val = (MPIDR_AFFINITY_LEVEL(cluster_id, 3) << 48  |
+               MPIDR_AFFINITY_LEVEL(cluster_id, 2) << 32  |
+               sgi << 24                                  |
+               MPIDR_AFFINITY_LEVEL(cluster_id, 1) << 16  |
+               tlist);
+
+        WRITE_SYSREG(val, ICC_SGI1R_EL1);
+    }
+    /* Force above writes to ICC_SGI1R_EL1 */
+    isb();
+}
+
+/* Shut down the per-CPU GIC interface */
+static void gicv3_disable_interface(void)
+{
+    spin_lock(&gicv3.lock);
+
+    gicv3_cpu_disable();
+    gicv3_hyp_disable();
+
+    spin_unlock(&gicv3.lock);
+}
+
+static void gicv3_update_lr(int lr, const struct pending_irq *p,
+                            unsigned int state)
+{
+    uint64_t grp = GICH_LR_GRP1;
+    uint64_t val = 0;
+
+    BUG_ON(lr >= gicv3_info.nr_lrs);
+    BUG_ON(lr < 0);
+
+    val =  (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
+    val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
+    val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
+
+   if ( p->desc != NULL )
+       val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
+                           << GICH_LR_PHYSICAL_SHIFT);
+
+    gicv3_ich_write_lr(lr, val);
+}
+
+static void gicv3_clear_lr(int lr)
+{
+    gicv3_ich_write_lr(lr, 0);
+}
+
+static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
+{
+    uint64_t lrv;
+
+    lrv = gicv3_ich_read_lr(lr);
+
+    lr_reg->pirq = (lrv >> GICH_LR_PHYSICAL_SHIFT) & GICH_LR_PHYSICAL_MASK;
+    lr_reg->virq = (lrv >> GICH_LR_VIRTUAL_SHIFT) & GICH_LR_VIRTUAL_MASK;
+
+    lr_reg->priority  = (lrv >> GICH_LR_PRIORITY_SHIFT) & GICH_LR_PRIORITY_MASK;
+    lr_reg->state     = (lrv >> GICH_LR_STATE_SHIFT) & GICH_LR_STATE_MASK;
+    lr_reg->hw_status = (lrv >> GICH_LR_HW_SHIFT) & GICH_LR_HW_MASK;
+    lr_reg->grp       = (lrv >> GICH_LR_GRP_SHIFT) & GICH_LR_GRP_MASK;
+}
+
+static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
+{
+    uint64_t lrv = 0;
+
+    lrv = ( ((u64)(lr->pirq & GICH_LR_PHYSICAL_MASK) << GICH_LR_PHYSICAL_SHIFT)|
+        ((u64)(lr->virq & GICH_LR_VIRTUAL_MASK)  << GICH_LR_VIRTUAL_SHIFT) |
+        ((u64)(lr->priority & GICH_LR_PRIORITY_MASK) << GICH_LR_PRIORITY_SHIFT)|
+        ((u64)(lr->state & GICH_LR_STATE_MASK) << GICH_LR_STATE_SHIFT) |
+        ((u64)(lr->hw_status & GICH_LR_HW_MASK) << GICH_LR_HW_SHIFT)  |
+        ((u64)(lr->grp & GICH_LR_GRP_MASK) << GICH_LR_GRP_SHIFT) );
+
+    gicv3_ich_write_lr(lr_reg, lrv);
+}
+
+static int gicv_v3_init(struct domain *d)
+{
+    int i;
+
+    /*
+     * Domain 0 gets the hardware address.
+     * Guests get the virtual platform layout.
+     */
+    if ( is_hardware_domain(d) )
+    {
+        d->arch.vgic.dbase = gicv3.dbase;
+        d->arch.vgic.dbase_size = gicv3.dbase_size;
+        for ( i = 0; i < gicv3.rdist_count; i++ )
+        {
+            d->arch.vgic.rbase[i] = gicv3.rdist_regions[i].base;
+            d->arch.vgic.rbase_size[i] = gicv3.rdist_regions[i].size;
+        }
+        d->arch.vgic.rdist_stride = gicv3.rdist_stride;
+        d->arch.vgic.rdist_count = gicv3.rdist_count;
+    }
+    else
+        d->arch.vgic.dbase = GUEST_GICD_BASE;
+
+    d->arch.vgic.nr_lines = 0;
+
+    return 0;
+}
+
+static void gicv3_hcr_status(uint32_t flag, bool_t status)
+{
+    uint32_t hcr;
+
+    hcr = READ_SYSREG32(ICH_HCR_EL2);
+    if ( status )
+        WRITE_SYSREG32(hcr | flag, ICH_HCR_EL2);
+    else
+        WRITE_SYSREG32(hcr & (~flag), ICH_HCR_EL2);
+    isb();
+}
+
+static unsigned int gicv3_read_vmcr_priority(void)
+{
+   return ((READ_SYSREG32(ICH_VMCR_EL2) >> GICH_VMCR_PRIORITY_SHIFT) &
+            GICH_VMCR_PRIORITY_MASK);
+}
+
+/* Only support reading GRP1 APRn registers */
+static unsigned int gicv3_read_apr(int apr_reg)
+{
+    switch ( apr_reg )
+    {
+    case 0:
+        ASSERT(gicv3.nr_priorities > 4 && gicv3.nr_priorities < 8);
+        return READ_SYSREG32(ICH_AP1R0_EL2);
+    case 1:
+        ASSERT(gicv3.nr_priorities > 5 && gicv3.nr_priorities < 8);
+        return READ_SYSREG32(ICH_AP1R1_EL2);
+    case 2:
+        ASSERT(gicv3.nr_priorities > 6 && gicv3.nr_priorities < 8);
+        return READ_SYSREG32(ICH_AP1R2_EL2);
+    default:
+        BUG();
+    }
+}
+
+static void gicv3_irq_enable(struct irq_desc *desc)
+{
+    unsigned long flags;
+
+    ASSERT(spin_is_locked(&desc->lock));
+
+    spin_lock_irqsave(&gicv3.lock, flags);
+    desc->status &= ~IRQ_DISABLED;
+    dsb(sy);
+    /* Enable routing */
+    gicv3_unmask_irq(desc);
+    spin_unlock_irqrestore(&gicv3.lock, flags);
+}
+
+static void gicv3_irq_disable(struct irq_desc *desc)
+{
+    unsigned long flags;
+
+    ASSERT(spin_is_locked(&desc->lock));
+
+    spin_lock_irqsave(&gicv3.lock, flags);
+    /* Disable routing */
+    gicv3_mask_irq(desc);
+    desc->status |= IRQ_DISABLED;
+    spin_unlock_irqrestore(&gicv3.lock, flags);
+}
+
+static unsigned int gicv3_irq_startup(struct irq_desc *desc)
+{
+    gicv3_irq_enable(desc);
+
+    return 0;
+}
+
+static void gicv3_irq_shutdown(struct irq_desc *desc)
+{
+    gicv3_irq_disable(desc);
+}
+
+static void gicv3_irq_ack(struct irq_desc *desc)
+{
+    /* No ACK -- reading IAR has done this for us */
+}
+
+static void gicv3_host_irq_end(struct irq_desc *desc)
+{
+    /* Lower the priority */
+    gicv3_eoi_irq(desc);
+    /* Deactivate */
+    gicv3_dir_irq(desc);
+}
+
+static void gicv3_guest_irq_end(struct irq_desc *desc)
+{
+    /* Lower the priority of the IRQ */
+    gicv3_eoi_irq(desc);
+    /* Deactivation happens in maintenance interrupt / via GICV */
+}
+
+static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+{
+    BUG();
+}
+
+static const hw_irq_controller gicv3_host_irq_type = {
+    .typename     = "gic-v3",
+    .startup      = gicv3_irq_startup,
+    .shutdown     = gicv3_irq_shutdown,
+    .enable       = gicv3_irq_enable,
+    .disable      = gicv3_irq_disable,
+    .ack          = gicv3_irq_ack,
+    .end          = gicv3_host_irq_end,
+    .set_affinity = gicv3_irq_set_affinity,
+};
+
+static const hw_irq_controller gicv3_guest_irq_type = {
+    .typename     = "gic-v3",
+    .startup      = gicv3_irq_startup,
+    .shutdown     = gicv3_irq_shutdown,
+    .enable       = gicv3_irq_enable,
+    .disable      = gicv3_irq_disable,
+    .ack          = gicv3_irq_ack,
+    .end          = gicv3_guest_irq_end,
+    .set_affinity = gicv3_irq_set_affinity,
+};
+
+static const struct gic_hw_operations gicv3_ops = {
+    .info                = &gicv3_info,
+    .save_state          = gicv3_save_state,
+    .restore_state       = gicv3_restore_state,
+    .dump_state          = gicv3_dump_state,
+    .gicv_setup          = gicv_v3_init,
+    .gic_host_irq_type   = &gicv3_host_irq_type,
+    .gic_guest_irq_type  = &gicv3_guest_irq_type,
+    .eoi_irq             = gicv3_eoi_irq,
+    .deactivate_irq      = gicv3_dir_irq,
+    .read_irq            = gicv3_read_irq,
+    .set_irq_properties  = gicv3_set_irq_properties,
+    .send_SGI            = gicv3_send_sgi,
+    .disable_interface   = gicv3_disable_interface,
+    .update_lr           = gicv3_update_lr,
+    .update_hcr_status   = gicv3_hcr_status,
+    .clear_lr            = gicv3_clear_lr,
+    .read_lr             = gicv3_read_lr,
+    .write_lr            = gicv3_write_lr,
+    .read_vmcr_priority  = gicv3_read_vmcr_priority,
+    .read_apr            = gicv3_read_apr,
+    .secondary_init      = gicv3_secondary_cpu_init,
+};
+
+/* Set up the GIC */
+static int __init gicv3_init(struct dt_device_node *node, const void *data)
+{
+    struct rdist_region *rdist_regs;
+    int res, i;
+    uint32_t reg;
+
+    dt_device_set_used_by(node, DOMID_XEN);
+
+    res = dt_device_get_address(node, 0, &gicv3.dbase, &gicv3.dbase_size);
+    if ( res || !gicv3.dbase )
+        panic("GICv3: Cannot find a valid distributor address");
+
+    if ( (gicv3.dbase & ~PAGE_MASK) || (gicv3.dbase_size & ~PAGE_MASK) )
+        panic("GICv3:  Found unaligned distributor address %"PRIpaddr"",
+              gicv3.dbase);
+
+    gicv3.map_dbase = ioremap_nocache(gicv3.dbase, gicv3.dbase_size);
+    if ( !gicv3.map_dbase )
+        panic("GICv3: Failed to ioremap for GIC distributor\n");
+
+    reg = readl_relaxed(GICD + GICD_PIDR2) & GICD_PIDR2_ARCH_REV_MASK;
+    if ( ((reg >> GICD_PIDR2_ARCH_REV_OFFSET) != GICD_PIDR2_ARCH_GICV3) )
+         panic("GICv3: no distributor detected\n");
+
+    if ( !dt_property_read_u32(node, "#redistributor-regions",
+                &gicv3.rdist_count) )
+        gicv3.rdist_count = 1;
+
+    if ( gicv3.rdist_count > MAX_RDIST_COUNT )
+        panic("GICv3: Number of redistributor regions is more than"
+              "%d (Increase MAX_RDIST_COUNT!!)\n", MAX_RDIST_COUNT);
+
+    rdist_regs = xzalloc_array(struct rdist_region, gicv3.rdist_count);
+    if ( !rdist_regs )
+        panic("GICv3: Failed to allocate memory for rdist regions\n");
+
+    for ( i = 0; i < gicv3.rdist_count; i++ )
+    {
+        uint64_t rdist_base, rdist_size;
+
+        res = dt_device_get_address(node, 1 + i, &rdist_base, &rdist_size);
+        if ( res || !rdist_base )
+            panic("GICv3: No rdist base found for region %d\n", i);
+
+        rdist_regs[i].base = rdist_base;
+        rdist_regs[i].size = rdist_size;
+    }
+
+    /* If stride is not set in dt. Set default to 2 * SZ_64K */
+    if ( !dt_property_read_u32(node, "redistributor-stride", &gicv3.rdist_stride) )
+        gicv3.rdist_stride = 0;
+
+    gicv3.rdist_regions= rdist_regs;
+
+    res = platform_get_irq(node, 0);
+    if ( res < 0 )
+        panic("GICv3: Cannot find the maintenance IRQ");
+    gicv3_info.maintenance_irq = res;
+
+    /* Set the GIC as the primary interrupt controller */
+    dt_interrupt_controller = node;
+
+    for ( i = 0; i < gicv3.rdist_count; i++ )
+    {
+        /* map dbase & rdist regions */
+        gicv3.rdist_regions[i].map_base =
+                ioremap_nocache(gicv3.rdist_regions[i].base,
+                                gicv3.rdist_regions[i].size);
+
+        if ( !gicv3.rdist_regions[i].map_base )
+            panic("GICv3: Failed to ioremap rdist region for region %d\n", i);
+    }
+
+    printk("GICv3 initialization:\n"
+           "      gic_dist_addr=%"PRIpaddr"\n"
+           "      gic_dist_size=%"PRIpaddr"\n"
+           "      gic_dist_mapaddr=%p\n"
+           "      gic_rdist_regions=%d\n"
+           "      gic_rdist_stride=%x\n"
+           "      gic_rdist_base=%"PRIpaddr"\n"
+           "      gic_rdist_base_size=%"PRIpaddr"\n"
+           "      gic_rdist_base_mapaddr=%p\n"
+           "      gic_maintenance_irq=%u\n",
+           gicv3.dbase, gicv3.dbase_size, gicv3.map_dbase, gicv3.rdist_count,
+           gicv3.rdist_stride, gicv3.rdist_regions[0].base,
+           gicv3.rdist_regions[0].size, gicv3.rdist_regions[0].map_base,
+           gicv3_info.maintenance_irq);
+
+    spin_lock_init(&gicv3.lock);
+
+    spin_lock(&gicv3.lock);
+
+    gicv3_dist_init();
+    res = gicv3_cpu_init();
+    gicv3_hyp_init();
+
+    gicv3_info.hw_version = GIC_V3;
+    /* Register hw ops*/
+    register_gic_ops(&gicv3_ops);
+
+    spin_unlock(&gicv3.lock);
+
+    return res;
+}
+
+static const char * const gicv3_dt_compat[] __initconst =
+{
+    DT_COMPAT_GIC_V3,
+    NULL
+};
+
+DT_DEVICE_START(gicv3, "GICv3", DEVICE_GIC)
+        .compatible = gicv3_dt_compat,
+        .init = gicv3_init,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 32d0554..787e93c 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -99,6 +99,14 @@ struct arch_domain
         /* Base address for guest GIC */
         paddr_t dbase; /* Distributor base address */
         paddr_t cbase; /* CPU base address */
+#ifdef CONFIG_ARM_64
+        /* GIC V3 addressing */
+        paddr_t dbase_size; /* Distributor base size */
+        paddr_t rbase[MAX_RDIST_COUNT];      /* Re-Distributor base address */
+        paddr_t rbase_size[MAX_RDIST_COUNT]; /* Re-Distributor size */
+        uint32_t rdist_stride;               /* Re-Distributor stride */
+        int rdist_count;                     /* No. of Re-Distributors */
+#endif
     } vgic;
 
     struct vuart {
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index a0c07bf..c4daf5b 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -18,6 +18,10 @@
 #ifndef __ASM_ARM_GIC_H__
 #define __ASM_ARM_GIC_H__
 
+#define NR_GIC_LOCAL_IRQS  NR_LOCAL_IRQS
+#define NR_GIC_SGI         16
+#define MAX_RDIST_COUNT    4
+
 #define GICD_CTLR       (0x000)
 #define GICD_TYPER      (0x004)
 #define GICD_IIDR       (0x008)
@@ -154,6 +158,20 @@
 #define DT_MATCH_GIC_V2 DT_MATCH_COMPATIBLE(DT_COMPAT_GIC_CORTEX_A15), \
                         DT_MATCH_COMPATIBLE(DT_COMPAT_GIC_CORTEX_A7)
 
+#define DT_COMPAT_GIC_V3             "arm,gic-v3"
+
+#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE(DT_COMPAT_GIC_V3)
+
+/*
+ * GICv3 registers that needs to be saved/restored
+ */
+struct gic_v3 {
+    uint32_t hcr, vmcr, sre_el1;
+    uint32_t apr0[4];
+    uint32_t apr1[4];
+    uint64_t lr[16];
+};
+
 /*
  * GICv2 register that needs to be saved/restored
  * on VCPU context switch
@@ -170,6 +188,7 @@ struct gic_v2 {
  */
 union gic_state_data {
     struct gic_v2 v2;
+    struct gic_v3 v3;
 };
 
 /*
@@ -189,6 +208,7 @@ struct gic_lr {
 
 enum gic_version {
     GIC_V2,
+    GIC_V3,
 };
 
 extern enum gic_version gic_hw_version(void);
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
new file mode 100644
index 0000000..115c998
--- /dev/null
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -0,0 +1,158 @@
+/*
+ * ARM Generic Interrupt Controller v3 definitions
+ *
+ * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>
+ * Copyright (c) 2014 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ASM_ARM_GIC_V3_DEFS_H__
+#define __ASM_ARM_GIC_V3_DEFS_H__
+
+/*
+ * Additional registers defined in GIC v3.
+ * Common GICD registers are defined in gic.h
+ */
+
+#define GICD_STATUSR                 (0x010)
+#define GICD_SETSPI_NSR              (0x040)
+#define GICD_CLRSPI_NSR              (0x048)
+#define GICD_SETSPI_SR               (0x050)
+#define GICD_CLRSPI_SR               (0x058)
+#define GICD_IROUTER                 (0x6000)
+#define GICD_IROUTER31               (0x60F8)
+#define GICD_IROUTER32               (0x6100)
+#define GICD_IROUTERN                (0x7FF8)
+#define GICD_PIDR0                   (0xFFE0)
+#define GICD_PIDR1                   (0xFFE4)
+#define GICD_PIDR2                   (0xFFE8)
+#define GICD_PIDR3                   (0xFFEC)
+#define GICD_PIDR4                   (0xFFD0)
+#define GICD_PIDR5                   (0xFFD4)
+#define GICD_PIDR7                   (0xFFDC)
+
+#define GICC_SRE_EL2_SRE             (1UL << 0)
+#define GICC_SRE_EL2_DFB             (1UL << 1)
+#define GICC_SRE_EL2_DIB             (1UL << 2)
+#define GICC_SRE_EL2_ENEL1           (1UL << 3)
+
+#define GICD_CTLR_RWP                (1UL << 31)
+#define GICD_CTLR_ARE_NS             (1U << 4)
+#define GICD_CTLR_ENABLE_G1A         (1U << 1)
+#define GICD_CTLR_ENABLE_G1          (1U << 0)
+#define GICD_IROUTER_SPI_MODE_ANY    (1UL << 31)
+
+#define GICC_CTLR_EL1_EOImode_drop   (1U << 1)
+
+#define GICR_WAKER_ProcessorSleep    (1U << 1)
+#define GICR_WAKER_ChildrenAsleep    (1U << 2)
+
+#define GICD_PIDR2_ARCH_REV_MASK     (0xf0)
+#define GICD_PIDR2_ARCH_REV_OFFSET   (0x4)
+#define GICD_PIDR2_ARCH_GICV3        (0x3)
+#define GICD_PIDR2_USES_JEP_MASK     (0x08)
+#define GICD_PIDR2_JEP_ID_MASK       (0x07)
+
+#define GICR_PIDR2_ARCH_REV_MASK     GICD_PIDR2_ARCH_REV_MASK
+#define GICR_PIDR2_ARCH_REV_OFFSET   GICD_PIDR2_ARCH_REV_OFFSET
+#define GICR_PIDR2_ARCH_GICV3        GICD_PIDR2_ARCH_GICV3
+#define GICR_PIDR2_USES_JEP_MASK     GICD_PIDR2_USES_JEP_MASK
+#define GICR_PIDR2_JEP_ID_MASK       GICD_PIDR2_JEP_ID_MASK
+
+#define GICR_SYNCR_NOT_BUSY          1
+/*
+ * Implementation defined value JEP106?
+ * use physical hw value for now
+ */
+#define GICV3_GICD_IIDR_VAL          0x34c
+#define GICV3_GICR_IIDR_VAL          GICV3_GICD_IIDR_VAL
+
+#define GICR_CTLR                    (0x0000)
+#define GICR_IIDR                    (0x0004)
+#define GICR_TYPER                   (0x0008)
+#define GICR_STATUSR                 (0x0010)
+#define GICR_WAKER                   (0x0014)
+#define GICR_SETLPIR                 (0x0040)
+#define GICR_CLRLPIR                 (0x0048)
+#define GICR_PROPBASER               (0x0070)
+#define GICR_PENDBASER               (0x0078)
+#define GICR_INVLPIR                 (0x00A0)
+#define GICR_INVALLR                 (0x00B0)
+#define GICR_SYNCR                   (0x00C0)
+#define GICR_MOVLPIR                 (0x100)
+#define GICR_MOVALLR                 (0x0110)
+#define GICR_PIDR0                   GICD_PIDR0
+#define GICR_PIDR1                   GICD_PIDR1
+#define GICR_PIDR2                   GICD_PIDR2
+#define GICR_PIDR3                   GICD_PIDR3
+#define GICR_PIDR4                   GICD_PIDR4
+#define GICR_PIDR5                   GICD_PIDR5
+#define GICR_PIDR7                   GICD_PIDR7
+
+/* GICR for SGI's & PPI's */
+
+#define GICR_IGROUPR0                (0x0080)
+#define GICR_IGRPMODR0               (0x0F80)
+#define GICR_ISENABLER0              (0x0100)
+#define GICR_ICENABLER0              (0x0180)
+#define GICR_ISPENDR0                (0x0200)
+#define GICR_ICPENDR0                (0x0280)
+#define GICR_ISACTIVER0              (0x0300)
+#define GICR_ICACTIVER0              (0x0380)
+#define GICR_IPRIORITYR0             (0x0400)
+#define GICR_IPRIORITYR7             (0x041C)
+#define GICR_ICFGR0                  (0x0C00)
+#define GICR_ICFGR1                  (0x0C04)
+#define GICR_NSACR                   (0x0E00)
+
+#define GICR_TYPER_PLPIS             (1U << 0)
+#define GICR_TYPER_VLPIS             (1U << 1)
+#define GICR_TYPER_LAST              (1U << 4)
+
+#define DEFAULT_PMR_VALUE            0xff
+
+#define GICH_VMCR_EOI                (1 << 9)
+#define GICH_VMCR_VENG1              (1 << 1)
+
+#define GICH_LR_VIRTUAL_MASK         0xffff
+#define GICH_LR_VIRTUAL_SHIFT        0
+#define GICH_LR_PHYSICAL_MASK        0x3ff
+#define GICH_LR_PHYSICAL_SHIFT       32
+#define GICH_LR_STATE_MASK           0x3
+#define GICH_LR_STATE_SHIFT          62
+#define GICH_LR_PRIORITY_MASK        0xff
+#define GICH_LR_PRIORITY_SHIFT       48
+#define GICH_LR_HW_MASK              0x1
+#define GICH_LR_HW_SHIFT             61
+#define GICH_LR_GRP_MASK             0x1
+#define GICH_LR_GRP_SHIFT            60
+#define GICH_LR_MAINTENANCE_IRQ      (1UL<<41)
+#define GICH_LR_GRP1                 (1UL<<60)
+#define GICH_LR_HW                   (1UL<<61)
+
+#define GICH_VTR_NRLRGS              0x3f
+#define GICH_VTR_PRIBITS_MASK        0x7
+#define GICH_VTR_PRIBITS_SHIFT       29
+
+#define GICH_VMCR_PRIORITY_MASK      0xff
+#define GICH_VMCR_PRIORITY_SHIFT     24
+
+#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index bdfff4e..ccaa9da 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -17,6 +17,20 @@
 #define MPIDR_HWID_MASK     _AC(0xffffff,U)
 #define MPIDR_INVALID       (~MPIDR_HWID_MASK)
 
+/*
+ * Macros to extract affinity level. picked from kernel
+ */
+
+#define MPIDR_LEVEL_BITS_SHIFT  3
+#define MPIDR_LEVEL_BITS        (1 << MPIDR_LEVEL_BITS_SHIFT)
+#define MPIDR_LEVEL_MASK        ((1 << MPIDR_LEVEL_BITS) - 1)
+
+#define MPIDR_LEVEL_SHIFT(level) \
+         (((1 << level) >> 1) << MPIDR_LEVEL_BITS_SHIFT)
+
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+         ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
+
 /* TTBCR Translation Table Base Control Register */
 #define TTBCR_EAE    _AC(0x80000000,U)
 #define TTBCR_N_MASK _AC(0x07,U)
diff --git a/xen/include/asm-arm/sysregs.h b/xen/include/asm-arm/sysregs.h
index b00871c..b4616ac 100644
--- a/xen/include/asm-arm/sysregs.h
+++ b/xen/include/asm-arm/sysregs.h
@@ -100,6 +100,59 @@
 #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
 #define HSR_SYSREG_CNTP_TVAL_EL0  HSR_SYSREG(3,3,c14,c2,0)
 
+/*
+ * GIC System register assembly aliases picked from kernel
+ */
+#define ICC_PMR_EL1               S3_0_C4_C6_0
+#define ICC_DIR_EL1               S3_0_C12_C11_1
+#define ICC_SGI1R_EL1             S3_0_C12_C11_5
+#define ICC_EOIR1_EL1             S3_0_C12_C12_1
+#define ICC_IAR1_EL1              S3_0_C12_C12_0
+#define ICC_BPR1_EL1              S3_0_C12_C12_3
+#define ICC_CTLR_EL1              S3_0_C12_C12_4
+#define ICC_SRE_EL1               S3_0_C12_C12_5
+#define ICC_IGRPEN1_EL1           S3_0_C12_C12_7
+
+#define ICH_VSEIR_EL2             S3_4_C12_C9_4
+#define ICC_SRE_EL2               S3_4_C12_C9_5
+#define ICH_HCR_EL2               S3_4_C12_C11_0
+#define ICH_VTR_EL2               S3_4_C12_C11_1
+#define ICH_MISR_EL2              S3_4_C12_C11_2
+#define ICH_EISR_EL2              S3_4_C12_C11_3
+#define ICH_ELSR_EL2              S3_4_C12_C11_5
+#define ICH_VMCR_EL2              S3_4_C12_C11_7
+
+#define __LR0_EL2(x)              S3_4_C12_C12_ ## x
+#define __LR8_EL2(x)              S3_4_C12_C13_ ## x
+
+#define ICH_LR0_EL2               __LR0_EL2(0)
+#define ICH_LR1_EL2               __LR0_EL2(1)
+#define ICH_LR2_EL2               __LR0_EL2(2)
+#define ICH_LR3_EL2               __LR0_EL2(3)
+#define ICH_LR4_EL2               __LR0_EL2(4)
+#define ICH_LR5_EL2               __LR0_EL2(5)
+#define ICH_LR6_EL2               __LR0_EL2(6)
+#define ICH_LR7_EL2               __LR0_EL2(7)
+#define ICH_LR8_EL2               __LR8_EL2(0)
+#define ICH_LR9_EL2               __LR8_EL2(1)
+#define ICH_LR10_EL2              __LR8_EL2(2)
+#define ICH_LR11_EL2              __LR8_EL2(3)
+#define ICH_LR12_EL2              __LR8_EL2(4)
+#define ICH_LR13_EL2              __LR8_EL2(5)
+#define ICH_LR14_EL2              __LR8_EL2(6)
+#define ICH_LR15_EL2              __LR8_EL2(7)
+
+#define __AP0Rx_EL2(x)            S3_4_C12_C8_ ## x
+#define ICH_AP0R0_EL2             __AP0Rx_EL2(0)
+#define ICH_AP0R1_EL2             __AP0Rx_EL2(1)
+#define ICH_AP0R2_EL2             __AP0Rx_EL2(2)
+#define ICH_AP0R3_EL2             __AP0Rx_EL2(3)
+
+#define __AP1Rx_EL2(x)            S3_4_C12_C9_ ## x
+#define ICH_AP1R0_EL2             __AP1Rx_EL2(0)
+#define ICH_AP1R1_EL2             __AP1Rx_EL2(1)
+#define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
+#define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
 
 #endif
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
                   ` (2 preceding siblings ...)
  2014-07-23 13:41 ` [PATCH v8 3/7] xen/arm: Add support for GIC v3 vijay.kilari
@ 2014-07-23 13:41 ` vijay.kilari
  2014-07-23 13:51   ` Vijay Kilari
  2014-07-23 13:41 ` [PATCH v8 5/7] xen/arm: Update Dom0 GIC dt node with GICv3 information vijay.kilari
                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Add virtual GICv3 driver support.
Also, with this patch vgic_irq_rank structure is modified to
hold GICv2 GICD_TARGET and GICv3 GICD_ROUTER registers under
union.

This patch adds only basic GICv3 support.
Does not support Interrupt Translation support (ITS)

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v8: - Fixed printk coding styles
    - Moved GICD_PIDRn and GICR_PIDRn macros to vgic-v3.c
      from header file
   - Check is made on return value of vgic_v3_init()
v7: Fixed coding style.
v6: Removed byte read access for IROUTERN register.
---
 xen/arch/arm/Makefile      |    1 +
 xen/arch/arm/vgic-v2.c     |    8 +-
 xen/arch/arm/vgic-v3.c     |  919 ++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic.c        |    6 +
 xen/include/asm-arm/vgic.h |   13 +-
 5 files changed, 942 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 1684c09..ba61e58 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -28,6 +28,7 @@ obj-y += smp.o
 obj-y += shutdown.o
 obj-y += traps.o
 obj-y += vgic.o vgic-v2.o
+obj-$(CONFIG_ARM_64) += vgic-v3.o
 obj-y += vtimer.o
 obj-y += vuart.o
 obj-y += hvm.o
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index 2102e43..d65146a 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -133,7 +133,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         if ( rank == NULL) goto read_as_zero;
 
         vgic_lock_rank(v, rank);
-        *r = rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+        *r = rank->v2.itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
                                            DABT_WORD)];
         if ( dabt.size == DABT_BYTE )
             *r = vgic_byte_read(*r, dabt.sign, gicd_reg);
@@ -361,10 +361,10 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
         if ( dabt.size == DABT_WORD )
-            rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+            rank->v2.itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
                                           DABT_WORD)] = *r;
         else
-            vgic_byte_write(&rank->itargets[REG_RANK_INDEX(8,
+            vgic_byte_write(&rank->v2.itargets[REG_RANK_INDEX(8,
                        gicd_reg - GICD_ITARGETSR, DABT_WORD)], *r, gicd_reg);
         vgic_unlock_rank(v, rank);
         return 1;
@@ -466,7 +466,7 @@ static int vgic_v2_vcpu_init(struct vcpu *v)
 
     /* For SGI and PPI the target is always this CPU */
     for ( i = 0 ; i < 8 ; i++ )
-        v->arch.vgic.private_irqs->itargets[i] =
+        v->arch.vgic.private_irqs->v2.itargets[i] =
               (1<<(v->vcpu_id+0))
             | (1<<(v->vcpu_id+8))
             | (1<<(v->vcpu_id+16))
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
new file mode 100644
index 0000000..6abaac3
--- /dev/null
+++ b/xen/arch/arm/vgic-v3.c
@@ -0,0 +1,919 @@
+/*
+ * xen/arch/arm/vgic-v3.c
+ *
+ * ARM Virtual Generic Interrupt Controller v3 support
+ * based on xen/arch/arm/vgic.c
+ *
+ * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>
+ * Copyright (c) 2014 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/bitops.h>
+#include <xen/config.h>
+#include <xen/lib.h>
+#include <xen/init.h>
+#include <xen/softirq.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+#include <asm/current.h>
+#include <asm/device.h>
+#include <asm/mmio.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/vgic.h>
+
+/* GICD_PIDRn register values for ARM implementations */
+#define GICV3_GICD_PIDR0  0x92
+#define GICV3_GICD_PIDR1  0xb4
+#define GICV3_GICD_PIDR2  0x3b
+#define GICV3_GICD_PIDR4  0x04
+
+/* GICR_PIDRn register values for ARM implementations */
+#define GICV3_GICR_PIDR0  0x93
+#define GICV3_GICR_PIDR1  GICV3_GICD_PIDR1
+#define GICV3_GICR_PIDR2  GICV3_GICD_PIDR2
+#define GICV3_GICR_PIDR4  GICV3_GICD_PIDR4
+
+static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
+                                         uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    uint64_t aff;
+
+    switch ( gicr_reg )
+    {
+    case GICR_CTLR:
+        /* We have not implemented LPI's, read zero */
+        goto read_as_zero;
+    case GICR_IIDR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_IIDR_VAL;
+        return 1;
+    case GICR_TYPER:
+        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+        /* TBD: Update processor id in [23:8] when ITS support is added */
+        aff = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 56 |
+               MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
+               MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
+               MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
+        *r = aff;
+        return 1;
+    case GICR_STATUSR:
+        /* Not implemented */
+        goto read_as_zero;
+    case GICR_WAKER:
+        /* Power management is not implemented */
+        goto read_as_zero;
+    case GICR_SETLPIR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+    case GICR_CLRLPIR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+    case GICR_PROPBASER:
+        /* LPI's not implemented */
+        goto read_as_zero_64;
+    case GICR_PENDBASER:
+        /* LPI's not implemented */
+        goto read_as_zero_64;
+    case GICR_INVLPIR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+    case GICR_INVALLR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+        return 0;
+    case GICR_SYNCR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* RO . But when read it always returns busy bito bit[0] */
+        *r = GICR_SYNCR_NOT_BUSY;
+        return 1;
+    case GICR_MOVLPIR:
+        /* WO Read as zero */
+        goto read_as_zero_64;
+    case GICR_MOVALLR:
+        /* WO Read as zero */
+        goto read_as_zero_64;
+    case GICR_PIDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR0;
+         return 1;
+    case GICR_PIDR1:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR1;
+         return 1;
+    case GICR_PIDR2:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR2;
+         return 1;
+    case GICR_PIDR3:
+        /* Manufacture/customer defined */
+        goto read_as_zero;
+    case GICR_PIDR4:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR4;
+         return 1;
+    case GICR_PIDR5 ... GICR_PIDR7:
+        /* Reserved0 */
+        goto read_as_zero;
+    default:
+        printk("vGICv3: vGICR: read r%d offset %#08x\n not found",
+               dabt.reg, gicr_reg);
+        return 0;
+    }
+bad_width:
+    printk("vGICv3: vGICR: bad read width %d r%d offset %#08x\n",
+           dabt.size, dabt.reg, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
+                                          uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+
+    switch ( gicr_reg )
+    {
+    case GICR_CTLR:
+        /* LPI's not implemented */
+        goto write_ignore;
+    case GICR_IIDR:
+        /* RO */
+        goto write_ignore;
+    case GICR_TYPER:
+        /* RO */
+        goto write_ignore_64;
+    case GICR_STATUSR:
+        /* Not implemented */
+        goto write_ignore;
+    case GICR_WAKER:
+        /* Power mgmt not implemented */
+        goto write_ignore;
+    case GICR_SETLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_CLRLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_PROPBASER:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_PENDBASER:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_INVLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_INVALLR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_SYNCR:
+        /* RO */
+        goto write_ignore;
+    case GICR_MOVLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_MOVALLR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_PIDR7... GICR_PIDR0:
+        /* RO */
+        goto write_ignore;
+    default:
+        printk("vGICR: write r%d offset %#08x\n not found", dabt.reg, gicr_reg);
+        return 0;
+    }
+bad_width:
+    printk("vGICR: bad write width %d r%d=%"PRIregister" offset %#08x\n",
+           dabt.size, dabt.reg, *r, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    return 1;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+}
+
+static int __vgic_v3_distr_common_mmio_read(struct vcpu *v, mmio_info_t *info,
+                                            uint32_t reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+
+    switch ( reg )
+    {
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ISENABLER, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->ienable;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ICENABLER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->ienable;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ISPENDR, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = vgic_byte_read(rank->ipend, dabt.sign, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ICPENDR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = vgic_byte_read(rank->ipend, dabt.sign, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ISACTIVER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->iactive;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ICACTIVER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->iactive;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 8, reg - GICD_IPRIORITYR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+
+        vgic_lock_rank(v, rank);
+        *r = rank->ipriority[REG_RANK_INDEX(8, reg - GICD_IPRIORITYR,
+                                            DABT_WORD)];
+        if ( dabt.size == DABT_BYTE )
+            *r = vgic_byte_read(*r, dabt.sign, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICFGR ... GICD_ICFGRN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 2, reg - GICD_ICFGR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGR, DABT_WORD)];
+        vgic_unlock_rank(v, rank);
+        return 1;
+    default:
+        printk("vGICv3: vGICD/vGICR: unhandled read r%d offset %#08x\n",
+               dabt.reg, reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR,
+            "vGICv3: vGICD/vGICR: bad read width %d r%d offset %#08x\n",
+            dabt.size, dabt.reg, reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int __vgic_v3_distr_common_mmio_write(struct vcpu *v, mmio_info_t *info,
+                                             uint32_t reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    uint32_t tr;
+
+    switch ( reg )
+    {
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ISENABLER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        tr = rank->ienable;
+        rank->ienable |= *r;
+        vgic_unlock_rank(v, rank);
+        /* The irq number is extracted from offset. so shift by register size */
+        vgic_enable_irqs(v, (*r) & (~tr), (reg - GICD_ISENABLER) >> DABT_WORD);
+        return 1;
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ICENABLER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        tr = rank->ienable;
+        rank->ienable &= ~*r;
+        vgic_unlock_rank(v, rank);
+        /* The irq number is extracted from offset. so shift by register size */
+        vgic_disable_irqs(v, (*r) & tr, (reg - GICD_ICENABLER) >> DABT_WORD);
+        return 1;
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ISPENDR, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->ipend = *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ICPENDR, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->ipend &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ISACTIVER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->iactive &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ICACTIVER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->iactive &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 8, reg - GICD_IPRIORITYR, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        if ( dabt.size == DABT_WORD )
+            rank->ipriority[REG_RANK_INDEX(8, reg - GICD_IPRIORITYR,
+                                           DABT_WORD)] = *r;
+        else
+            vgic_byte_write(&rank->ipriority[REG_RANK_INDEX(8,
+                       reg - GICD_IPRIORITYR, DABT_WORD)], *r, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICFGR: /* Restricted to configure SGIs */
+        goto write_ignore;
+    case GICD_ICFGR + 4 ... GICD_ICFGRN: /* PPI + SPIs */
+        /* ICFGR1 for PPI's, which is implementation defined
+           if ICFGR1 is programmable or not. We chose to program */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 2, reg - GICD_ICFGR, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGR, DABT_WORD)] = *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    default:
+        printk("vGICv3: vGICD/vGICR: unhandled write r%d "
+               "=%"PRIregister" offset %#08x\n", dabt.reg, *r, reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR,
+            "vGICv3: vGICD/vGICR: bad write width %d r%d=%"PRIregister" "
+            "offset %#08x\n", dabt.size, dabt.reg, *r, reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+}
+
+static int vgic_v3_rdistr_sgi_mmio_read(struct vcpu *v, mmio_info_t *info,
+                                        uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+
+    switch ( gicr_reg )
+    {
+    case GICR_IGRPMODR0:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+    case GICR_IGROUPR0:
+    case GICR_ISENABLER0:
+    case GICR_ICENABLER0:
+    case GICR_ISACTIVER0:
+    case GICR_ICACTIVER0:
+    case GICR_IPRIORITYR0...GICR_IPRIORITYR7:
+    case GICR_ICFGR0... GICR_ICFGR1:
+         /*
+          * Above registers offset are common with GICD.
+          * So handle in common with GICD handling
+          */
+        return __vgic_v3_distr_common_mmio_read(v, info, gicr_reg);
+    case GICR_ISPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ISPENDR0, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->pendsgi;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_ICPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ICPENDR0, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->pendsgi;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_NSACR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        return 1;
+    default:
+        printk("vGICv3: vGICR: read r%d offset %#08x\n not found",
+               dabt.reg, gicr_reg);
+        return 0;
+    }
+bad_width:
+    printk("vGICv3: vGICR: bad read width %d r%d offset %#08x\n",
+           dabt.size, dabt.reg, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int vgic_v3_rdistr_sgi_mmio_write(struct vcpu *v, mmio_info_t *info,
+                                         uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+
+    switch ( gicr_reg )
+    {
+    case GICR_IGRPMODR0:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    case GICR_IGROUPR0:
+    case GICR_ISENABLER0:
+    case GICR_ICENABLER0:
+    case GICR_ISACTIVER0:
+    case GICR_ICACTIVER0:
+    case GICR_ICFGR1:
+    case GICR_IPRIORITYR0...GICR_IPRIORITYR7:
+         /*
+          * Above registers offset are common with GICD.
+          * So handle common with GICD handling
+          */
+        return __vgic_v3_distr_common_mmio_write(v, info, gicr_reg);
+    case GICR_ISPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ISACTIVER0, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        /* TODO: we just store the SGI pending status. Handle it properly */
+        rank->pendsgi |= *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_ICPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ISACTIVER0, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        /* TODO: we just store the SGI pending status. Handle it properly */
+        rank->pendsgi &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_NSACR:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    default:
+        printk("vGICv3: vGICR SGI: write r%d offset %#08x\n not found",
+               dabt.reg, gicr_reg);
+        return 0;
+    }
+
+bad_width:
+    printk("vGICR SGI: bad write width %d r%d=%"PRIregister" offset %#08x\n",
+           dabt.size, dabt.reg, *r, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+}
+
+static int vgic_v3_rdistr_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+    uint32_t offset;
+
+    offset = info->gpa & (v->domain->arch.vgic.rdist_stride - 1);
+
+    if ( offset < SZ_64K )
+        return __vgic_v3_rdistr_rd_mmio_read(v, info, offset);
+    else  if ( (offset >= SZ_64K) && (offset < 2 * SZ_64K) )
+        return vgic_v3_rdistr_sgi_mmio_read(v, info, (offset - SZ_64K));
+    else
+        gdprintk(XENLOG_WARNING,
+                 "vGICv3: vGICR: unknown gpa read address %"PRIpaddr"\n",
+                 info->gpa);
+
+    return 0;
+}
+
+static int vgic_v3_rdistr_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+    uint32_t offset;
+
+    offset = info->gpa & (v->domain->arch.vgic.rdist_stride - 1);
+    if ( offset < SZ_64K )
+        return __vgic_v3_rdistr_rd_mmio_write(v, info, offset);
+    else  if ( (offset >= SZ_64K) && (offset < 2 * SZ_64K) )
+        return vgic_v3_rdistr_sgi_mmio_write(v, info, (offset - SZ_64K));
+    else
+        gdprintk(XENLOG_WARNING,
+                 "vGICV3: vGICR: unknown gpa write address %"PRIpaddr"\n",
+                 info->gpa);
+
+    return 0;
+}
+
+static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
+
+    switch ( gicd_reg )
+    {
+    case GICD_CTLR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        vgic_lock(v);
+        *r = v->domain->arch.vgic.ctlr;
+        vgic_unlock(v);
+        return 1;
+    case GICD_TYPER:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* No secure world support for guests. */
+        *r = (((v->domain->max_vcpus << 5) & GICD_TYPE_CPUS ) |
+              ((v->domain->arch.vgic.nr_lines / 32) & GICD_TYPE_LINES));
+        return 1;
+    case GICD_STATUSR:
+        /*
+         *  Optional, Not implemented for now.
+         *  Update to support guest debugging.
+         */
+        goto read_as_zero;
+    case GICD_IIDR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_IIDR_VAL;
+        return 1;
+    case 0x020 ... 0x03c:
+    case 0xc000 ... 0xffcc:
+        /* Implementation defined -- read as zero */
+        goto read_as_zero;
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+    case GICD_ICFGR ... GICD_ICFGRN:
+        /*
+         * Above all register are common with GICR and GICD
+         * Manage in common
+         */
+        return __vgic_v3_distr_common_mmio_read(v, info, gicd_reg);
+    case GICD_IROUTER ... GICD_IROUTER31:
+        /* SGI/PPI is RES0 */
+        goto read_as_zero_64;
+    case GICD_IROUTER32 ... GICD_IROUTERN:
+        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        /* IROUTER is 64 bit so, to make it byte size right shift by 3.
+           Here once. macro REG_RANK_INDEX will do it twice */
+        *r = rank->v3.irouter[REG_RANK_INDEX(64,
+                           (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)];
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_NSACR ... GICD_NSACRN:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+    case GICD_SGIR:
+        /* Read as ICH_SGIR system register with SRE set. So ignore */
+        goto read_as_zero;
+    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
+        /* Replaced with GICR_ICPENDR0. So ignore write */
+        goto read_as_zero;
+    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
+        /* Replaced with GICR_ISPENDR0. So ignore write */
+        goto read_as_zero;
+    case GICD_PIDR0:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR0;
+        return 1;
+    case GICD_PIDR1:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR1;
+        return 1;
+    case GICD_PIDR2:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR2;
+        return 1;
+    case GICD_PIDR3:
+        /* GICv3 identification value. Manufacturer/Customer defined */
+        goto read_as_zero;
+    case GICD_PIDR4:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR4;
+        return 1;
+    case GICD_PIDR5 ... GICD_PIDR7:
+        /* Reserved0 */
+        goto read_as_zero;
+    case 0x00c:
+    case 0x044:
+    case 0x04c:
+    case 0x05c ... 0x07c:
+    case 0xf30 ... 0x5fcc:
+    case 0x8000 ... 0xbfcc:
+        /* These are reserved register addresses */
+        printk("vGICv3: vGICD: read unknown 0x00c .. 0xfcc r%d offset %#08x\n",
+               dabt.reg, gicd_reg);
+        goto read_as_zero;
+    default:
+        printk("vGICv3: vGICD: unhandled read r%d offset %#08x\n",
+               dabt.reg, gicd_reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR, "vGICv3: vGICD: bad read width %d r%d offset %#08x\n",
+            dabt.size, dabt.reg, gicd_reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
+
+    switch ( gicd_reg )
+    {
+    case GICD_CTLR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* Ignore all but the enable bit */
+        v->domain->arch.vgic.ctlr = (*r) & GICD_CTL_ENABLE;
+        return 1;
+    case GICD_TYPER:
+        /* RO -- write ignored */
+        goto write_ignore;
+    case GICD_IIDR:
+        /* RO -- write ignored */
+        goto write_ignore;
+    case GICD_STATUSR:
+        /* RO -- write ignored */
+        goto write_ignore;
+    case GICD_SETSPI_NSR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case GICD_CLRSPI_NSR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case GICD_SETSPI_SR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case GICD_CLRSPI_SR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case 0x020 ... 0x03c:
+    case 0xc000 ... 0xffcc:
+        /* Implementation defined -- write ignored */
+        printk("vGICv3: vGICD: write unknown 0x020 - 0x03c r%d offset %#08x\n",
+               dabt.reg, gicd_reg);
+        goto write_ignore;
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+    case GICD_ICFGR ... GICD_ICFGRN:
+        /* Above registers are common with GICR and GICD
+         * Manage in common */
+        return __vgic_v3_distr_common_mmio_write(v, info, gicd_reg);
+    case GICD_IROUTER ... GICD_IROUTER31:
+        /* SGI/PPI is RES0 */
+        goto write_ignore_64;
+    case GICD_IROUTER32 ... GICD_IROUTERN:
+        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
+        if ( rank == NULL) goto write_ignore_64;
+        if ( *r )
+        {
+            /* TODO: Ignored. We don't support irq delivery for vcpu != 0 */
+            gdprintk(XENLOG_DEBUG,
+                     "SPI delivery to secondary cpus not supported\n");
+            goto write_ignore_64;
+        }
+        vgic_lock_rank(v, rank);
+        rank->v3.irouter[REG_RANK_INDEX(64,
+                      (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)] = *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_NSACR ... GICD_NSACRN:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    case GICD_SGIR:
+        /* it is accessed as system register in GICv3 */
+        goto write_ignore;
+    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
+        /* Replaced with GICR_ICPENDR0. So ignore write */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        return 0;
+    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
+        /* Replaced with GICR_ISPENDR0. So ignore write */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        return 0;
+    case GICD_PIDR7... GICD_PIDR0:
+        /* RO -- write ignore */
+        goto write_ignore;
+    case 0x00c:
+    case 0x044:
+    case 0x04c:
+    case 0x05c ... 0x07c:
+    case 0xf30 ... 0x5fcc:
+    case 0x8000 ... 0xbfcc:
+        /* Reserved register addresses */
+        printk("vGICv3: vGICD: write unknown 0x00c 0xfcc  r%d offset %#08x\n",
+                dabt.reg, gicd_reg);
+        goto write_ignore;
+    default:
+        printk("vGICv3: vGICD: unhandled write r%d=%"PRIregister" "
+               "offset %#08x\n", dabt.reg, *r, gicd_reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR,
+            "VGICv3: vGICD: bad write width %d r%d=%"PRIregister" "
+            "offset %#08x\n", dabt.size, dabt.reg, *r, gicd_reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+
+write_ignore_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    return 1;
+}
+
+static const struct mmio_handler_ops vgic_rdistr_mmio_handler = {
+    .read_handler  = vgic_v3_rdistr_mmio_read,
+    .write_handler = vgic_v3_rdistr_mmio_write,
+};
+
+static const struct mmio_handler_ops vgic_distr_mmio_handler = {
+    .read_handler  = vgic_v3_distr_mmio_read,
+    .write_handler = vgic_v3_distr_mmio_write,
+};
+
+static int vgicv3_vcpu_init(struct vcpu *v)
+{
+    int i;
+    uint64_t affinity;
+
+    /* For SGI and PPI the target is always this CPU */
+    affinity = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 32 |
+                MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 16 |
+                MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 8  |
+                MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0));
+
+    for ( i = 0 ; i < 32 ; i++ )
+        v->arch.vgic.private_irqs->v3.irouter[i] = affinity;
+
+    return 0;
+}
+
+static int vgicv3_domain_init(struct domain *d)
+{
+    int i;
+
+    /* We rely on gicv init to get dbase and size */
+    register_mmio_handler(d, &vgic_distr_mmio_handler, d->arch.vgic.dbase,
+                          d->arch.vgic.dbase_size);
+
+    /*
+     * Register mmio handler per redistributor region but not for
+     * every sgi rdist region which is per core.
+     * The redistributor region encompasses per core sgi region.
+     */
+    for ( i = 0; i < d->arch.vgic.rdist_count; i++ )
+        register_mmio_handler(d, &vgic_rdistr_mmio_handler,
+            d->arch.vgic.rbase[i], d->arch.vgic.rbase_size[i]);
+
+    return 0;
+}
+
+static const struct vgic_ops v3_ops = {
+    .vcpu_init   = vgicv3_vcpu_init,
+    .domain_init = vgicv3_domain_init,
+};
+
+int vgic_v3_init(struct domain *d)
+{
+    register_vgic_ops(d, &v3_ops);
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index d223db4..622219f 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -69,6 +69,12 @@ int domain_vgic_init(struct domain *d)
 
     switch ( gic_hw_version() )
     {
+#ifdef CONFIG_ARM_64
+    case GIC_V3:
+        if ( vgic_v3_init(d) )
+           return -ENODEV;
+        break;
+#endif
     case GIC_V2:
         if ( vgic_v2_init(d) )
             return -ENODEV;
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 19eed7e..f580b78 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -83,7 +83,14 @@ struct vgic_irq_rank {
     uint32_t ienable, iactive, ipend, pendsgi;
     uint32_t icfg[2];
     uint32_t ipriority[8];
-    uint32_t itargets[8];
+    union {
+        struct {
+            uint32_t itargets[8];
+        }v2;
+        struct {
+            uint64_t irouter[32];
+        }v3;
+    };
 };
 
 struct vgic_ops {
@@ -110,6 +117,9 @@ static inline int REG_RANK_NR(int b, uint32_t n)
 {
     switch ( b )
     {
+    case 64: return n >> 6;
+    case 32: return n >> 5;
+    case 16: return n >> 4;
     case 8: return n >> 3;
     case 4: return n >> 2;
     case 2: return n >> 1;
@@ -159,6 +169,7 @@ extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
 int vgic_v2_init(struct domain *d);
+int vgic_v3_init(struct domain *d);
 
 extern int vcpu_vgic_free(struct vcpu *v);
 extern int vgic_to_sgi(struct vcpu *v, register_t sgir,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [PATCH v8 5/7] xen/arm: Update Dom0 GIC dt node with GICv3 information
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
                   ` (3 preceding siblings ...)
  2014-07-23 13:41 ` [PATCH v8 4/7] xen/arm: Add virtual GICv3 support vijay.kilari
@ 2014-07-23 13:41 ` vijay.kilari
  2014-07-23 13:41 ` [PATCH v8 6/7] xen/arm: add SGI handling for GICv3 vijay.kilari
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Update GIC device tree node for DOM0 with GICv3
information. GIC hw specfic device tree information
is moved to respective GIC driver.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v8: - Remove GICV3 dt strings and introduced in patch 3
---
 xen/arch/arm/domain_build.c |   47 ++------------------------
 xen/arch/arm/gic-v2.c       |   51 ++++++++++++++++++++++++++++
 xen/arch/arm/gic-v3.c       |   78 +++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/gic.c          |    6 ++++
 xen/include/asm-arm/gic.h   |    4 +++
 5 files changed, 141 insertions(+), 45 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 8c850ca..40e83f3 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -526,9 +526,6 @@ static int make_gic_node(const struct domain *d, void *fdt,
                          const struct dt_device_node *node)
 {
     const struct dt_device_node *gic = dt_interrupt_controller;
-    const void *compatible = NULL;
-    u32 len;
-    __be32 *new_cells, *tmp;
     int res = 0;
 
     /*
@@ -543,48 +540,7 @@ static int make_gic_node(const struct domain *d, void *fdt,
 
     DPRINT("Create gic node\n");
 
-    compatible = dt_get_property(gic, "compatible", &len);
-    if ( !compatible )
-    {
-        dprintk(XENLOG_ERR, "Can't find compatible property for the gic node\n");
-        return -FDT_ERR_XEN(ENOENT);
-    }
-
-    res = fdt_begin_node(fdt, "interrupt-controller");
-    if ( res )
-        return res;
-
-    res = fdt_property(fdt, "compatible", compatible, len);
-    if ( res )
-        return res;
-
-    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
-    if ( res )
-        return res;
-
-    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
-
-    if ( res )
-        return res;
-
-    len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
-    len *= 2; /* GIC has two memory regions: Distributor + CPU interface */
-    new_cells = xzalloc_bytes(len);
-    if ( new_cells == NULL )
-        return -FDT_ERR_XEN(ENOMEM);
-
-    tmp = new_cells;
-    DPRINT("  Set Distributor Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
-           d->arch.vgic.dbase, d->arch.vgic.dbase + PAGE_SIZE - 1);
-    dt_set_range(&tmp, node, d->arch.vgic.dbase, PAGE_SIZE);
-
-    DPRINT("  Set Cpu Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
-           d->arch.vgic.cbase, d->arch.vgic.cbase + (PAGE_SIZE * 2) - 1);
-    dt_set_range(&tmp, node, d->arch.vgic.cbase, PAGE_SIZE * 2);
-
-    res = fdt_property(fdt, "reg", new_cells, len);
-    xfree(new_cells);
-
+    res = gic_make_node(d, node, fdt);
     if ( res )
         return res;
 
@@ -805,6 +761,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
     static const struct dt_device_match gic_matches[] __initconst =
     {
         DT_MATCH_GIC_V2,
+        DT_MATCH_GIC_V3,
         { /* sentinel */ },
     };
     static const struct dt_device_match timer_matches[] __initconst =
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index cc60af8..c1b9abe 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -27,6 +27,7 @@
 #include <xen/softirq.h>
 #include <xen/list.h>
 #include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
 #include <asm/p2m.h>
 #include <asm/domain.h>
 #include <asm/platform.h>
@@ -537,6 +538,55 @@ static void gicv2_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
     BUG();
 }
 
+static int gicv2_make_dt_node(const struct domain *d,
+                              const struct dt_device_node *node, void *fdt)
+{
+    const struct dt_device_node *gic = dt_interrupt_controller;
+    const void *compatible = NULL;
+    u32 len;
+    __be32 *new_cells, *tmp;
+    int res = 0;
+
+    compatible = dt_get_property(gic, "compatible", &len);
+    if ( !compatible )
+    {
+        dprintk(XENLOG_ERR, "Can't find compatible property for the gic node\n");
+        return -FDT_ERR_XEN(ENOENT);
+    }
+
+    res = fdt_begin_node(fdt, "interrupt-controller");
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "compatible", compatible, len);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+
+    if ( res )
+        return res;
+
+    len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+    len *= 2; /* GIC has two memory regions: Distributor + CPU interface */
+    new_cells = xzalloc_bytes(len);
+    if ( new_cells == NULL )
+        return -FDT_ERR_XEN(ENOMEM);
+
+    tmp = new_cells;
+    dt_set_range(&tmp, node, d->arch.vgic.dbase, PAGE_SIZE);
+    dt_set_range(&tmp, node, d->arch.vgic.cbase, PAGE_SIZE * 2);
+
+    res = fdt_property(fdt, "reg", new_cells, len);
+    xfree(new_cells);
+
+    return res;
+}
+
 /* XXX different for level vs edge */
 static hw_irq_controller gicv2_host_irq_type = {
     .typename     = "gic-v2",
@@ -582,6 +632,7 @@ const static struct gic_hw_operations gicv2_ops = {
     .write_lr            = gicv2_write_lr,
     .read_vmcr_priority  = gicv2_read_vmcr_priority,
     .read_apr            = gicv2_read_apr,
+    .make_dt_node        = gicv2_make_dt_node,
 };
 
 /* Set up the GIC */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 2b801f1..410d308 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -32,6 +32,7 @@
 #include <xen/delay.h>
 #include <xen/device_tree.h>
 #include <xen/sizes.h>
+#include <xen/libfdt/libfdt.h>
 #include <asm/p2m.h>
 #include <asm/domain.h>
 #include <asm/io.h>
@@ -1012,6 +1013,82 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
     BUG();
 }
 
+static int gicv3_make_dt_node(const struct domain *d,
+                              const struct dt_device_node *node, void *fdt)
+{
+    const struct dt_device_node *gic = dt_interrupt_controller;
+    const void *compatible = NULL;
+    uint32_t len;
+    __be32 *new_cells, *tmp;
+    uint32_t rd_stride = 0;
+    uint32_t rd_count = 0;
+
+    int i, res = 0;
+
+    compatible = dt_get_property(gic, "compatible", &len);
+    if ( !compatible )
+    {
+        dprintk(XENLOG_ERR, "Can't find compatible property for the gic node\n");
+        return -FDT_ERR_XEN(ENOENT);
+    }
+
+    res = fdt_begin_node(fdt, "interrupt-controller");
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "compatible", compatible, len);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    res = dt_property_read_u32(gic, "redistributor-stride", &rd_stride);
+    if ( !res )
+        rd_stride = 0;
+
+    res = dt_property_read_u32(gic, "#redistributor-regions", &rd_count);
+    if ( !res )
+        rd_count = 1;
+
+    res = fdt_property_cell(fdt, "redistributor-stride", rd_stride);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#redistributor-regions", rd_count);
+    if ( res )
+        return res;
+
+    len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+    /*
+     * GIC has two memory regions: Distributor + rdist regions
+     * CPU interface and virtual cpu interfaces accessesed as System registers
+     * So cells are created only for Distributor and rdist regions
+     */
+    len = len * (d->arch.vgic.rdist_count + 1);
+    new_cells = xzalloc_bytes(len);
+    if ( new_cells == NULL )
+        return -FDT_ERR_XEN(ENOMEM);
+
+    tmp = new_cells;
+
+    dt_set_range(&tmp, node, d->arch.vgic.dbase, d->arch.vgic.dbase_size);
+
+    for ( i = 0; i < d->arch.vgic.rdist_count; i++ )
+        dt_set_range(&tmp, node, d->arch.vgic.rbase[i],
+                     d->arch.vgic.rbase_size[i]);
+
+    res = fdt_property(fdt, "reg", new_cells, len);
+    xfree(new_cells);
+
+    return res;
+}
+
 static const hw_irq_controller gicv3_host_irq_type = {
     .typename     = "gic-v3",
     .startup      = gicv3_irq_startup,
@@ -1056,6 +1133,7 @@ static const struct gic_hw_operations gicv3_ops = {
     .read_vmcr_priority  = gicv3_read_vmcr_priority,
     .read_apr            = gicv3_read_apr,
     .secondary_init      = gicv3_secondary_cpu_init,
+    .make_dt_node        = gicv3_make_dt_node,
 };
 
 /* Set up the GIC */
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index c0d70b8..38aa7b5 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -616,6 +616,12 @@ void __cpuinit init_maintenance_interrupt(void)
                 "irq-maintenance", NULL);
 }
 
+int gic_make_node(const struct domain *d,const struct dt_device_node *node,
+                   void *fdt)
+{
+    return gic_hw_ops->make_dt_node(d, node, fdt);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index c4daf5b..51a0a26 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -336,9 +336,13 @@ struct gic_hw_operations {
     unsigned int (*read_apr)(int apr_reg);
     /* Secondary CPU init */
     int (*secondary_init)(void);
+    int (*make_dt_node)(const struct domain *d,
+                        const struct dt_device_node *node, void *fdt);
 };
 
 void register_gic_ops(const struct gic_hw_operations *ops);
+int gic_make_node(const struct domain *d,const struct dt_device_node *node,
+                  void *fdt);
 
 #endif /* __ASSEMBLY__ */
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [PATCH v8 6/7] xen/arm: add SGI handling for GICv3
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
                   ` (4 preceding siblings ...)
  2014-07-23 13:41 ` [PATCH v8 5/7] xen/arm: Update Dom0 GIC dt node with GICv3 information vijay.kilari
@ 2014-07-23 13:41 ` vijay.kilari
  2014-07-23 13:56   ` Stefano Stabellini
                     ` (2 more replies)
  2014-07-23 13:41 ` [PATCH v8 7/7] xen/arm: check for GICv3 platform support vijay.kilari
  2014-08-03 21:24 ` [PATCH v8 0/7] xen/arm: Add GICv3 support Julien Grall
  7 siblings, 3 replies; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

In ARMv8, write to ICC_SGI1R_EL1 register raises trap to EL2.
Handle the trap and inject SGI to vcpu.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v8: - Return 0 on wrong irqmode instead of BUG()
    - Add Comment on ignoring affinity 1,2,3 for SGI
    - Assert on emulate_sysreg callback
v7: - Introduced callback for sysreg emulation
    - Removed unused parameter in inject_undef_exception()
    - Use inject_undef64_exception for reporting sysreg
      handling failure

v6: - Removed forward declaration of vgic_to_sgi() in vgic-v3.c
    - Used vgic callback for SGI handling
    - Alignment changes
---
 xen/arch/arm/traps.c              |   15 +++++++++++
 xen/arch/arm/vgic-v3.c            |   54 +++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic.c               |    9 +++++++
 xen/include/asm-arm/gic_v3_defs.h |    7 +++++
 xen/include/asm-arm/sysregs.h     |    3 +++
 xen/include/asm-arm/vgic.h        |    3 +++
 6 files changed, 91 insertions(+)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 686d8b7..775bef1 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -41,6 +41,7 @@
 #include "decode.h"
 #include "vtimer.h"
 #include <asm/gic.h>
+#include <asm/vgic.h>
 
 /* The base of the stack must always be double-word aligned, which means
  * that both the kernel half of struct cpu_user_regs (which is pushed in
@@ -1641,6 +1642,20 @@ static void do_sysreg(struct cpu_user_regs *regs,
             domain_crash_synchronous();
         }
         break;
+    case HSR_SYSREG_ICC_SGI1R_EL1:
+        if ( !vgic_emulate(regs, hsr) )
+        {
+            dprintk(XENLOG_WARNING,
+                    "failed emulation of sysreg ICC_SGI1R_EL1 access\n");
+            inject_undef64_exception(regs, hsr.len);
+        }
+        break;
+    case HSR_SYSREG_ICC_SGI0R_EL1:
+    case HSR_SYSREG_ICC_ASGI1R_EL1:
+        /* TBD: Implement to support secure grp0/1 SGI forwarding */
+        dprintk(XENLOG_WARNING,
+                "Emulation of sysreg ICC_SGI0R_EL1/ASGI1R_EL1 not supported\n");
+        inject_undef64_exception(regs, hsr.len);
     default:
  bad_sysreg:
         {
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 6abaac3..8eb1193 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -850,6 +850,59 @@ write_ignore_64:
     return 1;
 }
 
+static int vgicv3_to_sgi(struct vcpu *v, register_t sgir)
+{
+    int virq;
+    int irqmode;
+    enum gic_sgi_mode sgi_mode;
+    unsigned long vcpu_mask = 0;
+
+    irqmode = (sgir >> ICH_SGI_IRQMODE_SHIFT) & ICH_SGI_IRQMODE_MASK;
+    virq = (sgir >> ICH_SGI_IRQ_SHIFT ) & ICH_SGI_IRQ_MASK;
+    /* SGI's are injected at Rdist level 0. ignoring affinity 1, 2, 3 */
+    vcpu_mask = sgir & ICH_SGI_TARGETLIST_MASK;
+
+    /* Map GIC sgi value to enum value */
+    switch ( irqmode )
+    {
+    case ICH_SGI_TARGET_LIST:
+        sgi_mode = SGI_TARGET_LIST;
+        break;
+    case ICH_SGI_TARGET_OTHERS:
+        sgi_mode = SGI_TARGET_OTHERS;
+        break;
+    default:
+        gdprintk(XENLOG_WARNING, "Wrong irq mode in SGI1R_EL1 register\n");
+        return 0;
+    }
+
+    return vgic_to_sgi(v, sgir, sgi_mode, virq, vcpu_mask);
+}
+
+static int vgicv3_emulate_sysreg(struct cpu_user_regs *regs, union hsr hsr)
+{
+    struct vcpu *v = current;
+    struct hsr_sysreg sysreg = hsr.sysreg;
+    register_t *r = select_user_reg(regs, sysreg.reg);
+
+    ASSERT (hsr.ec == HSR_EC_SYSREG);
+
+    switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
+    {
+    case HSR_SYSREG_ICC_SGI1R_EL1:
+        /* WO */
+        if ( !sysreg.read )
+            return vgicv3_to_sgi(v, *r);
+        else
+        {
+            gdprintk(XENLOG_WARNING, "Reading SGI1R_EL1 - WO register\n");
+            return 0;
+        }
+    default:
+        return 0;
+    }
+}
+
 static const struct mmio_handler_ops vgic_rdistr_mmio_handler = {
     .read_handler  = vgic_v3_rdistr_mmio_read,
     .write_handler = vgic_v3_rdistr_mmio_write,
@@ -900,6 +953,7 @@ static int vgicv3_domain_init(struct domain *d)
 static const struct vgic_ops v3_ops = {
     .vcpu_init   = vgicv3_vcpu_init,
     .domain_init = vgicv3_domain_init,
+    .emulate_sysreg  = vgicv3_emulate_sysreg,
 };
 
 int vgic_v3_init(struct domain *d)
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 622219f..ffdf574 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -344,6 +344,15 @@ out:
         smp_send_event_check_mask(cpumask_of(v->processor));
 }
 
+int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
+{
+    struct vcpu *v = current;
+ 
+    ASSERT(v->domain->arch.vgic.handler->emulate_sysreg != NULL);
+
+    return v->domain->arch.vgic.handler->emulate_sysreg(regs, hsr);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 115c998..93dd86b 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -146,6 +146,13 @@
 #define GICH_VMCR_PRIORITY_MASK      0xff
 #define GICH_VMCR_PRIORITY_SHIFT     24
 
+#define ICH_SGI_IRQMODE_SHIFT        40
+#define ICH_SGI_IRQMODE_MASK         0x1
+#define ICH_SGI_TARGET_OTHERS        1
+#define ICH_SGI_TARGET_LIST          0
+#define ICH_SGI_IRQ_SHIFT            24
+#define ICH_SGI_IRQ_MASK             0xf
+#define ICH_SGI_TARGETLIST_MASK      0xffff
 #endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
 
 /*
diff --git a/xen/include/asm-arm/sysregs.h b/xen/include/asm-arm/sysregs.h
index b4616ac..169b7ac 100644
--- a/xen/include/asm-arm/sysregs.h
+++ b/xen/include/asm-arm/sysregs.h
@@ -78,6 +78,9 @@
 #define HSR_SYSREG_PMINTENCLR_EL1 HSR_SYSREG(3,0,c9,c14,2)
 #define HSR_SYSREG_MAIR_EL1       HSR_SYSREG(3,0,c10,c2,0)
 #define HSR_SYSREG_AMAIR_EL1      HSR_SYSREG(3,0,c10,c3,0)
+#define HSR_SYSREG_ICC_SGI1R_EL1  HSR_SYSREG(3,0,c12,c11,5)
+#define HSR_SYSREG_ICC_ASGI1R_EL1 HSR_SYSREG(3,1,c12,c11,6)
+#define HSR_SYSREG_ICC_SGI0R_EL1  HSR_SYSREG(3,2,c12,c11,7)
 #define HSR_SYSREG_CONTEXTIDR_EL1 HSR_SYSREG(3,0,c13,c0,1)
 
 #define HSR_SYSREG_PMCR_EL0       HSR_SYSREG(3,3,c9,c12,0)
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index f580b78..63ecfa1 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -98,6 +98,8 @@ struct vgic_ops {
     int (*vcpu_init)(struct vcpu *v);
     /* Domain specific initialization of vGIC */
     int (*domain_init)(struct domain *d);
+    /* vGIC sysreg emulation */
+    int (*emulate_sysreg)(struct cpu_user_regs *regs, union hsr hsr);
 };
 
 /* Number of ranks of interrupt registers for a domain */
@@ -165,6 +167,7 @@ extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
 extern void vgic_clear_pending_irqs(struct vcpu *v);
 extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
 extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, int b, int n, int s);
+extern int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr);
 extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [PATCH v8 7/7] xen/arm: check for GICv3 platform support
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
                   ` (5 preceding siblings ...)
  2014-07-23 13:41 ` [PATCH v8 6/7] xen/arm: add SGI handling for GICv3 vijay.kilari
@ 2014-07-23 13:41 ` vijay.kilari
  2014-07-24 14:45   ` Julien Grall
  2014-08-03 21:24 ` [PATCH v8 0/7] xen/arm: Add GICv3 support Julien Grall
  7 siblings, 1 reply; 39+ messages in thread
From: vijay.kilari @ 2014-07-23 13:41 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

ID_AA64PFR0_EL1 register provides information about GIC support.
Check for this register in GICv3 driver.

Also print GICv3 support information in boot log

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v8: - Rearranged prf64 structure member variables
v7: - return -ENODEV on lack GICv3 platform support instead of
      panic
---
 xen/arch/arm/gic-v3.c            |    7 +++++++
 xen/arch/arm/setup.c             |    5 +++--
 xen/include/asm-arm/cpufeature.h |    1 +
 xen/include/asm-arm/processor.h  |    4 ++--
 4 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 410d308..9291a7a 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -39,6 +39,7 @@
 #include <asm/device.h>
 #include <asm/gic.h>
 #include <asm/gic_v3_defs.h>
+#include <asm/cpufeature.h>
 
 struct rdist_region {
     paddr_t base;
@@ -1143,6 +1144,12 @@ static int __init gicv3_init(struct dt_device_node *node, const void *data)
     int res, i;
     uint32_t reg;
 
+    if ( !cpu_has_gicv3 )
+    {
+        dprintk(XENLOG_ERR, "Platform does not support GICv3\n");
+        return -ENODEV;
+    }
+
     dt_device_set_used_by(node, DOMID_XEN);
 
     res = dt_device_get_address(node, 0, &gicv3.dbase, &gicv3.dbase_size);
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 78dc7f5..6a90c81 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -100,9 +100,10 @@ static void __init processor_id(void)
            cpu_has_el2_32 ? "64+32" : cpu_has_el2_64 ? "64" : "No",
            cpu_has_el1_32 ? "64+32" : cpu_has_el1_64 ? "64" : "No",
            cpu_has_el0_32 ? "64+32" : cpu_has_el0_64 ? "64" : "No");
-    printk("    Extensions:%s%s\n",
+    printk("    Extensions:%s%s%s\n",
            cpu_has_fp ? " FloatingPoint" : "",
-           cpu_has_simd ? " AdvancedSIMD" : "");
+           cpu_has_simd ? " AdvancedSIMD" : "",
+           cpu_has_gicv3 ? " GICv3" : "");
 
     printk("  Debug Features: %016"PRIx64" %016"PRIx64"\n",
            boot_cpu_data.dbg64.bits[0], boot_cpu_data.dbg64.bits[1]);
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 1d2c9c7..aec9173 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -15,6 +15,7 @@
 #define cpu_has_el3_64    (boot_cpu_feature64(el3) >= 1)
 #define cpu_has_fp        (boot_cpu_feature64(fp) == 0)
 #define cpu_has_simd      (boot_cpu_feature64(simd) == 0)
+#define cpu_has_gicv3     (boot_cpu_feature64(gic) == 1)
 #endif
 
 #define cpu_feature32(c, feat)         ((c)->pfr32.feat)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index ccaa9da..725c8b1 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -188,8 +188,8 @@ struct cpuinfo_arm {
             unsigned long el3:4;
             unsigned long fp:4;   /* Floating Point */
             unsigned long simd:4; /* Advanced SIMD */
-            unsigned long __res0:8;
-
+            unsigned long gic:4;  /* GIC support */
+            unsigned long __res0:4;
             unsigned long __res1;
         };
     } pfr64;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-23 13:41 ` [PATCH v8 4/7] xen/arm: Add virtual GICv3 support vijay.kilari
@ 2014-07-23 13:51   ` Vijay Kilari
  2014-07-28 14:35     ` Ian Campbell
  0 siblings, 1 reply; 39+ messages in thread
From: Vijay Kilari @ 2014-07-23 13:51 UTC (permalink / raw)
  To: Ian Campbell, Julien Grall, Stefano Stabellini,
	Stefano Stabellini, Tim Deegan, Jan Beulich, xen-devel
  Cc: Prasun Kapoor, Vijaya Kumar K, manish.jaggi, Vijay Kilari

Please ignore this patch.
There are some alignment issues reappearing

On Wed, Jul 23, 2014 at 7:11 PM,  <vijay.kilari@gmail.com> wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add virtual GICv3 driver support.
> Also, with this patch vgic_irq_rank structure is modified to
> hold GICv2 GICD_TARGET and GICv3 GICD_ROUTER registers under
> union.
>
> This patch adds only basic GICv3 support.
> Does not support Interrupt Translation support (ITS)
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v8: - Fixed printk coding styles
>     - Moved GICD_PIDRn and GICR_PIDRn macros to vgic-v3.c
>       from header file
>    - Check is made on return value of vgic_v3_init()
> v7: Fixed coding style.
> v6: Removed byte read access for IROUTERN register.
> ---
>  xen/arch/arm/Makefile      |    1 +
>  xen/arch/arm/vgic-v2.c     |    8 +-
>  xen/arch/arm/vgic-v3.c     |  919 ++++++++++++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vgic.c        |    6 +
>  xen/include/asm-arm/vgic.h |   13 +-
>  5 files changed, 942 insertions(+), 5 deletions(-)
>
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 1684c09..ba61e58 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -28,6 +28,7 @@ obj-y += smp.o
>  obj-y += shutdown.o
>  obj-y += traps.o
>  obj-y += vgic.o vgic-v2.o
> +obj-$(CONFIG_ARM_64) += vgic-v3.o
>  obj-y += vtimer.o
>  obj-y += vuart.o
>  obj-y += hvm.o
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index 2102e43..d65146a 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -133,7 +133,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
>          if ( rank == NULL) goto read_as_zero;
>
>          vgic_lock_rank(v, rank);
> -        *r = rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
> +        *r = rank->v2.itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
>                                             DABT_WORD)];
>          if ( dabt.size == DABT_BYTE )
>              *r = vgic_byte_read(*r, dabt.sign, gicd_reg);
> @@ -361,10 +361,10 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
>          if ( rank == NULL) goto write_ignore;
>          vgic_lock_rank(v, rank);
>          if ( dabt.size == DABT_WORD )
> -            rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
> +            rank->v2.itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
>                                            DABT_WORD)] = *r;
>          else
> -            vgic_byte_write(&rank->itargets[REG_RANK_INDEX(8,
> +            vgic_byte_write(&rank->v2.itargets[REG_RANK_INDEX(8,
>                         gicd_reg - GICD_ITARGETSR, DABT_WORD)], *r, gicd_reg);
>          vgic_unlock_rank(v, rank);
>          return 1;
> @@ -466,7 +466,7 @@ static int vgic_v2_vcpu_init(struct vcpu *v)
>
>      /* For SGI and PPI the target is always this CPU */
>      for ( i = 0 ; i < 8 ; i++ )
> -        v->arch.vgic.private_irqs->itargets[i] =
> +        v->arch.vgic.private_irqs->v2.itargets[i] =
>                (1<<(v->vcpu_id+0))
>              | (1<<(v->vcpu_id+8))
>              | (1<<(v->vcpu_id+16))
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> new file mode 100644
> index 0000000..6abaac3
> --- /dev/null
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -0,0 +1,919 @@
> +/*
> + * xen/arch/arm/vgic-v3.c
> + *
> + * ARM Virtual Generic Interrupt Controller v3 support
> + * based on xen/arch/arm/vgic.c
> + *
> + * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>
> + * Copyright (c) 2014 Cavium Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <xen/bitops.h>
> +#include <xen/config.h>
> +#include <xen/lib.h>
> +#include <xen/init.h>
> +#include <xen/softirq.h>
> +#include <xen/irq.h>
> +#include <xen/sched.h>
> +#include <xen/sizes.h>
> +#include <asm/current.h>
> +#include <asm/device.h>
> +#include <asm/mmio.h>
> +#include <asm/gic_v3_defs.h>
> +#include <asm/gic.h>
> +#include <asm/vgic.h>
> +
> +/* GICD_PIDRn register values for ARM implementations */
> +#define GICV3_GICD_PIDR0  0x92
> +#define GICV3_GICD_PIDR1  0xb4
> +#define GICV3_GICD_PIDR2  0x3b
> +#define GICV3_GICD_PIDR4  0x04
> +
> +/* GICR_PIDRn register values for ARM implementations */
> +#define GICV3_GICR_PIDR0  0x93
> +#define GICV3_GICR_PIDR1  GICV3_GICD_PIDR1
> +#define GICV3_GICR_PIDR2  GICV3_GICD_PIDR2
> +#define GICV3_GICR_PIDR4  GICV3_GICD_PIDR4
> +
> +static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                                         uint32_t gicr_reg)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    uint64_t aff;
> +
> +    switch ( gicr_reg )
> +    {
> +    case GICR_CTLR:
> +        /* We have not implemented LPI's, read zero */
> +        goto read_as_zero;
> +    case GICR_IIDR:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICR_IIDR_VAL;
> +        return 1;
> +    case GICR_TYPER:
> +        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +        /* TBD: Update processor id in [23:8] when ITS support is added */
> +        aff = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 56 |
> +               MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
> +               MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
> +               MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
> +        *r = aff;
> +        return 1;
> +    case GICR_STATUSR:
> +        /* Not implemented */
> +        goto read_as_zero;
> +    case GICR_WAKER:
> +        /* Power management is not implemented */
> +        goto read_as_zero;
> +    case GICR_SETLPIR:
> +        /* WO. Read as zero */
> +        goto read_as_zero_64;
> +    case GICR_CLRLPIR:
> +        /* WO. Read as zero */
> +        goto read_as_zero_64;
> +    case GICR_PROPBASER:
> +        /* LPI's not implemented */
> +        goto read_as_zero_64;
> +    case GICR_PENDBASER:
> +        /* LPI's not implemented */
> +        goto read_as_zero_64;
> +    case GICR_INVLPIR:
> +        /* WO. Read as zero */
> +        goto read_as_zero_64;
> +    case GICR_INVALLR:
> +        /* WO. Read as zero */
> +        goto read_as_zero_64;
> +        return 0;
> +    case GICR_SYNCR:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        /* RO . But when read it always returns busy bito bit[0] */
> +        *r = GICR_SYNCR_NOT_BUSY;
> +        return 1;
> +    case GICR_MOVLPIR:
> +        /* WO Read as zero */
> +        goto read_as_zero_64;
> +    case GICR_MOVALLR:
> +        /* WO Read as zero */
> +        goto read_as_zero_64;
> +    case GICR_PIDR0:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICR_PIDR0;
> +         return 1;
> +    case GICR_PIDR1:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICR_PIDR1;
> +         return 1;
> +    case GICR_PIDR2:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICR_PIDR2;
> +         return 1;
> +    case GICR_PIDR3:
> +        /* Manufacture/customer defined */
> +        goto read_as_zero;
> +    case GICR_PIDR4:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICR_PIDR4;
> +         return 1;
> +    case GICR_PIDR5 ... GICR_PIDR7:
> +        /* Reserved0 */
> +        goto read_as_zero;
> +    default:
> +        printk("vGICv3: vGICR: read r%d offset %#08x\n not found",
> +               dabt.reg, gicr_reg);
> +        return 0;
> +    }
> +bad_width:
> +    printk("vGICv3: vGICR: bad read width %d r%d offset %#08x\n",
> +           dabt.size, dabt.reg, gicr_reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +read_as_zero_64:
> +    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +    *r = 0;
> +    return 1;
> +
> +read_as_zero:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    *r = 0;
> +    return 1;
> +}
> +
> +static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                                          uint32_t gicr_reg)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +
> +    switch ( gicr_reg )
> +    {
> +    case GICR_CTLR:
> +        /* LPI's not implemented */
> +        goto write_ignore;
> +    case GICR_IIDR:
> +        /* RO */
> +        goto write_ignore;
> +    case GICR_TYPER:
> +        /* RO */
> +        goto write_ignore_64;
> +    case GICR_STATUSR:
> +        /* Not implemented */
> +        goto write_ignore;
> +    case GICR_WAKER:
> +        /* Power mgmt not implemented */
> +        goto write_ignore;
> +    case GICR_SETLPIR:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_CLRLPIR:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_PROPBASER:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_PENDBASER:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_INVLPIR:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_INVALLR:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_SYNCR:
> +        /* RO */
> +        goto write_ignore;
> +    case GICR_MOVLPIR:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_MOVALLR:
> +        /* LPI is not implemented */
> +        goto write_ignore_64;
> +    case GICR_PIDR7... GICR_PIDR0:
> +        /* RO */
> +        goto write_ignore;
> +    default:
> +        printk("vGICR: write r%d offset %#08x\n not found", dabt.reg, gicr_reg);
> +        return 0;
> +    }
> +bad_width:
> +    printk("vGICR: bad write width %d r%d=%"PRIregister" offset %#08x\n",
> +           dabt.size, dabt.reg, *r, gicr_reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +write_ignore_64:
> +    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +    return 1;
> +
> +write_ignore:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    return 1;
> +}
> +
> +static int __vgic_v3_distr_common_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                                            uint32_t reg)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    struct vgic_irq_rank *rank;
> +
> +    switch ( reg )
> +    {
> +    case GICD_IGROUPR ... GICD_IGROUPRN:
> +        /* We do not implement security extensions for guests, read zero */
> +        goto read_as_zero;
> +    case GICD_ISENABLER ... GICD_ISENABLERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ISENABLER, DABT_WORD);
> +        if ( rank == NULL ) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = rank->ienable;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICENABLER ... GICD_ICENABLERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ICENABLER, DABT_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = rank->ienable;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ISPENDR ... GICD_ISPENDRN:
> +        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ISPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = vgic_byte_read(rank->ipend, dabt.sign, reg);
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICPENDR ... GICD_ICPENDRN:
> +        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ICPENDR, DABT_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = vgic_byte_read(rank->ipend, dabt.sign, reg);
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ISACTIVER ... GICD_ISACTIVERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ISACTIVER, DABT_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = rank->iactive;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICACTIVER ... GICD_ICACTIVERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ICACTIVER, DABT_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = rank->iactive;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
> +        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 8, reg - GICD_IPRIORITYR, DABT_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +
> +        vgic_lock_rank(v, rank);
> +        *r = rank->ipriority[REG_RANK_INDEX(8, reg - GICD_IPRIORITYR,
> +                                            DABT_WORD)];
> +        if ( dabt.size == DABT_BYTE )
> +            *r = vgic_byte_read(*r, dabt.sign, reg);
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICFGR ... GICD_ICFGRN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 2, reg - GICD_ICFGR, DABT_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGR, DABT_WORD)];
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    default:
> +        printk("vGICv3: vGICD/vGICR: unhandled read r%d offset %#08x\n",
> +               dabt.reg, reg);
> +        return 0;
> +    }
> +
> +bad_width:
> +    dprintk(XENLOG_ERR,
> +            "vGICv3: vGICD/vGICR: bad read width %d r%d offset %#08x\n",
> +            dabt.size, dabt.reg, reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +read_as_zero:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    *r = 0;
> +    return 1;
> +}
> +
> +static int __vgic_v3_distr_common_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                                             uint32_t reg)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    struct vgic_irq_rank *rank;
> +    uint32_t tr;
> +
> +    switch ( reg )
> +    {
> +    case GICD_IGROUPR ... GICD_IGROUPRN:
> +        /* We do not implement security extensions for guests, write ignore */
> +        goto write_ignore;
> +    case GICD_ISENABLER ... GICD_ISENABLERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ISENABLER, DABT_WORD);
> +        if ( rank == NULL) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        tr = rank->ienable;
> +        rank->ienable |= *r;
> +        vgic_unlock_rank(v, rank);
> +        /* The irq number is extracted from offset. so shift by register size */
> +        vgic_enable_irqs(v, (*r) & (~tr), (reg - GICD_ISENABLER) >> DABT_WORD);
> +        return 1;
> +    case GICD_ICENABLER ... GICD_ICENABLERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ICENABLER, DABT_WORD);
> +        if ( rank == NULL) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        tr = rank->ienable;
> +        rank->ienable &= ~*r;
> +        vgic_unlock_rank(v, rank);
> +        /* The irq number is extracted from offset. so shift by register size */
> +        vgic_disable_irqs(v, (*r) & tr, (reg - GICD_ICENABLER) >> DABT_WORD);
> +        return 1;
> +    case GICD_ISPENDR ... GICD_ISPENDRN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ISPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->ipend = *r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICPENDR ... GICD_ICPENDRN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ICPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->ipend &= ~*r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ISACTIVER ... GICD_ISACTIVERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ISACTIVER, DABT_WORD);
> +        if ( rank == NULL) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->iactive &= ~*r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICACTIVER ... GICD_ICACTIVERN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ICACTIVER, DABT_WORD);
> +        if ( rank == NULL) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->iactive &= ~*r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
> +        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 8, reg - GICD_IPRIORITYR, DABT_WORD);
> +        if ( rank == NULL) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        if ( dabt.size == DABT_WORD )
> +            rank->ipriority[REG_RANK_INDEX(8, reg - GICD_IPRIORITYR,
> +                                           DABT_WORD)] = *r;
> +        else
> +            vgic_byte_write(&rank->ipriority[REG_RANK_INDEX(8,
> +                       reg - GICD_IPRIORITYR, DABT_WORD)], *r, reg);
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICFGR: /* Restricted to configure SGIs */
> +        goto write_ignore;
> +    case GICD_ICFGR + 4 ... GICD_ICFGRN: /* PPI + SPIs */
> +        /* ICFGR1 for PPI's, which is implementation defined
> +           if ICFGR1 is programmable or not. We chose to program */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 2, reg - GICD_ICFGR, DABT_WORD);
> +        if ( rank == NULL) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGR, DABT_WORD)] = *r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    default:
> +        printk("vGICv3: vGICD/vGICR: unhandled write r%d "
> +               "=%"PRIregister" offset %#08x\n", dabt.reg, *r, reg);
> +        return 0;
> +    }
> +
> +bad_width:
> +    dprintk(XENLOG_ERR,
> +            "vGICv3: vGICD/vGICR: bad write width %d r%d=%"PRIregister" "
> +            "offset %#08x\n", dabt.size, dabt.reg, *r, reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +write_ignore:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    return 1;
> +}
> +
> +static int vgic_v3_rdistr_sgi_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                                        uint32_t gicr_reg)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    struct vgic_irq_rank *rank;
> +
> +    switch ( gicr_reg )
> +    {
> +    case GICR_IGRPMODR0:
> +        /* We do not implement security extensions for guests, read zero */
> +        goto read_as_zero;
> +    case GICR_IGROUPR0:
> +    case GICR_ISENABLER0:
> +    case GICR_ICENABLER0:
> +    case GICR_ISACTIVER0:
> +    case GICR_ICACTIVER0:
> +    case GICR_IPRIORITYR0...GICR_IPRIORITYR7:
> +    case GICR_ICFGR0... GICR_ICFGR1:
> +         /*
> +          * Above registers offset are common with GICD.
> +          * So handle in common with GICD handling
> +          */
> +        return __vgic_v3_distr_common_mmio_read(v, info, gicr_reg);
> +    case GICR_ISPENDR0:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ISPENDR0, DABT_WORD);
> +        if ( rank == NULL ) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = rank->pendsgi;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICR_ICPENDR0:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ICPENDR0, DABT_WORD);
> +        if ( rank == NULL ) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = rank->pendsgi;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICR_NSACR:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        return 1;
> +    default:
> +        printk("vGICv3: vGICR: read r%d offset %#08x\n not found",
> +               dabt.reg, gicr_reg);
> +        return 0;
> +    }
> +bad_width:
> +    printk("vGICv3: vGICR: bad read width %d r%d offset %#08x\n",
> +           dabt.size, dabt.reg, gicr_reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +read_as_zero:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    *r = 0;
> +    return 1;
> +}
> +
> +static int vgic_v3_rdistr_sgi_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                                         uint32_t gicr_reg)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    struct vgic_irq_rank *rank;
> +
> +    switch ( gicr_reg )
> +    {
> +    case GICR_IGRPMODR0:
> +        /* We do not implement security extensions for guests, write ignore */
> +        goto write_ignore;
> +    case GICR_IGROUPR0:
> +    case GICR_ISENABLER0:
> +    case GICR_ICENABLER0:
> +    case GICR_ISACTIVER0:
> +    case GICR_ICACTIVER0:
> +    case GICR_ICFGR1:
> +    case GICR_IPRIORITYR0...GICR_IPRIORITYR7:
> +         /*
> +          * Above registers offset are common with GICD.
> +          * So handle common with GICD handling
> +          */
> +        return __vgic_v3_distr_common_mmio_write(v, info, gicr_reg);
> +    case GICR_ISPENDR0:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ISACTIVER0, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        /* TODO: we just store the SGI pending status. Handle it properly */
> +        rank->pendsgi |= *r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICR_ICPENDR0:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, gicr_reg - GICR_ISACTIVER0, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        /* TODO: we just store the SGI pending status. Handle it properly */
> +        rank->pendsgi &= ~*r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICR_NSACR:
> +        /* We do not implement security extensions for guests, write ignore */
> +        goto write_ignore;
> +    default:
> +        printk("vGICv3: vGICR SGI: write r%d offset %#08x\n not found",
> +               dabt.reg, gicr_reg);
> +        return 0;
> +    }
> +
> +bad_width:
> +    printk("vGICR SGI: bad write width %d r%d=%"PRIregister" offset %#08x\n",
> +           dabt.size, dabt.reg, *r, gicr_reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +write_ignore:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    return 1;
> +}
> +
> +static int vgic_v3_rdistr_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> +    uint32_t offset;
> +
> +    offset = info->gpa & (v->domain->arch.vgic.rdist_stride - 1);
> +
> +    if ( offset < SZ_64K )
> +        return __vgic_v3_rdistr_rd_mmio_read(v, info, offset);
> +    else  if ( (offset >= SZ_64K) && (offset < 2 * SZ_64K) )
> +        return vgic_v3_rdistr_sgi_mmio_read(v, info, (offset - SZ_64K));
> +    else
> +        gdprintk(XENLOG_WARNING,
> +                 "vGICv3: vGICR: unknown gpa read address %"PRIpaddr"\n",
> +                 info->gpa);
> +
> +    return 0;
> +}
> +
> +static int vgic_v3_rdistr_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> +    uint32_t offset;
> +
> +    offset = info->gpa & (v->domain->arch.vgic.rdist_stride - 1);
> +    if ( offset < SZ_64K )
> +        return __vgic_v3_rdistr_rd_mmio_write(v, info, offset);
> +    else  if ( (offset >= SZ_64K) && (offset < 2 * SZ_64K) )
> +        return vgic_v3_rdistr_sgi_mmio_write(v, info, (offset - SZ_64K));
> +    else
> +        gdprintk(XENLOG_WARNING,
> +                 "vGICV3: vGICR: unknown gpa write address %"PRIpaddr"\n",
> +                 info->gpa);
> +
> +    return 0;
> +}
> +
> +static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    struct vgic_irq_rank *rank;
> +    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
> +
> +    switch ( gicd_reg )
> +    {
> +    case GICD_CTLR:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        vgic_lock(v);
> +        *r = v->domain->arch.vgic.ctlr;
> +        vgic_unlock(v);
> +        return 1;
> +    case GICD_TYPER:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        /* No secure world support for guests. */
> +        *r = (((v->domain->max_vcpus << 5) & GICD_TYPE_CPUS ) |
> +              ((v->domain->arch.vgic.nr_lines / 32) & GICD_TYPE_LINES));
> +        return 1;
> +    case GICD_STATUSR:
> +        /*
> +         *  Optional, Not implemented for now.
> +         *  Update to support guest debugging.
> +         */
> +        goto read_as_zero;
> +    case GICD_IIDR:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICD_IIDR_VAL;
> +        return 1;
> +    case 0x020 ... 0x03c:
> +    case 0xc000 ... 0xffcc:
> +        /* Implementation defined -- read as zero */
> +        goto read_as_zero;
> +    case GICD_IGROUPR ... GICD_IGROUPRN:
> +    case GICD_ISENABLER ... GICD_ISENABLERN:
> +    case GICD_ICENABLER ... GICD_ICENABLERN:
> +    case GICD_ISPENDR ... GICD_ISPENDRN:
> +    case GICD_ICPENDR ... GICD_ICPENDRN:
> +    case GICD_ISACTIVER ... GICD_ISACTIVERN:
> +    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
> +    case GICD_ICFGR ... GICD_ICFGRN:
> +        /*
> +         * Above all register are common with GICR and GICD
> +         * Manage in common
> +         */
> +        return __vgic_v3_distr_common_mmio_read(v, info, gicd_reg);
> +    case GICD_IROUTER ... GICD_IROUTER31:
> +        /* SGI/PPI is RES0 */
> +        goto read_as_zero_64;
> +    case GICD_IROUTER32 ... GICD_IROUTERN:
> +        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        /* IROUTER is 64 bit so, to make it byte size right shift by 3.
> +           Here once. macro REG_RANK_INDEX will do it twice */
> +        *r = rank->v3.irouter[REG_RANK_INDEX(64,
> +                           (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)];
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_NSACR ... GICD_NSACRN:
> +        /* We do not implement security extensions for guests, read zero */
> +        goto read_as_zero;
> +    case GICD_SGIR:
> +        /* Read as ICH_SGIR system register with SRE set. So ignore */
> +        goto read_as_zero;
> +    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
> +        /* Replaced with GICR_ICPENDR0. So ignore write */
> +        goto read_as_zero;
> +    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
> +        /* Replaced with GICR_ISPENDR0. So ignore write */
> +        goto read_as_zero;
> +    case GICD_PIDR0:
> +        /* GICv3 identification value */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICD_PIDR0;
> +        return 1;
> +    case GICD_PIDR1:
> +        /* GICv3 identification value */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICD_PIDR1;
> +        return 1;
> +    case GICD_PIDR2:
> +        /* GICv3 identification value */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICD_PIDR2;
> +        return 1;
> +    case GICD_PIDR3:
> +        /* GICv3 identification value. Manufacturer/Customer defined */
> +        goto read_as_zero;
> +    case GICD_PIDR4:
> +        /* GICv3 identification value */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICD_PIDR4;
> +        return 1;
> +    case GICD_PIDR5 ... GICD_PIDR7:
> +        /* Reserved0 */
> +        goto read_as_zero;
> +    case 0x00c:
> +    case 0x044:
> +    case 0x04c:
> +    case 0x05c ... 0x07c:
> +    case 0xf30 ... 0x5fcc:
> +    case 0x8000 ... 0xbfcc:
> +        /* These are reserved register addresses */
> +        printk("vGICv3: vGICD: read unknown 0x00c .. 0xfcc r%d offset %#08x\n",
> +               dabt.reg, gicd_reg);
> +        goto read_as_zero;
> +    default:
> +        printk("vGICv3: vGICD: unhandled read r%d offset %#08x\n",
> +               dabt.reg, gicd_reg);
> +        return 0;
> +    }
> +
> +bad_width:
> +    dprintk(XENLOG_ERR, "vGICv3: vGICD: bad read width %d r%d offset %#08x\n",
> +            dabt.size, dabt.reg, gicd_reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +read_as_zero_64:
> +    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +    *r = 0;
> +    return 1;
> +
> +read_as_zero:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    *r = 0;
> +    return 1;
> +}
> +
> +static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    struct vgic_irq_rank *rank;
> +    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
> +
> +    switch ( gicd_reg )
> +    {
> +    case GICD_CTLR:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        /* Ignore all but the enable bit */
> +        v->domain->arch.vgic.ctlr = (*r) & GICD_CTL_ENABLE;
> +        return 1;
> +    case GICD_TYPER:
> +        /* RO -- write ignored */
> +        goto write_ignore;
> +    case GICD_IIDR:
> +        /* RO -- write ignored */
> +        goto write_ignore;
> +    case GICD_STATUSR:
> +        /* RO -- write ignored */
> +        goto write_ignore;
> +    case GICD_SETSPI_NSR:
> +        /* Message based SPI is not implemented */
> +        goto write_ignore;
> +    case GICD_CLRSPI_NSR:
> +        /* Message based SPI is not implemented */
> +        goto write_ignore;
> +    case GICD_SETSPI_SR:
> +        /* Message based SPI is not implemented */
> +        goto write_ignore;
> +    case GICD_CLRSPI_SR:
> +        /* Message based SPI is not implemented */
> +        goto write_ignore;
> +    case 0x020 ... 0x03c:
> +    case 0xc000 ... 0xffcc:
> +        /* Implementation defined -- write ignored */
> +        printk("vGICv3: vGICD: write unknown 0x020 - 0x03c r%d offset %#08x\n",
> +               dabt.reg, gicd_reg);
> +        goto write_ignore;
> +    case GICD_IGROUPR ... GICD_IGROUPRN:
> +    case GICD_ISENABLER ... GICD_ISENABLERN:
> +    case GICD_ICENABLER ... GICD_ICENABLERN:
> +    case GICD_ISPENDR ... GICD_ISPENDRN:
> +    case GICD_ICPENDR ... GICD_ICPENDRN:
> +    case GICD_ISACTIVER ... GICD_ISACTIVERN:
> +    case GICD_ICACTIVER ... GICD_ICACTIVERN:
> +    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
> +    case GICD_ICFGR ... GICD_ICFGRN:
> +        /* Above registers are common with GICR and GICD
> +         * Manage in common */
> +        return __vgic_v3_distr_common_mmio_write(v, info, gicd_reg);
> +    case GICD_IROUTER ... GICD_IROUTER31:
> +        /* SGI/PPI is RES0 */
> +        goto write_ignore_64;
> +    case GICD_IROUTER32 ... GICD_IROUTERN:
> +        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
> +        if ( rank == NULL) goto write_ignore_64;
> +        if ( *r )
> +        {
> +            /* TODO: Ignored. We don't support irq delivery for vcpu != 0 */
> +            gdprintk(XENLOG_DEBUG,
> +                     "SPI delivery to secondary cpus not supported\n");
> +            goto write_ignore_64;
> +        }
> +        vgic_lock_rank(v, rank);
> +        rank->v3.irouter[REG_RANK_INDEX(64,
> +                      (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)] = *r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_NSACR ... GICD_NSACRN:
> +        /* We do not implement security extensions for guests, write ignore */
> +        goto write_ignore;
> +    case GICD_SGIR:
> +        /* it is accessed as system register in GICv3 */
> +        goto write_ignore;
> +    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
> +        /* Replaced with GICR_ICPENDR0. So ignore write */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        return 0;
> +    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
> +        /* Replaced with GICR_ISPENDR0. So ignore write */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        return 0;
> +    case GICD_PIDR7... GICD_PIDR0:
> +        /* RO -- write ignore */
> +        goto write_ignore;
> +    case 0x00c:
> +    case 0x044:
> +    case 0x04c:
> +    case 0x05c ... 0x07c:
> +    case 0xf30 ... 0x5fcc:
> +    case 0x8000 ... 0xbfcc:
> +        /* Reserved register addresses */
> +        printk("vGICv3: vGICD: write unknown 0x00c 0xfcc  r%d offset %#08x\n",
> +                dabt.reg, gicd_reg);
> +        goto write_ignore;
> +    default:
> +        printk("vGICv3: vGICD: unhandled write r%d=%"PRIregister" "
> +               "offset %#08x\n", dabt.reg, *r, gicd_reg);
> +        return 0;
> +    }
> +
> +bad_width:
> +    dprintk(XENLOG_ERR,
> +            "VGICv3: vGICD: bad write width %d r%d=%"PRIregister" "
> +            "offset %#08x\n", dabt.size, dabt.reg, *r, gicd_reg);
> +    domain_crash_synchronous();
> +    return 0;
> +
> +write_ignore:
> +    if ( dabt.size != DABT_WORD ) goto bad_width;
> +    return 1;
> +
> +write_ignore_64:
> +    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +    return 1;
> +}
> +
> +static const struct mmio_handler_ops vgic_rdistr_mmio_handler = {
> +    .read_handler  = vgic_v3_rdistr_mmio_read,
> +    .write_handler = vgic_v3_rdistr_mmio_write,
> +};
> +
> +static const struct mmio_handler_ops vgic_distr_mmio_handler = {
> +    .read_handler  = vgic_v3_distr_mmio_read,
> +    .write_handler = vgic_v3_distr_mmio_write,
> +};
> +
> +static int vgicv3_vcpu_init(struct vcpu *v)
> +{
> +    int i;
> +    uint64_t affinity;
> +
> +    /* For SGI and PPI the target is always this CPU */
> +    affinity = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 32 |
> +                MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 16 |
> +                MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 8  |
> +                MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0));
> +
> +    for ( i = 0 ; i < 32 ; i++ )
> +        v->arch.vgic.private_irqs->v3.irouter[i] = affinity;
> +
> +    return 0;
> +}
> +
> +static int vgicv3_domain_init(struct domain *d)
> +{
> +    int i;
> +
> +    /* We rely on gicv init to get dbase and size */
> +    register_mmio_handler(d, &vgic_distr_mmio_handler, d->arch.vgic.dbase,
> +                          d->arch.vgic.dbase_size);
> +
> +    /*
> +     * Register mmio handler per redistributor region but not for
> +     * every sgi rdist region which is per core.
> +     * The redistributor region encompasses per core sgi region.
> +     */
> +    for ( i = 0; i < d->arch.vgic.rdist_count; i++ )
> +        register_mmio_handler(d, &vgic_rdistr_mmio_handler,
> +            d->arch.vgic.rbase[i], d->arch.vgic.rbase_size[i]);
> +
> +    return 0;
> +}
> +
> +static const struct vgic_ops v3_ops = {
> +    .vcpu_init   = vgicv3_vcpu_init,
> +    .domain_init = vgicv3_domain_init,
> +};
> +
> +int vgic_v3_init(struct domain *d)
> +{
> +    register_vgic_ops(d, &v3_ops);
> +
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index d223db4..622219f 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -69,6 +69,12 @@ int domain_vgic_init(struct domain *d)
>
>      switch ( gic_hw_version() )
>      {
> +#ifdef CONFIG_ARM_64
> +    case GIC_V3:
> +        if ( vgic_v3_init(d) )
> +           return -ENODEV;
> +        break;
> +#endif
>      case GIC_V2:
>          if ( vgic_v2_init(d) )
>              return -ENODEV;
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 19eed7e..f580b78 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -83,7 +83,14 @@ struct vgic_irq_rank {
>      uint32_t ienable, iactive, ipend, pendsgi;
>      uint32_t icfg[2];
>      uint32_t ipriority[8];
> -    uint32_t itargets[8];
> +    union {
> +        struct {
> +            uint32_t itargets[8];
> +        }v2;
> +        struct {
> +            uint64_t irouter[32];
> +        }v3;
> +    };
>  };
>
>  struct vgic_ops {
> @@ -110,6 +117,9 @@ static inline int REG_RANK_NR(int b, uint32_t n)
>  {
>      switch ( b )
>      {
> +    case 64: return n >> 6;
> +    case 32: return n >> 5;
> +    case 16: return n >> 4;
>      case 8: return n >> 3;
>      case 4: return n >> 2;
>      case 2: return n >> 1;
> @@ -159,6 +169,7 @@ extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
>  int vgic_v2_init(struct domain *d);
> +int vgic_v3_init(struct domain *d);
>
>  extern int vcpu_vgic_free(struct vcpu *v);
>  extern int vgic_to_sgi(struct vcpu *v, register_t sgir,
> --
> 1.7.9.5
>

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 6/7] xen/arm: add SGI handling for GICv3
  2014-07-23 13:41 ` [PATCH v8 6/7] xen/arm: add SGI handling for GICv3 vijay.kilari
@ 2014-07-23 13:56   ` Stefano Stabellini
  2014-07-24 14:37   ` Julien Grall
  2014-07-28 14:04   ` Ian Campbell
  2 siblings, 0 replies; 39+ messages in thread
From: Stefano Stabellini @ 2014-07-23 13:56 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, tim, xen-devel, stefano.stabellini, jbeulich,
	manish.jaggi



On Wed, 23 Jul 2014, vijay.kilari@gmail.com wrote:

> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> In ARMv8, write to ICC_SGI1R_EL1 register raises trap to EL2.
> Handle the trap and inject SGI to vcpu.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Looks good.

Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>


> v8: - Return 0 on wrong irqmode instead of BUG()
>     - Add Comment on ignoring affinity 1,2,3 for SGI
>     - Assert on emulate_sysreg callback
> v7: - Introduced callback for sysreg emulation
>     - Removed unused parameter in inject_undef_exception()
>     - Use inject_undef64_exception for reporting sysreg
>       handling failure
> 
> v6: - Removed forward declaration of vgic_to_sgi() in vgic-v3.c
>     - Used vgic callback for SGI handling
>     - Alignment changes
> ---
>  xen/arch/arm/traps.c              |   15 +++++++++++
>  xen/arch/arm/vgic-v3.c            |   54 +++++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vgic.c               |    9 +++++++
>  xen/include/asm-arm/gic_v3_defs.h |    7 +++++
>  xen/include/asm-arm/sysregs.h     |    3 +++
>  xen/include/asm-arm/vgic.h        |    3 +++
>  6 files changed, 91 insertions(+)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 686d8b7..775bef1 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -41,6 +41,7 @@
>  #include "decode.h"
>  #include "vtimer.h"
>  #include <asm/gic.h>
> +#include <asm/vgic.h>
>  
>  /* The base of the stack must always be double-word aligned, which means
>   * that both the kernel half of struct cpu_user_regs (which is pushed in
> @@ -1641,6 +1642,20 @@ static void do_sysreg(struct cpu_user_regs *regs,
>              domain_crash_synchronous();
>          }
>          break;
> +    case HSR_SYSREG_ICC_SGI1R_EL1:
> +        if ( !vgic_emulate(regs, hsr) )
> +        {
> +            dprintk(XENLOG_WARNING,
> +                    "failed emulation of sysreg ICC_SGI1R_EL1 access\n");
> +            inject_undef64_exception(regs, hsr.len);
> +        }
> +        break;
> +    case HSR_SYSREG_ICC_SGI0R_EL1:
> +    case HSR_SYSREG_ICC_ASGI1R_EL1:
> +        /* TBD: Implement to support secure grp0/1 SGI forwarding */
> +        dprintk(XENLOG_WARNING,
> +                "Emulation of sysreg ICC_SGI0R_EL1/ASGI1R_EL1 not supported\n");
> +        inject_undef64_exception(regs, hsr.len);
>      default:
>   bad_sysreg:
>          {
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 6abaac3..8eb1193 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -850,6 +850,59 @@ write_ignore_64:
>      return 1;
>  }
>  
> +static int vgicv3_to_sgi(struct vcpu *v, register_t sgir)
> +{
> +    int virq;
> +    int irqmode;
> +    enum gic_sgi_mode sgi_mode;
> +    unsigned long vcpu_mask = 0;
> +
> +    irqmode = (sgir >> ICH_SGI_IRQMODE_SHIFT) & ICH_SGI_IRQMODE_MASK;
> +    virq = (sgir >> ICH_SGI_IRQ_SHIFT ) & ICH_SGI_IRQ_MASK;
> +    /* SGI's are injected at Rdist level 0. ignoring affinity 1, 2, 3 */
> +    vcpu_mask = sgir & ICH_SGI_TARGETLIST_MASK;
> +
> +    /* Map GIC sgi value to enum value */
> +    switch ( irqmode )
> +    {
> +    case ICH_SGI_TARGET_LIST:
> +        sgi_mode = SGI_TARGET_LIST;
> +        break;
> +    case ICH_SGI_TARGET_OTHERS:
> +        sgi_mode = SGI_TARGET_OTHERS;
> +        break;
> +    default:
> +        gdprintk(XENLOG_WARNING, "Wrong irq mode in SGI1R_EL1 register\n");
> +        return 0;
> +    }
> +
> +    return vgic_to_sgi(v, sgir, sgi_mode, virq, vcpu_mask);
> +}
> +
> +static int vgicv3_emulate_sysreg(struct cpu_user_regs *regs, union hsr hsr)
> +{
> +    struct vcpu *v = current;
> +    struct hsr_sysreg sysreg = hsr.sysreg;
> +    register_t *r = select_user_reg(regs, sysreg.reg);
> +
> +    ASSERT (hsr.ec == HSR_EC_SYSREG);
> +
> +    switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
> +    {
> +    case HSR_SYSREG_ICC_SGI1R_EL1:
> +        /* WO */
> +        if ( !sysreg.read )
> +            return vgicv3_to_sgi(v, *r);
> +        else
> +        {
> +            gdprintk(XENLOG_WARNING, "Reading SGI1R_EL1 - WO register\n");
> +            return 0;
> +        }
> +    default:
> +        return 0;
> +    }
> +}
> +
>  static const struct mmio_handler_ops vgic_rdistr_mmio_handler = {
>      .read_handler  = vgic_v3_rdistr_mmio_read,
>      .write_handler = vgic_v3_rdistr_mmio_write,
> @@ -900,6 +953,7 @@ static int vgicv3_domain_init(struct domain *d)
>  static const struct vgic_ops v3_ops = {
>      .vcpu_init   = vgicv3_vcpu_init,
>      .domain_init = vgicv3_domain_init,
> +    .emulate_sysreg  = vgicv3_emulate_sysreg,
>  };
>  
>  int vgic_v3_init(struct domain *d)
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 622219f..ffdf574 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -344,6 +344,15 @@ out:
>          smp_send_event_check_mask(cpumask_of(v->processor));
>  }
>  
> +int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
> +{
> +    struct vcpu *v = current;
> + 
> +    ASSERT(v->domain->arch.vgic.handler->emulate_sysreg != NULL);
> +
> +    return v->domain->arch.vgic.handler->emulate_sysreg(regs, hsr);
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 115c998..93dd86b 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -146,6 +146,13 @@
>  #define GICH_VMCR_PRIORITY_MASK      0xff
>  #define GICH_VMCR_PRIORITY_SHIFT     24
>  
> +#define ICH_SGI_IRQMODE_SHIFT        40
> +#define ICH_SGI_IRQMODE_MASK         0x1
> +#define ICH_SGI_TARGET_OTHERS        1
> +#define ICH_SGI_TARGET_LIST          0
> +#define ICH_SGI_IRQ_SHIFT            24
> +#define ICH_SGI_IRQ_MASK             0xf
> +#define ICH_SGI_TARGETLIST_MASK      0xffff
>  #endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
>  
>  /*
> diff --git a/xen/include/asm-arm/sysregs.h b/xen/include/asm-arm/sysregs.h
> index b4616ac..169b7ac 100644
> --- a/xen/include/asm-arm/sysregs.h
> +++ b/xen/include/asm-arm/sysregs.h
> @@ -78,6 +78,9 @@
>  #define HSR_SYSREG_PMINTENCLR_EL1 HSR_SYSREG(3,0,c9,c14,2)
>  #define HSR_SYSREG_MAIR_EL1       HSR_SYSREG(3,0,c10,c2,0)
>  #define HSR_SYSREG_AMAIR_EL1      HSR_SYSREG(3,0,c10,c3,0)
> +#define HSR_SYSREG_ICC_SGI1R_EL1  HSR_SYSREG(3,0,c12,c11,5)
> +#define HSR_SYSREG_ICC_ASGI1R_EL1 HSR_SYSREG(3,1,c12,c11,6)
> +#define HSR_SYSREG_ICC_SGI0R_EL1  HSR_SYSREG(3,2,c12,c11,7)
>  #define HSR_SYSREG_CONTEXTIDR_EL1 HSR_SYSREG(3,0,c13,c0,1)
>  
>  #define HSR_SYSREG_PMCR_EL0       HSR_SYSREG(3,3,c9,c12,0)
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index f580b78..63ecfa1 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -98,6 +98,8 @@ struct vgic_ops {
>      int (*vcpu_init)(struct vcpu *v);
>      /* Domain specific initialization of vGIC */
>      int (*domain_init)(struct domain *d);
> +    /* vGIC sysreg emulation */
> +    int (*emulate_sysreg)(struct cpu_user_regs *regs, union hsr hsr);
>  };
>  
>  /* Number of ranks of interrupt registers for a domain */
> @@ -165,6 +167,7 @@ extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
>  extern void vgic_clear_pending_irqs(struct vcpu *v);
>  extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
>  extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, int b, int n, int s);
> +extern int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr);
>  extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros
  2014-07-23 13:41 ` [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros vijay.kilari
@ 2014-07-23 17:20   ` Julien Grall
  2014-07-28 13:56   ` Ian Campbell
  1 sibling, 0 replies; 39+ messages in thread
From: Julien Grall @ 2014-07-23 17:20 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar, manish.jaggi

Hi Vijay,

On 07/23/2014 02:41 PM, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> The register name parameter in {READ,WRITE}_SYSREG{32,64}
> macros is replaced as register name in mrs and msr assembly
> instruction.
> 
> If this register name is macro indirection, the register
> name is not replaced with designated macro. So replace the register
> name with __stringify macro, which replaces register name
> if it is macro
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Julien Grall <julien.grall@linaro.org>

Regards,

> ---
>  xen/include/asm-arm/arm64/processor.h |   10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/include/asm-arm/arm64/processor.h b/xen/include/asm-arm/arm64/processor.h
> index 5bf0867..3a9c0cd 100644
> --- a/xen/include/asm-arm/arm64/processor.h
> +++ b/xen/include/asm-arm/arm64/processor.h
> @@ -1,6 +1,8 @@
>  #ifndef __ASM_ARM_ARM64_PROCESSOR_H
>  #define __ASM_ARM_ARM64_PROCESSOR_H
>  
> +#include <xen/stringify.h>
> +
>  #ifndef __ASSEMBLY__
>  
>  /* Anonymous union includes both 32- and 64-bit names (e.g., r0/x0). */
> @@ -85,20 +87,20 @@ struct cpu_user_regs
>  
>  #define READ_SYSREG32(name) ({                          \
>      uint32_t _r;                                        \
> -    asm volatile("mrs  %0, "#name : "=r" (_r));         \
> +    asm volatile("mrs  %0, "__stringify(name) : "=r" (_r));         \
>      _r; })
>  #define WRITE_SYSREG32(v, name) do {                    \
>      uint32_t _r = v;                                    \
> -    asm volatile("msr "#name", %0" : : "r" (_r));       \
> +    asm volatile("msr "__stringify(name)", %0" : : "r" (_r));       \
>  } while (0)
>  
>  #define WRITE_SYSREG64(v, name) do {                    \
>      uint64_t _r = v;                                    \
> -    asm volatile("msr "#name", %0" : : "r" (_r));       \
> +    asm volatile("msr "__stringify(name)", %0" : : "r" (_r));       \
>  } while (0)
>  #define READ_SYSREG64(name) ({                          \
>      uint64_t _r;                                        \
> -    asm volatile("mrs  %0, "#name : "=r" (_r));         \
> +    asm volatile("mrs  %0, "__stringify(name) : "=r" (_r));         \
>      _r; })
>  
>  #define READ_SYSREG(name)     READ_SYSREG64(name)
> 


-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 3/7] xen/arm: Add support for GIC v3
  2014-07-23 13:41 ` [PATCH v8 3/7] xen/arm: Add support for GIC v3 vijay.kilari
@ 2014-07-23 17:28   ` Julien Grall
  2014-07-24  9:03     ` Ian Campbell
  0 siblings, 1 reply; 39+ messages in thread
From: Julien Grall @ 2014-07-23 17:28 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar, manish.jaggi

Hi Vijay,

On 07/23/2014 02:41 PM, vijay.kilari@gmail.com wrote:
> +#define GICD_PIDR2_ARCH_REV_MASK     (0xf0)
> +#define GICD_PIDR2_ARCH_REV_OFFSET   (0x4)

NIT: We tend to use SHIFT rather than OFFSET.

> +#define GICD_PIDR2_ARCH_GICV3        (0x3)
> +#define GICD_PIDR2_USES_JEP_MASK     (0x08)
> +#define GICD_PIDR2_JEP_ID_MASK       (0x07)

NIT: I don't see any usage of GICD_PIDR2_USES_JEP_MASK and
GICD_PIDR2_JEP_ID_MASK.

Hence, you don't define the SHIFT. So I would drop them.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 3/7] xen/arm: Add support for GIC v3
  2014-07-23 17:28   ` Julien Grall
@ 2014-07-24  9:03     ` Ian Campbell
  2014-07-24 10:26       ` Julien Grall
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-07-24  9:03 UTC (permalink / raw)
  To: Julien Grall
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi

On Wed, 2014-07-23 at 18:28 +0100, Julien Grall wrote:
> Hi Vijay,
> 
> On 07/23/2014 02:41 PM, vijay.kilari@gmail.com wrote:
> > +#define GICD_PIDR2_ARCH_REV_MASK     (0xf0)
> > +#define GICD_PIDR2_ARCH_REV_OFFSET   (0x4)
> 
> NIT: We tend to use SHIFT rather than OFFSET.
> 
> > +#define GICD_PIDR2_ARCH_GICV3        (0x3)
> > +#define GICD_PIDR2_USES_JEP_MASK     (0x08)
> > +#define GICD_PIDR2_JEP_ID_MASK       (0x07)
> 
> NIT: I don't see any usage of GICD_PIDR2_USES_JEP_MASK and
> GICD_PIDR2_JEP_ID_MASK.
> 
> Hence, you don't define the SHIFT. So I would drop them.

FWIW I don't think it is a problem define bits of a register which
aren't currently used on the principal that including the full set is a
reasonable choice.

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 3/7] xen/arm: Add support for GIC v3
  2014-07-24  9:03     ` Ian Campbell
@ 2014-07-24 10:26       ` Julien Grall
  0 siblings, 0 replies; 39+ messages in thread
From: Julien Grall @ 2014-07-24 10:26 UTC (permalink / raw)
  To: Ian Campbell
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi



On 24/07/14 10:03, Ian Campbell wrote:
> On Wed, 2014-07-23 at 18:28 +0100, Julien Grall wrote:
>> Hi Vijay,
>>
>> On 07/23/2014 02:41 PM, vijay.kilari@gmail.com wrote:
>>> +#define GICD_PIDR2_ARCH_REV_MASK     (0xf0)
>>> +#define GICD_PIDR2_ARCH_REV_OFFSET   (0x4)
>>
>> NIT: We tend to use SHIFT rather than OFFSET.
>>
>>> +#define GICD_PIDR2_ARCH_GICV3        (0x3)
>>> +#define GICD_PIDR2_USES_JEP_MASK     (0x08)
>>> +#define GICD_PIDR2_JEP_ID_MASK       (0x07)
>>
>> NIT: I don't see any usage of GICD_PIDR2_USES_JEP_MASK and
>> GICD_PIDR2_JEP_ID_MASK.
>>
>> Hence, you don't define the SHIFT. So I would drop them.
>
> FWIW I don't think it is a problem define bits of a register which
> aren't currently used on the principal that including the full set is a
> reasonable choice.

In this case, he has to be consistent and also define the SHIFT. It's 
pointless to only define a MASK.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 6/7] xen/arm: add SGI handling for GICv3
  2014-07-23 13:41 ` [PATCH v8 6/7] xen/arm: add SGI handling for GICv3 vijay.kilari
  2014-07-23 13:56   ` Stefano Stabellini
@ 2014-07-24 14:37   ` Julien Grall
  2014-07-28 14:04   ` Ian Campbell
  2 siblings, 0 replies; 39+ messages in thread
From: Julien Grall @ 2014-07-24 14:37 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar, manish.jaggi

Hi Vijay,

On 07/23/2014 02:41 PM, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> In ARMv8, write to ICC_SGI1R_EL1 register raises trap to EL2.
> Handle the trap and inject SGI to vcpu.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Acked-by: Julien Grall <julien.grall@linaro.org>

Regards,

> ---
> v8: - Return 0 on wrong irqmode instead of BUG()
>     - Add Comment on ignoring affinity 1,2,3 for SGI
>     - Assert on emulate_sysreg callback
> v7: - Introduced callback for sysreg emulation
>     - Removed unused parameter in inject_undef_exception()
>     - Use inject_undef64_exception for reporting sysreg
>       handling failure
> 
> v6: - Removed forward declaration of vgic_to_sgi() in vgic-v3.c
>     - Used vgic callback for SGI handling
>     - Alignment changes
> ---
>  xen/arch/arm/traps.c              |   15 +++++++++++
>  xen/arch/arm/vgic-v3.c            |   54 +++++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vgic.c               |    9 +++++++
>  xen/include/asm-arm/gic_v3_defs.h |    7 +++++
>  xen/include/asm-arm/sysregs.h     |    3 +++
>  xen/include/asm-arm/vgic.h        |    3 +++
>  6 files changed, 91 insertions(+)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 686d8b7..775bef1 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -41,6 +41,7 @@
>  #include "decode.h"
>  #include "vtimer.h"
>  #include <asm/gic.h>
> +#include <asm/vgic.h>
>  
>  /* The base of the stack must always be double-word aligned, which means
>   * that both the kernel half of struct cpu_user_regs (which is pushed in
> @@ -1641,6 +1642,20 @@ static void do_sysreg(struct cpu_user_regs *regs,
>              domain_crash_synchronous();
>          }
>          break;
> +    case HSR_SYSREG_ICC_SGI1R_EL1:
> +        if ( !vgic_emulate(regs, hsr) )
> +        {
> +            dprintk(XENLOG_WARNING,
> +                    "failed emulation of sysreg ICC_SGI1R_EL1 access\n");
> +            inject_undef64_exception(regs, hsr.len);
> +        }
> +        break;
> +    case HSR_SYSREG_ICC_SGI0R_EL1:
> +    case HSR_SYSREG_ICC_ASGI1R_EL1:
> +        /* TBD: Implement to support secure grp0/1 SGI forwarding */
> +        dprintk(XENLOG_WARNING,
> +                "Emulation of sysreg ICC_SGI0R_EL1/ASGI1R_EL1 not supported\n");
> +        inject_undef64_exception(regs, hsr.len);
>      default:
>   bad_sysreg:
>          {
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 6abaac3..8eb1193 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -850,6 +850,59 @@ write_ignore_64:
>      return 1;
>  }
>  
> +static int vgicv3_to_sgi(struct vcpu *v, register_t sgir)
> +{
> +    int virq;
> +    int irqmode;
> +    enum gic_sgi_mode sgi_mode;
> +    unsigned long vcpu_mask = 0;
> +
> +    irqmode = (sgir >> ICH_SGI_IRQMODE_SHIFT) & ICH_SGI_IRQMODE_MASK;
> +    virq = (sgir >> ICH_SGI_IRQ_SHIFT ) & ICH_SGI_IRQ_MASK;
> +    /* SGI's are injected at Rdist level 0. ignoring affinity 1, 2, 3 */
> +    vcpu_mask = sgir & ICH_SGI_TARGETLIST_MASK;
> +
> +    /* Map GIC sgi value to enum value */
> +    switch ( irqmode )
> +    {
> +    case ICH_SGI_TARGET_LIST:
> +        sgi_mode = SGI_TARGET_LIST;
> +        break;
> +    case ICH_SGI_TARGET_OTHERS:
> +        sgi_mode = SGI_TARGET_OTHERS;
> +        break;
> +    default:
> +        gdprintk(XENLOG_WARNING, "Wrong irq mode in SGI1R_EL1 register\n");
> +        return 0;
> +    }
> +
> +    return vgic_to_sgi(v, sgir, sgi_mode, virq, vcpu_mask);
> +}
> +
> +static int vgicv3_emulate_sysreg(struct cpu_user_regs *regs, union hsr hsr)
> +{
> +    struct vcpu *v = current;
> +    struct hsr_sysreg sysreg = hsr.sysreg;
> +    register_t *r = select_user_reg(regs, sysreg.reg);
> +
> +    ASSERT (hsr.ec == HSR_EC_SYSREG);
> +
> +    switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
> +    {
> +    case HSR_SYSREG_ICC_SGI1R_EL1:
> +        /* WO */
> +        if ( !sysreg.read )
> +            return vgicv3_to_sgi(v, *r);
> +        else
> +        {
> +            gdprintk(XENLOG_WARNING, "Reading SGI1R_EL1 - WO register\n");
> +            return 0;
> +        }
> +    default:
> +        return 0;
> +    }
> +}
> +
>  static const struct mmio_handler_ops vgic_rdistr_mmio_handler = {
>      .read_handler  = vgic_v3_rdistr_mmio_read,
>      .write_handler = vgic_v3_rdistr_mmio_write,
> @@ -900,6 +953,7 @@ static int vgicv3_domain_init(struct domain *d)
>  static const struct vgic_ops v3_ops = {
>      .vcpu_init   = vgicv3_vcpu_init,
>      .domain_init = vgicv3_domain_init,
> +    .emulate_sysreg  = vgicv3_emulate_sysreg,
>  };
>  
>  int vgic_v3_init(struct domain *d)
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 622219f..ffdf574 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -344,6 +344,15 @@ out:
>          smp_send_event_check_mask(cpumask_of(v->processor));
>  }
>  
> +int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
> +{
> +    struct vcpu *v = current;
> + 
> +    ASSERT(v->domain->arch.vgic.handler->emulate_sysreg != NULL);
> +
> +    return v->domain->arch.vgic.handler->emulate_sysreg(regs, hsr);
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 115c998..93dd86b 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -146,6 +146,13 @@
>  #define GICH_VMCR_PRIORITY_MASK      0xff
>  #define GICH_VMCR_PRIORITY_SHIFT     24
>  
> +#define ICH_SGI_IRQMODE_SHIFT        40
> +#define ICH_SGI_IRQMODE_MASK         0x1
> +#define ICH_SGI_TARGET_OTHERS        1
> +#define ICH_SGI_TARGET_LIST          0
> +#define ICH_SGI_IRQ_SHIFT            24
> +#define ICH_SGI_IRQ_MASK             0xf
> +#define ICH_SGI_TARGETLIST_MASK      0xffff
>  #endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
>  
>  /*
> diff --git a/xen/include/asm-arm/sysregs.h b/xen/include/asm-arm/sysregs.h
> index b4616ac..169b7ac 100644
> --- a/xen/include/asm-arm/sysregs.h
> +++ b/xen/include/asm-arm/sysregs.h
> @@ -78,6 +78,9 @@
>  #define HSR_SYSREG_PMINTENCLR_EL1 HSR_SYSREG(3,0,c9,c14,2)
>  #define HSR_SYSREG_MAIR_EL1       HSR_SYSREG(3,0,c10,c2,0)
>  #define HSR_SYSREG_AMAIR_EL1      HSR_SYSREG(3,0,c10,c3,0)
> +#define HSR_SYSREG_ICC_SGI1R_EL1  HSR_SYSREG(3,0,c12,c11,5)
> +#define HSR_SYSREG_ICC_ASGI1R_EL1 HSR_SYSREG(3,1,c12,c11,6)
> +#define HSR_SYSREG_ICC_SGI0R_EL1  HSR_SYSREG(3,2,c12,c11,7)
>  #define HSR_SYSREG_CONTEXTIDR_EL1 HSR_SYSREG(3,0,c13,c0,1)
>  
>  #define HSR_SYSREG_PMCR_EL0       HSR_SYSREG(3,3,c9,c12,0)
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index f580b78..63ecfa1 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -98,6 +98,8 @@ struct vgic_ops {
>      int (*vcpu_init)(struct vcpu *v);
>      /* Domain specific initialization of vGIC */
>      int (*domain_init)(struct domain *d);
> +    /* vGIC sysreg emulation */
> +    int (*emulate_sysreg)(struct cpu_user_regs *regs, union hsr hsr);
>  };
>  
>  /* Number of ranks of interrupt registers for a domain */
> @@ -165,6 +167,7 @@ extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
>  extern void vgic_clear_pending_irqs(struct vcpu *v);
>  extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
>  extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, int b, int n, int s);
> +extern int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr);
>  extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
> 


-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 7/7] xen/arm: check for GICv3 platform support
  2014-07-23 13:41 ` [PATCH v8 7/7] xen/arm: check for GICv3 platform support vijay.kilari
@ 2014-07-24 14:45   ` Julien Grall
  0 siblings, 0 replies; 39+ messages in thread
From: Julien Grall @ 2014-07-24 14:45 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar, manish.jaggi

On 07/23/2014 02:41 PM, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> ID_AA64PFR0_EL1 register provides information about GIC support.
> Check for this register in GICv3 driver.
> 
> Also print GICv3 support information in boot log
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v8: - Rearranged prf64 structure member variables
> v7: - return -ENODEV on lack GICv3 platform support instead of
>       panic
> ---
>  xen/arch/arm/gic-v3.c            |    7 +++++++
>  xen/arch/arm/setup.c             |    5 +++--
>  xen/include/asm-arm/cpufeature.h |    1 +
>  xen/include/asm-arm/processor.h  |    4 ++--
>  4 files changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 410d308..9291a7a 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -39,6 +39,7 @@
>  #include <asm/device.h>
>  #include <asm/gic.h>
>  #include <asm/gic_v3_defs.h>
> +#include <asm/cpufeature.h>
>  
>  struct rdist_region {
>      paddr_t base;
> @@ -1143,6 +1144,12 @@ static int __init gicv3_init(struct dt_device_node *node, const void *data)
>      int res, i;
>      uint32_t reg;
>  
> +    if ( !cpu_has_gicv3 )
> +    {
> +        dprintk(XENLOG_ERR, "Platform does not support GICv3\n");

AFAIU, this field indicates if we support GICv3 system registers or not.
It's entirely possible to use a GICv3 without system registers (see
4.4.1 System Register Access to CPU Interface Registers).

So I would be more specific in the error message. Smth like:

"The GICv3 drivers request system registers support.".

> +        return -ENODEV;
> +    }
> +
>      dt_device_set_used_by(node, DOMID_XEN);
>  
>      res = dt_device_get_address(node, 0, &gicv3.dbase, &gicv3.dbase_size);
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 78dc7f5..6a90c81 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -100,9 +100,10 @@ static void __init processor_id(void)
>             cpu_has_el2_32 ? "64+32" : cpu_has_el2_64 ? "64" : "No",
>             cpu_has_el1_32 ? "64+32" : cpu_has_el1_64 ? "64" : "No",
>             cpu_has_el0_32 ? "64+32" : cpu_has_el0_64 ? "64" : "No");
> -    printk("    Extensions:%s%s\n",
> +    printk("    Extensions:%s%s%s\n",
>             cpu_has_fp ? " FloatingPoint" : "",
> -           cpu_has_simd ? " AdvancedSIMD" : "");
> +           cpu_has_simd ? " AdvancedSIMD" : "",
> +           cpu_has_gicv3 ? " GICv3" : "");

I would rename into GICv3SysReg.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 1/7] xen/arm: Introduce sizes.h
  2014-07-23 13:41 ` [PATCH v8 1/7] xen/arm: Introduce sizes.h vijay.kilari
@ 2014-07-28 13:55   ` Ian Campbell
  0 siblings, 0 replies; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 13:55 UTC (permalink / raw)
  To: vijay.kilari
  Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, julien.grall,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi

On Wed, 2014-07-23 at 19:11 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> Add sizes.h file similar to include/linux/sizes.h of
> kernel code. These size macros helps in porting
> kernel drivers to Xen
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros
  2014-07-23 13:41 ` [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros vijay.kilari
  2014-07-23 17:20   ` Julien Grall
@ 2014-07-28 13:56   ` Ian Campbell
  1 sibling, 0 replies; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 13:56 UTC (permalink / raw)
  To: vijay.kilari
  Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, julien.grall,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi

On Wed, 2014-07-23 at 19:11 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> The register name parameter in {READ,WRITE}_SYSREG{32,64}
> macros is replaced as register name in mrs and msr assembly
> instruction.
> 
> If this register name is macro indirection, the register
> name is not replaced with designated macro. So replace the register
> name with __stringify macro, which replaces register name
> if it is macro
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 6/7] xen/arm: add SGI handling for GICv3
  2014-07-23 13:41 ` [PATCH v8 6/7] xen/arm: add SGI handling for GICv3 vijay.kilari
  2014-07-23 13:56   ` Stefano Stabellini
  2014-07-24 14:37   ` Julien Grall
@ 2014-07-28 14:04   ` Ian Campbell
  2 siblings, 0 replies; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 14:04 UTC (permalink / raw)
  To: vijay.kilari
  Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, julien.grall,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi

On Wed, 2014-07-23 at 19:11 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> In ARMv8, write to ICC_SGI1R_EL1 register raises trap to EL2.
> Handle the trap and inject SGI to vcpu.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-23 13:51   ` Vijay Kilari
@ 2014-07-28 14:35     ` Ian Campbell
  2014-07-28 14:55       ` Ian Campbell
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 14:35 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Julien Grall,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On Wed, 2014-07-23 at 19:21 +0530, Vijay Kilari wrote:
> Please ignore this patch.
> There are some alignment issues reappearing

OK.

I suppose there isn't much point in taking #3 without this?

FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
tracked it down to an actual location yet though. I'll let you know what
I find.

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 14:35     ` Ian Campbell
@ 2014-07-28 14:55       ` Ian Campbell
  2014-07-28 15:08         ` Julien Grall
  2014-07-28 15:35         ` Julien Grall
  0 siblings, 2 replies; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 14:55 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Julien Grall,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On Mon, 2014-07-28 at 15:35 +0100, Ian Campbell wrote:
> FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
> tracked it down to an actual location yet though. I'll let you know what
> I find.

The access to ICC_BPR1_EL1 in gicv3_cpu_init is faulting for some
reason. do_hyp_trap is then access current-> but it is too soon to do
that, hence a data abort occurs which is what I was seeing.

This is on a fast model with the boot-wrapper, perhaps the "firmware"
failed to setup something?

BTW, ICC_BPR1_EL1 is the secure alias of the NS version of the register.
Why not just use the NS version directly?

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 14:55       ` Ian Campbell
@ 2014-07-28 15:08         ` Julien Grall
  2014-07-28 15:26           ` Ian Campbell
  2014-07-28 15:35         ` Julien Grall
  1 sibling, 1 reply; 39+ messages in thread
From: Julien Grall @ 2014-07-28 15:08 UTC (permalink / raw)
  To: Ian Campbell, Vijay Kilari
  Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
	xen-devel, Stefano Stabellini, Jan Beulich, manish.jaggi

On 07/28/2014 03:55 PM, Ian Campbell wrote:
> On Mon, 2014-07-28 at 15:35 +0100, Ian Campbell wrote:
>> FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
>> tracked it down to an actual location yet though. I'll let you know what
>> I find.
> 
> The access to ICC_BPR1_EL1 in gicv3_cpu_init is faulting for some
> reason. do_hyp_trap is then access current-> but it is too soon to do
> that, hence a data abort occurs which is what I was seeing.

In which "case" does Xen end up? The sysreg one?
If so, it looks wrong to me that an error who come from the hypervisor
is using the current variable.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 15:08         ` Julien Grall
@ 2014-07-28 15:26           ` Ian Campbell
  2014-07-28 15:29             ` Julien Grall
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 15:26 UTC (permalink / raw)
  To: Julien Grall
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On Mon, 2014-07-28 at 16:08 +0100, Julien Grall wrote:
> On 07/28/2014 03:55 PM, Ian Campbell wrote:
> > On Mon, 2014-07-28 at 15:35 +0100, Ian Campbell wrote:
> >> FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
> >> tracked it down to an actual location yet though. I'll let you know what
> >> I find.
> > 
> > The access to ICC_BPR1_EL1 in gicv3_cpu_init is faulting for some
> > reason. do_hyp_trap is then access current-> but it is too soon to do
> > that, hence a data abort occurs which is what I was seeing.
> 
> In which "case" does Xen end up? The sysreg one?

Yes.

> If so, it looks wrong to me that an error who come from the hypervisor
> is using the current variable.

Correct, although it's an unexpected occurrence in the normal case.

If we intended to "probe" for system registers then we'd need to handle
this case explicitly. As it stands it just lead to a rather odd failure
mode. We could perhaps add some debugging checks on regs->cpsr mode.

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 15:26           ` Ian Campbell
@ 2014-07-28 15:29             ` Julien Grall
  0 siblings, 0 replies; 39+ messages in thread
From: Julien Grall @ 2014-07-28 15:29 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On 07/28/2014 04:26 PM, Ian Campbell wrote:
> On Mon, 2014-07-28 at 16:08 +0100, Julien Grall wrote:
>> On 07/28/2014 03:55 PM, Ian Campbell wrote:
>>> On Mon, 2014-07-28 at 15:35 +0100, Ian Campbell wrote:
>>>> FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
>>>> tracked it down to an actual location yet though. I'll let you know what
>>>> I find.
>>>
>>> The access to ICC_BPR1_EL1 in gicv3_cpu_init is faulting for some
>>> reason. do_hyp_trap is then access current-> but it is too soon to do
>>> that, hence a data abort occurs which is what I was seeing.
>>
>> In which "case" does Xen end up? The sysreg one?
> 
> Yes.
> 
>> If so, it looks wrong to me that an error who come from the hypervisor
>> is using the current variable.
> 
> Correct, although it's an unexpected occurrence in the normal case.
> 
> If we intended to "probe" for system registers then we'd need to handle
> this case explicitly. As it stands it just lead to a rather odd failure
> mode. We could perhaps add some debugging checks on regs->cpsr mode.

I think it's a good idea to add debugging checks. It would avoid
spending time to understand what's going.

But it's not part of this series.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 14:55       ` Ian Campbell
  2014-07-28 15:08         ` Julien Grall
@ 2014-07-28 15:35         ` Julien Grall
  2014-07-28 16:02           ` Ian Campbell
  1 sibling, 1 reply; 39+ messages in thread
From: Julien Grall @ 2014-07-28 15:35 UTC (permalink / raw)
  To: Ian Campbell, Vijay Kilari
  Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
	xen-devel, Stefano Stabellini, Jan Beulich, manish.jaggi

On 07/28/2014 03:55 PM, Ian Campbell wrote:
> On Mon, 2014-07-28 at 15:35 +0100, Ian Campbell wrote:
>> FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
>> tracked it down to an actual location yet though. I'll let you know what
>> I find.
> 
> The access to ICC_BPR1_EL1 in gicv3_cpu_init is faulting for some
> reason. do_hyp_trap is then access current-> but it is too soon to do
> that, hence a data abort occurs which is what I was seeing.
> 
> This is on a fast model with the boot-wrapper, perhaps the "firmware"
> failed to setup something?

IIRC, the "firmware" has to configure some register in secure. I suspect
it's ICC_SRE_EL1.

I guess your bootwrapper already contains this stuff.

> BTW, ICC_BPR1_EL1 is the secure alias of the NS version of the register.
> Why not just use the NS version directly?


AFAIU the spec, the name is not different between secure and non-secure.
The processor will know it following the state.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 15:35         ` Julien Grall
@ 2014-07-28 16:02           ` Ian Campbell
  2014-07-28 16:11             ` Julien Grall
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 16:02 UTC (permalink / raw)
  To: Julien Grall
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On Mon, 2014-07-28 at 16:35 +0100, Julien Grall wrote:
> On 07/28/2014 03:55 PM, Ian Campbell wrote:
> > On Mon, 2014-07-28 at 15:35 +0100, Ian Campbell wrote:
> >> FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
> >> tracked it down to an actual location yet though. I'll let you know what
> >> I find.
> > 
> > The access to ICC_BPR1_EL1 in gicv3_cpu_init is faulting for some
> > reason. do_hyp_trap is then access current-> but it is too soon to do
> > that, hence a data abort occurs which is what I was seeing.
> > 
> > This is on a fast model with the boot-wrapper, perhaps the "firmware"
> > failed to setup something?
> 
> IIRC, the "firmware" has to configure some register in secure. I suspect
> it's ICC_SRE_EL1.

_EL3, but otherwise I think you are right.

> I guess your bootwrapper already contains this stuff.

It turns out it does not because I'm using an old version of the
upstream bootwrapper modified for Xen.

Unfortunately upstream has moved on in ways which make rebasing the Xen
support a case of a complete rewrite so I'd been putting it off. Looks
like it just got bumped up the TODO a bit...

> > BTW, ICC_BPR1_EL1 is the secure alias of the NS version of the register.
> > Why not just use the NS version directly?
> 
> 
> AFAIU the spec, the name is not different between secure and non-secure.
> The processor will know it following the state.

The spec says:
        This register is an alias of the Non-secure view of
        ICC_BPR0_EL1, and a Secure access to this register is identical
        to a Non-secure access to ICC_BPR0_EL1.

So for Xen (necessarily running NS) ICC_BPR0_EL1 and ICC_BPR1_EL1 are
the same, but accessing BPR0 would seem more natural.

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 16:02           ` Ian Campbell
@ 2014-07-28 16:11             ` Julien Grall
  2014-07-28 16:25               ` Ian Campbell
  0 siblings, 1 reply; 39+ messages in thread
From: Julien Grall @ 2014-07-28 16:11 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On 07/28/2014 05:02 PM, Ian Campbell wrote:
> On Mon, 2014-07-28 at 16:35 +0100, Julien Grall wrote:
>> On 07/28/2014 03:55 PM, Ian Campbell wrote:
>>> On Mon, 2014-07-28 at 15:35 +0100, Ian Campbell wrote:
>>>> FYI I'm seeing a trap in gicv3_init when running on a model. I haven't
>>>> tracked it down to an actual location yet though. I'll let you know what
>>>> I find.
>>>
>>> The access to ICC_BPR1_EL1 in gicv3_cpu_init is faulting for some
>>> reason. do_hyp_trap is then access current-> but it is too soon to do
>>> that, hence a data abort occurs which is what I was seeing.
>>>
>>> This is on a fast model with the boot-wrapper, perhaps the "firmware"
>>> failed to setup something?
>>
>> IIRC, the "firmware" has to configure some register in secure. I suspect
>> it's ICC_SRE_EL1.
> 
> _EL3, but otherwise I think you are right.

It might be worse to add a check in Xen that ICC_SRE_EL3 as the SRE bit
enabled.

>> I guess your bootwrapper already contains this stuff.
> 
> It turns out it does not because I'm using an old version of the
> upstream bootwrapper modified for Xen.
> 
> Unfortunately upstream has moved on in ways which make rebasing the Xen
> support a case of a complete rewrite so I'd been putting it off. Looks
> like it just got bumped up the TODO a bit...
> 
>>> BTW, ICC_BPR1_EL1 is the secure alias of the NS version of the register.
>>> Why not just use the NS version directly?
>>
>>
>> AFAIU the spec, the name is not different between secure and non-secure.
>> The processor will know it following the state.
> 
> The spec says:
>         This register is an alias of the Non-secure view of
>         ICC_BPR0_EL1, and a Secure access to this register is identical
>         to a Non-secure access to ICC_BPR0_EL1.
> 
> So for Xen (necessarily running NS) ICC_BPR0_EL1 and ICC_BPR1_EL1 are
> the same, but accessing BPR0 would seem more natural.

You are right. I read quickly the specification.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 16:11             ` Julien Grall
@ 2014-07-28 16:25               ` Ian Campbell
  2014-07-28 16:30                 ` Julien Grall
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-07-28 16:25 UTC (permalink / raw)
  To: Julien Grall
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On Mon, 2014-07-28 at 17:11 +0100, Julien Grall wrote:

> It might be worse to add a check in Xen that ICC_SRE_EL3 as the SRE bit
> enabled.

I don't think it is readable from EL2.

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 4/7] xen/arm: Add virtual GICv3 support
  2014-07-28 16:25               ` Ian Campbell
@ 2014-07-28 16:30                 ` Julien Grall
  0 siblings, 0 replies; 39+ messages in thread
From: Julien Grall @ 2014-07-28 16:30 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On 07/28/2014 05:25 PM, Ian Campbell wrote:
> On Mon, 2014-07-28 at 17:11 +0100, Julien Grall wrote:
> 
>> It might be worse to add a check in Xen that ICC_SRE_EL3 as the SRE bit
>> enabled.
> 
> I don't think it is readable from EL2.

Oh yes.

FWIW, the Linux documentation say:

"For systems with a GICv3 interrupt controller:
  - If EL3 is present:
    ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1.
    ICC_SRE_EL3.SRE (bit 0) must be initialised to 0b1.
  - If the kernel is entered at EL1:
    ICC.SRE_EL2.Enable (bit 3) must be initialised to 0b1
    ICC_SRE_EL2.SRE (bit 0) must be initialised to 0b1.
"

I guess if the firmware doesn't set up correctly those values, then it's
the problem of the firmware developper.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
                   ` (6 preceding siblings ...)
  2014-07-23 13:41 ` [PATCH v8 7/7] xen/arm: check for GICv3 platform support vijay.kilari
@ 2014-08-03 21:24 ` Julien Grall
  2014-08-04 10:06   ` Ian Campbell
  7 siblings, 1 reply; 39+ messages in thread
From: Julien Grall @ 2014-08-03 21:24 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, tim, jbeulich, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar, manish.jaggi

Hi Vijay,

On 23/07/14 14:41, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add GICv3 support for arm64 platform.
>  From v6 version, this patch series is split into v6a and v6.
> All the GICv2 and VGIC code refactoring patches of v6a are merged.
>
> The v8 version of GICv3 patches are posted here.
>
> Tested with ARM64 simulator with multicore core
> and booted Dom0 kernel.

Do you have any plan to add support for guest? This actual series looks 
pointless to me if we are only able to boot DOM0... Hence, this series 
doesn't support GICv2 emulation on GICv3.

It would be nice if you fix the toolstack as soon as possible.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-08-03 21:24 ` [PATCH v8 0/7] xen/arm: Add GICv3 support Julien Grall
@ 2014-08-04 10:06   ` Ian Campbell
  2014-08-04 10:25     ` Julien Grall
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-08-04 10:06 UTC (permalink / raw)
  To: Julien Grall
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi

On Sun, 2014-08-03 at 22:24 +0100, Julien Grall wrote:
> Hi Vijay,
> 
> On 23/07/14 14:41, vijay.kilari@gmail.com wrote:
> > From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >
> > Add GICv3 support for arm64 platform.
> >  From v6 version, this patch series is split into v6a and v6.
> > All the GICv2 and VGIC code refactoring patches of v6a are merged.
> >
> > The v8 version of GICv3 patches are posted here.
> >
> > Tested with ARM64 simulator with multicore core
> > and booted Dom0 kernel.
> 
> Do you have any plan to add support for guest? This actual series looks 
> pointless to me if we are only able to boot DOM0...

That's not pointless, it's a necessary step along the way to supporting
GICv3.

Remember: Xen wouldn't boot on these systems at all without GICv3
support (since they lack v2 compatibility in h/w), so getting to a dom0
prompt is a useful step in its own right.

>  Hence, this series 
> doesn't support GICv2 emulation on GICv3.
> 
> It would be nice if you fix the toolstack as soon as possible.

A not for application bodge at the end of the series would be nice, but
not critical IMHO. Proper toolstack level support for gic v3 can come
next.

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-08-04 10:06   ` Ian Campbell
@ 2014-08-04 10:25     ` Julien Grall
  2014-08-04 10:42       ` Ian Campbell
  0 siblings, 1 reply; 39+ messages in thread
From: Julien Grall @ 2014-08-04 10:25 UTC (permalink / raw)
  To: Ian Campbell
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi

On 08/04/2014 11:06 AM, Ian Campbell wrote:
> On Sun, 2014-08-03 at 22:24 +0100, Julien Grall wrote:
>> Hi Vijay,
>>
>> On 23/07/14 14:41, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Add GICv3 support for arm64 platform.
>>>  From v6 version, this patch series is split into v6a and v6.
>>> All the GICv2 and VGIC code refactoring patches of v6a are merged.
>>>
>>> The v8 version of GICv3 patches are posted here.
>>>
>>> Tested with ARM64 simulator with multicore core
>>> and booted Dom0 kernel.
>>
>> Do you have any plan to add support for guest? This actual series looks 
>> pointless to me if we are only able to boot DOM0...
> 
> That's not pointless, it's a necessary step along the way to supporting
> GICv3.
> 
> Remember: Xen wouldn't boot on these systems at all without GICv3
> support (since they lack v2 compatibility in h/w), so getting to a dom0
> prompt is a useful step in its own right.
> 
>>  Hence, this series 
>> doesn't support GICv2 emulation on GICv3.
>>
>> It would be nice if you fix the toolstack as soon as possible.
> 
> A not for application bodge at the end of the series would be nice, but
> not critical IMHO. Proper toolstack level support for gic v3 can come
> next.

I'm mostly worry on the fact in 8 versions of this series we didn't see
any support/testing for GICv3 guest.

If it doesn't happen that mean we will have a Xen 4.5 with GICv3 support
only able to boot DOM0...

I want to make sure that won't happen.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-08-04 10:25     ` Julien Grall
@ 2014-08-04 10:42       ` Ian Campbell
       [not found]         ` <CALicx6ucRFNP4A3a2uBPTmhYzKtNWNOvwmPLUfy4Z2DoYDVr3g@mail.gmail.com>
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-08-04 10:42 UTC (permalink / raw)
  To: Julien Grall
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	tim, xen-devel, stefano.stabellini, jbeulich, manish.jaggi

On Mon, 2014-08-04 at 11:25 +0100, Julien Grall wrote:
> I'm mostly worry on the fact in 8 versions of this series we didn't see
> any support/testing for GICv3 guest.
> 
> If it doesn't happen that mean we will have a Xen 4.5 with GICv3 support
> only able to boot DOM0...

That would indeed to unfortunate, even if it doesn't block this series
as it is.

Vijay, are you planning to work on GICv3 guest support once the basic
Host support is in?

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
       [not found]         ` <CALicx6ucRFNP4A3a2uBPTmhYzKtNWNOvwmPLUfy4Z2DoYDVr3g@mail.gmail.com>
@ 2014-08-04 15:48           ` Ian Campbell
  2014-08-06 14:52             ` Vijay Kilari
  0 siblings, 1 reply; 39+ messages in thread
From: Ian Campbell @ 2014-08-04 15:48 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
	stefano.stabellini, jbeulich, manish.jaggi

(reinstating the cc)

On Mon, 2014-08-04 at 20:19 +0530, Vijay Kilari wrote:
> On Mon, Aug 4, 2014 at 4:12 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> Yes, Already I have booted domU with GICv3 patches with changes to
> xl tool.

Excellent!

>  Need some time to make it generic and send patches.

Perhaps you could send out whatever dirty version you have now for
people who want to try GICv3 to play with?

>  BTW, I have following questions
> 
> 1) The device tree is generated by xl tool for domU is hardcoding
> GICv2 node. How to generate device tree for GICv2 & GICv3 based on DomU?
> Should we always generate GIC node based on HW support  (by query hw
> support via domctl) ?

Something like that, yes.

There's a few different aspects which need to be taken care of, first of
all what the hardware+Xen actual support (v2 only, v3 only, v3 w/ v2
compat, vN+1 etc) and what the guest cfg actually asks for, what to do
if the guest cfg asks for something which Xen can't manage (e.g. v2 on a
v3 only system etc).

I think Julien is adding a domctl to control the number of SPIs in his
passthrough series. It probably makes sense to extend that into a more
generic "gic cfg" domctl with a field for requesting the version of gic
to support for this guest, with an explicit "default" setting for
letting Xen choose. I think it would be preferable to do all the gic
setup in one domctl, rather than proliferate loads of them.

By default I suppose Xen should be able to pick the latest GIC, unless
the guest cfg file says otherwise.

Which means we need some mechanism to find out the actual version which
Xen picked, so that the toolstack can generate the appropriate DTB node.
Perhaps that is as simple as having the domctl field be IN/OUT i.e.
updated with the actual selection.

I think we can get away with not exposing what Xen supports by simply
failing domctl requests for things which can't be done. If we really do
need this then we can perhaps use XENVER_capabilities or something new
which exposes which versions of h/w we support. I suppose we'll need
something like that sooner or later.

> 2) The GIC regs are hardcoded to physical base address for DOMU in
> xen/include/public/arch-arm.h
> 
> /* Physical Address Space */
> #define GUEST_GICD_BASE   0x03001000ULL
> #define GUEST_GICD_SIZE   0x00001000ULL
> #define GUEST_GICC_BASE   0x03002000ULL
> #define GUEST_GICC_SIZE   0x00000100ULL
> 
> any idea why it is kept same as physical address?

Same as what physical address?

The guest physical address space is static (and unrelated to any
particular hardware). So far we haven't had a need to make it dynamic.

Note that these are shared only between the Xen hypervisor (which
implements the "hardware") and Xen toolstack (which generates the DTB
describing the "hardware"). These defines are surrounded by ifdefs so
that e.g. guest kernels do not see them.

> How about for GICv3?. can we encode physical base address for domU's?

Yes. Just need to reserve some space, it can reuse the existing spaces
or extend it as needed. I think there is plenty of room in the vicinity
of 0x03xxxxxx.

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-08-04 15:48           ` Ian Campbell
@ 2014-08-06 14:52             ` Vijay Kilari
  2014-08-07 14:11               ` Julien Grall
  0 siblings, 1 reply; 39+ messages in thread
From: Vijay Kilari @ 2014-08-06 14:52 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
	xen-devel, Stefano Stabellini, Jan Beulich, manish.jaggi

Hi Ian,

On Mon, Aug 4, 2014 at 9:18 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> (reinstating the cc)
>
> On Mon, 2014-08-04 at 20:19 +0530, Vijay Kilari wrote:
>> On Mon, Aug 4, 2014 at 4:12 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>
>> Yes, Already I have booted domU with GICv3 patches with changes to
>> xl tool.
>
> Excellent!
>
>>  Need some time to make it generic and send patches.
>
> Perhaps you could send out whatever dirty version you have now for
> people who want to try GICv3 to play with?

If there are no further comments on GICv3 patches,
I can fix pending NITs and send new version, which you can merge.
This helps to test DomUs and your 48 bit support RFC patches.

>
>>  BTW, I have following questions
>>
>> 1) The device tree is generated by xl tool for domU is hardcoding
>> GICv2 node. How to generate device tree for GICv2 & GICv3 based on DomU?
>> Should we always generate GIC node based on HW support  (by query hw
>> support via domctl) ?
>
> Something like that, yes.
>
> There's a few different aspects which need to be taken care of, first of
> all what the hardware+Xen actual support (v2 only, v3 only, v3 w/ v2
> compat, vN+1 etc) and what the guest cfg actually asks for, what to do
> if the guest cfg asks for something which Xen can't manage (e.g. v2 on a
> v3 only system etc).
>
> I think Julien is adding a domctl to control the number of SPIs in his
> passthrough series. It probably makes sense to extend that into a more
> generic "gic cfg" domctl with a field for requesting the version of gic
> to support for this guest, with an explicit "default" setting for
> letting Xen choose. I think it would be preferable to do all the gic
> setup in one domctl, rather than proliferate loads of them.
>

 Any reference to Julien's patches for the same?

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-08-06 14:52             ` Vijay Kilari
@ 2014-08-07 14:11               ` Julien Grall
  2014-08-28 11:36                 ` Vijay Kilari
  0 siblings, 1 reply; 39+ messages in thread
From: Julien Grall @ 2014-08-07 14:11 UTC (permalink / raw)
  To: Vijay Kilari, Ian Campbell
  Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
	xen-devel, Stefano Stabellini, Jan Beulich, manish.jaggi

Hello Vijay,

On 08/06/2014 03:52 PM, Vijay Kilari wrote:
> Hi Ian,
> 
> On Mon, Aug 4, 2014 at 9:18 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> (reinstating the cc)
>>
>> On Mon, 2014-08-04 at 20:19 +0530, Vijay Kilari wrote:
>>> On Mon, Aug 4, 2014 at 4:12 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>
>>> Yes, Already I have booted domU with GICv3 patches with changes to
>>> xl tool.
>>
>> Excellent!
>>
>>>  Need some time to make it generic and send patches.
>>
>> Perhaps you could send out whatever dirty version you have now for
>> people who want to try GICv3 to play with?
> 
> If there are no further comments on GICv3 patches,
> I can fix pending NITs and send new version, which you can merge.
> This helps to test DomUs and your 48 bit support RFC patches.

I think Ian was asking to have a dirty version of your patch to support
GICv3 on guest.

It would be nice if you can send it. So people can give a try of the
support on the ARM64 model.

>>
>>>  BTW, I have following questions
>>>
>>> 1) The device tree is generated by xl tool for domU is hardcoding
>>> GICv2 node. How to generate device tree for GICv2 & GICv3 based on DomU?
>>> Should we always generate GIC node based on HW support  (by query hw
>>> support via domctl) ?
>>
>> Something like that, yes.
>>
>> There's a few different aspects which need to be taken care of, first of
>> all what the hardware+Xen actual support (v2 only, v3 only, v3 w/ v2
>> compat, vN+1 etc) and what the guest cfg actually asks for, what to do
>> if the guest cfg asks for something which Xen can't manage (e.g. v2 on a
>> v3 only system etc).
>>
>> I think Julien is adding a domctl to control the number of SPIs in his
>> passthrough series. It probably makes sense to extend that into a more
>> generic "gic cfg" domctl with a field for requesting the version of gic
>> to support for this guest, with an explicit "default" setting for
>> letting Xen choose. I think it would be preferable to do all the gic
>> setup in one domctl, rather than proliferate loads of them.

The plan was to allow the hypercall to be extended. I'm sure we would
need to configure other things in the future.

>>
> 
>  Any reference to Julien's patches for the same?

https://patches.linaro.org/34664/

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-08-07 14:11               ` Julien Grall
@ 2014-08-28 11:36                 ` Vijay Kilari
  2014-09-02 23:18                   ` Stefano Stabellini
  0 siblings, 1 reply; 39+ messages in thread
From: Vijay Kilari @ 2014-08-28 11:36 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On Thu, Aug 7, 2014 at 7:41 PM, Julien Grall <julien.grall@linaro.org> wrote:
> Hello Vijay,
>
> On 08/06/2014 03:52 PM, Vijay Kilari wrote:
>> Hi Ian,
>>
>> On Mon, Aug 4, 2014 at 9:18 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>> (reinstating the cc)
>>>
>>> On Mon, 2014-08-04 at 20:19 +0530, Vijay Kilari wrote:
>>>> On Mon, Aug 4, 2014 at 4:12 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>>
>>>> Yes, Already I have booted domU with GICv3 patches with changes to
>>>> xl tool.
>>>
>>> Excellent!
>>>
>>>>  Need some time to make it generic and send patches.
>>>
>>> Perhaps you could send out whatever dirty version you have now for
>>> people who want to try GICv3 to play with?
>>
>> If there are no further comments on GICv3 patches,
>> I can fix pending NITs and send new version, which you can merge.
>> This helps to test DomUs and your 48 bit support RFC patches.
>
> I think Ian was asking to have a dirty version of your patch to support
> GICv3 on guest.
>
> It would be nice if you can send it. So people can give a try of the
> support on the ARM64 model.

I have sent trivial patch for testing  GICv3 with domU.
Any further comments on GICv3 patch series?.

Regards
Vijay

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-08-28 11:36                 ` Vijay Kilari
@ 2014-09-02 23:18                   ` Stefano Stabellini
  2014-09-03 12:15                     ` Ian Campbell
  0 siblings, 1 reply; 39+ messages in thread
From: Stefano Stabellini @ 2014-09-02 23:18 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Julien Grall, Tim Deegan, xen-devel, Stefano Stabellini,
	Jan Beulich, manish.jaggi

On Thu, 28 Aug 2014, Vijay Kilari wrote:
> On Thu, Aug 7, 2014 at 7:41 PM, Julien Grall <julien.grall@linaro.org> wrote:
> > Hello Vijay,
> >
> > On 08/06/2014 03:52 PM, Vijay Kilari wrote:
> >> Hi Ian,
> >>
> >> On Mon, Aug 4, 2014 at 9:18 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >>> (reinstating the cc)
> >>>
> >>> On Mon, 2014-08-04 at 20:19 +0530, Vijay Kilari wrote:
> >>>> On Mon, Aug 4, 2014 at 4:12 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >>>
> >>>> Yes, Already I have booted domU with GICv3 patches with changes to
> >>>> xl tool.
> >>>
> >>> Excellent!
> >>>
> >>>>  Need some time to make it generic and send patches.
> >>>
> >>> Perhaps you could send out whatever dirty version you have now for
> >>> people who want to try GICv3 to play with?
> >>
> >> If there are no further comments on GICv3 patches,
> >> I can fix pending NITs and send new version, which you can merge.
> >> This helps to test DomUs and your 48 bit support RFC patches.
> >
> > I think Ian was asking to have a dirty version of your patch to support
> > GICv3 on guest.
> >
> > It would be nice if you can send it. So people can give a try of the
> > support on the ARM64 model.
> 
> I have sent trivial patch for testing  GICv3 with domU.
> Any further comments on GICv3 patch series?.

Sorry for the late reply but we are all currently traveling or on
vacation.  There are still a few minor unanswered points on v8, maybe
you could go ahead and send a v9?

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v8 0/7] xen/arm: Add GICv3 support
  2014-09-02 23:18                   ` Stefano Stabellini
@ 2014-09-03 12:15                     ` Ian Campbell
  0 siblings, 0 replies; 39+ messages in thread
From: Ian Campbell @ 2014-09-03 12:15 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Vijay Kilari, Prasun Kapoor, Vijaya Kumar K, Julien Grall,
	Tim Deegan, xen-devel, Stefano Stabellini, Jan Beulich,
	manish.jaggi

On Wed, 2014-09-03 at 00:18 +0100, Stefano Stabellini wrote:
> Sorry for the late reply but we are all currently traveling or on
> vacation.  There are still a few minor unanswered points on v8, maybe
> you could go ahead and send a v9?

Indeed. I've just applied the first two patches since they were
standalone but from the third onwards there were some outstanding (but
minor seeming) comments. Most concerning was Vijay's comment on #4 I think?

Ian.

^ permalink raw reply	[flat|nested] 39+ messages in thread

end of thread, other threads:[~2014-09-03 12:15 UTC | newest]

Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-07-23 13:41 [PATCH v8 0/7] xen/arm: Add GICv3 support vijay.kilari
2014-07-23 13:41 ` [PATCH v8 1/7] xen/arm: Introduce sizes.h vijay.kilari
2014-07-28 13:55   ` Ian Campbell
2014-07-23 13:41 ` [PATCH v8 2/7] xen/arm: Stringify the register name in sysreg read write macros vijay.kilari
2014-07-23 17:20   ` Julien Grall
2014-07-28 13:56   ` Ian Campbell
2014-07-23 13:41 ` [PATCH v8 3/7] xen/arm: Add support for GIC v3 vijay.kilari
2014-07-23 17:28   ` Julien Grall
2014-07-24  9:03     ` Ian Campbell
2014-07-24 10:26       ` Julien Grall
2014-07-23 13:41 ` [PATCH v8 4/7] xen/arm: Add virtual GICv3 support vijay.kilari
2014-07-23 13:51   ` Vijay Kilari
2014-07-28 14:35     ` Ian Campbell
2014-07-28 14:55       ` Ian Campbell
2014-07-28 15:08         ` Julien Grall
2014-07-28 15:26           ` Ian Campbell
2014-07-28 15:29             ` Julien Grall
2014-07-28 15:35         ` Julien Grall
2014-07-28 16:02           ` Ian Campbell
2014-07-28 16:11             ` Julien Grall
2014-07-28 16:25               ` Ian Campbell
2014-07-28 16:30                 ` Julien Grall
2014-07-23 13:41 ` [PATCH v8 5/7] xen/arm: Update Dom0 GIC dt node with GICv3 information vijay.kilari
2014-07-23 13:41 ` [PATCH v8 6/7] xen/arm: add SGI handling for GICv3 vijay.kilari
2014-07-23 13:56   ` Stefano Stabellini
2014-07-24 14:37   ` Julien Grall
2014-07-28 14:04   ` Ian Campbell
2014-07-23 13:41 ` [PATCH v8 7/7] xen/arm: check for GICv3 platform support vijay.kilari
2014-07-24 14:45   ` Julien Grall
2014-08-03 21:24 ` [PATCH v8 0/7] xen/arm: Add GICv3 support Julien Grall
2014-08-04 10:06   ` Ian Campbell
2014-08-04 10:25     ` Julien Grall
2014-08-04 10:42       ` Ian Campbell
     [not found]         ` <CALicx6ucRFNP4A3a2uBPTmhYzKtNWNOvwmPLUfy4Z2DoYDVr3g@mail.gmail.com>
2014-08-04 15:48           ` Ian Campbell
2014-08-06 14:52             ` Vijay Kilari
2014-08-07 14:11               ` Julien Grall
2014-08-28 11:36                 ` Vijay Kilari
2014-09-02 23:18                   ` Stefano Stabellini
2014-09-03 12:15                     ` Ian Campbell

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.