All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v4 00/16] xen/arm: Add GICv3 support
@ 2014-05-26 10:26 vijay.kilari
  2014-05-26 10:26 ` [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder vijay.kilari
                   ` (16 more replies)
  0 siblings, 17 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Made changes to existing gic and vgic drivers
to make it generic and added support for GIC v3
hw version

Tested with ARM64 simulator with multicore core
and booted Dom0 kernel.

Major changes in v4:
 - Changed io handlers to take mmio address and size as
   parameters
 - ioremap is used instead of fixmap to map GICv2 address
   space. Removed /4 in GICv2 register definitions
 - vGIC driver now uses register size to calculate IRQ rank
 - GICv2 LR register definitions are declared locally in GICv2 driver
 - GICR & GICD common register handling in vgic-v3 driver are segregated
   in one common function
 - irq_hw_controller definition is managed in GICv2 and GICv3 drivers
 - Made irq_ops const
 - GIC DT node is updated in respective drivers
 - Comments and coding style fixes
 - Rebased on remotes/origin/no_maintenance_interrupts-v8 + Julien's
   patch set

Major changes in v3:
 - Moved io.h from xen/arch/arm to xen/include/asm-arm
   and renamed as mmio.h
 - Made all the io handlers as domain specific and removed
   check handlers. Instead address and size are passed
 - Cleaned up sgi handling functions in V2 gic driver and
   made generic
 - Removed unused parameters in do_sgi call
 - VGIC driver is chosen based on GIC driver initialized
 - Implemented SGI handling for GICv3 to boot multicore
 - Missing coding style fixes
 - Rebased to remotes/origin/no_maintenance_interrupts-v6
   branch of Stephano's unstable tree

Major Changes in v2:
 - Added per domain io handler
 - Use device api to initialize gic driver
 - Avoid use of void pointer to access common data
 - Moved vgic address information data from domain.h
 - Coding style


Vijaya Kumar K (16):
  xen/arm: move io.h as mmio.h to include folder
  xen/arm: make mmio handlers domain specific
  xen/arm: make sgi handling generic
  xen/arm: remove unused parameter in do_sgi call
  xen/arm: use ioremap to map gic-v2 registers
  xen/arm: segregate and split GIC low level functionality
  arm/xen: move GIC context data structure to gic driver
  xen/arm: use device api to detect GIC version
  xen/arm: move vgic rank data to gic header file
  xen/arm: move vgic defines to vgic header file
  xen/arm: calculate vgic irq rank based on register size
  xen/arm: split vgic driver into generic and vgic-v2 driver
  xen/arm: Add support for GIC v3
  xen/arm: Add virtual GICv3 support
  xen/arm: Update Dom0 GIC dt node with GICv3 information
  xen/arm: add SGI handling for GICv3

 xen/arch/arm/Makefile                         |    6 +-
 xen/arch/arm/domain.c                         |    5 +
 xen/arch/arm/domain_build.c                   |   49 +-
 xen/arch/arm/gic-v2.c                         |  735 +++++++++++++++
 xen/arch/arm/gic-v3.c                         | 1202 +++++++++++++++++++++++++
 xen/arch/arm/gic.c                            |  534 ++---------
 xen/arch/arm/io.c                             |   59 +-
 xen/arch/arm/irq.c                            |    1 +
 xen/arch/arm/time.c                           |    1 +
 xen/arch/arm/traps.c                          |   32 +-
 xen/arch/arm/vgic-v2.c                        |  487 ++++++++++
 xen/arch/arm/vgic-v3.c                        |  932 +++++++++++++++++++
 xen/arch/arm/vgic.c                           |  647 ++-----------
 xen/arch/arm/vpsci.c                          |    1 +
 xen/arch/arm/vtimer.c                         |    1 +
 xen/arch/arm/vuart.c                          |   53 +-
 xen/include/asm-arm/device.h                  |    3 +-
 xen/include/asm-arm/domain.h                  |   30 +-
 xen/include/asm-arm/gic.h                     |  270 ++++--
 xen/include/asm-arm/gic_v3_defs.h             |  164 ++++
 xen/{arch/arm/io.h => include/asm-arm/mmio.h} |   30 +-
 xen/include/asm-arm/processor.h               |   22 +
 xen/include/asm-arm/sysregs.h                 |    3 +
 xen/include/asm-arm/vgic.h                    |  145 +++
 xen/include/xen/lib.h                         |    2 +
 xen/include/xen/sched.h                       |   16 +
 26 files changed, 4242 insertions(+), 1188 deletions(-)
 create mode 100644 xen/arch/arm/gic-v2.c
 create mode 100644 xen/arch/arm/gic-v3.c
 create mode 100644 xen/arch/arm/vgic-v2.c
 create mode 100644 xen/arch/arm/vgic-v3.c
 create mode 100644 xen/include/asm-arm/gic_v3_defs.h
 rename xen/{arch/arm/io.h => include/asm-arm/mmio.h} (66%)
 create mode 100644 xen/include/asm-arm/vgic.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 78+ messages in thread

* [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 11:28   ` Julien Grall
  2014-05-28 13:55   ` Stefano Stabellini
  2014-05-26 10:26 ` [PATCH v4 02/16] xen/arm: make mmio handlers domain specific vijay.kilari
                   ` (15 subsequent siblings)
  16 siblings, 2 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

io.h is local to arch/arm folder. move this file as mmio.h
file to include/asm-arm folder as it might be
required for inclusion in other header files in future.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/io.c                             |    3 +--
 xen/arch/arm/traps.c                          |    2 +-
 xen/arch/arm/vgic.c                           |    2 +-
 xen/arch/arm/vuart.c                          |    2 +-
 xen/{arch/arm/io.h => include/asm-arm/mmio.h} |    8 ++++----
 5 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index a6db00b..ada1918 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -19,8 +19,7 @@
 #include <xen/config.h>
 #include <xen/lib.h>
 #include <asm/current.h>
-
-#include "io.h"
+#include <asm/mmio.h>
 
 static const struct mmio_handler *const mmio_handlers[] =
 {
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index a4bdaaa..9348147 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -36,9 +36,9 @@
 #include <asm/regs.h>
 #include <asm/cpregs.h>
 #include <asm/psci.h>
+#include <asm/mmio.h>
 
 #include "decode.h"
-#include "io.h"
 #include "vtimer.h"
 #include <asm/gic.h>
 
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 4869b87..4962e70 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -27,7 +27,7 @@
 
 #include <asm/current.h>
 
-#include "io.h"
+#include <asm/mmio.h>
 #include <asm/gic.h>
 
 #define REG(n) (n/4)
diff --git a/xen/arch/arm/vuart.c b/xen/arch/arm/vuart.c
index c02a8a9..953cd46 100644
--- a/xen/arch/arm/vuart.c
+++ b/xen/arch/arm/vuart.c
@@ -38,9 +38,9 @@
 #include <xen/errno.h>
 #include <xen/ctype.h>
 #include <xen/serial.h>
+#include <asm/mmio.h>
 
 #include "vuart.h"
-#include "io.h"
 
 #define domain_has_vuart(d) ((d)->arch.vuart.info != NULL)
 
diff --git a/xen/arch/arm/io.h b/xen/include/asm-arm/mmio.h
similarity index 91%
rename from xen/arch/arm/io.h
rename to xen/include/asm-arm/mmio.h
index 8d252c0..5870985 100644
--- a/xen/arch/arm/io.h
+++ b/xen/include/asm-arm/mmio.h
@@ -1,5 +1,5 @@
 /*
- * xen/arch/arm/io.h
+ * xen/include/asm-arm/mmio.h
  *
  * ARM I/O handlers
  *
@@ -16,8 +16,8 @@
  * GNU General Public License for more details.
  */
 
-#ifndef __ARCH_ARM_IO_H__
-#define __ARCH_ARM_IO_H__
+#ifndef __ASM_ARM_MMIO_H__
+#define __ASM_ARM_MMIO_H__
 
 #include <xen/lib.h>
 #include <asm/processor.h>
@@ -45,7 +45,7 @@ extern const struct mmio_handler vuart_mmio_handler;
 
 extern int handle_mmio(mmio_info_t *info);
 
-#endif
+#endif  /* __ASM_ARM_MMIO_H__ */
 
 /*
  * Local variables:
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 02/16] xen/arm: make mmio handlers domain specific
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
  2014-05-26 10:26 ` [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 12:33   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 03/16] xen/arm: make sgi handling generic vijay.kilari
                   ` (14 subsequent siblings)
  16 siblings, 1 reply; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

register mmio handlers at runtime and make mmio handlers
domain specific and check handlers are removed.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/domain.c        |    3 ++
 xen/arch/arm/io.c            |   56 +++++++++++++++++++++++-------
 xen/arch/arm/vgic.c          |   79 ++++++++++++++++++------------------------
 xen/arch/arm/vuart.c         |   51 ++++++++++++---------------
 xen/include/asm-arm/domain.h |    2 ++
 xen/include/asm-arm/mmio.h   |   22 +++++++++---
 6 files changed, 123 insertions(+), 90 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 33141e3..40acfb3 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -508,6 +508,9 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags)
     share_xen_page_with_guest(
         virt_to_page(d->shared_info), d, XENSHARE_writable);
 
+    if ( (rc = domain_io_init(d)) != 0 )
+        goto fail;
+
     if ( (rc = p2m_alloc_table(d)) != 0 )
         goto fail;
 
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index ada1918..220488a 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -1,5 +1,5 @@
 /*
- * xen/arch/arm/io.h
+ * xen/arch/arm/io.c
  *
  * ARM I/O handlers
  *
@@ -18,29 +18,61 @@
 
 #include <xen/config.h>
 #include <xen/lib.h>
+#include <xen/spinlock.h>
+#include <xen/sched.h>
 #include <asm/current.h>
 #include <asm/mmio.h>
 
-static const struct mmio_handler *const mmio_handlers[] =
-{
-    &vgic_distr_mmio_handler,
-    &vuart_mmio_handler,
-};
-#define MMIO_HANDLER_NR ARRAY_SIZE(mmio_handlers)
-
 int handle_mmio(mmio_info_t *info)
 {
     struct vcpu *v = current;
     int i;
+    struct mmio_handler *mmio_handler;
+    struct io_handler *io_handlers = &v->domain->arch.io_handlers;
 
-    for ( i = 0; i < MMIO_HANDLER_NR; i++ )
-        if ( mmio_handlers[i]->check_handler(v, info->gpa) )
+    for ( i = 0; i < io_handlers->num_entries; i++ )
+    {
+        mmio_handler = &io_handlers->mmio_handlers[i];
+
+        if ( (info->gpa >= mmio_handler->addr) &&
+             (info->gpa < (mmio_handler->addr + mmio_handler->size)) )
+        {
             return info->dabt.write ?
-                mmio_handlers[i]->write_handler(v, info) :
-                mmio_handlers[i]->read_handler(v, info);
+                mmio_handler->mmio_handler_ops->write_handler(v, info) :
+                mmio_handler->mmio_handler_ops->read_handler(v, info);
+        }
+    }
 
     return 0;
 }
+
+void register_mmio_handler(struct domain *d,
+                           const struct mmio_handler_ops *handle,
+                           paddr_t addr, paddr_t size)
+{
+    struct io_handler *handler = &d->arch.io_handlers;
+
+    BUG_ON(handler->num_entries >= MAX_IO_HANDLER);
+
+    spin_lock(&handler->lock);
+
+    handler->mmio_handlers[handler->num_entries].mmio_handler_ops = handle;
+    handler->mmio_handlers[handler->num_entries].addr = addr;
+    handler->mmio_handlers[handler->num_entries].size = size;
+    handler->num_entries++;
+    dsb(sy);
+
+    spin_unlock(&handler->lock);
+}
+
+int domain_io_init(struct domain *d)
+{
+   spin_lock_init(&d->arch.io_handlers.lock);
+   d->arch.io_handlers.num_entries = 0;
+
+   return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 4962e70..151ec3e 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -73,43 +73,6 @@ static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
         return NULL;
 }
 
-int domain_vgic_init(struct domain *d)
-{
-    int i;
-
-    d->arch.vgic.ctlr = 0;
-
-    /* Currently nr_lines in vgic and gic doesn't have the same meanings
-     * Here nr_lines = number of SPIs
-     */
-    if ( is_hardware_domain(d) )
-        d->arch.vgic.nr_lines = gic_number_lines() - 32;
-    else
-        d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
-
-    d->arch.vgic.shared_irqs =
-        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
-    if ( d->arch.vgic.shared_irqs == NULL )
-        return -ENOMEM;
-
-    d->arch.vgic.pending_irqs =
-        xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
-    if ( d->arch.vgic.pending_irqs == NULL )
-    {
-        xfree(d->arch.vgic.shared_irqs);
-        return -ENOMEM;
-    }
-
-    for (i=0; i<d->arch.vgic.nr_lines; i++)
-    {
-        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
-        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
-    }
-    for (i=0; i<DOMAIN_NR_RANKS(d); i++)
-        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
-    return 0;
-}
-
 void domain_vgic_free(struct domain *d)
 {
     xfree(d->arch.vgic.shared_irqs);
@@ -676,15 +639,7 @@ write_ignore:
     return 1;
 }
 
-static int vgic_distr_mmio_check(struct vcpu *v, paddr_t addr)
-{
-    struct domain *d = v->domain;
-
-    return (addr >= (d->arch.vgic.dbase)) && (addr < (d->arch.vgic.dbase + PAGE_SIZE));
-}
-
-const struct mmio_handler vgic_distr_mmio_handler = {
-    .check_handler = vgic_distr_mmio_check,
+const struct mmio_handler_ops vgic_distr_mmio_handler = {
     .read_handler  = vgic_distr_mmio_read,
     .write_handler = vgic_distr_mmio_write,
 };
@@ -766,6 +721,38 @@ out:
         smp_send_event_check_mask(cpumask_of(v->processor));
 }
 
+int domain_vgic_init(struct domain *d)
+{
+    int i;
+
+    d->arch.vgic.ctlr = 0;
+
+    /* Currently nr_lines in vgic and gic doesn't have the same meanings
+     * Here nr_lines = number of SPIs
+     */
+    if ( d->domain_id == 0 )
+        d->arch.vgic.nr_lines = gic_number_lines() - 32;
+    else
+        d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
+
+    d->arch.vgic.shared_irqs =
+        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
+    d->arch.vgic.pending_irqs =
+        xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
+    for (i=0; i<d->arch.vgic.nr_lines; i++)
+    {
+        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
+        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
+    }
+    for (i=0; i<DOMAIN_NR_RANKS(d); i++)
+        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
+
+    register_mmio_handler(d, &vgic_distr_mmio_handler,
+                          d->arch.vgic.dbase, PAGE_SIZE);
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vuart.c b/xen/arch/arm/vuart.c
index 953cd46..52f3259 100644
--- a/xen/arch/arm/vuart.c
+++ b/xen/arch/arm/vuart.c
@@ -44,24 +44,6 @@
 
 #define domain_has_vuart(d) ((d)->arch.vuart.info != NULL)
 
-int domain_vuart_init(struct domain *d)
-{
-    ASSERT( is_hardware_domain(d) );
-
-    d->arch.vuart.info = serial_vuart_info(SERHND_DTUART);
-    if ( !d->arch.vuart.info )
-        return 0;
-
-    spin_lock_init(&d->arch.vuart.lock);
-    d->arch.vuart.idx = 0;
-
-    d->arch.vuart.buf = xzalloc_array(char, VUART_BUF_SIZE);
-    if ( !d->arch.vuart.buf )
-        return -ENOMEM;
-
-    return 0;
-}
-
 void domain_vuart_free(struct domain *d)
 {
     if ( !domain_has_vuart(d) )
@@ -92,14 +74,6 @@ static void vuart_print_char(struct vcpu *v, char c)
     spin_unlock(&uart->lock);
 }
 
-static int vuart_mmio_check(struct vcpu *v, paddr_t addr)
-{
-    const struct vuart_info *info = v->domain->arch.vuart.info;
-
-    return (domain_has_vuart(v->domain) && addr >= info->base_addr &&
-            addr <= (info->base_addr + info->size));
-}
-
 static int vuart_mmio_read(struct vcpu *v, mmio_info_t *info)
 {
     struct domain *d = v->domain;
@@ -133,12 +107,33 @@ static int vuart_mmio_write(struct vcpu *v, mmio_info_t *info)
     return 1;
 }
 
-const struct mmio_handler vuart_mmio_handler = {
-    .check_handler = vuart_mmio_check,
+const struct mmio_handler_ops vuart_mmio_handler = {
     .read_handler  = vuart_mmio_read,
     .write_handler = vuart_mmio_write,
 };
 
+int domain_vuart_init(struct domain *d)
+{
+    ASSERT( !d->domain_id );
+
+    d->arch.vuart.info = serial_vuart_info(SERHND_DTUART);
+    if ( !d->arch.vuart.info )
+        return 0;
+
+    spin_lock_init(&d->arch.vuart.lock);
+    d->arch.vuart.idx = 0;
+
+    d->arch.vuart.buf = xzalloc_array(char, VUART_BUF_SIZE);
+    if ( !d->arch.vuart.buf )
+        return -ENOMEM;
+
+    register_mmio_handler(d, &vuart_mmio_handler,
+                          d->arch.vuart.info->base_addr,
+                          d->arch.vuart.info->size);
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index f47f928..61a498f 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -7,6 +7,7 @@
 #include <asm/page.h>
 #include <asm/p2m.h>
 #include <asm/vfp.h>
+#include <asm/mmio.h>
 #include <public/hvm/params.h>
 #include <xen/serial.h>
 
@@ -117,6 +118,7 @@ struct arch_domain
     struct hvm_domain hvm_domain;
     xen_pfn_t *grant_table_gpfn;
 
+    struct io_handler io_handlers;
     /* Continuable domain_relinquish_resources(). */
     enum {
         RELMEM_not_started,
diff --git a/xen/include/asm-arm/mmio.h b/xen/include/asm-arm/mmio.h
index 5870985..0160f09 100644
--- a/xen/include/asm-arm/mmio.h
+++ b/xen/include/asm-arm/mmio.h
@@ -23,6 +23,8 @@
 #include <asm/processor.h>
 #include <asm/regs.h>
 
+#define MAX_IO_HANDLER  16
+
 typedef struct
 {
     struct hsr_dabt dabt;
@@ -34,16 +36,28 @@ typedef int (*mmio_read_t)(struct vcpu *v, mmio_info_t *info);
 typedef int (*mmio_write_t)(struct vcpu *v, mmio_info_t *info);
 typedef int (*mmio_check_t)(struct vcpu *v, paddr_t addr);
 
-struct mmio_handler {
-    mmio_check_t check_handler;
+struct mmio_handler_ops {
     mmio_read_t read_handler;
     mmio_write_t write_handler;
 };
 
-extern const struct mmio_handler vgic_distr_mmio_handler;
-extern const struct mmio_handler vuart_mmio_handler;
+struct mmio_handler {
+    paddr_t addr;
+    paddr_t size;
+    const struct mmio_handler_ops *mmio_handler_ops;
+};
+
+struct io_handler {
+    int num_entries;
+    spinlock_t lock;
+    struct mmio_handler mmio_handlers[MAX_IO_HANDLER];
+};
 
 extern int handle_mmio(mmio_info_t *info);
+void register_mmio_handler(struct domain *d,
+                           const struct mmio_handler_ops *handle,
+                           paddr_t addr, paddr_t size);
+int domain_io_init(struct domain *d);
 
 #endif  /* __ASM_ARM_MMIO_H__ */
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 03/16] xen/arm: make sgi handling generic
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
  2014-05-26 10:26 ` [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder vijay.kilari
  2014-05-26 10:26 ` [PATCH v4 02/16] xen/arm: make mmio handlers domain specific vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 12:41   ` Julien Grall
  2014-05-28 14:10   ` Stefano Stabellini
  2014-05-26 10:26 ` [PATCH v4 04/16] xen/arm: remove unused parameter in do_sgi call vijay.kilari
                   ` (13 subsequent siblings)
  16 siblings, 2 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

move all the hw specific sgi handling functionality
to one function and use it.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/gic.c        |   37 ++++++++++++++++++++++++++++---------
 xen/include/asm-arm/gic.h |    8 ++++++++
 2 files changed, 36 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index efcd785..f8e49df 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -483,21 +483,40 @@ void __init gic_init(void)
     spin_unlock(&gic.lock);
 }
 
-void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
+static void send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
+                     uint8_t irqmode)
 {
     unsigned int mask = 0;
+
+    switch ( irqmode )
+    {
+    case SGI_TARGET_OTHERS:
+        GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS | sgi;
+        break;
+    case SGI_TARGET_SELF:
+        GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF | sgi;
+        break;
+    case SGI_TARGET_LIST:
+        mask = gic_cpu_mask(cpu_mask);
+        GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST |
+                          (mask<<GICD_SGI_TARGET_SHIFT) | sgi;
+        break;
+    default:
+        BUG_ON(1);
+    }
+}
+
+void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
+{
     cpumask_t online_mask;
 
     ASSERT(sgi < 16); /* There are only 16 SGIs */
 
     cpumask_and(&online_mask, cpumask, &cpu_online_map);
-    mask = gic_cpu_mask(&online_mask);
 
     dsb(sy);
 
-    GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST
-        | (mask<<GICD_SGI_TARGET_SHIFT)
-        | sgi;
+    send_SGI(&online_mask, sgi, SGI_TARGET_LIST);
 }
 
 void send_SGI_one(unsigned int cpu, enum gic_sgi sgi)
@@ -512,18 +531,18 @@ void send_SGI_self(enum gic_sgi sgi)
 
     dsb(sy);
 
-    GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF
-        | sgi;
+    send_SGI(cpumask_of(smp_processor_id()), sgi, SGI_TARGET_SELF);
 }
 
 void send_SGI_allbutself(enum gic_sgi sgi)
 {
+   cpumask_t all_others_mask;
    ASSERT(sgi < 16); /* There are only 16 SGIs */
 
+   cpumask_andnot(&all_others_mask, &cpu_possible_map, cpumask_of(smp_processor_id()));
    dsb(sy);
 
-   GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS
-       | sgi;
+   send_SGI(&all_others_mask, sgi, SGI_TARGET_OTHERS);
 }
 
 void smp_send_state_dump(unsigned int cpu)
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 8e37ccf..c7b7368 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -210,6 +210,14 @@ enum gic_sgi {
     GIC_SGI_DUMP_STATE  = 1,
     GIC_SGI_CALL_FUNCTION = 2,
 };
+
+/* SGI irq mode types */
+enum gic_sgi_mode {
+    SGI_TARGET_LIST,
+    SGI_TARGET_OTHERS,
+    SGI_TARGET_SELF,
+};
+
 extern void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi);
 extern void send_SGI_one(unsigned int cpu, enum gic_sgi sgi);
 extern void send_SGI_self(enum gic_sgi sgi);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 04/16] xen/arm: remove unused parameter in do_sgi call
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (2 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 03/16] xen/arm: make sgi handling generic vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 12:48   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers vijay.kilari
                   ` (12 subsequent siblings)
  16 siblings, 1 reply; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

othercpu parameter in do_sgi is unused

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/gic.c |    5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index f8e49df..ce2269f 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -856,7 +856,7 @@ void gic_inject(void)
 
 }
 
-static void do_sgi(struct cpu_user_regs *regs, int othercpu, enum gic_sgi sgi)
+static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
 {
     /* Lower the priority */
     GICC[GICC_EOIR] = sgi;
@@ -900,8 +900,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
         }
         else if (unlikely(irq < 16))
         {
-            unsigned int cpu = (intack & GICC_IA_CPU_MASK) >> GICC_IA_CPU_SHIFT;
-            do_sgi(regs, cpu, irq);
+            do_sgi(regs, irq);
         }
         else
         {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (3 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 04/16] xen/arm: remove unused parameter in do_sgi call vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 13:10   ` Julien Grall
  2014-05-28 14:26   ` Stefano Stabellini
  2014-05-26 10:26 ` [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality vijay.kilari
                   ` (11 subsequent siblings)
  16 siblings, 2 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

gic-v2 driver uses fixmap to map the registers.
Instead use ioremap to access mmio registers.

With this patch, gic-v2 register definitions are updated
to use obsolute offset address instead of dividing the
register offset by 4.

Update vgic driver logic to compute using obsolute register
address offsets

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/gic.c        |  150 ++++++++++++++++++++++++---------------------
 xen/arch/arm/vgic.c       |   17 ++---
 xen/include/asm-arm/gic.h |  104 +++++++++++++++----------------
 3 files changed, 142 insertions(+), 129 deletions(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index ce2269f..470e6c0 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -31,20 +31,23 @@
 #include <asm/p2m.h>
 #include <asm/domain.h>
 #include <asm/platform.h>
-
+#include <asm/io.h>
 #include <asm/gic.h>
 
-/* Access to the GIC Distributor registers through the fixmap */
-#define GICD ((volatile uint32_t *) FIXMAP_ADDR(FIXMAP_GICD))
-#define GICC ((volatile uint32_t *) FIXMAP_ADDR(FIXMAP_GICC1))
-#define GICH ((volatile uint32_t *) FIXMAP_ADDR(FIXMAP_GICH))
+#define GICD (gic.map_dbase)
+#define GICC (gic.map_cbase)
+#define GICH (gic.map_hbase)
+
 static void gic_restore_pending_irqs(struct vcpu *v);
 
 /* Global state */
 static struct {
     paddr_t dbase;       /* Address of distributor registers */
+    void __iomem * map_dbase;  /* IO mapped Address of distributor registers */
     paddr_t cbase;       /* Address of CPU interface registers */
+    void __iomem * map_cbase; /* IO mapped Address of CPU interface registers*/
     paddr_t hbase;       /* Address of virtual interface registers */
+    void __iomem * map_hbase; /* IO Address of virtual interface registers */
     paddr_t vbase;       /* Address of virtual cpu interface registers */
     unsigned int lines;  /* Number of interrupts (SPIs + PPIs + SGIs) */
     unsigned int maintenance_irq; /* IRQ maintenance */
@@ -101,12 +104,12 @@ void gic_save_state(struct vcpu *v)
      * accessed simultaneously by another pCPU.
      */
     for ( i=0; i<nr_lrs; i++)
-        v->arch.gic_lr[i] = GICH[GICH_LR + i];
+        v->arch.gic_lr[i] = readl_relaxed(GICH + GICH_LR + i * 4);
     v->arch.lr_mask = this_cpu(lr_mask);
-    v->arch.gic_apr = GICH[GICH_APR];
-    v->arch.gic_vmcr = GICH[GICH_VMCR];
+    v->arch.gic_apr = readl_relaxed(GICH + GICH_APR);
+    v->arch.gic_vmcr = readl_relaxed(GICH + GICH_VMCR);
     /* Disable until next VCPU scheduled */
-    GICH[GICH_HCR] = 0;
+    writel_relaxed(0, GICH + GICH_HCR);
     isb();
 }
 
@@ -120,10 +123,10 @@ void gic_restore_state(struct vcpu *v)
 
     this_cpu(lr_mask) = v->arch.lr_mask;
     for ( i=0; i<nr_lrs; i++)
-        GICH[GICH_LR + i] = v->arch.gic_lr[i];
-    GICH[GICH_APR] = v->arch.gic_apr;
-    GICH[GICH_VMCR] = v->arch.gic_vmcr;
-    GICH[GICH_HCR] = GICH_HCR_EN;
+        writel_relaxed(v->arch.gic_lr[i], GICH + GICH_LR + i * 4);
+    writel_relaxed(v->arch.gic_apr, GICH + GICH_APR);
+    writel_relaxed(v->arch.gic_vmcr, GICH + GICH_VMCR);
+    writel_relaxed(GICH_HCR_EN, GICH + GICH_HCR);
     isb();
 
     gic_restore_pending_irqs(v);
@@ -139,8 +142,7 @@ static void gic_irq_enable(struct irq_desc *desc)
     spin_lock_irqsave(&gic.lock, flags);
     desc->status &= ~IRQ_DISABLED;
     dsb(sy);
-    /* Enable routing */
-    GICD[GICD_ISENABLER + irq / 32] = (1u << (irq % 32));
+    writel_relaxed((1u << (irq % 32)), GICD + GICD_ISENABLER + (irq / 32) * 4);
     spin_unlock_irqrestore(&gic.lock, flags);
 }
 
@@ -153,7 +155,7 @@ static void gic_irq_disable(struct irq_desc *desc)
 
     spin_lock_irqsave(&gic.lock, flags);
     /* Disable routing */
-    GICD[GICD_ICENABLER + irq / 32] = (1u << (irq % 32));
+    writel_relaxed(1u << (irq % 32), GICD + GICD_ICENABLER + (irq / 32) * 4);
     desc->status |= IRQ_DISABLED;
     spin_unlock_irqrestore(&gic.lock, flags);
 }
@@ -179,16 +181,16 @@ static void gic_host_irq_end(struct irq_desc *desc)
 {
     int irq = desc->irq;
     /* Lower the priority */
-    GICC[GICC_EOIR] = irq;
+    writel_relaxed(irq, GICC + GICC_EOIR);
     /* Deactivate */
-    GICC[GICC_DIR] = irq;
+    writel_relaxed(irq, GICC + GICC_DIR);
 }
 
 static void gic_guest_irq_end(struct irq_desc *desc)
 {
     int irq = desc->irq;
     /* Lower the priority of the IRQ */
-    GICC[GICC_EOIR] = irq;
+    writel_relaxed(irq, GICC + GICC_EOIR);
     /* Deactivation happens in maintenance interrupt / via GICV */
 }
 
@@ -243,13 +245,13 @@ static void gic_set_irq_properties(struct irq_desc *desc,
     mask = gic_cpu_mask(cpu_mask);
 
     /* Set edge / level */
-    cfg = GICD[GICD_ICFGR + irq / 16];
+    cfg = readl_relaxed(GICD + GICD_ICFGR + (irq / 16) * 4);
     edgebit = 2u << (2 * (irq % 16));
     if ( type & DT_IRQ_TYPE_LEVEL_MASK )
         cfg &= ~edgebit;
     else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
         cfg |= edgebit;
-    GICD[GICD_ICFGR + irq / 16] = cfg;
+    writel_relaxed(cfg, GICD + GICD_ICFGR + (irq / 16) * 4);
 
     /* Set target CPU mask (RAZ/WI on uniprocessor) */
     bytereg = (unsigned char *) (GICD + GICD_ITARGETSR);
@@ -303,87 +305,91 @@ static void __init gic_dist_init(void)
     uint32_t cpumask;
     int i;
 
-    cpumask = GICD[GICD_ITARGETSR] & 0xff;
+    cpumask = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
     cpumask |= cpumask << 8;
     cpumask |= cpumask << 16;
 
     /* Disable the distributor */
-    GICD[GICD_CTLR] = 0;
+    writel_relaxed(0, GICD + GICD_CTLR);
 
-    type = GICD[GICD_TYPER];
+    type = readl_relaxed(GICD + GICD_TYPER);
     gic.lines = 32 * ((type & GICD_TYPE_LINES) + 1);
     gic.cpus = 1 + ((type & GICD_TYPE_CPUS) >> 5);
     printk("GIC: %d lines, %d cpu%s%s (IID %8.8x).\n",
            gic.lines, gic.cpus, (gic.cpus == 1) ? "" : "s",
            (type & GICD_TYPE_SEC) ? ", secure" : "",
-           GICD[GICD_IIDR]);
+           readl_relaxed(GICD + GICD_IIDR));
 
     /* Default all global IRQs to level, active low */
     for ( i = 32; i < gic.lines; i += 16 )
-        GICD[GICD_ICFGR + i / 16] = 0x0;
+        writel_relaxed(0x0, GICD + GICD_ICFGR + (i / 16) * 4);
 
     /* Route all global IRQs to this CPU */
     for ( i = 32; i < gic.lines; i += 4 )
-        GICD[GICD_ITARGETSR + i / 4] = cpumask;
+        writel_relaxed(cpumask, GICD + GICD_ITARGETSR + (i / 4) * 4);
 
     /* Default priority for global interrupts */
     for ( i = 32; i < gic.lines; i += 4 )
-        GICD[GICD_IPRIORITYR + i / 4] =
-            GIC_PRI_IRQ<<24 | GIC_PRI_IRQ<<16 | GIC_PRI_IRQ<<8 | GIC_PRI_IRQ;
+        writel_relaxed (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
+                        GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
+                        GICD + GICD_IPRIORITYR + (i / 4) * 4);
 
     /* Disable all global interrupts */
     for ( i = 32; i < gic.lines; i += 32 )
-        GICD[GICD_ICENABLER + i / 32] = (uint32_t)~0ul;
+        writel_relaxed(~0x0, GICD + GICD_ICENABLER + (i / 32) * 4);
 
     /* Turn on the distributor */
-    GICD[GICD_CTLR] = GICD_CTL_ENABLE;
+    writel_relaxed(GICD_CTL_ENABLE, GICD + GICD_CTLR);
 }
 
 static void __cpuinit gic_cpu_init(void)
 {
     int i;
 
-    this_cpu(gic_cpu_id) = GICD[GICD_ITARGETSR] & 0xff;
+    this_cpu(gic_cpu_id) = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
 
     /* The first 32 interrupts (PPI and SGI) are banked per-cpu, so
      * even though they are controlled with GICD registers, they must
      * be set up here with the other per-cpu state. */
-    GICD[GICD_ICENABLER] = 0xffff0000; /* Disable all PPI */
-    GICD[GICD_ISENABLER] = 0x0000ffff; /* Enable all SGI */
+    writel_relaxed(0xffff0000, GICD + GICD_ICENABLER); /* Disable all PPI */
+    writel_relaxed(0x0000ffff, GICD + GICD_ISENABLER); /* Enable all SGI */
+
     /* Set SGI priorities */
     for (i = 0; i < 16; i += 4)
-        GICD[GICD_IPRIORITYR + i / 4] =
-            GIC_PRI_IPI<<24 | GIC_PRI_IPI<<16 | GIC_PRI_IPI<<8 | GIC_PRI_IPI;
+        writel_relaxed(GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 |
+                       GIC_PRI_IPI << 8 | GIC_PRI_IPI,
+                       GICD + GICD_IPRIORITYR + (i / 4) * 4);
     /* Set PPI priorities */
     for (i = 16; i < 32; i += 4)
-        GICD[GICD_IPRIORITYR + i / 4] =
-            GIC_PRI_IRQ<<24 | GIC_PRI_IRQ<<16 | GIC_PRI_IRQ<<8 | GIC_PRI_IRQ;
+        writel_relaxed(GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
+                      GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
+                      GICD + GICD_IPRIORITYR + (i / 4) * 4);
 
     /* Local settings: interface controller */
-    GICC[GICC_PMR] = 0xff;                /* Don't mask by priority */
-    GICC[GICC_BPR] = 0;                   /* Finest granularity of priority */
-    GICC[GICC_CTLR] = GICC_CTL_ENABLE|GICC_CTL_EOI;    /* Turn on delivery */
+    writel_relaxed(0xff, GICC + GICC_PMR);                /* Don't mask by priority */
+    writel_relaxed(0x0, GICC + GICC_BPR);                   /* Finest granularity of priority */
+    writel_relaxed(GICC_CTL_ENABLE|GICC_CTL_EOI, GICC + GICC_CTLR);    /* Turn on delivery */
 }
 
 static void gic_cpu_disable(void)
 {
-    GICC[GICC_CTLR] = 0;
+    writel_relaxed(0x0, GICC + GICC_CTLR);
 }
 
 static void __cpuinit gic_hyp_init(void)
 {
     uint32_t vtr;
 
-    vtr = GICH[GICH_VTR];
+    vtr = readl_relaxed(GICH + GICH_VTR);
     nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
 
-    GICH[GICH_MISR] = GICH_MISR_EOI;
+    writel_relaxed(GICH_MISR_EOI, GICH + GICH_MISR);
     this_cpu(lr_mask) = 0ULL;
 }
 
 static void __cpuinit gic_hyp_disable(void)
 {
-    GICH[GICH_HCR] = 0;
+    writel_relaxed(0, GICH + GICH_HCR);
 }
 
 int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
@@ -462,15 +468,21 @@ void __init gic_init(void)
          (gic.hbase & ~PAGE_MASK) || (gic.vbase & ~PAGE_MASK) )
         panic("GIC interfaces not page aligned");
 
-    set_fixmap(FIXMAP_GICD, gic.dbase >> PAGE_SHIFT, DEV_SHARED);
-    BUILD_BUG_ON(FIXMAP_ADDR(FIXMAP_GICC1) !=
-                 FIXMAP_ADDR(FIXMAP_GICC2)-PAGE_SIZE);
-    set_fixmap(FIXMAP_GICC1, gic.cbase >> PAGE_SHIFT, DEV_SHARED);
+    gic.map_dbase = ioremap_nocache(gic.dbase, PAGE_SIZE);
+    if ( !gic.map_dbase )
+        panic("Failed to ioremap for GIC distributor\n");
+
     if ( platform_has_quirk(PLATFORM_QUIRK_GIC_64K_STRIDE) )
-        set_fixmap(FIXMAP_GICC2, (gic.cbase >> PAGE_SHIFT) + 0x10, DEV_SHARED);
+        gic.map_cbase = ioremap_nocache(gic.cbase, PAGE_SIZE * 0x10);
     else
-        set_fixmap(FIXMAP_GICC2, (gic.cbase >> PAGE_SHIFT) + 0x1, DEV_SHARED);
-    set_fixmap(FIXMAP_GICH, gic.hbase >> PAGE_SHIFT, DEV_SHARED);
+        gic.map_cbase = ioremap_nocache(gic.cbase, PAGE_SIZE * 2);
+
+    if ( !gic.map_cbase )
+        panic("Failed to ioremap for GIC CPU interface\n");
+
+    gic.map_hbase = ioremap_nocache(gic.hbase, PAGE_SIZE);
+    if ( !gic.map_hbase )
+        panic("Failed to ioremap for GIC Virtual interface\n");
 
     /* Global settings: interrupt distributor */
     spin_lock_init(&gic.lock);
@@ -491,15 +503,16 @@ static void send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
     switch ( irqmode )
     {
     case SGI_TARGET_OTHERS:
-        GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS | sgi;
+        writel_relaxed(GICD_SGI_TARGET_OTHERS | sgi, GICD + GICD_SGIR);
         break;
     case SGI_TARGET_SELF:
-        GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF | sgi;
+        writel_relaxed(GICD_SGI_TARGET_SELF | sgi, GICD + GICD_SGIR);
         break;
     case SGI_TARGET_LIST:
         mask = gic_cpu_mask(cpu_mask);
-        GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST |
-                          (mask<<GICD_SGI_TARGET_SHIFT) | sgi;
+        writel_relaxed(GICD_SGI_TARGET_LIST |
+                       (mask << GICD_SGI_TARGET_SHIFT) | sgi,
+                       GICD + GICD_SGIR);
         break;
     default:
         BUG_ON(1);
@@ -585,7 +598,7 @@ static inline void gic_set_lr(int lr, struct pending_irq *p,
     if ( p->desc != NULL )
         lr_val |= GICH_LR_HW | (p->desc->irq << GICH_LR_PHYSICAL_SHIFT);
 
-    GICH[GICH_LR + lr] = lr_val;
+    writel_relaxed(lr_reg, GICH + GICH_LR + lr * 4);
 
     set_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
     clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
@@ -677,7 +690,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
     ASSERT(spin_is_locked(&v->arch.vgic.lock));
     ASSERT(!local_irq_is_enabled());
 
-    lr = GICH[GICH_LR + i];
+    lr = readl_relaxed(GICH + GICH_LR + i * 4);
     irq = (lr >> GICH_LR_VIRTUAL_SHIFT) & GICH_LR_VIRTUAL_MASK;
     p = irq_to_pending(v, irq);
     if ( lr & GICH_LR_ACTIVE )
@@ -687,7 +700,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
              test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
         {
             if ( p->desc == NULL )
-                GICH[GICH_LR + i] = lr | GICH_LR_PENDING;
+                writel_relaxed(lr | GICH_LR_PENDING, GICH + GICH_LR + i * 4);
             else
                 gdprintk(XENLOG_WARNING, "unable to inject hw irq=%d into d%dv%d: already active in LR%d\n",
                          irq, v->domain->domain_id, v->vcpu_id, i);
@@ -700,7 +713,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
                     irq, v->domain->domain_id, v->vcpu_id, i);
 #endif
     } else {
-        GICH[GICH_LR + i] = 0;
+        writel_relaxed(0, GICH + GICH_LR + i * 4);
         clear_bit(i, &this_cpu(lr_mask));
 
         if ( p->desc != NULL )
@@ -808,7 +821,7 @@ int gic_events_need_delivery(void)
     struct pending_irq *p;
     unsigned long flags;
 
-    mask_priority = (GICH[GICH_VMCR] >> GICH_VMCR_PRIORITY_SHIFT) & GICH_VMCR_PRIORITY_MASK;
+    mask_priority = (readl_relaxed(GICH + GICH_VMCR) >> GICH_VMCR_PRIORITY_SHIFT) & GICH_VMCR_PRIORITY_MASK;
     mask_priority = mask_priority << 3;
 
     spin_lock_irqsave(&v->arch.vgic.lock, flags);
@@ -844,22 +857,21 @@ int gic_events_need_delivery(void)
 
 void gic_inject(void)
 {
+    uint32_t hcr;
     ASSERT(!local_irq_is_enabled());
 
     gic_restore_pending_irqs(current);
 
-
     if ( !list_empty(&current->arch.vgic.lr_pending) && lr_all_full() )
-        GICH[GICH_HCR] |= GICH_HCR_UIE;
+        writel_relaxed(hcr | GICH_HCR_UIE, GICH + GICH_HCR);
     else
-        GICH[GICH_HCR] &= ~GICH_HCR_UIE;
-
+        writel_relaxed(hcr & ~GICH_HCR_UIE, GICH + GICH_HCR);
 }
 
 static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
 {
     /* Lower the priority */
-    GICC[GICC_EOIR] = sgi;
+    writel_relaxed(sgi, GICC + GICC_EOIR);
 
     switch (sgi)
     {
@@ -878,7 +890,7 @@ static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
     }
 
     /* Deactivate */
-    GICC[GICC_DIR] = sgi;
+    writel_relaxed(sgi, GICC + GICC_DIR);
 }
 
 /* Accept an interrupt from the GIC and dispatch its handler */
@@ -889,7 +901,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
 
 
     do  {
-        intack = GICC[GICC_IAR];
+        intack = readl_relaxed(GICC + GICC_IAR);
         irq = intack & GICC_IA_IRQ;
 
         if ( likely(irq >= 16 && irq < 1021) )
@@ -976,7 +988,7 @@ void gic_dump_info(struct vcpu *v)
     if ( v == current )
     {
         for ( i = 0; i < nr_lrs; i++ )
-            printk("   HW_LR[%d]=%x\n", i, GICH[GICH_LR + i]);
+            printk("   HW_LR[%d]=%x\n", i, readl_relaxed(GICH + GICH_LR + i * 4));
     } else {
         for ( i = 0; i < nr_lrs; i++ )
             printk("   VCPU_LR[%d]=%x\n", i, v->arch.gic_lr[i]);
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 151ec3e..b56f9d1 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -30,7 +30,7 @@
 #include <asm/mmio.h>
 #include <asm/gic.h>
 
-#define REG(n) (n/4)
+#define REG(n) (n)
 
 /* Number of ranks of interrupt registers for a domain */
 #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
@@ -55,7 +55,7 @@ static inline int REG_RANK_NR(int b, uint32_t n)
  * Offset of GICD_<FOO><n> with its rank, for GICD_<FOO> with
  * <b>-bits-per-interrupt.
  */
-#define REG_RANK_INDEX(b, n) ((n) & ((b)-1))
+#define REG_RANK_INDEX(b, n) (((n) >> 2) & ((b)-1))
 
 /*
  * Returns rank corresponding to a GICD_<FOO><n> register for
@@ -63,7 +63,9 @@ static inline int REG_RANK_NR(int b, uint32_t n)
  */
 static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
 {
-    int rank = REG_RANK_NR(b, n);
+    int rank;
+    n = n >> 2;
+    rank = REG_RANK_NR(b, n);
 
     if ( rank == 0 )
         return &v->arch.vgic.private_irqs;
@@ -492,7 +494,7 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         tr = rank->ienable;
         rank->ienable |= *r;
         vgic_unlock_rank(v, rank);
-        vgic_enable_irqs(v, (*r) & (~tr), gicd_reg - GICD_ISENABLER);
+        vgic_enable_irqs(v, (*r) & (~tr), (gicd_reg - GICD_ISENABLER) >> 2);
         return 1;
 
     case GICD_ICENABLER ... GICD_ICENABLERN:
@@ -503,7 +505,7 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         tr = rank->ienable;
         rank->ienable &= ~*r;
         vgic_unlock_rank(v, rank);
-        vgic_disable_irqs(v, (*r) & tr, gicd_reg - GICD_ICENABLER);
+        vgic_disable_irqs(v, (*r) & tr, (gicd_reg - GICD_ICENABLER) >> 2);
         return 1;
 
     case GICD_ISPENDR ... GICD_ISPENDRN:
@@ -670,9 +672,8 @@ void vgic_clear_pending_irqs(struct vcpu *v)
 
 void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
 {
-    int idx = irq >> 2, byte = irq & 0x3;
     uint8_t priority;
-    struct vgic_irq_rank *rank = vgic_irq_rank(v, 8, idx);
+    struct vgic_irq_rank *rank = vgic_irq_rank(v, 8, irq);
     struct pending_irq *iter, *n = irq_to_pending(v, irq);
     unsigned long flags;
     bool_t running;
@@ -693,7 +694,7 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
         return;
     }
 
-    priority = byte_read(rank->ipriority[REG_RANK_INDEX(8, idx)], 0, byte);
+    priority = byte_read(rank->ipriority[REG_RANK_INDEX(8, irq)], 0, irq & 0x3);
 
     n->irq = irq;
     set_bit(GIC_IRQ_GUEST_QUEUED, &n->status);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index c7b7368..7fa3b95 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -18,37 +18,37 @@
 #ifndef __ASM_ARM_GIC_H__
 #define __ASM_ARM_GIC_H__
 
-#define GICD_CTLR       (0x000/4)
-#define GICD_TYPER      (0x004/4)
-#define GICD_IIDR       (0x008/4)
-#define GICD_IGROUPR    (0x080/4)
-#define GICD_IGROUPRN   (0x0FC/4)
-#define GICD_ISENABLER  (0x100/4)
-#define GICD_ISENABLERN (0x17C/4)
-#define GICD_ICENABLER  (0x180/4)
-#define GICD_ICENABLERN (0x1fC/4)
-#define GICD_ISPENDR    (0x200/4)
-#define GICD_ISPENDRN   (0x27C/4)
-#define GICD_ICPENDR    (0x280/4)
-#define GICD_ICPENDRN   (0x2FC/4)
-#define GICD_ISACTIVER  (0x300/4)
-#define GICD_ISACTIVERN (0x37C/4)
-#define GICD_ICACTIVER  (0x380/4)
-#define GICD_ICACTIVERN (0x3FC/4)
-#define GICD_IPRIORITYR (0x400/4)
-#define GICD_IPRIORITYRN (0x7F8/4)
-#define GICD_ITARGETSR  (0x800/4)
-#define GICD_ITARGETSRN (0xBF8/4)
-#define GICD_ICFGR      (0xC00/4)
-#define GICD_ICFGRN     (0xCFC/4)
-#define GICD_NSACR      (0xE00/4)
-#define GICD_NSACRN     (0xEFC/4)
-#define GICD_SGIR       (0xF00/4)
-#define GICD_CPENDSGIR  (0xF10/4)
-#define GICD_CPENDSGIRN (0xF1C/4)
-#define GICD_SPENDSGIR  (0xF20/4)
-#define GICD_SPENDSGIRN (0xF2C/4)
-#define GICD_ICPIDR2    (0xFE8/4)
+#define GICD_CTLR       (0x000)
+#define GICD_TYPER      (0x004)
+#define GICD_IIDR       (0x008)
+#define GICD_IGROUPR    (0x080)
+#define GICD_IGROUPRN   (0x0FC)
+#define GICD_ISENABLER  (0x100)
+#define GICD_ISENABLERN (0x17C)
+#define GICD_ICENABLER  (0x180)
+#define GICD_ICENABLERN (0x1fC)
+#define GICD_ISPENDR    (0x200)
+#define GICD_ISPENDRN   (0x27C)
+#define GICD_ICPENDR    (0x280)
+#define GICD_ICPENDRN   (0x2FC)
+#define GICD_ISACTIVER  (0x300)
+#define GICD_ISACTIVERN (0x37C)
+#define GICD_ICACTIVER  (0x380)
+#define GICD_ICACTIVERN (0x3FC)
+#define GICD_IPRIORITYR (0x400)
+#define GICD_IPRIORITYRN (0x7F8)
+#define GICD_ITARGETSR  (0x800)
+#define GICD_ITARGETSRN (0xBF8)
+#define GICD_ICFGR      (0xC00)
+#define GICD_ICFGRN     (0xCFC)
+#define GICD_NSACR      (0xE00)
+#define GICD_NSACRN     (0xEFC)
+#define GICD_SGIR       (0xF00)
+#define GICD_CPENDSGIR  (0xF10)
+#define GICD_CPENDSGIRN (0xF1C)
+#define GICD_SPENDSGIR  (0xF20)
+#define GICD_SPENDSGIRN (0xF2C)
+#define GICD_ICPIDR2    (0xFE8)
 
 #define GICD_SGI_TARGET_LIST_SHIFT   (24)
 #define GICD_SGI_TARGET_LIST_MASK    (0x3UL << GICD_SGI_TARGET_LIST_SHIFT)
@@ -60,27 +60,27 @@
 #define GICD_SGI_GROUP1              (1UL<<15)
 #define GICD_SGI_INTID_MASK          (0xFUL)
 
-#define GICC_CTLR       (0x0000/4)
-#define GICC_PMR        (0x0004/4)
-#define GICC_BPR        (0x0008/4)
-#define GICC_IAR        (0x000C/4)
-#define GICC_EOIR       (0x0010/4)
-#define GICC_RPR        (0x0014/4)
-#define GICC_HPPIR      (0x0018/4)
-#define GICC_APR        (0x00D0/4)
-#define GICC_NSAPR      (0x00E0/4)
-#define GICC_DIR        (0x1000/4)
-
-#define GICH_HCR        (0x00/4)
-#define GICH_VTR        (0x04/4)
-#define GICH_VMCR       (0x08/4)
-#define GICH_MISR       (0x10/4)
-#define GICH_EISR0      (0x20/4)
-#define GICH_EISR1      (0x24/4)
-#define GICH_ELSR0      (0x30/4)
-#define GICH_ELSR1      (0x34/4)
-#define GICH_APR        (0xF0/4)
-#define GICH_LR         (0x100/4)
+#define GICC_CTLR       (0x0000)
+#define GICC_PMR        (0x0004)
+#define GICC_BPR        (0x0008)
+#define GICC_IAR        (0x000C)
+#define GICC_EOIR       (0x0010)
+#define GICC_RPR        (0x0014)
+#define GICC_HPPIR      (0x0018)
+#define GICC_APR        (0x00D0)
+#define GICC_NSAPR      (0x00E0)
+#define GICC_DIR        (0x1000)
+
+#define GICH_HCR        (0x00)
+#define GICH_VTR        (0x04)
+#define GICH_VMCR       (0x08)
+#define GICH_MISR       (0x10)
+#define GICH_EISR0      (0x20)
+#define GICH_EISR1      (0x24)
+#define GICH_ELSR0      (0x30)
+#define GICH_ELSR1      (0x34)
+#define GICH_APR        (0xF0)
+#define GICH_LR         (0x100)
 
 /* Register bits */
 #define GICD_CTL_ENABLE 0x1
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (4 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 14:09   ` Julien Grall
                     ` (2 more replies)
  2014-05-26 10:26 ` [PATCH v4 07/16] arm/xen: move GIC context data structure to gic driver vijay.kilari
                   ` (10 subsequent siblings)
  16 siblings, 3 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

GIC driver contains both generic and hardware specific low
level functionality in gic.c file.

With this patch, low level functionality is moved to separate
file gic-v2.c and generic code is kept in gic.c file

Callbacks are registered by low level driver with generic driver
and are called whereever required.

The locking mechanism is not changed.

This helps to separate generic and hardware functionality
and implement future hardware version drivers.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/Makefile     |    2 +-
 xen/arch/arm/gic-v2.c     |  679 +++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/gic.c        |  534 ++++-------------------------------
 xen/include/asm-arm/gic.h |  112 ++++++--
 4 files changed, 832 insertions(+), 495 deletions(-)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 63e0460..969ee52 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -10,7 +10,7 @@ obj-y += vpsci.o
 obj-y += domctl.o
 obj-y += sysctl.o
 obj-y += domain_build.o
-obj-y += gic.o
+obj-y += gic.o gic-v2.o
 obj-y += io.o
 obj-y += irq.o
 obj-y += kernel.o
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
new file mode 100644
index 0000000..0628b5d
--- /dev/null
+++ b/xen/arch/arm/gic-v2.c
@@ -0,0 +1,679 @@
+/*
+ * xen/arch/arm/gic-v2.c
+ *
+ * ARM Generic Interrupt Controller support v2
+ *
+ * Tim Deegan <tim@xen.org>
+ * Copyright (c) 2011 Citrix Systems.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/config.h>
+#include <xen/lib.h>
+#include <xen/init.h>
+#include <xen/mm.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/errno.h>
+#include <xen/softirq.h>
+#include <xen/list.h>
+#include <xen/device_tree.h>
+#include <asm/p2m.h>
+#include <asm/domain.h>
+#include <asm/platform.h>
+
+#include <asm/io.h>
+#include <asm/gic.h>
+
+/*
+ * LR register definitions are GIC v2 specific.
+ * Moved these definitions from header file to here
+ */
+#define GICH_V2_LR_VIRTUAL_MASK    0x3ff
+#define GICH_V2_LR_VIRTUAL_SHIFT   0
+#define GICH_V2_LR_PHYSICAL_MASK   0x3ff
+#define GICH_V2_LR_PHYSICAL_SHIFT  10
+#define GICH_V2_LR_STATE_MASK      0x3
+#define GICH_V2_LR_STATE_SHIFT     28
+#define GICH_V2_LR_PRIORITY_SHIFT  23
+#define GICH_V2_LR_PRIORITY_MASK   0x1f
+#define GICH_V2_LR_HW_SHIFT        31
+#define GICH_V2_LR_HW_MASK         0x1
+#define GICH_V2_LR_GRP_SHIFT       30
+#define GICH_V2_LR_GRP_MASK        0x1
+#define GICH_V2_LR_MAINTENANCE_IRQ (1<<19)
+#define GICH_V2_LR_GRP1            (1<<30)
+#define GICH_V2_LR_HW              (1<<31)
+#define GICH_V2_LR_CPUID_SHIFT     9
+#define GICH_V2_VTR_NRLRGS         0x3f
+
+#define GICH_V2_VMCR_PRIORITY_MASK   0x1f
+#define GICH_V2_VMCR_PRIORITY_SHIFT  27
+
+#define GICD (gicv2.map_dbase)
+#define GICC (gicv2.map_cbase)
+#define GICH (gicv2.map_hbase)
+
+/* Global state */
+static struct {
+    paddr_t dbase;            /* Address of distributor registers */
+    void __iomem * map_dbase; /* IO mapped Address of distributor registers */
+    paddr_t cbase;            /* Address of CPU interface registers */
+    void __iomem * map_cbase; /* IO mapped Address of CPU interface registers*/
+    paddr_t hbase;            /* Address of virtual interface registers */
+    void __iomem * map_hbase; /* IO Address of virtual interface registers */
+    paddr_t vbase;            /* Address of virtual cpu interface registers */
+    spinlock_t lock;
+} gicv2;
+
+static struct gic_info gicv2_info;
+
+/* The GIC mapping of CPU interfaces does not necessarily match the
+ * logical CPU numbering. Let's use mapping as returned by the GIC
+ * itself
+ */
+static DEFINE_PER_CPU(u8, gic_cpu_id);
+
+/* Maximum cpu interface per GIC */
+#define NR_GIC_CPU_IF 8
+
+static unsigned int gicv2_cpu_mask(const cpumask_t *cpumask)
+{
+    unsigned int cpu;
+    unsigned int mask = 0;
+    cpumask_t possible_mask;
+
+    cpumask_and(&possible_mask, cpumask, &cpu_possible_map);
+    for_each_cpu( cpu, &possible_mask )
+    {
+        ASSERT(cpu < NR_GIC_CPU_IF);
+        mask |= per_cpu(gic_cpu_id, cpu);
+    }
+
+    return mask;
+}
+
+static void gicv2_save_state(struct vcpu *v)
+{
+    int i;
+
+    /* No need for spinlocks here because interrupts are disabled around
+     * this call and it only accesses struct vcpu fields that cannot be
+     * accessed simultaneously by another pCPU.
+     */
+    for ( i = 0; i < gicv2_info.nr_lrs; i++ )
+        v->arch.gic_lr[i] = readl_relaxed(GICH + GICH_LR + i * 4);
+
+    v->arch.gic_apr = readl_relaxed(GICH + GICH_APR);
+    v->arch.gic_vmcr = readl_relaxed(GICH + GICH_VMCR);
+    /* Disable until next VCPU scheduled */
+    writel_relaxed(0, GICH + GICH_HCR);
+}
+
+static void gicv2_restore_state(const struct vcpu *v)
+{
+    int i;
+
+    for ( i = 0; i < gicv2_info.nr_lrs; i++ )
+        writel_relaxed(v->arch.gic_lr[i], GICH + GICH_LR + i * 4);
+
+    writel_relaxed(v->arch.gic_apr, GICH + GICH_APR);
+    writel_relaxed(v->arch.gic_vmcr, GICH + GICH_VMCR);
+    writel_relaxed(GICH_HCR_EN, GICH + GICH_HCR);
+}
+
+static void gicv2_dump_state(const struct vcpu *v)
+{
+    int i;
+
+    if ( v == current )
+    {
+        for ( i = 0; i < gicv2_info.nr_lrs; i++ )
+            printk("   HW_LR[%d]=%x\n", i,
+                   readl_relaxed(GICH + GICH_LR + i * 4));
+    }
+    else
+    {
+        for ( i = 0; i < gicv2_info.nr_lrs; i++ )
+            printk("   VCPU_LR[%d]=%x\n", i, v->arch.gic_lr[i]);
+    }
+}
+
+static void gicv2_enable_irq(struct irq_desc *irqd)
+{
+    int irq = irqd->irq;
+    /* Enable routing */
+    writel_relaxed((1u << (irq % 32)), GICD + GICD_ISENABLER + (irq / 32) * 4);
+}
+
+static void gicv2_disable_irq(struct irq_desc *irqd)
+{
+    int irq = irqd->irq;
+    /* Disable routing */
+    writel_relaxed(1u << (irq % 32), GICD + GICD_ICENABLER + (irq / 32) * 4);
+}
+
+static void gicv2_eoi_irq(struct irq_desc *irqd)
+{
+    int irq = irqd->irq;
+    /* Lower the priority */
+    writel_relaxed(irq, GICC + GICC_EOIR);
+}
+
+static void gicv2_dir_irq(int irq)
+{
+    /* Deactivate */
+    writel_relaxed(irq, GICC + GICC_DIR);
+}
+
+static unsigned int gicv2_read_irq(void)
+{
+    return (readl_relaxed(GICC + GICC_IAR) & GICC_IA_IRQ);
+}
+
+/*
+ * needs to be called with a valid cpu_mask, ie each cpu in the mask has
+ * already called gic_cpu_init
+ */
+static void gicv2_set_irq_properties(struct irq_desc *desc,
+                                   const cpumask_t *cpu_mask,
+                                   unsigned int priority)
+{
+    volatile unsigned char *bytereg;
+    uint32_t cfg, edgebit;
+    unsigned int mask = gicv2_cpu_mask(cpu_mask);
+    unsigned int irq = desc->irq;
+    unsigned int type = desc->arch.type;
+
+    spin_lock(&gicv2.lock);
+    /* Set edge / level */
+    cfg = readl_relaxed(GICD + GICD_ICFGR + (irq / 16) * 4);
+    edgebit = 2u << (2 * (irq % 16));
+    if ( type & DT_IRQ_TYPE_LEVEL_MASK )
+        cfg &= ~edgebit;
+    else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
+        cfg |= edgebit;
+    writel_relaxed(cfg, GICD + GICD_ICFGR + (irq / 16) * 4);
+
+    /* Set target CPU mask (RAZ/WI on uniprocessor) */
+    bytereg = (unsigned char *) (GICD + GICD_ITARGETSR);
+    bytereg[irq] = mask;
+
+    /* Set priority */
+    bytereg = (unsigned char *) (GICD + GICD_IPRIORITYR);
+    bytereg[irq] = priority;
+
+    spin_unlock(&gicv2.lock);
+}
+
+static void __init gicv2_dist_init(void)
+{
+    uint32_t type;
+    uint32_t cpumask;
+    uint32_t gic_cpus;
+    int i;
+
+    cpumask = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
+    cpumask |= cpumask << 8;
+    cpumask |= cpumask << 16;
+
+    /* Disable the distributor */
+    writel_relaxed(0, GICD + GICD_CTLR);
+
+    type = readl_relaxed(GICD + GICD_TYPER);
+    gicv2_info.nr_lines = 32 * ((type & GICD_TYPE_LINES) + 1);
+    gic_cpus = 1 + ((type & GICD_TYPE_CPUS) >> 5);
+    printk("GICv2: %d lines, %d cpu%s%s (IID %8.8x).\n",
+           gicv2_info.nr_lines, gic_cpus, (gic_cpus == 1) ? "" : "s",
+           (type & GICD_TYPE_SEC) ? ", secure" : "",
+           readl_relaxed(GICD + GICD_IIDR));
+
+    /* Default all global IRQs to level, active low */
+    for ( i = 32; i < gicv2_info.nr_lines; i += 16 )
+        writel_relaxed(0x0, GICD + GICD_ICFGR + (i / 16) * 4);
+
+    /* Route all global IRQs to this CPU */
+    for ( i = 32; i < gicv2_info.nr_lines; i += 4 )
+        writel_relaxed(cpumask, GICD + GICD_ITARGETSR + (i / 4) * 4);
+
+    /* Default priority for global interrupts */
+    for ( i = 32; i < gicv2_info.nr_lines; i += 4 )
+        writel_relaxed (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
+                        GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
+                        GICD + GICD_IPRIORITYR + (i / 4) * 4);
+
+    /* Disable all global interrupts */
+    for ( i = 32; i < gicv2_info.nr_lines; i += 32 )
+        writel_relaxed(~0x0, GICD + GICD_ICENABLER + (i / 32) * 4);
+
+    /* Turn on the distributor */
+    writel_relaxed(GICD_CTL_ENABLE, GICD + GICD_CTLR);
+}
+
+static void __cpuinit gicv2_cpu_init(void)
+{
+    int i;
+
+    this_cpu(gic_cpu_id) = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
+
+    /* The first 32 interrupts (PPI and SGI) are banked per-cpu, so
+     * even though they are controlled with GICD registers, they must
+     * be set up here with the other per-cpu state. */
+    writel_relaxed(0xffff0000, GICD + GICD_ICENABLER); /* Disable all PPI */
+    writel_relaxed(0x0000ffff, GICD + GICD_ISENABLER); /* Enable all SGI */
+
+    /* Set SGI priorities */
+    for ( i = 0; i < 16; i += 4 )
+        writel_relaxed(GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 |
+                       GIC_PRI_IPI << 8 | GIC_PRI_IPI,
+                       GICD + GICD_IPRIORITYR + (i / 4) * 4);
+
+    /* Set PPI priorities */
+    for ( i = 16; i < 32; i += 4 )
+        writel_relaxed(GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
+                      GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
+                      GICD + GICD_IPRIORITYR + (i / 4) * 4);
+
+    /* Local settings: interface controller */
+    writel_relaxed(0xff, GICC + GICC_PMR);  /* Don't mask by priority */
+    writel_relaxed(0x0, GICC + GICC_BPR);   /* Finest granularity of priority */
+    writel_relaxed(GICC_CTL_ENABLE|GICC_CTL_EOI, GICC + GICC_CTLR);
+}
+
+static void gicv2_cpu_disable(void)
+{
+    writel_relaxed(0x0, GICC + GICC_CTLR);
+}
+
+static void __cpuinit gicv2_hyp_init(void)
+{
+    uint32_t vtr, nr_lrs;
+
+    vtr = readl_relaxed(GICH + GICH_VTR);
+    nr_lrs  = (vtr & GICH_V2_VTR_NRLRGS) + 1;
+    gicv2_info.nr_lrs = nr_lrs;
+
+    writel_relaxed(GICH_MISR_EOI, GICH + GICH_MISR);
+    update_cpu_lr_mask();
+}
+
+static void __cpuinit gicv2_hyp_disable(void)
+{
+    writel_relaxed(0, GICH + GICH_HCR);
+}
+
+static int gicv2_secondary_cpu_init(void)
+{
+    spin_lock(&gicv2.lock);
+
+    gicv2_cpu_init();
+    gicv2_hyp_init();
+
+    spin_unlock(&gicv2.lock);
+
+    return 0;
+}
+
+static void gicv2_send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
+                           uint8_t irqmode)
+{
+    unsigned int mask = 0;
+
+    switch ( irqmode )
+    {
+    case SGI_TARGET_OTHERS:
+        writel_relaxed(GICD_SGI_TARGET_OTHERS | sgi, GICD + GICD_SGIR);
+        break;
+    case SGI_TARGET_SELF:
+        writel_relaxed(GICD_SGI_TARGET_SELF | sgi, GICD + GICD_SGIR);
+        break;
+    case SGI_TARGET_LIST:
+        mask = gicv2_cpu_mask(cpu_mask);
+        writel_relaxed(GICD_SGI_TARGET_LIST |
+                       (mask << GICD_SGI_TARGET_SHIFT) | sgi,
+                       GICD + GICD_SGIR);
+        break;
+    default:
+        BUG_ON(1);
+    }
+}
+
+/* Shut down the per-CPU GIC interface */
+static void gicv2_disable_interface(void)
+{
+    spin_lock(&gicv2.lock);
+    gicv2_cpu_disable();
+    gicv2_hyp_disable();
+    spin_unlock(&gicv2.lock);
+}
+
+static void gicv2_update_lr(int lr, const struct pending_irq *p,
+                            unsigned int state)
+{
+    uint32_t lr_reg;
+
+    BUG_ON(lr >= gicv2_info.nr_lrs);
+    BUG_ON(lr < 0);
+    BUG_ON(state & ~(GICH_V2_LR_STATE_MASK << GICH_V2_LR_STATE_SHIFT));
+
+    lr_reg = state | (GIC_PRI_TO_GUEST(p->priority) << GICH_V2_LR_PRIORITY_SHIFT) |
+             ((p->irq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT);
+
+    if ( p->desc != NULL )
+        lr_reg |= GICH_V2_LR_HW | (p->desc->irq << GICH_V2_LR_PHYSICAL_SHIFT);
+
+    writel_relaxed(lr_reg, GICH + GICH_LR + lr * 4);
+}
+
+static void gicv2_clear_lr(int lr)
+{
+    writel_relaxed(0, GICH + GICH_LR + lr * 4);
+}
+
+static int gicv_v2_init(struct domain *d)
+{
+    int ret;
+
+    /*
+     * Domain 0 gets the hardware address.
+     * Guests get the virtual platform layout.
+     */
+    if ( d->domain_id == 0 )
+    {
+        d->arch.vgic.dbase = gicv2.dbase;
+        d->arch.vgic.cbase = gicv2.cbase;
+    }
+    else
+    {
+        d->arch.vgic.dbase = GUEST_GICD_BASE;
+        d->arch.vgic.cbase = GUEST_GICC_BASE;
+    }
+
+    d->arch.vgic.nr_lines = 0;
+
+    /*
+     * Map the gic virtual cpu interface in the gic cpu interface
+     * region of the guest.
+     *
+     * The second page is always mapped at +4K irrespective of the
+     * GIC_64K_STRIDE quirk. The DTB passed to the guest reflects this.
+     */
+    ret = map_mmio_regions(d, d->arch.vgic.cbase,
+                           d->arch.vgic.cbase + PAGE_SIZE - 1,
+                           gicv2.vbase);
+    if ( ret )
+        return ret;
+
+    if ( !platform_has_quirk(PLATFORM_QUIRK_GIC_64K_STRIDE) )
+        ret = map_mmio_regions(d, d->arch.vgic.cbase + PAGE_SIZE,
+                               d->arch.vgic.cbase + (2 * PAGE_SIZE) - 1,
+                               gicv2.vbase + PAGE_SIZE);
+    else
+        ret = map_mmio_regions(d, d->arch.vgic.cbase + PAGE_SIZE,
+                               d->arch.vgic.cbase + (2 * PAGE_SIZE) - 1,
+                               gicv2.vbase + 16*PAGE_SIZE);
+
+    return ret;
+}
+
+static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
+{
+    uint32_t lrv;
+
+    lrv          = readl_relaxed(GICH + GICH_LR + lr * 4);
+    lr_reg->pirq = (lrv >> GICH_V2_LR_PHYSICAL_SHIFT) & GICH_V2_LR_PHYSICAL_MASK;
+    lr_reg->virq = (lrv >> GICH_V2_LR_VIRTUAL_SHIFT) & GICH_V2_LR_VIRTUAL_MASK;
+    lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
+    lr_reg->state     = (lrv >> GICH_V2_LR_STATE_SHIFT) & GICH_V2_LR_STATE_MASK;
+    lr_reg->hw_status = (lrv >> GICH_V2_LR_HW_SHIFT) & GICH_V2_LR_HW_MASK;
+    lr_reg->grp       = (lrv >> GICH_V2_LR_GRP_SHIFT) & GICH_V2_LR_GRP_MASK;
+}
+
+static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
+{
+    uint32_t lrv = 0;
+
+    lrv = ( ((lr_reg->pirq & GICH_V2_LR_PHYSICAL_MASK) << GICH_V2_LR_PHYSICAL_SHIFT) |
+          ((lr_reg->virq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT)   |
+          ((uint32_t)(lr_reg->priority & GICH_V2_LR_PRIORITY_MASK)
+                                      << GICH_V2_LR_PRIORITY_SHIFT) |
+          ((uint32_t)(lr_reg->state & GICH_V2_LR_STATE_MASK)
+                                   << GICH_V2_LR_STATE_SHIFT) |
+          ((uint32_t)(lr_reg->hw_status & GICH_V2_LR_HW_MASK)
+                                       << GICH_V2_LR_HW_SHIFT)  |
+          ((uint32_t)(lr_reg->grp & GICH_V2_LR_GRP_MASK) << GICH_V2_LR_GRP_SHIFT) );
+
+    writel_relaxed(lrv, GICH + GICH_LR + lr * 4);
+}
+
+static void gicv2_hcr_status(uint32_t flag, bool_t status)
+{
+    uint32_t hcr = readl_relaxed(GICH + GICH_HCR);
+
+    if ( status )
+        hcr |= flag;
+    else
+        hcr &= (~flag);
+
+    writel_relaxed(hcr, GICH + GICH_HCR);
+}
+
+static unsigned int gicv2_read_vmcr_priority(void)
+{
+   return ((readl_relaxed(GICH + GICH_VMCR) >> GICH_V2_VMCR_PRIORITY_SHIFT)
+           & GICH_V2_VMCR_PRIORITY_MASK);
+}
+
+static void gicv2_irq_enable(struct irq_desc *desc)
+{
+    unsigned long flags;
+
+    spin_lock_irqsave(&desc->lock, flags);
+    spin_lock(&gicv2.lock);
+    desc->status &= ~IRQ_DISABLED;
+    dsb(sy);
+    /* Enable routing */
+    gicv2_enable_irq(desc);
+    spin_unlock(&gicv2.lock);
+    spin_unlock_irqrestore(&desc->lock, flags);
+}
+
+static void gicv2_irq_disable(struct irq_desc *desc)
+{
+    unsigned long flags;
+
+    spin_lock_irqsave(&desc->lock, flags);
+    spin_lock(&gicv2.lock);
+    /* Disable routing */
+    gicv2_disable_irq(desc);
+    desc->status |= IRQ_DISABLED;
+    spin_unlock(&gicv2.lock);
+    spin_unlock_irqrestore(&desc->lock, flags);
+}
+
+static unsigned int gicv2_irq_startup(struct irq_desc *desc)
+{
+    gicv2_irq_enable(desc);
+
+    return 0;
+}
+
+static void gicv2_irq_shutdown(struct irq_desc *desc)
+{
+    gicv2_irq_disable(desc);
+}
+
+static void gicv2_irq_ack(struct irq_desc *desc)
+{
+    /* No ACK -- reading IAR has done this for us */
+}
+
+static void gicv2_host_irq_end(struct irq_desc *desc)
+{
+    /* Lower the priority */
+    gicv2_eoi_irq(desc);
+    /* Deactivate */
+    gicv2_dir_irq(desc->irq);
+}
+
+static void gicv2_guest_irq_end(struct irq_desc *desc)
+{
+    /* Lower the priority of the IRQ */
+    gicv2_eoi_irq(desc);
+    /* Deactivation happens in maintenance interrupt / via GICV */
+}
+
+static void gicv2_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+{
+    BUG();
+}
+
+/* XXX different for level vs edge */
+static hw_irq_controller gicv2_host_irq_type = {
+    .typename     = "gic-v2",
+    .startup      = gicv2_irq_startup,
+    .shutdown     = gicv2_irq_shutdown,
+    .enable       = gicv2_irq_enable,
+    .disable      = gicv2_irq_disable,
+    .ack          = gicv2_irq_ack,
+    .end          = gicv2_host_irq_end,
+    .set_affinity = gicv2_irq_set_affinity,
+};
+
+static hw_irq_controller gicv2_guest_irq_type = {
+    .typename     = "gic-v2",
+    .startup      = gicv2_irq_startup,
+    .shutdown     = gicv2_irq_shutdown,
+    .enable       = gicv2_irq_enable,
+    .disable      = gicv2_irq_disable,
+    .ack          = gicv2_irq_ack,
+    .end          = gicv2_guest_irq_end,
+    .set_affinity = gicv2_irq_set_affinity,
+};
+
+const static struct gic_hw_operations gicv2_ops = {
+    .info                = &gicv2_info,
+    .secondary_init      = gicv2_secondary_cpu_init,
+    .save_state          = gicv2_save_state,
+    .restore_state       = gicv2_restore_state,
+    .dump_state          = gicv2_dump_state,
+    .gicv_setup          = gicv_v2_init,
+    .gic_host_irq_type   = &gicv2_host_irq_type,
+    .gic_guest_irq_type  = &gicv2_guest_irq_type,
+    .eoi_irq             = gicv2_eoi_irq,
+    .deactivate_irq      = gicv2_dir_irq,
+    .read_irq            = gicv2_read_irq,
+    .set_irq_properties  = gicv2_set_irq_properties,
+    .send_SGI            = gicv2_send_SGI,
+    .disable_interface   = gicv2_disable_interface,
+    .update_lr           = gicv2_update_lr,
+    .update_hcr_status   = gicv2_hcr_status,
+    .clear_lr            = gicv2_clear_lr,
+    .read_lr             = gicv2_read_lr,
+    .write_lr            = gicv2_write_lr,
+    .read_vmcr_priority  = gicv2_read_vmcr_priority,
+};
+
+/* Set up the GIC */
+void __init gicv2_init(void)
+{
+    static const struct dt_device_match gic_ids[] __initconst =
+    {
+        DT_MATCH_GIC,
+        { /* sentinel */ },
+    };
+    struct dt_device_node *node;
+    int res;
+
+    node = dt_find_interrupt_controller(gic_ids);
+    if ( !node )
+        panic("GICv2: Unable to find compatible GIC in the device tree");
+
+    dt_device_set_used_by(node, DOMID_XEN);
+
+    res = dt_device_get_address(node, 0, &gicv2.dbase, NULL);
+    if ( res || !gicv2.dbase || (gicv2.dbase & ~PAGE_MASK) )
+        panic("GICv2: Cannot find a valid address for the distributor");
+
+    res = dt_device_get_address(node, 1, &gicv2.cbase, NULL);
+    if ( res || !gicv2.cbase || (gicv2.cbase & ~PAGE_MASK) )
+        panic("GICv2: Cannot find a valid address for the CPU");
+
+    res = dt_device_get_address(node, 2, &gicv2.hbase, NULL);
+    if ( res || !gicv2.hbase || (gicv2.hbase & ~PAGE_MASK) )
+        panic("GICv2: Cannot find a valid address for the hypervisor");
+
+    res = dt_device_get_address(node, 3, &gicv2.vbase, NULL);
+    if ( res || !gicv2.vbase || (gicv2.vbase & ~PAGE_MASK) )
+        panic("GICv2: Cannot find a valid address for the virtual CPU");
+
+    res = platform_get_irq(node, 0);
+    if ( res < 0 )
+        panic("GICv2: Cannot find the maintenance IRQ");
+    gicv2_info.maintenance_irq = res;
+
+    /* Set the GIC as the primary interrupt controller */
+    dt_interrupt_controller = node;
+
+    /* TODO: Add check on distributor, cpu size */
+
+    printk("GICv2 initialization:\n"
+              "        gic_dist_addr=%"PRIpaddr"\n"
+              "        gic_cpu_addr=%"PRIpaddr"\n"
+              "        gic_hyp_addr=%"PRIpaddr"\n"
+              "        gic_vcpu_addr=%"PRIpaddr"\n"
+              "        gic_maintenance_irq=%u\n",
+              gicv2.dbase, gicv2.cbase, gicv2.hbase, gicv2.vbase,
+              gicv2_info.maintenance_irq);
+
+    if ( (gicv2.dbase & ~PAGE_MASK) || (gicv2.cbase & ~PAGE_MASK) ||
+         (gicv2.hbase & ~PAGE_MASK) || (gicv2.vbase & ~PAGE_MASK) )
+        panic("GICv2 interfaces not page aligned");
+
+    gicv2.map_dbase = ioremap_nocache(gicv2.dbase, PAGE_SIZE);
+    if ( !gicv2.map_dbase )
+        panic("GICv2: Failed to ioremap for GIC distributor\n");
+
+    if ( platform_has_quirk(PLATFORM_QUIRK_GIC_64K_STRIDE) )
+        gicv2.map_cbase = ioremap_nocache(gicv2.cbase, PAGE_SIZE * 0x10);
+    else
+        gicv2.map_cbase = ioremap_nocache(gicv2.cbase, PAGE_SIZE * 2);
+
+    if ( !gicv2.map_cbase )
+        panic("GICv2: Failed to ioremap for GIC CPU interface\n");
+
+    gicv2.map_hbase = ioremap_nocache(gicv2.hbase, PAGE_SIZE);
+    if ( !gicv2.map_hbase )
+        panic("GICv2: Failed to ioremap for GIC Virtual interface\n");
+
+    /* Global settings: interrupt distributor */
+    spin_lock_init(&gicv2.lock);
+    spin_lock(&gicv2.lock);
+
+    gicv2_dist_init();
+    gicv2_cpu_init();
+    gicv2_hyp_init();
+
+    spin_unlock(&gicv2.lock);
+
+    gicv2_info.hw_version = GIC_V2;
+    register_gic_ops(&gicv2_ops);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 470e6c0..975b10c 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -24,7 +24,6 @@
 #include <xen/irq.h>
 #include <xen/sched.h>
 #include <xen/errno.h>
-#include <xen/serial.h>
 #include <xen/softirq.h>
 #include <xen/list.h>
 #include <xen/device_tree.h>
@@ -34,195 +33,70 @@
 #include <asm/io.h>
 #include <asm/gic.h>
 
-#define GICD (gic.map_dbase)
-#define GICC (gic.map_cbase)
-#define GICH (gic.map_hbase)
-
 static void gic_restore_pending_irqs(struct vcpu *v);
 
-/* Global state */
-static struct {
-    paddr_t dbase;       /* Address of distributor registers */
-    void __iomem * map_dbase;  /* IO mapped Address of distributor registers */
-    paddr_t cbase;       /* Address of CPU interface registers */
-    void __iomem * map_cbase; /* IO mapped Address of CPU interface registers*/
-    paddr_t hbase;       /* Address of virtual interface registers */
-    void __iomem * map_hbase; /* IO Address of virtual interface registers */
-    paddr_t vbase;       /* Address of virtual cpu interface registers */
-    unsigned int lines;  /* Number of interrupts (SPIs + PPIs + SGIs) */
-    unsigned int maintenance_irq; /* IRQ maintenance */
-    unsigned int cpus;
-    spinlock_t lock;
-} gic;
-
 static DEFINE_PER_CPU(uint64_t, lr_mask);
 
-static uint8_t nr_lrs;
-#define lr_all_full() (this_cpu(lr_mask) == ((1 << nr_lrs) - 1))
-
-/* The GIC mapping of CPU interfaces does not necessarily match the
- * logical CPU numbering. Let's use mapping as returned by the GIC
- * itself
- */
-static DEFINE_PER_CPU(u8, gic_cpu_id);
-
-/* Maximum cpu interface per GIC */
-#define NR_GIC_CPU_IF 8
+#define lr_all_full() (this_cpu(lr_mask) == ((1 << gic_hw_ops->info->nr_lrs) - 1))
 
 #undef GIC_DEBUG
 
 static void gic_update_one_lr(struct vcpu *v, int i);
 
-static unsigned int gic_cpu_mask(const cpumask_t *cpumask)
+static const struct gic_hw_operations *gic_hw_ops;
+
+void register_gic_ops(const struct gic_hw_operations *ops)
 {
-    unsigned int cpu;
-    unsigned int mask = 0;
-    cpumask_t possible_mask;
+    gic_hw_ops = ops;
+}
 
-    cpumask_and(&possible_mask, cpumask, &cpu_possible_map);
-    for_each_cpu(cpu, &possible_mask)
-    {
-        ASSERT(cpu < NR_GIC_CPU_IF);
-        mask |= per_cpu(gic_cpu_id, cpu);
-    }
+void update_cpu_lr_mask(void)
+{
+    this_cpu(lr_mask) = 0ULL;
+}
 
-    return mask;
+int gic_hw_version(void)
+{
+   return gic_hw_ops->info->hw_version;
 }
 
 unsigned int gic_number_lines(void)
 {
-    return gic.lines;
+    return gic_hw_ops->info->nr_lines;
 }
 
 void gic_save_state(struct vcpu *v)
 {
-    int i;
     ASSERT(!local_irq_is_enabled());
 
+    if ( is_idle_vcpu(v) )
+        return;
     /* No need for spinlocks here because interrupts are disabled around
      * this call and it only accesses struct vcpu fields that cannot be
      * accessed simultaneously by another pCPU.
      */
-    for ( i=0; i<nr_lrs; i++)
-        v->arch.gic_lr[i] = readl_relaxed(GICH + GICH_LR + i * 4);
     v->arch.lr_mask = this_cpu(lr_mask);
-    v->arch.gic_apr = readl_relaxed(GICH + GICH_APR);
-    v->arch.gic_vmcr = readl_relaxed(GICH + GICH_VMCR);
-    /* Disable until next VCPU scheduled */
-    writel_relaxed(0, GICH + GICH_HCR);
+    gic_hw_ops->save_state(v);
     isb();
 }
 
 void gic_restore_state(struct vcpu *v)
 {
-    int i;
     ASSERT(!local_irq_is_enabled());
 
     if ( is_idle_vcpu(v) )
         return;
 
     this_cpu(lr_mask) = v->arch.lr_mask;
-    for ( i=0; i<nr_lrs; i++)
-        writel_relaxed(v->arch.gic_lr[i], GICH + GICH_LR + i * 4);
-    writel_relaxed(v->arch.gic_apr, GICH + GICH_APR);
-    writel_relaxed(v->arch.gic_vmcr, GICH + GICH_VMCR);
-    writel_relaxed(GICH_HCR_EN, GICH + GICH_HCR);
+    gic_hw_ops->restore_state(v);
+
     isb();
 
     gic_restore_pending_irqs(v);
 }
 
-static void gic_irq_enable(struct irq_desc *desc)
-{
-    int irq = desc->irq;
-    unsigned long flags;
-
-    ASSERT(spin_is_locked(&desc->lock));
-
-    spin_lock_irqsave(&gic.lock, flags);
-    desc->status &= ~IRQ_DISABLED;
-    dsb(sy);
-    writel_relaxed((1u << (irq % 32)), GICD + GICD_ISENABLER + (irq / 32) * 4);
-    spin_unlock_irqrestore(&gic.lock, flags);
-}
-
-static void gic_irq_disable(struct irq_desc *desc)
-{
-    int irq = desc->irq;
-    unsigned long flags;
-
-    ASSERT(spin_is_locked(&desc->lock));
-
-    spin_lock_irqsave(&gic.lock, flags);
-    /* Disable routing */
-    writel_relaxed(1u << (irq % 32), GICD + GICD_ICENABLER + (irq / 32) * 4);
-    desc->status |= IRQ_DISABLED;
-    spin_unlock_irqrestore(&gic.lock, flags);
-}
-
-static unsigned int gic_irq_startup(struct irq_desc *desc)
-{
-    gic_irq_enable(desc);
-
-    return 0;
-}
-
-static void gic_irq_shutdown(struct irq_desc *desc)
-{
-    gic_irq_disable(desc);
-}
-
-static void gic_irq_ack(struct irq_desc *desc)
-{
-    /* No ACK -- reading IAR has done this for us */
-}
-
-static void gic_host_irq_end(struct irq_desc *desc)
-{
-    int irq = desc->irq;
-    /* Lower the priority */
-    writel_relaxed(irq, GICC + GICC_EOIR);
-    /* Deactivate */
-    writel_relaxed(irq, GICC + GICC_DIR);
-}
-
-static void gic_guest_irq_end(struct irq_desc *desc)
-{
-    int irq = desc->irq;
-    /* Lower the priority of the IRQ */
-    writel_relaxed(irq, GICC + GICC_EOIR);
-    /* Deactivation happens in maintenance interrupt / via GICV */
-}
-
-static void gic_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
-{
-    BUG();
-}
-
-/* XXX different for level vs edge */
-static hw_irq_controller gic_host_irq_type = {
-    .typename = "gic",
-    .startup = gic_irq_startup,
-    .shutdown = gic_irq_shutdown,
-    .enable = gic_irq_enable,
-    .disable = gic_irq_disable,
-    .ack = gic_irq_ack,
-    .end = gic_host_irq_end,
-    .set_affinity = gic_irq_set_affinity,
-};
-static hw_irq_controller gic_guest_irq_type = {
-    .typename = "gic",
-    .startup = gic_irq_startup,
-    .shutdown = gic_irq_shutdown,
-    .enable = gic_irq_enable,
-    .disable = gic_irq_disable,
-    .ack = gic_irq_ack,
-    .end = gic_guest_irq_end,
-    .set_affinity = gic_irq_set_affinity,
-};
-
 /*
- * - needs to be called with a valid cpu_mask, ie each cpu in the mask has
+ * needs to be called with a valid cpu_mask, ie each cpu in the mask has
  * already called gic_cpu_init
  * - desc.lock must be held
  * - arch.type must be valid (i.e != DT_IRQ_TYPE_INVALID)
@@ -231,37 +105,7 @@ static void gic_set_irq_properties(struct irq_desc *desc,
                                    const cpumask_t *cpu_mask,
                                    unsigned int priority)
 {
-    volatile unsigned char *bytereg;
-    uint32_t cfg, edgebit;
-    unsigned int mask;
-    unsigned int irq = desc->irq;
-    unsigned int type = desc->arch.type;
-
-    ASSERT(type != DT_IRQ_TYPE_INVALID);
-    ASSERT(spin_is_locked(&desc->lock));
-
-    spin_lock(&gic.lock);
-
-    mask = gic_cpu_mask(cpu_mask);
-
-    /* Set edge / level */
-    cfg = readl_relaxed(GICD + GICD_ICFGR + (irq / 16) * 4);
-    edgebit = 2u << (2 * (irq % 16));
-    if ( type & DT_IRQ_TYPE_LEVEL_MASK )
-        cfg &= ~edgebit;
-    else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
-        cfg |= edgebit;
-    writel_relaxed(cfg, GICD + GICD_ICFGR + (irq / 16) * 4);
-
-    /* Set target CPU mask (RAZ/WI on uniprocessor) */
-    bytereg = (unsigned char *) (GICD + GICD_ITARGETSR);
-    bytereg[irq] = mask;
-
-    /* Set priority */
-    bytereg = (unsigned char *) (GICD + GICD_IPRIORITYR);
-    bytereg[irq] = priority;
-
-    spin_unlock(&gic.lock);
+   gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
 }
 
 /* Program the GIC to route an interrupt to the host (i.e. Xen)
@@ -271,11 +115,11 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
                           unsigned int priority)
 {
     ASSERT(priority <= 0xff);     /* Only 8 bits of priority */
-    ASSERT(desc->irq < gic.lines);/* Can't route interrupts that don't exist */
+    ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
     ASSERT(desc->status & IRQ_DISABLED);
     ASSERT(spin_is_locked(&desc->lock));
 
-    desc->handler = &gic_host_irq_type;
+    desc->handler = gic_hw_ops->gic_host_irq_type;
 
     gic_set_irq_properties(desc, cpu_mask, priority);
 }
@@ -289,7 +133,7 @@ void gic_route_irq_to_guest(struct domain *d, struct irq_desc *desc,
     struct pending_irq *p;
     ASSERT(spin_is_locked(&desc->lock));
 
-    desc->handler = &gic_guest_irq_type;
+    desc->handler = gic_hw_ops->gic_guest_irq_type;
     desc->status |= IRQ_GUEST;
 
     gic_set_irq_properties(desc, cpumask_of(smp_processor_id()), GIC_PRI_IRQ);
@@ -299,99 +143,6 @@ void gic_route_irq_to_guest(struct domain *d, struct irq_desc *desc,
     p->desc = desc;
 }
 
-static void __init gic_dist_init(void)
-{
-    uint32_t type;
-    uint32_t cpumask;
-    int i;
-
-    cpumask = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
-    cpumask |= cpumask << 8;
-    cpumask |= cpumask << 16;
-
-    /* Disable the distributor */
-    writel_relaxed(0, GICD + GICD_CTLR);
-
-    type = readl_relaxed(GICD + GICD_TYPER);
-    gic.lines = 32 * ((type & GICD_TYPE_LINES) + 1);
-    gic.cpus = 1 + ((type & GICD_TYPE_CPUS) >> 5);
-    printk("GIC: %d lines, %d cpu%s%s (IID %8.8x).\n",
-           gic.lines, gic.cpus, (gic.cpus == 1) ? "" : "s",
-           (type & GICD_TYPE_SEC) ? ", secure" : "",
-           readl_relaxed(GICD + GICD_IIDR));
-
-    /* Default all global IRQs to level, active low */
-    for ( i = 32; i < gic.lines; i += 16 )
-        writel_relaxed(0x0, GICD + GICD_ICFGR + (i / 16) * 4);
-
-    /* Route all global IRQs to this CPU */
-    for ( i = 32; i < gic.lines; i += 4 )
-        writel_relaxed(cpumask, GICD + GICD_ITARGETSR + (i / 4) * 4);
-
-    /* Default priority for global interrupts */
-    for ( i = 32; i < gic.lines; i += 4 )
-        writel_relaxed (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
-                        GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
-                        GICD + GICD_IPRIORITYR + (i / 4) * 4);
-
-    /* Disable all global interrupts */
-    for ( i = 32; i < gic.lines; i += 32 )
-        writel_relaxed(~0x0, GICD + GICD_ICENABLER + (i / 32) * 4);
-
-    /* Turn on the distributor */
-    writel_relaxed(GICD_CTL_ENABLE, GICD + GICD_CTLR);
-}
-
-static void __cpuinit gic_cpu_init(void)
-{
-    int i;
-
-    this_cpu(gic_cpu_id) = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
-
-    /* The first 32 interrupts (PPI and SGI) are banked per-cpu, so
-     * even though they are controlled with GICD registers, they must
-     * be set up here with the other per-cpu state. */
-    writel_relaxed(0xffff0000, GICD + GICD_ICENABLER); /* Disable all PPI */
-    writel_relaxed(0x0000ffff, GICD + GICD_ISENABLER); /* Enable all SGI */
-
-    /* Set SGI priorities */
-    for (i = 0; i < 16; i += 4)
-        writel_relaxed(GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 |
-                       GIC_PRI_IPI << 8 | GIC_PRI_IPI,
-                       GICD + GICD_IPRIORITYR + (i / 4) * 4);
-    /* Set PPI priorities */
-    for (i = 16; i < 32; i += 4)
-        writel_relaxed(GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
-                      GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
-                      GICD + GICD_IPRIORITYR + (i / 4) * 4);
-
-    /* Local settings: interface controller */
-    writel_relaxed(0xff, GICC + GICC_PMR);                /* Don't mask by priority */
-    writel_relaxed(0x0, GICC + GICC_BPR);                   /* Finest granularity of priority */
-    writel_relaxed(GICC_CTL_ENABLE|GICC_CTL_EOI, GICC + GICC_CTLR);    /* Turn on delivery */
-}
-
-static void gic_cpu_disable(void)
-{
-    writel_relaxed(0x0, GICC + GICC_CTLR);
-}
-
-static void __cpuinit gic_hyp_init(void)
-{
-    uint32_t vtr;
-
-    vtr = readl_relaxed(GICH + GICH_VTR);
-    nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
-
-    writel_relaxed(GICH_MISR_EOI, GICH + GICH_MISR);
-    this_cpu(lr_mask) = 0ULL;
-}
-
-static void __cpuinit gic_hyp_disable(void)
-{
-    writel_relaxed(0, GICH + GICH_HCR);
-}
-
 int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
                   unsigned int *out_hwirq,
                   unsigned int *out_type)
@@ -415,108 +166,7 @@ int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
 /* Set up the GIC */
 void __init gic_init(void)
 {
-    static const struct dt_device_match gic_ids[] __initconst =
-    {
-        DT_MATCH_GIC,
-        { /* sentinel */ },
-    };
-    struct dt_device_node *node;
-    int res;
-
-    node = dt_find_interrupt_controller(gic_ids);
-    if ( !node )
-        panic("Unable to find compatible GIC in the device tree");
-
-    dt_device_set_used_by(node, DOMID_XEN);
-
-    res = dt_device_get_address(node, 0, &gic.dbase, NULL);
-    if ( res || !gic.dbase || (gic.dbase & ~PAGE_MASK) )
-        panic("GIC: Cannot find a valid address for the distributor");
-
-    res = dt_device_get_address(node, 1, &gic.cbase, NULL);
-    if ( res || !gic.cbase || (gic.cbase & ~PAGE_MASK) )
-        panic("GIC: Cannot find a valid address for the CPU");
-
-    res = dt_device_get_address(node, 2, &gic.hbase, NULL);
-    if ( res || !gic.hbase || (gic.hbase & ~PAGE_MASK) )
-        panic("GIC: Cannot find a valid address for the hypervisor");
-
-    res = dt_device_get_address(node, 3, &gic.vbase, NULL);
-    if ( res || !gic.vbase || (gic.vbase & ~PAGE_MASK) )
-        panic("GIC: Cannot find a valid address for the virtual CPU");
-
-    res = platform_get_irq(node, 0);
-    if ( res < 0 )
-        panic("GIC: Cannot find the maintenance IRQ");
-    gic.maintenance_irq = res;
-
-    /* Set the GIC as the primary interrupt controller */
-    dt_interrupt_controller = node;
-
-    /* TODO: Add check on distributor, cpu size */
-
-    printk("GIC initialization:\n"
-              "        gic_dist_addr=%"PRIpaddr"\n"
-              "        gic_cpu_addr=%"PRIpaddr"\n"
-              "        gic_hyp_addr=%"PRIpaddr"\n"
-              "        gic_vcpu_addr=%"PRIpaddr"\n"
-              "        gic_maintenance_irq=%u\n",
-              gic.dbase, gic.cbase, gic.hbase, gic.vbase,
-              gic.maintenance_irq);
-
-    if ( (gic.dbase & ~PAGE_MASK) || (gic.cbase & ~PAGE_MASK) ||
-         (gic.hbase & ~PAGE_MASK) || (gic.vbase & ~PAGE_MASK) )
-        panic("GIC interfaces not page aligned");
-
-    gic.map_dbase = ioremap_nocache(gic.dbase, PAGE_SIZE);
-    if ( !gic.map_dbase )
-        panic("Failed to ioremap for GIC distributor\n");
-
-    if ( platform_has_quirk(PLATFORM_QUIRK_GIC_64K_STRIDE) )
-        gic.map_cbase = ioremap_nocache(gic.cbase, PAGE_SIZE * 0x10);
-    else
-        gic.map_cbase = ioremap_nocache(gic.cbase, PAGE_SIZE * 2);
-
-    if ( !gic.map_cbase )
-        panic("Failed to ioremap for GIC CPU interface\n");
-
-    gic.map_hbase = ioremap_nocache(gic.hbase, PAGE_SIZE);
-    if ( !gic.map_hbase )
-        panic("Failed to ioremap for GIC Virtual interface\n");
-
-    /* Global settings: interrupt distributor */
-    spin_lock_init(&gic.lock);
-    spin_lock(&gic.lock);
-
-    gic_dist_init();
-    gic_cpu_init();
-    gic_hyp_init();
-
-    spin_unlock(&gic.lock);
-}
-
-static void send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
-                     uint8_t irqmode)
-{
-    unsigned int mask = 0;
-
-    switch ( irqmode )
-    {
-    case SGI_TARGET_OTHERS:
-        writel_relaxed(GICD_SGI_TARGET_OTHERS | sgi, GICD + GICD_SGIR);
-        break;
-    case SGI_TARGET_SELF:
-        writel_relaxed(GICD_SGI_TARGET_SELF | sgi, GICD + GICD_SGIR);
-        break;
-    case SGI_TARGET_LIST:
-        mask = gic_cpu_mask(cpu_mask);
-        writel_relaxed(GICD_SGI_TARGET_LIST |
-                       (mask << GICD_SGI_TARGET_SHIFT) | sgi,
-                       GICD + GICD_SGIR);
-        break;
-    default:
-        BUG_ON(1);
-    }
+    gicv2_init();
 }
 
 void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
@@ -529,12 +179,11 @@ void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
 
     dsb(sy);
 
-    send_SGI(&online_mask, sgi, SGI_TARGET_LIST);
+    gic_hw_ops->send_SGI(&online_mask, sgi, SGI_TARGET_LIST);
 }
 
 void send_SGI_one(unsigned int cpu, enum gic_sgi sgi)
 {
-    ASSERT(cpu < NR_GIC_CPU_IF);  /* Targets bitmap only supports 8 CPUs */
     send_SGI_mask(cpumask_of(cpu), sgi);
 }
 
@@ -544,7 +193,7 @@ void send_SGI_self(enum gic_sgi sgi)
 
     dsb(sy);
 
-    send_SGI(cpumask_of(smp_processor_id()), sgi, SGI_TARGET_SELF);
+    gic_hw_ops->send_SGI(cpumask_of(smp_processor_id()), sgi, SGI_TARGET_SELF);
 }
 
 void send_SGI_allbutself(enum gic_sgi sgi)
@@ -555,7 +204,7 @@ void send_SGI_allbutself(enum gic_sgi sgi)
    cpumask_andnot(&all_others_mask, &cpu_possible_map, cpumask_of(smp_processor_id()));
    dsb(sy);
 
-   send_SGI(&all_others_mask, sgi, SGI_TARGET_OTHERS);
+   gic_hw_ops->send_SGI(&all_others_mask, sgi, SGI_TARGET_OTHERS);
 }
 
 void smp_send_state_dump(unsigned int cpu)
@@ -566,10 +215,7 @@ void smp_send_state_dump(unsigned int cpu)
 /* Set up the per-CPU parts of the GIC for a secondary CPU */
 void __cpuinit gic_init_secondary_cpu(void)
 {
-    spin_lock(&gic.lock);
-    gic_cpu_init();
-    gic_hyp_init();
-    spin_unlock(&gic.lock);
+    gic_hw_ops->secondary_init();
 }
 
 /* Shut down the per-CPU GIC interface */
@@ -577,28 +223,14 @@ void gic_disable_cpu(void)
 {
     ASSERT(!local_irq_is_enabled());
 
-    spin_lock(&gic.lock);
-    gic_cpu_disable();
-    gic_hyp_disable();
-    spin_unlock(&gic.lock);
+    gic_hw_ops->disable_interface();
 }
 
 static inline void gic_set_lr(int lr, struct pending_irq *p,
-        unsigned int state)
+                              unsigned int state)
 {
-    uint32_t lr_val;
-
     ASSERT(!local_irq_is_enabled());
-    BUG_ON(lr >= nr_lrs);
-    BUG_ON(lr < 0);
-    BUG_ON(state & ~(GICH_LR_STATE_MASK<<GICH_LR_STATE_SHIFT));
-
-    lr_val = state | (GIC_PRI_TO_GUEST(p->priority) << GICH_LR_PRIORITY_SHIFT) |
-        ((p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT);
-    if ( p->desc != NULL )
-        lr_val |= GICH_LR_HW | (p->desc->irq << GICH_LR_PHYSICAL_SHIFT);
-
-    writel_relaxed(lr_reg, GICH + GICH_LR + lr * 4);
+    gic_hw_ops->update_lr(lr, p, state);
 
     set_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
     clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
@@ -665,6 +297,7 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
         unsigned int priority)
 {
     int i;
+    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
 
     ASSERT(spin_is_locked(&v->arch.vgic.lock));
 
@@ -684,28 +317,31 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
 static void gic_update_one_lr(struct vcpu *v, int i)
 {
     struct pending_irq *p;
-    uint32_t lr;
     int irq;
+    struct gic_lr lr_val;
 
     ASSERT(spin_is_locked(&v->arch.vgic.lock));
     ASSERT(!local_irq_is_enabled());
 
-    lr = readl_relaxed(GICH + GICH_LR + i * 4);
-    irq = (lr >> GICH_LR_VIRTUAL_SHIFT) & GICH_LR_VIRTUAL_MASK;
+    gic_hw_ops->read_lr(i, &lr_val);
+    irq = lr_val.virq;
     p = irq_to_pending(v, irq);
-    if ( lr & GICH_LR_ACTIVE )
+    if ( lr_val.state & GICH_LR_ACTIVE )
     {
         set_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
         if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
              test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
         {
             if ( p->desc == NULL )
-                writel_relaxed(lr | GICH_LR_PENDING, GICH + GICH_LR + i * 4);
+            {
+                 lr_val.state |= GICH_LR_PENDING;
+                 gic_hw_ops->write_lr(i, &lr_val);
+            }
             else
                 gdprintk(XENLOG_WARNING, "unable to inject hw irq=%d into d%dv%d: already active in LR%d\n",
                          irq, v->domain->domain_id, v->vcpu_id, i);
         }
-    } else if ( lr & GICH_LR_PENDING ) {
+    } else if ( lr_val.state & GICH_LR_PENDING ) {
         int q __attribute__ ((unused)) = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
 #ifdef GIC_DEBUG
         if ( q )
@@ -713,7 +349,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
                     irq, v->domain->domain_id, v->vcpu_id, i);
 #endif
     } else {
-        writel_relaxed(0, GICH + GICH_LR + i * 4);
+        gic_hw_ops->clear_lr(i);
         clear_bit(i, &this_cpu(lr_mask));
 
         if ( p->desc != NULL )
@@ -733,6 +369,7 @@ void gic_clear_lrs(struct vcpu *v)
 {
     int i = 0;
     unsigned long flags;
+    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
 
     if ( is_idle_vcpu(v) )
         return;
@@ -750,11 +387,13 @@ void gic_clear_lrs(struct vcpu *v)
 
 static void gic_restore_pending_irqs(struct vcpu *v)
 {
-    int lr = 0, lrs = nr_lrs;
+    int lr = 0, lrs;
     struct pending_irq *p, *t, *p_r;
     struct list_head *inflight_r;
     unsigned long flags;
+    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
 
+    lrs = nr_lrs;
     spin_lock_irqsave(&v->arch.vgic.lock, flags);
 
     if ( list_empty(&v->arch.vgic.lr_pending) )
@@ -815,13 +454,15 @@ void gic_clear_pending_irqs(struct vcpu *v)
 
 int gic_events_need_delivery(void)
 {
-    int mask_priority, lrs = nr_lrs;
+    int mask_priority, lrs;
     int max_priority = 0xff, active_priority = 0xff;
     struct vcpu *v = current;
     struct pending_irq *p;
     unsigned long flags;
+    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
+    lrs = nr_lrs;
 
-    mask_priority = (readl_relaxed(GICH + GICH_VMCR) >> GICH_VMCR_PRIORITY_SHIFT) & GICH_VMCR_PRIORITY_MASK;
+    mask_priority = gic_hw_ops->read_vmcr_priority();
     mask_priority = mask_priority << 3;
 
     spin_lock_irqsave(&v->arch.vgic.lock, flags);
@@ -857,21 +498,21 @@ int gic_events_need_delivery(void)
 
 void gic_inject(void)
 {
-    uint32_t hcr;
     ASSERT(!local_irq_is_enabled());
 
     gic_restore_pending_irqs(current);
 
     if ( !list_empty(&current->arch.vgic.lr_pending) && lr_all_full() )
-        writel_relaxed(hcr | GICH_HCR_UIE, GICH + GICH_HCR);
+        gic_hw_ops->update_hcr_status(GICH_HCR_UIE, 1);
     else
-        writel_relaxed(hcr & ~GICH_HCR_UIE, GICH + GICH_HCR);
+        gic_hw_ops->update_hcr_status(GICH_HCR_UIE, 0);
 }
 
 static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
 {
     /* Lower the priority */
-    writel_relaxed(sgi, GICC + GICC_EOIR);
+    struct irq_desc *desc = irq_to_desc(sgi);
+    gic_hw_ops->eoi_irq(desc);
 
     switch (sgi)
     {
@@ -890,19 +531,17 @@ static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
     }
 
     /* Deactivate */
-    writel_relaxed(sgi, GICC + GICC_DIR);
+    gic_hw_ops->deactivate_irq(sgi);
 }
 
 /* Accept an interrupt from the GIC and dispatch its handler */
 void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
 {
-    uint32_t intack;
     unsigned int irq;
 
-
     do  {
-        intack = readl_relaxed(GICC + GICC_IAR);
-        irq = intack & GICC_IA_IRQ;
+        /* Reading IRQ will ACK it */
+        irq = gic_hw_ops->read_irq();
 
         if ( likely(irq >= 16 && irq < 1021) )
         {
@@ -924,49 +563,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
 
 int gicv_setup(struct domain *d)
 {
-    int ret;
-
-    /*
-     * The hardware domain gets the hardware address.
-     * Guests get the virtual platform layout.
-     */
-    if ( is_hardware_domain(d) )
-    {
-        d->arch.vgic.dbase = gic.dbase;
-        d->arch.vgic.cbase = gic.cbase;
-    }
-    else
-    {
-        d->arch.vgic.dbase = GUEST_GICD_BASE;
-        d->arch.vgic.cbase = GUEST_GICC_BASE;
-    }
-
-    d->arch.vgic.nr_lines = 0;
-
-    /*
-     * Map the gic virtual cpu interface in the gic cpu interface
-     * region of the guest.
-     *
-     * The second page is always mapped at +4K irrespective of the
-     * GIC_64K_STRIDE quirk. The DTB passed to the guest reflects this.
-     */
-    ret = map_mmio_regions(d, d->arch.vgic.cbase,
-                           d->arch.vgic.cbase + PAGE_SIZE - 1,
-                           gic.vbase);
-    if (ret)
-        return ret;
-
-    if ( !platform_has_quirk(PLATFORM_QUIRK_GIC_64K_STRIDE) )
-        ret = map_mmio_regions(d, d->arch.vgic.cbase + PAGE_SIZE,
-                               d->arch.vgic.cbase + (2 * PAGE_SIZE) - 1,
-                               gic.vbase + PAGE_SIZE);
-    else
-        ret = map_mmio_regions(d, d->arch.vgic.cbase + PAGE_SIZE,
-                               d->arch.vgic.cbase + (2 * PAGE_SIZE) - 1,
-                               gic.vbase + 16*PAGE_SIZE);
-
-    return ret;
-
+    return gic_hw_ops->gicv_setup(d);
 }
 
 static void maintenance_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
@@ -981,18 +578,10 @@ static void maintenance_interrupt(int irq, void *dev_id, struct cpu_user_regs *r
 
 void gic_dump_info(struct vcpu *v)
 {
-    int i;
     struct pending_irq *p;
 
     printk("GICH_LRs (vcpu %d) mask=%"PRIx64"\n", v->vcpu_id, v->arch.lr_mask);
-    if ( v == current )
-    {
-        for ( i = 0; i < nr_lrs; i++ )
-            printk("   HW_LR[%d]=%x\n", i, readl_relaxed(GICH + GICH_LR + i * 4));
-    } else {
-        for ( i = 0; i < nr_lrs; i++ )
-            printk("   VCPU_LR[%d]=%x\n", i, v->arch.gic_lr[i]);
-    }
+    gic_hw_ops->dump_state(v);
 
     list_for_each_entry ( p, &v->arch.vgic.inflight_irqs, inflight )
     {
@@ -1003,12 +592,11 @@ void gic_dump_info(struct vcpu *v)
     {
         printk("Pending irq=%d\n", p->irq);
     }
-
 }
 
 void __cpuinit init_maintenance_interrupt(void)
 {
-    request_irq(gic.maintenance_irq, 0, maintenance_interrupt,
+    request_irq(gic_hw_ops->info->maintenance_irq, 0, maintenance_interrupt,
                 "irq-maintenance", NULL);
 }
 
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 7fa3b95..f119bcd 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -114,24 +114,6 @@
 #define GICH_MISR_VGRP1E  (1 << 6)
 #define GICH_MISR_VGRP1D  (1 << 7)
 
-#define GICH_LR_VIRTUAL_MASK    0x3ff
-#define GICH_LR_VIRTUAL_SHIFT   0
-#define GICH_LR_PHYSICAL_MASK   0x3ff
-#define GICH_LR_PHYSICAL_SHIFT  10
-#define GICH_LR_STATE_MASK      0x3
-#define GICH_LR_STATE_SHIFT     28
-#define GICH_LR_PRIORITY_SHIFT  23
-#define GICH_LR_MAINTENANCE_IRQ (1<<19)
-#define GICH_LR_PENDING         (1<<28)
-#define GICH_LR_ACTIVE          (1<<29)
-#define GICH_LR_GRP1            (1<<30)
-#define GICH_LR_HW              (1<<31)
-#define GICH_LR_CPUID_SHIFT     9
-#define GICH_VTR_NRLRGS         0x3f
-
-#define GICH_VMCR_PRIORITY_MASK   0x1f
-#define GICH_VMCR_PRIORITY_SHIFT  27
-
 /*
  * The minimum GICC_BPR is required to be in the range 0-3. We set
  * GICC_BPR to 0 but we must expect that it might be 3. This means we
@@ -155,6 +137,8 @@
 #define GIC_PRI_TO_GUEST(pri) (pri >> 3) /* GICH_LR and GICH_VMCR only support
                                             5 bits for guest irq priority */
 
+#define GICH_LR_PENDING         1
+#define GICH_LR_ACTIVE          2
 
 #ifndef __ASSEMBLY__
 #include <xen/device_tree.h>
@@ -163,6 +147,24 @@
 #define DT_MATCH_GIC    DT_MATCH_COMPATIBLE("arm,cortex-a15-gic"), \
                         DT_MATCH_COMPATIBLE("arm,cortex-a7-gic")
 
+/*
+ * Decode LR register content.
+ * The LR register format is different for GIC HW version
+ */
+struct gic_lr {
+   /* Physical IRQ */
+   uint32_t pirq;
+   /* Virtual IRQ */
+   uint32_t virq;
+   uint8_t priority;
+   uint8_t state;
+   uint8_t hw_status;
+   uint8_t grp;
+};
+
+extern int gic_hw_version(void);
+extern void gicv2_init(void);
+
 extern int domain_vgic_init(struct domain *d);
 extern void domain_vgic_free(struct domain *d);
 
@@ -213,9 +215,9 @@ enum gic_sgi {
 
 /* SGI irq mode types */
 enum gic_sgi_mode {
-    SGI_TARGET_LIST,
-    SGI_TARGET_OTHERS,
-    SGI_TARGET_SELF,
+    SGI_TARGET_LIST   = 0,
+    SGI_TARGET_OTHERS = 1,
+    SGI_TARGET_SELF   = 2,
 };
 
 extern void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi);
@@ -234,6 +236,74 @@ int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
                   unsigned int *out_hwirq, unsigned int *out_type);
 void gic_clear_lrs(struct vcpu *v);
 
+enum gic_version {
+    GIC_V2 = 2,
+};
+
+struct gic_info {
+    /* GIC version */
+    enum gic_version hw_version;
+    /* Number of GIC lines supported */
+    unsigned int nr_lines;
+    /* Number of LR registers */
+    unsigned int nr_lrs;
+    /* Maintenance irq number */
+    unsigned int maintenance_irq;
+};
+
+struct gic_hw_operations {
+    /* Hold GIC HW information */
+    struct gic_info *info;
+    /* Save GIC registers */
+    void (*save_state)(struct vcpu *);
+    /* Restore GIC registers */
+    void (*restore_state)(const struct vcpu *);
+    /* Dump GIC LR register information */
+    void (*dump_state)(const struct vcpu *);
+    /* Map MMIO region of GIC */
+    int (*gicv_setup)(struct domain *);
+
+    /* hw_irq_controller to enable/disable/eoi host irq */
+    hw_irq_controller *gic_host_irq_type;
+
+    /* hw_irq_controller to enable/disable/eoi guest irq */
+    hw_irq_controller *gic_guest_irq_type;
+
+    /* End of Interrupt */
+    void (*eoi_irq)(struct irq_desc *irqd);
+    /* Deactivate/reduce priority of irq */
+    void (*deactivate_irq)(int);
+    /* Read IRQ id and Ack */
+    unsigned int (*read_irq)(void);
+    /* Set IRQ property */
+    void (*set_irq_properties)(struct irq_desc *desc,
+                               const cpumask_t *cpu_mask,
+                               unsigned int priority);
+    /* Send SGI */
+    void (*send_SGI)(const cpumask_t *online_mask,
+                     enum gic_sgi sgi, uint8_t irqmode);
+    /* Disable CPU physical and virtual interfaces */
+    void (*disable_interface)(void);
+    /* Update LR register with state and priority */
+    void (*update_lr)(int lr, const struct pending_irq *pending_irq,
+                      unsigned int state);
+    /* Update HCR status register */
+    void (*update_hcr_status)(uint32_t flag, bool_t set);
+    /* Clear LR register */
+    void (*clear_lr)(int lr);
+    /* Read LR register and populate gic_lr structure */
+    void (*read_lr)(int lr, struct gic_lr *);
+    /* Write LR register from gic_lr structure */
+    void (*write_lr)(int lr, const struct gic_lr *);
+    /* Read VMCR priority */
+    unsigned int (*read_vmcr_priority)(void);
+    /* Secondary CPU init */
+    int (*secondary_init)(void);
+};
+
+void register_gic_ops(const struct gic_hw_operations *ops);
+extern void update_cpu_lr_mask(void);
+
 #endif /* __ASSEMBLY__ */
 #endif
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 07/16] arm/xen: move GIC context data structure to gic driver
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (5 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 14:32   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 08/16] xen/arm: use device api to detect GIC version vijay.kilari
                   ` (9 subsequent siblings)
  16 siblings, 1 reply; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

arch_domain in domain.h defines all the GIC registers
that needs to be saved/restored directly.

These GIC registers are GIC HW version specific. The number
of registers and size of registers varies with GIC version.
So move these registers to gic.h file and make a union of these
registers. This helps to define GIC HW version specific
structure to this union for later GIC versions.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/gic-v2.c        |   14 +++++++-------
 xen/include/asm-arm/domain.h |    7 +++++--
 xen/include/asm-arm/gic.h    |   18 ++++++++++++++++++
 3 files changed, 30 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 0628b5d..3595cbe 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -111,10 +111,10 @@ static void gicv2_save_state(struct vcpu *v)
      * accessed simultaneously by another pCPU.
      */
     for ( i = 0; i < gicv2_info.nr_lrs; i++ )
-        v->arch.gic_lr[i] = readl_relaxed(GICH + GICH_LR + i * 4);
+        v->arch.gic.v2.lr[i] = readl_relaxed(GICH + GICH_LR + i * 4);
 
-    v->arch.gic_apr = readl_relaxed(GICH + GICH_APR);
-    v->arch.gic_vmcr = readl_relaxed(GICH + GICH_VMCR);
+    v->arch.gic.v2.apr = readl_relaxed(GICH + GICH_APR);
+    v->arch.gic.v2.vmcr = readl_relaxed(GICH + GICH_VMCR);
     /* Disable until next VCPU scheduled */
     writel_relaxed(0, GICH + GICH_HCR);
 }
@@ -124,10 +124,10 @@ static void gicv2_restore_state(const struct vcpu *v)
     int i;
 
     for ( i = 0; i < gicv2_info.nr_lrs; i++ )
-        writel_relaxed(v->arch.gic_lr[i], GICH + GICH_LR + i * 4);
+        writel_relaxed(v->arch.gic.v2.lr[i], GICH + GICH_LR + i * 4);
 
-    writel_relaxed(v->arch.gic_apr, GICH + GICH_APR);
-    writel_relaxed(v->arch.gic_vmcr, GICH + GICH_VMCR);
+    writel_relaxed(v->arch.gic.v2.apr, GICH + GICH_APR);
+    writel_relaxed(v->arch.gic.v2.vmcr, GICH + GICH_VMCR);
     writel_relaxed(GICH_HCR_EN, GICH + GICH_HCR);
 }
 
@@ -144,7 +144,7 @@ static void gicv2_dump_state(const struct vcpu *v)
     else
     {
         for ( i = 0; i < gicv2_info.nr_lrs; i++ )
-            printk("   VCPU_LR[%d]=%x\n", i, v->arch.gic_lr[i]);
+            printk("   VCPU_LR[%d]=%x\n", i, v->arch.gic.v2.lr[i]);
     }
 }
 
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 61a498f..f46b631 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -8,6 +8,7 @@
 #include <asm/p2m.h>
 #include <asm/vfp.h>
 #include <asm/mmio.h>
+#include <asm/gic.h>
 #include <public/hvm/params.h>
 #include <xen/serial.h>
 
@@ -261,8 +262,10 @@ struct arch_vcpu
     uint32_t csselr;
     register_t vmpidr;
 
-    uint32_t gic_hcr, gic_vmcr, gic_apr;
-    uint32_t gic_lr[64];
+    /* Holds gic context data */
+    union gic_state_data gic;
+
+    uint64_t event_mask;
     uint64_t lr_mask;
 
     struct {
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index f119bcd..b53f403 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -148,6 +148,24 @@
                         DT_MATCH_COMPATIBLE("arm,cortex-a7-gic")
 
 /*
+ * GICv2 register that needs to be saved/restored
+ * on VCPU context switch
+ */
+struct gic_v2 {
+    uint32_t hcr;
+    uint32_t vmcr;
+    uint32_t apr;
+    uint32_t lr[64];
+};
+
+/*
+ * Union to hold underlying hw version context information
+ */ 
+union gic_state_data {
+    struct gic_v2 v2;
+};
+
+/*
  * Decode LR register content.
  * The LR register format is different for GIC HW version
  */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 08/16] xen/arm: use device api to detect GIC version
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (6 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 07/16] arm/xen: move GIC context data structure to gic driver vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 14:39   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file vijay.kilari
                   ` (8 subsequent siblings)
  16 siblings, 1 reply; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Run through the device tree to detect
compatible GIC version and initialize GIC driver

Also change DT_MATCH_GIC to DT_MATCH_GIC_V2 to point
the GIC HW version to add later GIC versions

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/domain_build.c  |    2 +-
 xen/arch/arm/gic-v2.c        |   27 ++++++++++++++++-----------
 xen/arch/arm/gic.c           |   24 +++++++++++++++++++++++-
 xen/include/asm-arm/device.h |    3 ++-
 xen/include/asm-arm/gic.h    |    8 +++++---
 5 files changed, 47 insertions(+), 17 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index fd35223..15ed97b 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -780,7 +780,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
     };
     static const struct dt_device_match gic_matches[] __initconst =
     {
-        DT_MATCH_GIC,
+        DT_MATCH_GIC_V2,
         { /* sentinel */ },
     };
     static const struct dt_device_match timer_matches[] __initconst =
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 3595cbe..8060de3 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -30,6 +30,7 @@
 #include <asm/p2m.h>
 #include <asm/domain.h>
 #include <asm/platform.h>
+#include <asm/device.h>
 
 #include <asm/io.h>
 #include <asm/gic.h>
@@ -584,20 +585,10 @@ const static struct gic_hw_operations gicv2_ops = {
 };
 
 /* Set up the GIC */
-void __init gicv2_init(void)
+static int __init gicv2_init(struct dt_device_node *node, const void *data)
 {
-    static const struct dt_device_match gic_ids[] __initconst =
-    {
-        DT_MATCH_GIC,
-        { /* sentinel */ },
-    };
-    struct dt_device_node *node;
     int res;
 
-    node = dt_find_interrupt_controller(gic_ids);
-    if ( !node )
-        panic("GICv2: Unable to find compatible GIC in the device tree");
-
     dt_device_set_used_by(node, DOMID_XEN);
 
     res = dt_device_get_address(node, 0, &gicv2.dbase, NULL);
@@ -667,8 +658,22 @@ void __init gicv2_init(void)
 
     gicv2_info.hw_version = GIC_V2;
     register_gic_ops(&gicv2_ops);
+
+    return 0;
 }
 
+static const char * const gicv2_dt_compat[] __initconst =
+{
+    DT_MATCH_GIC_V2_STRING_1,
+    DT_MATCH_GIC_V2_STRING_2,
+    NULL
+};
+
+DT_DEVICE_START(gicv2, "GICv2:", DEVICE_GIC)
+        .compatible = gicv2_dt_compat,
+        .init = gicv2_init,
+DT_DEVICE_END
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 975b10c..9331216 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -30,6 +30,7 @@
 #include <asm/p2m.h>
 #include <asm/domain.h>
 #include <asm/platform.h>
+#include <asm/device.h>
 #include <asm/io.h>
 #include <asm/gic.h>
 
@@ -166,7 +167,28 @@ int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
 /* Set up the GIC */
 void __init gic_init(void)
 {
-    gicv2_init();
+    int rc;
+    struct dt_device_node *node;
+    bool_t num_gics = 0;
+
+    dt_for_each_device_node( dt_host, node )
+    {
+        if ( !dt_get_property(node, "interrupt-controller", NULL) )
+            continue;
+
+        if ( !dt_get_parent(node) )
+            continue;
+
+        rc = device_init(node, DEVICE_GIC, NULL);
+        if ( !rc )
+        {
+            /* NOTE: Only one GIC is supported */
+            num_gics = 1;
+            break;
+        }
+    }
+    if ( !num_gics )
+        panic("Unable to find compatible GIC in the device tree");
 }
 
 void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index 9e47ca6..30668fa 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -6,7 +6,8 @@
 
 enum device_type
 {
-    DEVICE_SERIAL
+    DEVICE_SERIAL,
+    DEVICE_GIC,
 };
 
 struct device_desc {
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index b53f403..f4c467a 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -144,8 +144,11 @@
 #include <xen/device_tree.h>
 #include <xen/irq.h>
 
-#define DT_MATCH_GIC    DT_MATCH_COMPATIBLE("arm,cortex-a15-gic"), \
-                        DT_MATCH_COMPATIBLE("arm,cortex-a7-gic")
+#define DT_MATCH_GIC_V2_STRING_1     "arm,cortex-a15-gic"
+#define DT_MATCH_GIC_V2_STRING_2     "arm,cortex-a7-gic"
+
+#define DT_MATCH_GIC_V2 DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V2_STRING_1), \
+                        DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V2_STRING_2)
 
 /*
  * GICv2 register that needs to be saved/restored
@@ -181,7 +184,6 @@ struct gic_lr {
 };
 
 extern int gic_hw_version(void);
-extern void gicv2_init(void);
 
 extern int domain_vgic_init(struct domain *d);
 extern void domain_vgic_free(struct domain *d);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (7 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 08/16] xen/arm: use device api to detect GIC version vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-27 11:32   ` Julien Grall
  2014-05-28 14:54   ` Stefano Stabellini
  2014-05-26 10:26 ` [PATCH v4 10/16] xen/arm: move vgic defines to vgic " vijay.kilari
                   ` (7 subsequent siblings)
  16 siblings, 2 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

vgic_irq_rank structure contains gic specific data elements.
Move this out of domain.h to new vgic header file vgic.h
Allocate memory dynamically in vgic driver.

This patch reduces the size of domain struct and helps to
keep domain struct within PAGE_SIZE when future GIC hw versions
are added

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/domain.c        |    2 ++
 xen/arch/arm/vgic.c          |   18 ++++++++++++++----
 xen/include/asm-arm/domain.h |   11 +----------
 xen/include/asm-arm/vgic.h   |   40 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 57 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 40acfb3..bccbeda 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -31,6 +31,7 @@
 #include <asm/procinfo.h>
 
 #include <asm/gic.h>
+#include <asm/vgic.h>
 #include <asm/platform.h>
 #include "vtimer.h"
 #include "vuart.h"
@@ -481,6 +482,7 @@ int vcpu_initialise(struct vcpu *v)
 void vcpu_destroy(struct vcpu *v)
 {
     vcpu_timer_destroy(v);
+    vcpu_vgic_free(v);
     free_xenheap_pages(v->arch.stack, STACK_ORDER);
 }
 
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index b56f9d1..d2a9e34 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -29,6 +29,7 @@
 
 #include <asm/mmio.h>
 #include <asm/gic.h>
+#include <asm/vgic.h>
 
 #define REG(n) (n)
 
@@ -68,7 +69,7 @@ static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
     rank = REG_RANK_NR(b, n);
 
     if ( rank == 0 )
-        return &v->arch.vgic.private_irqs;
+        return v->arch.vgic.private_irqs;
     else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
         return &v->domain->arch.vgic.shared_irqs[rank - 1];
     else
@@ -84,9 +85,12 @@ void domain_vgic_free(struct domain *d)
 int vcpu_vgic_init(struct vcpu *v)
 {
     int i;
-    memset(&v->arch.vgic.private_irqs, 0, sizeof(v->arch.vgic.private_irqs));
 
-    spin_lock_init(&v->arch.vgic.private_irqs.lock);
+    v->arch.vgic.private_irqs = xzalloc(struct vgic_irq_rank);
+    if ( v->arch.vgic.private_irqs == NULL )
+      return -ENOMEM;
+
+    spin_lock_init(&v->arch.vgic.private_irqs->lock);
 
     memset(&v->arch.vgic.pending_irqs, 0, sizeof(v->arch.vgic.pending_irqs));
     for (i = 0; i < 32; i++)
@@ -97,7 +101,7 @@ int vcpu_vgic_init(struct vcpu *v)
 
     /* For SGI and PPI the target is always this CPU */
     for ( i = 0 ; i < 8 ; i++ )
-        v->arch.vgic.private_irqs.itargets[i] =
+        v->arch.vgic.private_irqs->itargets[i] =
               (1<<(v->vcpu_id+0))
             | (1<<(v->vcpu_id+8))
             | (1<<(v->vcpu_id+16))
@@ -109,6 +113,12 @@ int vcpu_vgic_init(struct vcpu *v)
     return 0;
 }
 
+int vcpu_vgic_free(struct vcpu *v)
+{
+    xfree(v->arch.vgic.private_irqs);
+    return 0;
+}
+
 #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
 #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
 
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index f46b631..fe84ce5 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -12,15 +12,6 @@
 #include <public/hvm/params.h>
 #include <xen/serial.h>
 
-/* Represents state corresponding to a block of 32 interrupts */
-struct vgic_irq_rank {
-    spinlock_t lock; /* Covers access to all other members of this struct */
-    uint32_t ienable, iactive, ipend, pendsgi;
-    uint32_t icfg[2];
-    uint32_t ipriority[8];
-    uint32_t itargets[8];
-};
-
 struct pending_irq
 {
     /*
@@ -274,7 +265,7 @@ struct arch_vcpu
          * struct arch_domain.
          */
         struct pending_irq pending_irqs[32];
-        struct vgic_irq_rank private_irqs;
+        struct vgic_irq_rank *private_irqs;
 
         /* This list is ordered by IRQ priority and it is used to keep
          * track of the IRQs that the VGIC injected into the guest.
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
new file mode 100644
index 0000000..104a87d
--- /dev/null
+++ b/xen/include/asm-arm/vgic.h
@@ -0,0 +1,40 @@
+/*
+ * ARM Virtual Generic Interrupt Controller support
+ *
+ * Ian Campbell <ian.campbell@citrix.com>
+ * Copyright (c) 2011 Citrix Systems.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ASM_ARM_VGIC_H__
+#define __ASM_ARM_VGIC_H__
+
+/* Represents state corresponding to a block of 32 interrupts */
+struct vgic_irq_rank {
+    spinlock_t lock; /* Covers access to all other members of this struct */
+    uint32_t ienable, iactive, ipend, pendsgi;
+    uint32_t icfg[2];
+    uint32_t ipriority[8];
+    uint32_t itargets[8];
+};
+
+extern int vcpu_vgic_free(struct vcpu *v);
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 10/16] xen/arm: move vgic defines to vgic header file
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (8 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-27 11:49   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size vijay.kilari
                   ` (6 subsequent siblings)
  16 siblings, 1 reply; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

gic.h contains defines of vgic. Move these to vgic
header file. Also move inline functions and defines/macros
in vgic.c to vgic.h. These vgic functions and macros
can be used by other GIC HW specific VGIC drivers,

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/irq.c         |    1 +
 xen/arch/arm/time.c        |    1 +
 xen/arch/arm/vgic.c        |   87 --------------------------------------------
 xen/arch/arm/vpsci.c       |    1 +
 xen/arch/arm/vtimer.c      |    1 +
 xen/include/asm-arm/gic.h  |    7 ----
 xen/include/asm-arm/vgic.h |   79 ++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/sched.h    |   16 ++++++++
 8 files changed, 99 insertions(+), 94 deletions(-)

diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 9c141bc..3a8acbf 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -26,6 +26,7 @@
 #include <xen/sched.h>
 
 #include <asm/gic.h>
+#include <asm/vgic.h>
 
 static unsigned int local_irqs_type[NR_LOCAL_IRQS];
 static DEFINE_SPINLOCK(local_irqs_type_lock);
diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c
index 06cc709..4c3e1a6 100644
--- a/xen/arch/arm/time.c
+++ b/xen/arch/arm/time.c
@@ -32,6 +32,7 @@
 #include <asm/system.h>
 #include <asm/time.h>
 #include <asm/gic.h>
+#include <asm/vgic.h>
 #include <asm/cpufeature.h>
 #include <asm/platform.h>
 
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index d2a9e34..597fb52 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -33,49 +33,6 @@
 
 #define REG(n) (n)
 
-/* Number of ranks of interrupt registers for a domain */
-#define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
-
-/*
- * Rank containing GICD_<FOO><n> for GICD_<FOO> with
- * <b>-bits-per-interrupt
- */
-static inline int REG_RANK_NR(int b, uint32_t n)
-{
-    switch ( b )
-    {
-    case 8: return n >> 3;
-    case 4: return n >> 2;
-    case 2: return n >> 1;
-    case 1: return n;
-    default: BUG();
-    }
-}
-
-/*
- * Offset of GICD_<FOO><n> with its rank, for GICD_<FOO> with
- * <b>-bits-per-interrupt.
- */
-#define REG_RANK_INDEX(b, n) (((n) >> 2) & ((b)-1))
-
-/*
- * Returns rank corresponding to a GICD_<FOO><n> register for
- * GICD_<FOO> with <b>-bits-per-interrupt.
- */
-static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
-{
-    int rank;
-    n = n >> 2;
-    rank = REG_RANK_NR(b, n);
-
-    if ( rank == 0 )
-        return v->arch.vgic.private_irqs;
-    else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
-        return &v->domain->arch.vgic.shared_irqs[rank - 1];
-    else
-        return NULL;
-}
-
 void domain_vgic_free(struct domain *d)
 {
     xfree(d->arch.vgic.shared_irqs);
@@ -119,34 +76,6 @@ int vcpu_vgic_free(struct vcpu *v)
     return 0;
 }
 
-#define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
-#define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
-
-#define vgic_lock_rank(v, r) spin_lock(&(r)->lock)
-#define vgic_unlock_rank(v, r) spin_unlock(&(r)->lock)
-
-static uint32_t byte_read(uint32_t val, int sign, int offset)
-{
-    int byte = offset & 0x3;
-
-    val = val >> (8*byte);
-    if ( sign && (val & 0x80) )
-        val |= 0xffffff00;
-    else
-        val &= 0x000000ff;
-    return val;
-}
-
-static void byte_write(uint32_t *reg, uint32_t var, int offset)
-{
-    int byte = offset & 0x3;
-
-    var &= (0xff << (8*byte));
-
-    *reg &= ~(0xff << (8*byte));
-    *reg |= var;
-}
-
 static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
 {
     struct hsr_dabt dabt = info->dabt;
@@ -400,22 +329,6 @@ static void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n)
     }
 }
 
-static inline int is_vcpu_running(struct domain *d, int vcpuid)
-{
-    struct vcpu *v;
-
-    if ( vcpuid >= d->max_vcpus )
-        return 0;
-
-    v = d->vcpu[vcpuid];
-    if ( v == NULL )
-        return 0;
-    if (test_bit(_VPF_down, &v->pause_flags) )
-        return 0;
-
-    return 1;
-}
-
 static int vgic_to_sgi(struct vcpu *v, register_t sgir)
 {
     struct domain *d = v->domain;
diff --git a/xen/arch/arm/vpsci.c b/xen/arch/arm/vpsci.c
index 1ceb8cb..80678f6 100644
--- a/xen/arch/arm/vpsci.c
+++ b/xen/arch/arm/vpsci.c
@@ -16,6 +16,7 @@
 
 #include <asm/current.h>
 #include <asm/gic.h>
+#include <asm/vgic.h>
 #include <asm/psci.h>
 
 int do_psci_cpu_on(uint32_t vcpuid, register_t entry_point)
diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
index b751692..7016989 100644
--- a/xen/arch/arm/vtimer.c
+++ b/xen/arch/arm/vtimer.c
@@ -24,6 +24,7 @@
 #include <asm/irq.h>
 #include <asm/time.h>
 #include <asm/gic.h>
+#include <asm/vgic.h>
 #include <asm/regs.h>
 
 extern s_time_t ticks_to_ns(uint64_t ticks);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index f4c467a..2aadcb6 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -185,13 +185,6 @@ struct gic_lr {
 
 extern int gic_hw_version(void);
 
-extern int domain_vgic_init(struct domain *d);
-extern void domain_vgic_free(struct domain *d);
-
-extern int vcpu_vgic_init(struct vcpu *v);
-
-extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
-extern void vgic_clear_pending_irqs(struct vcpu *v);
 extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
 
 /* Program the GIC to route an interrupt */
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 104a87d..9be15b4 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -18,6 +18,8 @@
 #ifndef __ASM_ARM_VGIC_H__
 #define __ASM_ARM_VGIC_H__
 
+#include <xen/bitops.h>
+
 /* Represents state corresponding to a block of 32 interrupts */
 struct vgic_irq_rank {
     spinlock_t lock; /* Covers access to all other members of this struct */
@@ -27,6 +29,83 @@ struct vgic_irq_rank {
     uint32_t itargets[8];
 };
 
+/* Number of ranks of interrupt registers for a domain */
+#define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
+
+#define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
+#define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
+
+#define vgic_lock_rank(v, r) spin_lock(&(r)->lock)
+#define vgic_unlock_rank(v, r) spin_unlock(&(r)->lock)
+
+/*
+ * Rank containing GICD_<FOO><n> for GICD_<FOO> with
+ * <b>-bits-per-interrupt
+ */
+static inline int REG_RANK_NR(int b, uint32_t n)
+{
+    switch ( b )
+    {
+    case 8: return n >> 3;
+    case 4: return n >> 2;
+    case 2: return n >> 1;
+    case 1: return n;
+    default: BUG();
+    }
+}
+
+static inline uint32_t byte_read(uint32_t val, int sign, int offset)
+{
+    int byte = offset & 0x3;
+
+    val = val >> (8*byte);
+    if ( sign && (val & 0x80) )
+        val |= 0xffffff00;
+    else
+        val &= 0x000000ff;
+    return val;
+}
+
+static inline void byte_write(uint32_t *reg, uint32_t var, int offset)
+{
+    int byte = offset & 0x3;
+
+    var &= (0xff << (8*byte));
+
+    *reg &= ~(0xff << (8*byte));
+    *reg |= var;
+}
+
+/*
+ * Offset of GICD_<FOO><n> with its rank, for GICD_<FOO> with
+ * <b>-bits-per-interrupt.
+ */
+#define REG_RANK_INDEX(b, n) (((n) >> 2) & ((b)-1))
+
+/*
+ * Returns rank corresponding to a GICD_<FOO><n> register for
+ * GICD_<FOO> with <b>-bits-per-interrupt.
+ */
+static inline struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
+{
+    int rank;
+    n = n >> 2;
+    rank = REG_RANK_NR(b, n);
+
+    if ( rank == 0 )
+        return v->arch.vgic.private_irqs;
+    else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
+        return &v->domain->arch.vgic.shared_irqs[rank - 1];
+    else
+        return NULL;
+}
+
+extern int domain_vgic_init(struct domain *d);
+extern void domain_vgic_free(struct domain *d);
+extern int vcpu_vgic_init(struct vcpu *v);
+extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
+extern void vgic_clear_pending_irqs(struct vcpu *v);
+
 extern int vcpu_vgic_free(struct vcpu *v);
 #endif
 
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 44851ae..03416b5 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -826,6 +826,22 @@ void watchdog_domain_destroy(struct domain *d);
 #define need_iommu(d)    (0)
 #endif
 
+static inline int is_vcpu_running(struct domain *d, int vcpuid)
+{
+    struct vcpu *v;
+
+    if ( vcpuid >= d->max_vcpus )
+        return 0;
+
+    v = d->vcpu[vcpuid];
+    if ( v == NULL )
+        return 0;
+    if (test_bit(_VPF_down, &v->pause_flags) )
+        return 0;
+
+    return 1;
+}
+
 void set_vcpu_migration_delay(unsigned int delay);
 unsigned int get_vcpu_migration_delay(void);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (9 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 10/16] xen/arm: move vgic defines to vgic " vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-27 11:56   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 12/16] xen/arm: split vgic driver into generic and vgic-v2 driver vijay.kilari
                   ` (5 subsequent siblings)
  16 siblings, 1 reply; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

vGIC irq rank was computed assuming the register offset is byte size.
With this patch, pass the register size as parameter and compute
irq rank. This makes generic and can be used with any register
size.

Use the HSR  abort address size in calculating register size

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/vgic.c             |  142 ++++++++++++++++++++-------------------
 xen/include/asm-arm/processor.h |    8 +++
 xen/include/asm-arm/vgic.h      |   11 +--
 3 files changed, 88 insertions(+), 73 deletions(-)

diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 597fb52..487c708 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -88,21 +88,21 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
     switch ( gicd_reg )
     {
     case GICD_CTLR:
-        if ( dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_WORD ) goto bad_width;
         vgic_lock(v);
         *r = v->domain->arch.vgic.ctlr;
         vgic_unlock(v);
         return 1;
     case GICD_TYPER:
-        if ( dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_WORD ) goto bad_width;
         /* No secure world support for guests. */
         vgic_lock(v);
-        *r = ( (v->domain->max_vcpus<<5) & GICD_TYPE_CPUS )
-            |( ((v->domain->arch.vgic.nr_lines/32)) & GICD_TYPE_LINES );
+        *r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
+            |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );
         vgic_unlock(v);
         return 1;
     case GICD_IIDR:
-        if ( dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_WORD ) goto bad_width;
         /*
          * XXX Do we need a JEP106 manufacturer ID?
          * Just use the physical h/w value for now
@@ -119,8 +119,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         goto read_as_zero;
 
     case GICD_ISENABLER ... GICD_ISENABLERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = rank->ienable;
@@ -128,8 +128,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_ICENABLER ... GICD_ICENABLERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = rank->ienable;
@@ -137,8 +137,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_ISPENDR ... GICD_ISPENDRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISPENDR);
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISPENDR, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = byte_read(rank->ipend, dabt.sign, offset);
@@ -146,8 +146,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_ICPENDR ... GICD_ICPENDRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICPENDR);
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICPENDR, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = byte_read(rank->ipend, dabt.sign, offset);
@@ -155,8 +155,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_ISACTIVER ... GICD_ISACTIVERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = rank->iactive;
@@ -164,8 +164,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_ICACTIVER ... GICD_ICACTIVERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = rank->iactive;
@@ -173,35 +173,37 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_ITARGETSR ... GICD_ITARGETSRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR);
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
 
         vgic_lock_rank(v, rank);
-        *r = rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR)];
-        if ( dabt.size == 0 )
+        *r = rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+                                           DABT_WORD)];
+        if ( dabt.size == DABT_BYTE )
             *r = byte_read(*r, dabt.sign, offset);
         vgic_unlock_rank(v, rank);
         return 1;
 
     case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR);
+        if ( dabt.size != 0 && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
 
         vgic_lock_rank(v, rank);
-        *r = rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR)];
-        if ( dabt.size == 0 )
+        *r = rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR,
+                                            DABT_WORD)];
+        if ( dabt.size == DABT_BYTE )
             *r = byte_read(*r, dabt.sign, offset);
         vgic_unlock_rank(v, rank);
         return 1;
 
     case GICD_ICFGR ... GICD_ICFGRN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
-        *r = rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR)];
+        *r = rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR, DABT_WORD)];
         vgic_unlock_rank(v, rank);
         return 1;
 
@@ -210,14 +212,14 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         goto read_as_zero;
 
     case GICD_SGIR:
-        if ( dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_WORD ) goto bad_width;
         /* Write only -- read unknown */
         *r = 0xdeadbeef;
         return 1;
 
     case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_CPENDSGIR);
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_CPENDSGIR, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = byte_read(rank->pendsgi, dabt.sign, offset);
@@ -225,8 +227,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_SPENDSGIR);
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_SPENDSGIR, DABT_WORD);
         if ( rank == NULL) goto read_as_zero;
         vgic_lock_rank(v, rank);
         *r = byte_read(rank->pendsgi, dabt.sign, offset);
@@ -238,7 +240,7 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         goto read_as_zero;
 
     case GICD_ICPIDR2:
-        if ( dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_WORD ) goto bad_width;
         printk("vGICD: unhandled read from ICPIDR2\n");
         return 0;
 
@@ -268,7 +270,7 @@ bad_width:
     return 0;
 
 read_as_zero:
-    if ( dabt.size != 2 ) goto bad_width;
+    if ( dabt.size != DABT_WORD ) goto bad_width;
     *r = 0;
     return 1;
 }
@@ -391,7 +393,7 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
     switch ( gicd_reg )
     {
     case GICD_CTLR:
-        if ( dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_WORD ) goto bad_width;
         /* Ignore all but the enable bit */
         v->domain->arch.vgic.ctlr = (*r) & GICD_CTL_ENABLE;
         return 1;
@@ -410,42 +412,44 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         goto write_ignore;
 
     case GICD_ISENABLER ... GICD_ISENABLERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
         tr = rank->ienable;
         rank->ienable |= *r;
         vgic_unlock_rank(v, rank);
-        vgic_enable_irqs(v, (*r) & (~tr), (gicd_reg - GICD_ISENABLER) >> 2);
+        vgic_enable_irqs(v, (*r) & (~tr),
+                         (gicd_reg - GICD_ISENABLER) >> DABT_WORD);
         return 1;
 
     case GICD_ICENABLER ... GICD_ICENABLERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER, DABT_WORD);
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
         tr = rank->ienable;
         rank->ienable &= ~*r;
         vgic_unlock_rank(v, rank);
-        vgic_disable_irqs(v, (*r) & tr, (gicd_reg - GICD_ICENABLER) >> 2);
+        vgic_disable_irqs(v, (*r) & tr,
+                          (gicd_reg - GICD_ICENABLER) >> DABT_WORD);
         return 1;
 
     case GICD_ISPENDR ... GICD_ISPENDRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
         printk("vGICD: unhandled %s write %#"PRIregister" to ISPENDR%d\n",
                dabt.size ? "word" : "byte", *r, gicd_reg - GICD_ISPENDR);
         return 0;
 
     case GICD_ICPENDR ... GICD_ICPENDRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
         printk("vGICD: unhandled %s write %#"PRIregister" to ICPENDR%d\n",
                dabt.size ? "word" : "byte", *r, gicd_reg - GICD_ICPENDR);
         return 0;
 
     case GICD_ISACTIVER ... GICD_ISACTIVERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER, DABT_WORD);
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
         rank->iactive &= ~*r;
@@ -453,8 +457,8 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         return 1;
 
     case GICD_ICACTIVER ... GICD_ICACTIVERN:
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER, DABT_WORD);
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
         rank->iactive &= ~*r;
@@ -466,28 +470,30 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         goto write_ignore;
 
     case GICD_ITARGETSR + 8 ... GICD_ITARGETSRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR);
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
-        if ( dabt.size == 2 )
-            rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR)] = *r;
+        if ( dabt.size == DABT_WORD )
+            rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+                                          DABT_WORD)] = *r;
         else
-            byte_write(&rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR)],
-                       *r, offset);
+            byte_write(&rank->itargets[REG_RANK_INDEX(8,
+                        gicd_reg - GICD_ITARGETSR, DABT_WORD)], *r, offset);
         vgic_unlock_rank(v, rank);
         return 1;
 
     case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR);
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR, DABT_WORD);
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
-        if ( dabt.size == 2 )
-            rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR)] = *r;
+        if ( dabt.size == DABT_WORD )
+            rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR,
+                                           DABT_WORD)] = *r;
         else
-            byte_write(&rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR)],
-                       *r, offset);
+            byte_write(&rank->ipriority[REG_RANK_INDEX(8,
+                       gicd_reg - GICD_IPRIORITYR, DABT_WORD)], *r, offset);
         vgic_unlock_rank(v, rank);
         return 1;
 
@@ -497,11 +503,11 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         /* It is implementation defined if these are writeable. We chose not */
         goto write_ignore;
     case GICD_ICFGR + 2 ... GICD_ICFGRN: /* SPIs */
-        if ( dabt.size != 2 ) goto bad_width;
-        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR);
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR, DABT_WORD);
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
-        rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR)] = *r;
+        rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR, DABT_WORD)] = *r;
         vgic_unlock_rank(v, rank);
         return 1;
 
@@ -515,13 +521,13 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         return vgic_to_sgi(v, *r);
 
     case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
         printk("vGICD: unhandled %s write %#"PRIregister" to ICPENDSGIR%d\n",
                dabt.size ? "word" : "byte", *r, gicd_reg - GICD_CPENDSGIR);
         return 0;
 
     case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
-        if ( dabt.size != 0 && dabt.size != 2 ) goto bad_width;
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
         printk("vGICD: unhandled %s write %#"PRIregister" to ISPENDSGIR%d\n",
                dabt.size ? "word" : "byte", *r, gicd_reg - GICD_SPENDSGIR);
         return 0;
@@ -560,7 +566,7 @@ bad_width:
     return 0;
 
 write_ignore:
-    if ( dabt.size != 2 ) goto bad_width;
+    if ( dabt.size != DABT_WORD ) goto bad_width;
     return 1;
 }
 
@@ -596,7 +602,7 @@ void vgic_clear_pending_irqs(struct vcpu *v)
 void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
 {
     uint8_t priority;
-    struct vgic_irq_rank *rank = vgic_irq_rank(v, 8, irq);
+    struct vgic_irq_rank *rank = vgic_irq_rank(v, 8, irq, DABT_WORD);
     struct pending_irq *iter, *n = irq_to_pending(v, irq);
     unsigned long flags;
     bool_t running;
@@ -617,7 +623,7 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
         return;
     }
 
-    priority = byte_read(rank->ipriority[REG_RANK_INDEX(8, irq)], 0, irq & 0x3);
+    priority = byte_read(rank->ipriority[REG_RANK_INDEX(8, irq, DABT_WORD)], 0, irq & 0x3);
 
     n->irq = irq;
     set_bit(GIC_IRQ_GUEST_QUEUED, &n->status);
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 9267c1b..3662749 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -251,6 +251,14 @@ extern struct cpuinfo_arm cpu_data[];
 extern u32 __cpu_logical_map[];
 #define cpu_logical_map(cpu) __cpu_logical_map[cpu]
 
+/* HSR data abort size definition */
+enum dabt_size {
+    DABT_BYTE        = 0,
+    DABT_HALF_WORD   = 1,
+    DABT_WORD        = 2,
+    DABT_DOUBLE_WORD = 3,
+};
+
 union hsr {
     uint32_t bits;
     struct {
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 9be15b4..9bcd0c3 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -77,19 +77,20 @@ static inline void byte_write(uint32_t *reg, uint32_t var, int offset)
 }
 
 /*
- * Offset of GICD_<FOO><n> with its rank, for GICD_<FOO> with
+ * Offset of GICD_<FOO><n> with its rank, for GICD_<FOO> size s with
  * <b>-bits-per-interrupt.
  */
-#define REG_RANK_INDEX(b, n) (((n) >> 2) & ((b)-1))
+#define REG_RANK_INDEX(b, n, s) ((((n) >> s) & ((b)-1)) % 32)
 
 /*
  * Returns rank corresponding to a GICD_<FOO><n> register for
- * GICD_<FOO> with <b>-bits-per-interrupt.
+ * GICD_<FOO> size s with <b>-bits-per-interrupt.
  */
-static inline struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
+static inline struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b,
+                     int n, int s)
 {
     int rank;
-    n = n >> 2;
+    n = n >> s;
     rank = REG_RANK_NR(b, n);
 
     if ( rank == 0 )
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 12/16] xen/arm: split vgic driver into generic and vgic-v2 driver
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (10 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-27 16:50   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 13/16] xen/arm: Add support for GIC v3 vijay.kilari
                   ` (4 subsequent siblings)
  16 siblings, 1 reply; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Existing vGIC driver has both generic code and hw specific
code. Segregate vGIC low level driver into vgic-v2.c and
keep generic code in existing vgic.c file

some static generic functions in vgic.c is made as non-static
so that these generic functions can be used in vGIC v2 driver.

vGIC v2 driver registers required callbacks to generic vGIC
driver. This helps to plug in next version of vGIC drivers
like vGIC v3. These callbacks are registered per domain

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/Makefile        |    2 +-
 xen/arch/arm/vgic-v2.c       |  487 ++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic.c          |  485 +++++------------------------------------
 xen/include/asm-arm/domain.h |    2 +
 xen/include/asm-arm/vgic.h   |   14 ++
 5 files changed, 553 insertions(+), 437 deletions(-)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 969ee52..20f59f4 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -26,7 +26,7 @@ obj-y += smpboot.o
 obj-y += smp.o
 obj-y += shutdown.o
 obj-y += traps.o
-obj-y += vgic.o
+obj-y += vgic.o vgic-v2.o
 obj-y += vtimer.o
 obj-y += vuart.o
 obj-y += hvm.o
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
new file mode 100644
index 0000000..b146efe
--- /dev/null
+++ b/xen/arch/arm/vgic-v2.c
@@ -0,0 +1,487 @@
+/*
+ * xen/arch/arm/vgic-v2.c
+ *
+ * ARM Virtual Generic Interrupt Controller support v2
+ *
+ * Ian Campbell <ian.campbell@citrix.com>
+ * Copyright (c) 2011 Citrix Systems.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/bitops.h>
+#include <xen/config.h>
+#include <xen/lib.h>
+#include <xen/init.h>
+#include <xen/softirq.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+
+#include <asm/current.h>
+#include <asm/device.h>
+
+#include <asm/mmio.h>
+#include <asm/gic.h>
+#include <asm/vgic.h>
+
+static int vgic_v2_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
+
+    switch ( gicd_reg )
+    {
+    case GICD_CTLR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        vgic_lock(v);
+        *r = v->domain->arch.vgic.ctlr;
+        vgic_unlock(v);
+        return 1;
+    case GICD_TYPER:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* No secure world support for guests. */
+        vgic_lock(v);
+        *r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
+            |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );
+        vgic_unlock(v);
+        return 1;
+    case GICD_IIDR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /*
+         * XXX Do we need a JEP106 manufacturer ID?
+         * Just use the physical h/w value for now
+         */
+        *r = 0x0000043b;
+        return 1;
+
+    /* Implementation defined -- read as zero */
+    case 0x020 ... 0x03c:
+        goto read_as_zero;
+
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->ienable;
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->ienable;
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISPENDR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = byte_read(rank->ipend, dabt.sign, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICPENDR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = byte_read(rank->ipend, dabt.sign, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->iactive;
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->iactive;
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ITARGETSR ... GICD_ITARGETSRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+
+        vgic_lock_rank(v, rank);
+        *r = rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+                                           DABT_WORD)];
+        if ( dabt.size == DABT_BYTE )
+            *r = byte_read(*r, dabt.sign, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+
+        vgic_lock_rank(v, rank);
+        *r = rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR,
+                                            DABT_WORD)];
+        if ( dabt.size == DABT_BYTE )
+            *r = byte_read(*r, dabt.sign, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ICFGR ... GICD_ICFGRN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR, DABT_WORD)];
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_NSACR ... GICD_NSACRN:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+
+    case GICD_SGIR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* Write only -- read unknown */
+        *r = 0xdeadbeef;
+        return 1;
+
+    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_CPENDSGIR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = byte_read(rank->pendsgi, dabt.sign, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_SPENDSGIR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = byte_read(rank->pendsgi, dabt.sign, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    /* Implementation defined -- read as zero */
+    case 0xfd0 ... 0xfe4:
+        goto read_as_zero;
+
+    case GICD_ICPIDR2:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        printk("vGICD: unhandled read from ICPIDR2\n");
+        return 0;
+
+    /* Implementation defined -- read as zero */
+    case 0xfec ... 0xffc:
+        goto read_as_zero;
+
+    /* Reserved -- read as zero */
+    case 0x00c ... 0x01c:
+    case 0x040 ... 0x07c:
+    case 0x7fc:
+    case 0xbfc:
+    case 0xf04 ... 0xf0c:
+    case 0xf30 ... 0xfcc:
+        goto read_as_zero;
+
+    default:
+        printk("vGICD: unhandled read r%d offset %#08x\n",
+               dabt.reg, gicd_reg);
+        return 0;
+    }
+
+bad_width:
+    printk("vGICD: bad read width %d r%d offset %#08x\n",
+           dabt.size, dabt.reg, gicd_reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int vgic_v2_to_sgi(struct vcpu *v, register_t sgir)
+{
+
+    int virq;
+    int irqmode;
+    unsigned long vcpu_mask = 0;
+
+    irqmode = (sgir  >> GICD_SGI_TARGET_LIST_SHIFT) & GICD_SGI_TARGET_LIST_MASK;
+    virq = (sgir & GICD_SGI_INTID_MASK);
+    vcpu_mask = (sgir & GICD_SGI_TARGET_MASK) >> GICD_SGI_TARGET_SHIFT;
+
+    return vgic_to_sgi(v, sgir, irqmode, virq, vcpu_mask);
+}
+
+static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
+    uint32_t tr;
+
+    switch ( gicd_reg )
+    {
+    case GICD_CTLR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* Ignore all but the enable bit */
+        v->domain->arch.vgic.ctlr = (*r) & GICD_CTL_ENABLE;
+        return 1;
+
+    /* R/O -- write ignored */
+    case GICD_TYPER:
+    case GICD_IIDR:
+        goto write_ignore;
+
+    /* Implementation defined -- write ignored */
+    case 0x020 ... 0x03c:
+        goto write_ignore;
+
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        tr = rank->ienable;
+        rank->ienable |= *r;
+        vgic_unlock_rank(v, rank);
+        /* The virtual irq is derived from register offset.
+         * The register difference is word difference. So divide by 2(DABT_WORD)
+         * to get Virtual irq number */
+        vgic_enable_irqs(v, (*r) & (~tr),
+                         (gicd_reg - GICD_ISENABLER) >> DABT_WORD);
+        return 1;
+
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        tr = rank->ienable;
+        rank->ienable &= ~*r;
+        vgic_unlock_rank(v, rank);
+        /* The virtual irq is derived from register offset.
+         * The register difference is word difference. So divide by 2(DABT_WORD)
+         * to get  Virtual irq number */
+        vgic_disable_irqs(v, (*r) & tr,
+                         (gicd_reg - GICD_ICENABLER) >> DABT_WORD);
+        return 1;
+
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        printk("vGICD: unhandled %s write %#"PRIregister" to ISPENDR%d\n",
+               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_ISPENDR);
+        return 0;
+
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        printk("vGICD: unhandled %s write %#"PRIregister" to ICPENDR%d\n",
+               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_ICPENDR);
+        return 0;
+
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->iactive &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->iactive &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ITARGETSR ... GICD_ITARGETSR + 7:
+        /* SGI/PPI target is read only */
+        goto write_ignore;
+
+    case GICD_ITARGETSR + 8 ... GICD_ITARGETSRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        if ( dabt.size == DABT_WORD )
+            rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+                                          DABT_WORD)] = *r;
+        else
+            byte_write(&rank->itargets[REG_RANK_INDEX(8,
+                       gicd_reg - GICD_ITARGETSR, DABT_WORD)], *r, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        if ( dabt.size == DABT_WORD )
+            rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR,
+                                           DABT_WORD)] = *r;
+        else
+            byte_write(&rank->ipriority[REG_RANK_INDEX(8,
+                        gicd_reg - GICD_IPRIORITYR, DABT_WORD)], *r, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_ICFGR: /* SGIs */
+        goto write_ignore;
+    case GICD_ICFGR + 1: /* PPIs */
+        /* It is implementation defined if these are writeable. We chose not */
+        goto write_ignore;
+    case GICD_ICFGR + 2 ... GICD_ICFGRN: /* SPIs */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR, DABT_WORD);
+        vgic_lock_rank(v, rank);
+        if ( rank == NULL) goto write_ignore;
+        rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR, DABT_WORD)] = *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+
+    case GICD_NSACR ... GICD_NSACRN:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+
+    case GICD_SGIR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        return vgic_v2_to_sgi(v, *r);
+
+    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        printk("vGICD: unhandled %s write %#"PRIregister" to ICPENDSGIR%d\n",
+               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_CPENDSGIR);
+        return 0;
+
+    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        printk("vGICD: unhandled %s write %#"PRIregister" to ISPENDSGIR%d\n",
+               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_SPENDSGIR);
+        return 0;
+
+    /* Implementation defined -- write ignored */
+    case 0xfd0 ... 0xfe4:
+        goto write_ignore;
+
+    /* R/O -- write ignore */
+    case GICD_ICPIDR2:
+        goto write_ignore;
+
+    /* Implementation defined -- write ignored */
+    case 0xfec ... 0xffc:
+        goto write_ignore;
+
+    /* Reserved -- write ignored */
+    case 0x00c ... 0x01c:
+    case 0x040 ... 0x07c:
+    case 0x7fc:
+    case 0xbfc:
+    case 0xf04 ... 0xf0c:
+    case 0xf30 ... 0xfcc:
+        goto write_ignore;
+
+    default:
+        printk("vGICD: unhandled write r%d=%"PRIregister" offset %#08x\n",
+               dabt.reg, *r, gicd_reg);
+        return 0;
+    }
+
+bad_width:
+    printk("vGICD: bad write width %d r%d=%"PRIregister" offset %#08x\n",
+           dabt.size, dabt.reg, *r, gicd_reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+}
+
+const struct mmio_handler_ops vgic_v2_distr_mmio_handler = {
+    .read_handler  = vgic_v2_distr_mmio_read,
+    .write_handler = vgic_v2_distr_mmio_write,
+};
+
+static int vgic_v2_vcpu_init(struct vcpu *v)
+{
+    int i;
+
+    /* For SGI and PPI the target is always this CPU */
+    for ( i = 0 ; i < 8 ; i++ )
+        v->arch.vgic.private_irqs->itargets[i] =
+              (1<<(v->vcpu_id+0))
+            | (1<<(v->vcpu_id+8))
+            | (1<<(v->vcpu_id+16))
+            | (1<<(v->vcpu_id+24));
+
+    return 0;
+}
+
+static int vgic_v2_domain_init(struct domain *d)
+{
+    register_mmio_handler(d, &vgic_v2_distr_mmio_handler, d->arch.vgic.dbase,
+                          PAGE_SIZE);
+    return 0;
+}
+
+const static struct vgic_ops vgic_v2_ops = {
+    .vcpu_init   = vgic_v2_vcpu_init,
+    .domain_init = vgic_v2_domain_init,
+};
+
+int vgic_v2_init(struct domain *d)
+{
+    register_vgic_ops(d, &vgic_v2_ops);
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 487c708..3fa0857 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -31,7 +31,10 @@
 #include <asm/gic.h>
 #include <asm/vgic.h>
 
-#define REG(n) (n)
+void register_vgic_ops(struct domain *d, const struct vgic_ops *ops)
+{
+   d->arch.vgic.handler = ops;
+}
 
 void domain_vgic_free(struct domain *d)
 {
@@ -49,6 +52,8 @@ int vcpu_vgic_init(struct vcpu *v)
 
     spin_lock_init(&v->arch.vgic.private_irqs->lock);
 
+    v->domain->arch.vgic.handler->vcpu_init(v);
+
     memset(&v->arch.vgic.pending_irqs, 0, sizeof(v->arch.vgic.pending_irqs));
     for (i = 0; i < 32; i++)
     {
@@ -56,13 +61,6 @@ int vcpu_vgic_init(struct vcpu *v)
         INIT_LIST_HEAD(&v->arch.vgic.pending_irqs[i].lr_queue);
     }
 
-    /* For SGI and PPI the target is always this CPU */
-    for ( i = 0 ; i < 8 ; i++ )
-        v->arch.vgic.private_irqs->itargets[i] =
-              (1<<(v->vcpu_id+0))
-            | (1<<(v->vcpu_id+8))
-            | (1<<(v->vcpu_id+16))
-            | (1<<(v->vcpu_id+24));
     INIT_LIST_HEAD(&v->arch.vgic.inflight_irqs);
     INIT_LIST_HEAD(&v->arch.vgic.lr_pending);
     spin_lock_init(&v->arch.vgic.lock);
@@ -76,206 +74,7 @@ int vcpu_vgic_free(struct vcpu *v)
     return 0;
 }
 
-static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
-{
-    struct hsr_dabt dabt = info->dabt;
-    struct cpu_user_regs *regs = guest_cpu_user_regs();
-    register_t *r = select_user_reg(regs, dabt.reg);
-    struct vgic_irq_rank *rank;
-    int offset = (int)(info->gpa - v->domain->arch.vgic.dbase);
-    int gicd_reg = REG(offset);
-
-    switch ( gicd_reg )
-    {
-    case GICD_CTLR:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        vgic_lock(v);
-        *r = v->domain->arch.vgic.ctlr;
-        vgic_unlock(v);
-        return 1;
-    case GICD_TYPER:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        /* No secure world support for guests. */
-        vgic_lock(v);
-        *r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
-            |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );
-        vgic_unlock(v);
-        return 1;
-    case GICD_IIDR:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        /*
-         * XXX Do we need a JEP106 manufacturer ID?
-         * Just use the physical h/w value for now
-         */
-        *r = 0x0000043b;
-        return 1;
-
-    /* Implementation defined -- read as zero */
-    case REG(0x020) ... REG(0x03c):
-        goto read_as_zero;
-
-    case GICD_IGROUPR ... GICD_IGROUPRN:
-        /* We do not implement security extensions for guests, read zero */
-        goto read_as_zero;
-
-    case GICD_ISENABLER ... GICD_ISENABLERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = rank->ienable;
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ICENABLER ... GICD_ICENABLERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = rank->ienable;
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ISPENDR ... GICD_ISPENDRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISPENDR, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = byte_read(rank->ipend, dabt.sign, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ICPENDR ... GICD_ICPENDRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICPENDR, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = byte_read(rank->ipend, dabt.sign, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ISACTIVER ... GICD_ISACTIVERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = rank->iactive;
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ICACTIVER ... GICD_ICACTIVERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = rank->iactive;
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ITARGETSR ... GICD_ITARGETSRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-
-        vgic_lock_rank(v, rank);
-        *r = rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
-                                           DABT_WORD)];
-        if ( dabt.size == DABT_BYTE )
-            *r = byte_read(*r, dabt.sign, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
-        if ( dabt.size != 0 && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-
-        vgic_lock_rank(v, rank);
-        *r = rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR,
-                                            DABT_WORD)];
-        if ( dabt.size == DABT_BYTE )
-            *r = byte_read(*r, dabt.sign, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ICFGR ... GICD_ICFGRN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR, DABT_WORD)];
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_NSACR ... GICD_NSACRN:
-        /* We do not implement security extensions for guests, read zero */
-        goto read_as_zero;
-
-    case GICD_SGIR:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        /* Write only -- read unknown */
-        *r = 0xdeadbeef;
-        return 1;
-
-    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_CPENDSGIR, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = byte_read(rank->pendsgi, dabt.sign, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_SPENDSGIR, DABT_WORD);
-        if ( rank == NULL) goto read_as_zero;
-        vgic_lock_rank(v, rank);
-        *r = byte_read(rank->pendsgi, dabt.sign, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    /* Implementation defined -- read as zero */
-    case REG(0xfd0) ... REG(0xfe4):
-        goto read_as_zero;
-
-    case GICD_ICPIDR2:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        printk("vGICD: unhandled read from ICPIDR2\n");
-        return 0;
-
-    /* Implementation defined -- read as zero */
-    case REG(0xfec) ... REG(0xffc):
-        goto read_as_zero;
-
-    /* Reserved -- read as zero */
-    case REG(0x00c) ... REG(0x01c):
-    case REG(0x040) ... REG(0x07c):
-    case REG(0x7fc):
-    case REG(0xbfc):
-    case REG(0xf04) ... REG(0xf0c):
-    case REG(0xf30) ... REG(0xfcc):
-        goto read_as_zero;
-
-    default:
-        printk("vGICD: unhandled read r%d offset %#08x\n",
-               dabt.reg, offset);
-        return 0;
-    }
-
-bad_width:
-    printk("vGICD: bad read width %d r%d offset %#08x\n",
-           dabt.size, dabt.reg, offset);
-    domain_crash_synchronous();
-    return 0;
-
-read_as_zero:
-    if ( dabt.size != DABT_WORD ) goto bad_width;
-    *r = 0;
-    return 1;
-}
-
-static void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n)
+void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n)
 {
     const unsigned long mask = r;
     struct pending_irq *p;
@@ -298,7 +97,7 @@ static void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n)
     }
 }
 
-static void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n)
+void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n)
 {
     const unsigned long mask = r;
     struct pending_irq *p;
@@ -331,250 +130,51 @@ static void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n)
     }
 }
 
-static int vgic_to_sgi(struct vcpu *v, register_t sgir)
+int vgic_to_sgi(struct vcpu *v, register_t sgir, int irqmode, int virq,
+                unsigned long vcpu_mask)
 {
     struct domain *d = v->domain;
-    int virtual_irq;
-    int filter;
     int vcpuid;
     int i;
-    unsigned long vcpu_mask = 0;
 
     ASSERT(d->max_vcpus < 8*sizeof(vcpu_mask));
 
-    filter = (sgir & GICD_SGI_TARGET_LIST_MASK);
-    virtual_irq = (sgir & GICD_SGI_INTID_MASK);
-    ASSERT( virtual_irq < 16 );
+    ASSERT( virq < 16 );
 
-    switch ( filter )
+    switch ( irqmode )
     {
-        case GICD_SGI_TARGET_LIST:
-            vcpu_mask = (sgir & GICD_SGI_TARGET_MASK) >> GICD_SGI_TARGET_SHIFT;
-            break;
-        case GICD_SGI_TARGET_OTHERS:
-            for ( i = 0; i < d->max_vcpus; i++ )
-            {
-                if ( i != current->vcpu_id && is_vcpu_running(d, i) )
-                    set_bit(i, &vcpu_mask);
-            }
-            break;
-        case GICD_SGI_TARGET_SELF:
-            set_bit(current->vcpu_id, &vcpu_mask);
-            break;
-        default:
-            gdprintk(XENLOG_WARNING, "vGICD: unhandled GICD_SGIR write %"PRIregister" with wrong TargetListFilter field\n",
-                     sgir);
-            return 0;
+    case SGI_TARGET_LIST:
+        break;
+    case SGI_TARGET_OTHERS:
+        for ( i = 0; i < d->max_vcpus; i++ )
+        {
+            if ( i != current->vcpu_id && is_vcpu_running(d, i) )
+                set_bit(i, &vcpu_mask);
+        }
+        break;
+    case SGI_TARGET_SELF:
+        set_bit(current->vcpu_id, &vcpu_mask);
+        break;
+    default:
+        gdprintk(XENLOG_WARNING, "vGICD: unhandled GICD_SGIR write %"PRIregister" with wrong mode\n",
+                 sgir);
+       return 0;
     }
 
     for_each_set_bit( vcpuid, &vcpu_mask, d->max_vcpus )
     {
         if ( !is_vcpu_running(d, vcpuid) )
         {
-            gdprintk(XENLOG_WARNING, "vGICD: GICD_SGIR write r=%"PRIregister" vcpu_mask=%lx, wrong CPUTargetList\n",
+            gdprintk(XENLOG_WARNING, " write r=%"PRIregister" vcpu_mask=%lx, wrong CPUTargetList\n",
                      sgir, vcpu_mask);
             continue;
         }
-        vgic_vcpu_inject_irq(d->vcpu[vcpuid], virtual_irq);
-    }
-    return 1;
-}
-
-static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
-{
-    struct hsr_dabt dabt = info->dabt;
-    struct cpu_user_regs *regs = guest_cpu_user_regs();
-    register_t *r = select_user_reg(regs, dabt.reg);
-    struct vgic_irq_rank *rank;
-    int offset = (int)(info->gpa - v->domain->arch.vgic.dbase);
-    int gicd_reg = REG(offset);
-    uint32_t tr;
-
-    switch ( gicd_reg )
-    {
-    case GICD_CTLR:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        /* Ignore all but the enable bit */
-        v->domain->arch.vgic.ctlr = (*r) & GICD_CTL_ENABLE;
-        return 1;
-
-    /* R/O -- write ignored */
-    case GICD_TYPER:
-    case GICD_IIDR:
-        goto write_ignore;
-
-    /* Implementation defined -- write ignored */
-    case REG(0x020) ... REG(0x03c):
-        goto write_ignore;
-
-    case GICD_IGROUPR ... GICD_IGROUPRN:
-        /* We do not implement security extensions for guests, write ignore */
-        goto write_ignore;
-
-    case GICD_ISENABLER ... GICD_ISENABLERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
-        if ( rank == NULL) goto write_ignore;
-        vgic_lock_rank(v, rank);
-        tr = rank->ienable;
-        rank->ienable |= *r;
-        vgic_unlock_rank(v, rank);
-        vgic_enable_irqs(v, (*r) & (~tr),
-                         (gicd_reg - GICD_ISENABLER) >> DABT_WORD);
-        return 1;
-
-    case GICD_ICENABLER ... GICD_ICENABLERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICENABLER, DABT_WORD);
-        if ( rank == NULL) goto write_ignore;
-        vgic_lock_rank(v, rank);
-        tr = rank->ienable;
-        rank->ienable &= ~*r;
-        vgic_unlock_rank(v, rank);
-        vgic_disable_irqs(v, (*r) & tr,
-                          (gicd_reg - GICD_ICENABLER) >> DABT_WORD);
-        return 1;
-
-    case GICD_ISPENDR ... GICD_ISPENDRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        printk("vGICD: unhandled %s write %#"PRIregister" to ISPENDR%d\n",
-               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_ISPENDR);
-        return 0;
-
-    case GICD_ICPENDR ... GICD_ICPENDRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        printk("vGICD: unhandled %s write %#"PRIregister" to ICPENDR%d\n",
-               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_ICPENDR);
-        return 0;
-
-    case GICD_ISACTIVER ... GICD_ISACTIVERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISACTIVER, DABT_WORD);
-        if ( rank == NULL) goto write_ignore;
-        vgic_lock_rank(v, rank);
-        rank->iactive &= ~*r;
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ICACTIVER ... GICD_ICACTIVERN:
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ICACTIVER, DABT_WORD);
-        if ( rank == NULL) goto write_ignore;
-        vgic_lock_rank(v, rank);
-        rank->iactive &= ~*r;
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ITARGETSR ... GICD_ITARGETSR + 7:
-        /* SGI/PPI target is read only */
-        goto write_ignore;
-
-    case GICD_ITARGETSR + 8 ... GICD_ITARGETSRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
-        if ( rank == NULL) goto write_ignore;
-        vgic_lock_rank(v, rank);
-        if ( dabt.size == DABT_WORD )
-            rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
-                                          DABT_WORD)] = *r;
-        else
-            byte_write(&rank->itargets[REG_RANK_INDEX(8,
-                        gicd_reg - GICD_ITARGETSR, DABT_WORD)], *r, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 8, gicd_reg - GICD_IPRIORITYR, DABT_WORD);
-        if ( rank == NULL) goto write_ignore;
-        vgic_lock_rank(v, rank);
-        if ( dabt.size == DABT_WORD )
-            rank->ipriority[REG_RANK_INDEX(8, gicd_reg - GICD_IPRIORITYR,
-                                           DABT_WORD)] = *r;
-        else
-            byte_write(&rank->ipriority[REG_RANK_INDEX(8,
-                       gicd_reg - GICD_IPRIORITYR, DABT_WORD)], *r, offset);
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_ICFGR: /* SGIs */
-        goto write_ignore;
-    case GICD_ICFGR + 1: /* PPIs */
-        /* It is implementation defined if these are writeable. We chose not */
-        goto write_ignore;
-    case GICD_ICFGR + 2 ... GICD_ICFGRN: /* SPIs */
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR, DABT_WORD);
-        if ( rank == NULL) goto write_ignore;
-        vgic_lock_rank(v, rank);
-        rank->icfg[REG_RANK_INDEX(2, gicd_reg - GICD_ICFGR, DABT_WORD)] = *r;
-        vgic_unlock_rank(v, rank);
-        return 1;
-
-    case GICD_NSACR ... GICD_NSACRN:
-        /* We do not implement security extensions for guests, write ignore */
-        goto write_ignore;
-
-    case GICD_SGIR:
-        if ( dabt.size != 2 )
-            goto bad_width;
-        return vgic_to_sgi(v, *r);
-
-    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        printk("vGICD: unhandled %s write %#"PRIregister" to ICPENDSGIR%d\n",
-               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_CPENDSGIR);
-        return 0;
-
-    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
-        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-        printk("vGICD: unhandled %s write %#"PRIregister" to ISPENDSGIR%d\n",
-               dabt.size ? "word" : "byte", *r, gicd_reg - GICD_SPENDSGIR);
-        return 0;
-
-    /* Implementation defined -- write ignored */
-    case REG(0xfd0) ... REG(0xfe4):
-        goto write_ignore;
-
-    /* R/O -- write ignore */
-    case GICD_ICPIDR2:
-        goto write_ignore;
-
-    /* Implementation defined -- write ignored */
-    case REG(0xfec) ... REG(0xffc):
-        goto write_ignore;
-
-    /* Reserved -- write ignored */
-    case REG(0x00c) ... REG(0x01c):
-    case REG(0x040) ... REG(0x07c):
-    case REG(0x7fc):
-    case REG(0xbfc):
-    case REG(0xf04) ... REG(0xf0c):
-    case REG(0xf30) ... REG(0xfcc):
-        goto write_ignore;
-
-    default:
-        printk("vGICD: unhandled write r%d=%"PRIregister" offset %#08x\n",
-               dabt.reg, *r, offset);
-        return 0;
+        vgic_vcpu_inject_irq(d->vcpu[vcpuid], virq);
     }
 
-bad_width:
-    printk("vGICD: bad write width %d r%d=%"PRIregister" offset %#08x\n",
-           dabt.size, dabt.reg, *r, offset);
-    domain_crash_synchronous();
-    return 0;
-
-write_ignore:
-    if ( dabt.size != DABT_WORD ) goto bad_width;
     return 1;
 }
 
-const struct mmio_handler_ops vgic_distr_mmio_handler = {
-    .read_handler  = vgic_distr_mmio_read,
-    .write_handler = vgic_distr_mmio_write,
-};
-
 struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
 {
     struct pending_irq *n;
@@ -654,7 +254,6 @@ out:
 int domain_vgic_init(struct domain *d)
 {
     int i;
-
     d->arch.vgic.ctlr = 0;
 
     /* Currently nr_lines in vgic and gic doesn't have the same meanings
@@ -665,20 +264,34 @@ int domain_vgic_init(struct domain *d)
     else
         d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
 
+    if ( gic_hw_version() == GIC_V2 )
+        vgic_v2_init(d);
+    else
+        panic("No VGIC found\n");
+
     d->arch.vgic.shared_irqs =
         xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
+    if ( d->arch.vgic.shared_irqs == NULL )
+        return -ENOMEM;
+
+    for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
+        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
+
     d->arch.vgic.pending_irqs =
         xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
-    for (i=0; i<d->arch.vgic.nr_lines; i++)
+    if ( d->arch.vgic.pending_irqs == NULL )
+    {
+        xfree(d->arch.vgic.shared_irqs);
+        return -ENOMEM;
+    }
+
+    for ( i = 0; i < d->arch.vgic.nr_lines; i++)
     {
         INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
         INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
     }
-    for (i=0; i<DOMAIN_NR_RANKS(d); i++)
-        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
 
-    register_mmio_handler(d, &vgic_distr_mmio_handler,
-                          d->arch.vgic.dbase, PAGE_SIZE);
+    d->arch.vgic.handler->domain_init(d);
 
     return 0;
 }
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index fe84ce5..8a00aa1 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -131,6 +131,8 @@ struct arch_domain
     } virt_timer_base;
 
     struct {
+        /* GIC HW version specific vGIC driver handler */
+        const struct vgic_ops *handler;
         /*
          * Covers access to other members of this struct _except_ for
          * shared_irqs where each member contains its own locking.
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 9bcd0c3..aa4e94d 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -29,6 +29,13 @@ struct vgic_irq_rank {
     uint32_t itargets[8];
 };
 
+struct vgic_ops {
+    /* Initialize vGIC */
+    int (*vcpu_init)(struct vcpu *v);
+    /* Domain specific initialization of vGIC */
+    int (*domain_init)(struct domain *d);
+};
+
 /* Number of ranks of interrupt registers for a domain */
 #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
 
@@ -106,6 +113,13 @@ extern void domain_vgic_free(struct domain *d);
 extern int vcpu_vgic_init(struct vcpu *v);
 extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
 extern void vgic_clear_pending_irqs(struct vcpu *v);
+extern int vgic_to_sgi(struct vcpu *v, register_t sgir, int irqmode, int virq,
+                       unsigned long vcpu_mask);
+
+extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
+extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
+extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
+int vgic_v2_init(struct domain *d);
 
 extern int vcpu_vgic_free(struct vcpu *v);
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (11 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 12/16] xen/arm: split vgic driver into generic and vgic-v2 driver vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-27 19:47   ` Julien Grall
  2014-06-02 17:33   ` Stefano Stabellini
  2014-05-26 10:26 ` [PATCH v4 14/16] xen/arm: Add virtual GICv3 support vijay.kilari
                   ` (3 subsequent siblings)
  16 siblings, 2 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Add support for GIC v3 specification System register access(SRE)
is enabled to access cpu and virtual interface regiseters based
on kernel GICv3 driver.

This patch adds only basic v3 support.
Does not support Interrupt Translation support (ITS)

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/Makefile             |    1 +
 xen/arch/arm/gic-v3.c             | 1125 +++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/domain.h      |    8 +
 xen/include/asm-arm/gic.h         |   20 +
 xen/include/asm-arm/gic_v3_defs.h |  157 ++++++
 xen/include/asm-arm/processor.h   |   14 +
 xen/include/xen/lib.h             |    2 +
 7 files changed, 1327 insertions(+)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 20f59f4..1684c09 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -11,6 +11,7 @@ obj-y += domctl.o
 obj-y += sysctl.o
 obj-y += domain_build.o
 obj-y += gic.o gic-v2.o
+obj-$(CONFIG_ARM_64) += gic-v3.o
 obj-y += io.o
 obj-y += irq.o
 obj-y += kernel.o
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
new file mode 100644
index 0000000..7543fb4
--- /dev/null
+++ b/xen/arch/arm/gic-v3.c
@@ -0,0 +1,1125 @@
+/*
+ * xen/arch/arm/gic-v3.c
+ *
+ * ARM Generic Interrupt Controller support v3 version
+ * based on xen/arch/arm/gic-v2.c and kernel GICv3 driver
+ *
+ * Copyright (C) 2012,2013 - ARM Ltd
+ * Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>, Cavium Inc
+ * ported to Xen
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/config.h>
+#include <xen/lib.h>
+#include <xen/init.h>
+#include <xen/cpu.h>
+#include <xen/mm.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/errno.h>
+#include <xen/delay.h>
+#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
+#include <asm/p2m.h>
+#include <asm/domain.h>
+#include <asm/platform.h>
+#include <asm/io.h>
+#include <asm/device.h>
+#include <asm/gic.h>
+#include <asm/gic_v3_defs.h>
+
+struct rdist_region {
+    paddr_t base;
+    paddr_t size;
+    void __iomem *map_base;
+};
+
+/* Global state */
+static struct {
+    paddr_t dbase;            /* Address of distributor registers */
+    paddr_t dbase_size;
+    void __iomem *map_dbase;  /* Mapped address of distributor registers */
+    struct rdist_region *rdist_regions;
+    uint32_t  rdist_stride;
+    unsigned int rdist_count; /* Number of rdist regions count */
+    unsigned int nr_priorities;
+    spinlock_t lock;
+}gicv3;
+
+static struct gic_info gicv3_info;
+
+/* per-cpu re-distributor base */
+static DEFINE_PER_CPU(void __iomem*, rbase);
+
+#define GICD                   (gicv3.map_dbase)
+#define GICD_RDIST_BASE        (this_cpu(rbase))
+#define GICD_RDIST_SGI_BASE    (GICD_RDIST_BASE + SZ_64K)
+
+/*
+ * Saves all 16(Max) LR registers. Though number of LRs implemented
+ * is implementation specific.
+ */
+static inline void gicv3_save_lr(int nr_lrs, struct vcpu *v)
+{
+    /* Fall through for all the cases */
+    switch ( nr_lrs )
+    {
+    case 16:
+        v->arch.gic.v3.lr[15] = READ_SYSREG(ICH_LR15_EL2);
+    case 15:
+        v->arch.gic.v3.lr[14] = READ_SYSREG(ICH_LR14_EL2);
+    case 14:
+        v->arch.gic.v3.lr[13] = READ_SYSREG(ICH_LR13_EL2);
+    case 13:
+        v->arch.gic.v3.lr[12] = READ_SYSREG(ICH_LR12_EL2);
+    case 12:
+        v->arch.gic.v3.lr[11] = READ_SYSREG(ICH_LR11_EL2);
+    case 11:
+        v->arch.gic.v3.lr[10] = READ_SYSREG(ICH_LR10_EL2);
+    case 10:
+        v->arch.gic.v3.lr[9] = READ_SYSREG(ICH_LR9_EL2);
+    case 9:
+        v->arch.gic.v3.lr[8] = READ_SYSREG(ICH_LR8_EL2);
+    case 8:
+        v->arch.gic.v3.lr[7] = READ_SYSREG(ICH_LR7_EL2);
+    case 7:
+        v->arch.gic.v3.lr[6] = READ_SYSREG(ICH_LR6_EL2);
+    case 6:
+        v->arch.gic.v3.lr[5] = READ_SYSREG(ICH_LR5_EL2);
+    case 5:
+        v->arch.gic.v3.lr[4] = READ_SYSREG(ICH_LR4_EL2);
+    case 4:
+        v->arch.gic.v3.lr[3] = READ_SYSREG(ICH_LR3_EL2);
+    case 3:
+        v->arch.gic.v3.lr[2] = READ_SYSREG(ICH_LR2_EL2);
+    case 2:
+        v->arch.gic.v3.lr[1] = READ_SYSREG(ICH_LR1_EL2);
+    case 1:
+         v->arch.gic.v3.lr[0] = READ_SYSREG(ICH_LR0_EL2);
+         break;
+    default:
+         BUG();
+    }
+}
+
+/*
+ * Restores all 16(Max) LR registers. Though number of LRs implemented
+ * is implementation specific.
+ */
+static inline void gicv3_restore_lr(int nr_lrs, const struct vcpu *v)
+{
+    /* Fall through for all the cases */
+    switch ( nr_lrs )
+    {
+    case 16:
+        WRITE_SYSREG(v->arch.gic.v3.lr[15], ICH_LR15_EL2);
+    case 15:
+        WRITE_SYSREG(v->arch.gic.v3.lr[14], ICH_LR14_EL2);
+    case 14:
+        WRITE_SYSREG(v->arch.gic.v3.lr[13], ICH_LR13_EL2);
+    case 13:
+        WRITE_SYSREG(v->arch.gic.v3.lr[12], ICH_LR12_EL2);
+    case 12:
+        WRITE_SYSREG(v->arch.gic.v3.lr[11], ICH_LR11_EL2);
+    case 11:
+        WRITE_SYSREG(v->arch.gic.v3.lr[10], ICH_LR10_EL2);
+    case 10:
+        WRITE_SYSREG(v->arch.gic.v3.lr[9], ICH_LR9_EL2);
+    case 9:
+        WRITE_SYSREG(v->arch.gic.v3.lr[8], ICH_LR8_EL2);
+    case 8:
+        WRITE_SYSREG(v->arch.gic.v3.lr[7], ICH_LR7_EL2);
+    case 7:
+        WRITE_SYSREG(v->arch.gic.v3.lr[6], ICH_LR6_EL2);
+    case 6:
+        WRITE_SYSREG(v->arch.gic.v3.lr[5], ICH_LR5_EL2);
+    case 5:
+        WRITE_SYSREG(v->arch.gic.v3.lr[4], ICH_LR4_EL2);
+    case 4:
+        WRITE_SYSREG(v->arch.gic.v3.lr[3], ICH_LR3_EL2);
+    case 3:
+        WRITE_SYSREG(v->arch.gic.v3.lr[2], ICH_LR2_EL2);
+    case 2:
+        WRITE_SYSREG(v->arch.gic.v3.lr[1], ICH_LR1_EL2);
+    case 1:
+        WRITE_SYSREG(v->arch.gic.v3.lr[0], ICH_LR0_EL2);
+         break;
+    default:
+         BUG();
+    }
+}
+
+static uint64_t gicv3_ich_read_lr(int lr)
+{
+    switch ( lr )
+    {
+    case 0: return READ_SYSREG(ICH_LR0_EL2);
+    case 1: return READ_SYSREG(ICH_LR1_EL2);
+    case 2: return READ_SYSREG(ICH_LR2_EL2);
+    case 3: return READ_SYSREG(ICH_LR3_EL2);
+    case 4: return READ_SYSREG(ICH_LR4_EL2);
+    case 5: return READ_SYSREG(ICH_LR5_EL2);
+    case 6: return READ_SYSREG(ICH_LR6_EL2);
+    case 7: return READ_SYSREG(ICH_LR7_EL2);
+    case 8: return READ_SYSREG(ICH_LR8_EL2);
+    case 9: return READ_SYSREG(ICH_LR9_EL2);
+    case 10: return READ_SYSREG(ICH_LR10_EL2);
+    case 11: return READ_SYSREG(ICH_LR11_EL2);
+    case 12: return READ_SYSREG(ICH_LR12_EL2);
+    case 13: return READ_SYSREG(ICH_LR13_EL2);
+    case 14: return READ_SYSREG(ICH_LR14_EL2);
+    case 15: return READ_SYSREG(ICH_LR15_EL2);
+    default:
+        BUG();
+    }
+}
+
+static void gicv3_ich_write_lr(int lr, uint64_t val)
+{
+    switch ( lr )
+    {
+    case 0:
+        WRITE_SYSREG(val, ICH_LR0_EL2);
+        break;
+    case 1:
+        WRITE_SYSREG(val, ICH_LR1_EL2);
+        break;
+    case 2:
+        WRITE_SYSREG(val, ICH_LR2_EL2);
+        break;
+    case 3:
+        WRITE_SYSREG(val, ICH_LR3_EL2);
+        break;
+    case 4:
+        WRITE_SYSREG(val, ICH_LR4_EL2);
+        break;
+    case 5:
+        WRITE_SYSREG(val, ICH_LR5_EL2);
+        break;
+    case 6:
+        WRITE_SYSREG(val, ICH_LR6_EL2);
+        break;
+    case 7:
+        WRITE_SYSREG(val, ICH_LR7_EL2);
+        break;
+    case 8:
+        WRITE_SYSREG(val, ICH_LR8_EL2);
+        break;
+    case 9:
+        WRITE_SYSREG(val, ICH_LR9_EL2);
+        break;
+    case 10:
+        WRITE_SYSREG(val, ICH_LR10_EL2);
+        break;
+    case 11:
+        WRITE_SYSREG(val, ICH_LR11_EL2);
+        break;
+    case 12:
+        WRITE_SYSREG(val, ICH_LR12_EL2);
+        break;
+    case 13:
+        WRITE_SYSREG(val, ICH_LR13_EL2);
+        break;
+    case 14:
+        WRITE_SYSREG(val, ICH_LR14_EL2);
+        break;
+    case 15:
+        WRITE_SYSREG(val, ICH_LR15_EL2);
+        break;
+    default:
+        return;
+    }
+}
+
+/*
+ * System Register Enable (SRE). Enable to access CPU & Virtual
+ * interface registers as system registers in EL2
+ */
+static void gicv3_enable_sre(void)
+{
+    uint32_t val;
+
+    val = READ_SYSREG32(ICC_SRE_EL2);
+    val |= GICC_SRE_EL2_SRE | GICC_SRE_EL2_ENEL1 |
+           GICC_SRE_EL2_DFB | GICC_SRE_EL2_DIB;
+
+    WRITE_SYSREG32(val, ICC_SRE_EL2);
+    isb();
+}
+
+/* Wait for completion of a distributor change */
+static void gicv3_do_wait_for_rwp(void __iomem *base)
+{
+    uint32_t val;
+    bool_t timeout = 0;
+    s_time_t deadline = NOW() + MILLISECS(1000);
+
+    do {
+        val = readl_relaxed(base + GICD_CTLR);
+        if ( !(val & GICD_CTLR_RWP) )
+            break;
+        if ( NOW() > deadline )
+        {
+            timeout = 1;
+            break;
+        }
+        cpu_relax();
+        udelay(1);
+    } while ( 1 );
+
+    if ( timeout )
+        dprintk(XENLOG_ERR, "RWP timeout\n");
+}
+
+static void gicv3_dist_wait_for_rwp(void)
+{
+    gicv3_do_wait_for_rwp(GICD);
+}
+
+static void gicv3_redist_wait_for_rwp(void)
+{
+    gicv3_do_wait_for_rwp(GICD_RDIST_BASE);
+}
+
+static void gicv3_wait_for_rwp(int irq)
+{
+    if ( irq < NR_LOCAL_IRQS )
+         gicv3_redist_wait_for_rwp();
+    else
+         gicv3_dist_wait_for_rwp();
+}
+
+static unsigned int gicv3_get_cpu_from_mask(const cpumask_t *cpumask)
+{
+    unsigned int cpu;
+    cpumask_t possible_mask;
+
+    cpumask_and(&possible_mask, cpumask, &cpu_possible_map);
+    cpu = cpumask_any(&possible_mask);
+
+    return cpu;
+}
+
+static void restore_aprn_regs(const union gic_state_data *d)
+{
+    /* Write APRn register based on number of priorities
+       platform has implemented */
+    switch ( gicv3.nr_priorities )
+    {
+    case 7:
+        WRITE_SYSREG32(d->v3.apr0[2], ICH_AP0R2_EL2);
+        WRITE_SYSREG32(d->v3.apr1[2], ICH_AP1R2_EL2);
+        /* Fall through */
+    case 6:
+        WRITE_SYSREG32(d->v3.apr0[1], ICH_AP0R1_EL2);
+        WRITE_SYSREG32(d->v3.apr1[1], ICH_AP1R1_EL2);
+        /* Fall through */
+    case 5:
+        WRITE_SYSREG32(d->v3.apr0[0], ICH_AP0R0_EL2);
+        WRITE_SYSREG32(d->v3.apr1[0], ICH_AP1R0_EL2);
+        break;
+    default:
+        BUG();
+    }
+}
+
+static void save_aprn_regs(union gic_state_data *d)
+{
+    /* Read APRn register based on number of priorities
+       platform has implemented */
+    switch ( gicv3.nr_priorities )
+    {
+    case 7:
+        d->v3.apr0[2] = READ_SYSREG32(ICH_AP0R2_EL2);
+        d->v3.apr1[2] = READ_SYSREG32(ICH_AP1R2_EL2);
+        /* Fall through */
+    case 6:
+        d->v3.apr0[1] = READ_SYSREG32(ICH_AP0R1_EL2);
+        d->v3.apr1[1] = READ_SYSREG32(ICH_AP1R1_EL2);
+        /* Fall through */
+    case 5:
+        d->v3.apr0[0] = READ_SYSREG32(ICH_AP0R0_EL2);
+        d->v3.apr1[0] = READ_SYSREG32(ICH_AP1R0_EL2);
+        break;
+    default:
+        BUG();
+    }
+}
+
+static void gicv3_save_state(struct vcpu *v)
+{
+
+    /* No need for spinlocks here because interrupts are disabled around
+     * this call and it only accesses struct vcpu fields that cannot be
+     * accessed simultaneously by another pCPU.
+     */
+    dsb(sy);
+    gicv3_save_lr(gicv3_info.nr_lrs, v);
+    save_aprn_regs(&v->arch.gic);
+    v->arch.gic.v3.vmcr = READ_SYSREG32(ICH_VMCR_EL2);
+}
+
+static void gicv3_restore_state(const struct vcpu *v)
+{
+    gicv3_restore_lr(gicv3_info.nr_lrs, v);
+    restore_aprn_regs(&v->arch.gic);
+    WRITE_SYSREG32(v->arch.gic.v3.vmcr, ICH_VMCR_EL2);
+}
+
+static void gicv3_dump_state(const struct vcpu *v)
+{
+    int i;
+
+    if ( v == current )
+    {
+        for ( i = 0; i < gicv3_info.nr_lrs; i++ )
+            printk("   HW_LR[%d]=%lx\n", i, gicv3_ich_read_lr(i));
+    }
+    else
+    {
+        for ( i = 0; i < gicv3_info.nr_lrs; i++ )
+            printk("   VCPU_LR[%d]=%lx\n", i, v->arch.gic.v3.lr[i]);
+    }
+}
+
+static void gicv3_enable_irq(struct irq_desc *irqd)
+{
+    int irq = irqd->irq;
+
+    /* Enable routing */
+    if ( irq < NR_GIC_LOCAL_IRQS )
+        writel_relaxed((1u << irq), GICD_RDIST_SGI_BASE + GICR_ISENABLER0);
+    else
+        writel_relaxed((1u << (irq % 32)),
+                        GICD + GICD_ISENABLER + (irq / 32) * 4);
+
+    gicv3_wait_for_rwp(irq);
+}
+
+static void gicv3_disable_irq(struct irq_desc *irqd)
+{
+    /* Disable routing */
+    if ( irqd->irq < NR_GIC_LOCAL_IRQS )
+        writel_relaxed((1u << irqd->irq),
+                       GICD_RDIST_SGI_BASE + GICR_ICENABLER0);
+    else
+        writel_relaxed(1u << (irqd->irq % 32),
+                       GICD + GICD_ICENABLER + ((irqd->irq / 32) * 4));
+}
+
+static void gicv3_eoi_irq(struct irq_desc *irqd)
+{
+    /* Lower the priority */
+    WRITE_SYSREG32(irqd->irq, ICC_EOIR1_EL1);
+}
+
+static void gicv3_dir_irq(int irq)
+{
+    /* Deactivate */
+    WRITE_SYSREG32(irq, ICC_DIR_EL1);
+}
+
+static unsigned int gicv3_read_irq(void)
+{
+    return (READ_SYSREG32(ICC_IAR1_EL1) & GICC_IA_IRQ);
+}
+
+static inline uint64_t gicv3_mpidr_to_affinity(uint64_t mpidr)
+{
+     return (MPIDR_AFFINITY_LEVEL(mpidr, 3) << 32 |
+             MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16 |
+             MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8  |
+             MPIDR_AFFINITY_LEVEL(mpidr, 0));
+}
+
+static void gicv3_set_irq_properties(struct irq_desc *desc,
+                                     const cpumask_t *cpu_mask,
+                                     unsigned int priority)
+{
+    unsigned long flags;
+    uint32_t cfg, edgebit;
+    uint64_t affinity;
+    void __iomem *base;
+    unsigned int cpu = gicv3_get_cpu_from_mask(cpu_mask);
+    unsigned int irq = desc->irq;
+    unsigned int type = desc->arch.type;
+
+    spin_lock_irqsave(&gicv3.lock, flags);
+
+    /* SGI's are always edge-triggered not need to call GICD_ICFGR0 */
+    if ( irq >= NR_GIC_SGI )
+    {
+        if ( irq < NR_GIC_LOCAL_IRQS)
+            base = GICD + GICD_ICFGR + (irq / 16) * 4;
+        else
+            base = GICD_RDIST_SGI_BASE + GICR_ICFGR1;
+
+        cfg = readl_relaxed(base);
+
+        edgebit = 2u << (2 * (irq % 16));
+        if ( type & DT_IRQ_TYPE_LEVEL_MASK )
+           cfg &= ~edgebit;
+        else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
+           cfg |= edgebit;
+
+        writel_relaxed(cfg, base);
+    }
+
+    affinity = gicv3_mpidr_to_affinity(cpu_logical_map(cpu));
+    /* Make sure we don't broadcast the interrupt */
+    affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
+
+    if ( irq >= NR_GIC_LOCAL_IRQS )
+        writeq_relaxed(affinity, (GICD + GICD_IROUTER + irq * 8));
+
+    /* Set priority */
+    if ( irq < NR_GIC_LOCAL_IRQS )
+        writeb_relaxed(priority, GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irq);
+    else
+        writeb_relaxed(priority, GICD + GICD_IPRIORITYR + irq);
+
+    spin_unlock_irqrestore(&gicv3.lock, flags);
+}
+
+static void __init gicv3_dist_init(void)
+{
+    uint32_t type;
+    uint32_t priority;
+    uint64_t affinity;
+    int i;
+
+    /* Disable the distributor */
+    writel_relaxed(0, GICD + GICD_CTLR);
+
+    type = readl_relaxed(GICD + GICD_TYPER);
+    gicv3_info.nr_lines = 32 * ((type & GICD_TYPE_LINES) + 1);
+
+    printk("GICv3: %d lines, (IID %8.8x).\n",
+           gicv3_info.nr_lines, readl_relaxed(GICD + GICD_IIDR));
+
+    /* Default all global IRQs to level, active low */
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 16 )
+        writel_relaxed(0, GICD + GICD_ICFGR + (i / 16) * 4);
+
+    /* Default priority for global interrupts */
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 4 )
+    {
+        priority = (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
+                    GIC_PRI_IRQ << 8 | GIC_PRI_IRQ);
+        writel_relaxed(priority, GICD + GICD_IPRIORITYR + (i / 4) * 4);
+    }
+
+    /* Disable all global interrupts */
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 32 )
+        writel_relaxed(0xffffffff, GICD + GICD_ICENABLER + (i / 32) * 4);
+
+    gicv3_dist_wait_for_rwp();
+
+    /* Turn on the distributor */
+    writel_relaxed(GICD_CTL_ENABLE | GICD_CTLR_ARE_NS |
+                GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1, GICD + GICD_CTLR);
+
+    /* Route all global IRQs to this CPU */
+    affinity = gicv3_mpidr_to_affinity(cpu_logical_map(smp_processor_id()));
+    /* Make sure we don't broadcast the interrupt */
+    affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
+
+    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i++ )
+        writeq_relaxed(affinity, GICD + GICD_IROUTER + i * 8);
+}
+
+static int gicv3_enable_redist(void)
+{
+    uint32_t val;
+    bool_t timeout = 0;
+    s_time_t deadline = NOW() + MILLISECS(1000);
+
+    /* Wake up this CPU redistributor */
+    val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
+    val &= ~GICR_WAKER_ProcessorSleep;
+    writel_relaxed(val, GICD_RDIST_BASE + GICR_WAKER);
+
+    do {
+        val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
+        if ( !(val & GICR_WAKER_ChildrenAsleep) )
+            break;
+        if ( NOW() > deadline )
+        {
+            timeout = 1;
+            break;
+        }
+        cpu_relax();
+        udelay(1);
+    } while ( timeout );
+
+    if ( timeout )
+        dprintk(XENLOG_ERR, "GICv3: Redist enable RWP timeout\n");
+
+    return 0;
+}
+
+static int __init gicv3_populate_rdist(void)
+{
+    int i;
+    uint32_t aff;
+    uint32_t reg;
+    uint64_t typer;
+    uint64_t mpidr = cpu_logical_map(smp_processor_id());
+
+    aff = gicv3_mpidr_to_affinity(mpidr);
+
+    for ( i = 0; i < gicv3.rdist_count; i++ )
+    {
+        void __iomem *ptr = gicv3.rdist_regions[i].map_base;
+
+        reg = readl_relaxed(ptr + GICR_PIDR2) & GICR_PIDR2_ARCH_MASK;
+        if ( reg  != (GICV3_GICR_PIDR2 & GICR_PIDR2_ARCH_MASK) )
+        {
+            dprintk(XENLOG_ERR,
+                    "GICv3: No redistributor present @%"PRIpaddr"\n",
+                    gicv3.rdist_regions[i].base);
+            break;
+        }
+
+        do {
+            typer = readq_relaxed(ptr + GICR_TYPER);
+
+            if ( (typer >> 32) == aff )
+            {
+                this_cpu(rbase) = ptr;
+                printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
+                        smp_processor_id(), i, ptr);
+                return 0;
+            }
+
+            ptr += gicv3.rdist_stride;
+            if ( typer & GICR_TYPER_VLPIS )
+                    ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+
+        } while ( !(typer & GICR_TYPER_LAST) );
+    }
+
+    dprintk(XENLOG_ERR, "GICv3: CPU%d: mpidr %lx has no re-distributor!\n",
+                  smp_processor_id(), mpidr);
+    return -ENODEV;
+}
+
+static int __cpuinit gicv3_cpu_init(void)
+{
+    int i;
+    uint32_t priority;
+
+    /* Register ourselves with the rest of the world */
+    if ( gicv3_populate_rdist() )
+        return -ENODEV;
+
+    if ( gicv3_enable_redist() )
+        return -ENODEV;
+
+    /* Set priority on PPI and SGI interrupts */
+    priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
+                GIC_PRI_IPI);
+    for (i = 0; i < NR_GIC_SGI; i += 4)
+        writel_relaxed(priority,
+                GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
+
+    priority = (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 | GIC_PRI_IRQ << 8 |
+                GIC_PRI_IRQ);
+    for (i = NR_GIC_SGI; i < NR_GIC_LOCAL_IRQS; i += 4)
+        writel_relaxed(priority,
+                GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
+
+    /*
+     * Disable all PPI interrupts, ensure all SGI interrupts are
+     * enabled.
+     */
+    writel_relaxed(0xffff0000, GICD_RDIST_SGI_BASE + GICR_ICENABLER0);
+    writel_relaxed(0x0000ffff, GICD_RDIST_SGI_BASE + GICR_ISENABLER0);
+
+    gicv3_redist_wait_for_rwp();
+
+    /* Enable system registers */
+    gicv3_enable_sre();
+
+    /* No priority grouping */
+    WRITE_SYSREG32(0, ICC_BPR1_EL1);
+
+    /* Set priority mask register */
+    WRITE_SYSREG32(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+
+    /* EOI drops priority too (mode 0) */
+    WRITE_SYSREG32(GICC_CTLR_EL1_EOImode_drop, ICC_CTLR_EL1);
+
+    /* Enable Group1 interrupts */
+    WRITE_SYSREG32(1, ICC_IGRPEN1_EL1);
+
+    return 0;
+}
+
+static void gicv3_cpu_disable(void)
+{
+    WRITE_SYSREG32(0, ICC_CTLR_EL1);
+}
+
+static void __cpuinit gicv3_hyp_init(void)
+{
+    uint32_t vtr;
+
+    vtr = READ_SYSREG32(ICH_VTR_EL2);
+    gicv3_info.nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
+    gicv3.nr_priorities = ((vtr >> GICH_VTR_PRIBITS_SHIFT) &
+                          GICH_VTR_PRIBITS_MASK) + 1;
+
+    ASSERT((gicv3.nr_priorities > 4 && gicv3.nr_priorities < 8));
+
+    WRITE_SYSREG32(GICH_VMCR_EOI | GICH_VMCR_VENG1, ICH_VMCR_EL2);
+    WRITE_SYSREG32(GICH_HCR_EN, ICH_HCR_EL2);
+
+    update_cpu_lr_mask();
+}
+
+/* Set up the per-CPU parts of the GIC for a secondary CPU */
+static int gicv3_secondary_cpu_init(void)
+{
+    int res;
+
+    spin_lock(&gicv3.lock);
+
+    res = gicv3_cpu_init();
+    gicv3_hyp_init();
+
+    spin_unlock(&gicv3.lock);
+
+    return res;
+}
+
+static void __cpuinit gicv3_hyp_disable(void)
+{
+    uint32_t vtr;
+
+    vtr = READ_SYSREG32(ICH_HCR_EL2);
+    vtr &= ~0x1;
+    WRITE_SYSREG32( vtr, ICH_HCR_EL2);
+}
+
+static u16 gicv3_compute_target_list(int *base_cpu, const struct cpumask *mask,
+                                     uint64_t cluster_id)
+{
+    int cpu = *base_cpu;
+    uint64_t mpidr = cpu_logical_map(cpu);
+    u16 tlist = 0;
+
+    while ( cpu < nr_cpu_ids )
+    {
+        /*
+         * If we ever get a cluster of more than 16 CPUs, just
+         * scream and skip that CPU.
+         */
+        if ( (mpidr & 0xff) >= 16 )
+        {
+            dprintk(XENLOG_WARNING, "GICv3:Cluster with more than 16's cpus\n");
+            goto out;
+        }
+        tlist |= 1 << (mpidr & 0xf);
+
+        cpu = cpumask_next(cpu, mask);
+        if ( cpu == nr_cpu_ids )
+        {
+            cpu--;
+            goto out;
+        }
+
+        mpidr = cpu_logical_map(cpu);
+        if ( cluster_id != (mpidr & ~0xffUL) ) {
+            cpu--;
+            goto out;
+        }
+    }
+out:
+    *base_cpu = cpu;
+
+    return tlist;
+}
+
+static void gicv3_send_sgi(const cpumask_t *cpumask, enum gic_sgi sgi,
+                           uint8_t mode)
+{
+    int cpu = 0;
+    uint64_t val;
+
+    dsb(sy);
+
+    for_each_cpu(cpu, cpumask)
+    {
+        uint64_t cluster_id = cpu_logical_map(cpu) & ~0xffUL;
+        u16 tlist;
+
+        /* Get targetlist for the cluster to send SGI */
+        tlist = gicv3_compute_target_list(&cpu, cpumask, cluster_id);
+
+        /*
+         * Prepare affinity path of the cluster for which SGI is generated
+         * along with SGI number
+         */
+        val = (MPIDR_AFFINITY_LEVEL(cluster_id, 3) << 48  |
+               MPIDR_AFFINITY_LEVEL(cluster_id, 2) << 32  |
+               sgi << 24                                  |
+               MPIDR_AFFINITY_LEVEL(cluster_id, 1) << 16  |
+               tlist);
+
+        WRITE_SYSREG(val, ICC_SGI1R_EL1);
+    }
+}
+
+/* Shut down the per-CPU GIC interface */
+static void gicv3_disable_interface(void)
+{
+    spin_lock(&gicv3.lock);
+
+    gicv3_cpu_disable();
+    gicv3_hyp_disable();
+
+    spin_unlock(&gicv3.lock);
+}
+
+static void gicv3_update_lr(int lr, const struct pending_irq *p,
+                            unsigned int state)
+{
+    uint64_t grp = GICH_LR_GRP1;
+    uint64_t val = 0;
+
+    BUG_ON(lr >= gicv3_info.nr_lrs);
+    BUG_ON(lr < 0);
+
+    val =  (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
+    val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
+    val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
+
+   if ( p->desc != NULL )
+       val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
+                           << GICH_LR_PHYSICAL_SHIFT);
+
+    gicv3_ich_write_lr(lr, val);
+}
+
+static void gicv3_clear_lr(int lr)
+{
+    gicv3_ich_write_lr(lr, 0);
+}
+
+static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
+{
+    uint64_t lrv;
+
+    lrv = gicv3_ich_read_lr(lr);
+
+    lr_reg->pirq = (lrv >> GICH_LR_PHYSICAL_SHIFT) & GICH_LR_PHYSICAL_MASK;
+    lr_reg->virq = (lrv >> GICH_LR_VIRTUAL_SHIFT) & GICH_LR_VIRTUAL_MASK;
+
+    lr_reg->priority  = (lrv >> GICH_LR_PRIORITY_SHIFT) & GICH_LR_PRIORITY_MASK;
+    lr_reg->state     = (lrv >> GICH_LR_STATE_SHIFT) & GICH_LR_STATE_MASK;
+    lr_reg->hw_status = (lrv >> GICH_LR_HW_SHIFT) & GICH_LR_HW_MASK;
+    lr_reg->grp       = (lrv >> GICH_LR_GRP_SHIFT) & GICH_LR_GRP_MASK;
+}
+
+static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
+{
+    uint64_t lrv = 0;
+
+    lrv = ( ((u64)(lr->pirq & GICH_LR_PHYSICAL_MASK) << GICH_LR_PHYSICAL_SHIFT)|
+        ((u64)(lr->virq & GICH_LR_VIRTUAL_MASK)  << GICH_LR_VIRTUAL_SHIFT) |
+        ((u64)(lr->priority & GICH_LR_PRIORITY_MASK) << GICH_LR_PRIORITY_SHIFT)|
+        ((u64)(lr->state & GICH_LR_STATE_MASK) << GICH_LR_STATE_SHIFT) |
+        ((u64)(lr->hw_status & GICH_LR_HW_MASK) << GICH_LR_HW_SHIFT)  |
+        ((u64)(lr->grp & GICH_LR_GRP_MASK) << GICH_LR_GRP_SHIFT) );
+
+    gicv3_ich_write_lr(lr_reg, lrv);
+}
+
+static int gicv_v3_init(struct domain *d)
+{
+    int i;
+
+    /*
+     * Domain 0 gets the hardware address.
+     * Guests get the virtual platform layout.
+     */
+    if ( d->domain_id == 0 )
+    {
+        d->arch.vgic.dbase = gicv3.dbase;
+        d->arch.vgic.dbase_size = gicv3.dbase_size;
+        for ( i = 0; i < gicv3.rdist_count; i++ )
+        {
+            d->arch.vgic.rbase[i] = gicv3.rdist_regions[i].base;
+            d->arch.vgic.rbase_size[i] = gicv3.rdist_regions[i].size;
+        }
+        d->arch.vgic.rdist_stride = gicv3.rdist_stride;
+        d->arch.vgic.rdist_count = gicv3.rdist_count;
+    }
+    else
+        d->arch.vgic.dbase = GUEST_GICD_BASE;
+
+    d->arch.vgic.nr_lines = 0;
+
+    return 0;
+}
+
+static void gicv3_hcr_status(uint32_t flag, bool_t status)
+{
+    uint32_t hcr;
+
+    hcr = READ_SYSREG32(ICH_HCR_EL2);
+    if ( status )
+        WRITE_SYSREG32(hcr | flag, ICH_HCR_EL2);
+    else
+        WRITE_SYSREG32(hcr & (~flag), ICH_HCR_EL2);
+}
+
+static unsigned int gicv3_read_vmcr_priority(void)
+{
+   return ((READ_SYSREG32(ICH_VMCR_EL2) >> GICH_VMCR_PRIORITY_SHIFT) &
+            GICH_VMCR_PRIORITY_MASK);
+}
+
+static void gicv3_irq_enable(struct irq_desc *desc)
+{
+    unsigned long flags;
+
+    spin_lock_irqsave(&gicv3.lock, flags);
+    desc->status &= ~IRQ_DISABLED;
+    dsb(sy);
+    /* Enable routing */
+    gicv3_enable_irq(desc);
+    spin_unlock_irqrestore(&gicv3.lock, flags);
+}
+
+static void gicv3_irq_disable(struct irq_desc *desc)
+{
+    unsigned long flags;
+
+    spin_lock_irqsave(&gicv3.lock, flags);
+    /* Disable routing */
+    gicv3_disable_irq(desc);
+    desc->status |= IRQ_DISABLED;
+    spin_unlock_irqrestore(&gicv3.lock, flags);
+}
+
+static unsigned int gicv3_irq_startup(struct irq_desc *desc)
+{
+    gicv3_irq_enable(desc);
+
+    return 0;
+}
+
+static void gicv3_irq_shutdown(struct irq_desc *desc)
+{
+    gicv3_irq_disable(desc);
+}
+
+static void gicv3_irq_ack(struct irq_desc *desc)
+{
+    /* No ACK -- reading IAR has done this for us */
+}
+
+static void gicv3_host_irq_end(struct irq_desc *desc)
+{
+    /* Lower the priority */
+    gicv3_eoi_irq(desc);
+    /* Deactivate */
+    gicv3_dir_irq(desc->irq);
+}
+
+static void gicv3_guest_irq_end(struct irq_desc *desc)
+{
+    /* Lower the priority of the IRQ */
+    gicv3_eoi_irq(desc);
+    /* Deactivation happens in maintenance interrupt / via GICV */
+}
+
+static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+{
+    BUG();
+}
+
+const static hw_irq_controller gicv3_host_irq_type = {
+    .typename     = "gic-v3",
+    .startup      = gicv3_irq_startup,
+    .shutdown     = gicv3_irq_shutdown,
+    .enable       = gicv3_irq_enable,
+    .disable      = gicv3_irq_disable,
+    .ack          = gicv3_irq_ack,
+    .end          = gicv3_host_irq_end,
+    .set_affinity = gicv3_irq_set_affinity,
+};
+
+const static hw_irq_controller gicv3_guest_irq_type = {
+    .typename     = "gic-v3",
+    .startup      = gicv3_irq_startup,
+    .shutdown     = gicv3_irq_shutdown,
+    .enable       = gicv3_irq_enable,
+    .disable      = gicv3_irq_disable,
+    .ack          = gicv3_irq_ack,
+    .end          = gicv3_guest_irq_end,
+    .set_affinity = gicv3_irq_set_affinity,
+};
+
+const static struct gic_hw_operations gicv3_ops = {
+    .info                = &gicv3_info,
+    .save_state          = gicv3_save_state,
+    .restore_state       = gicv3_restore_state,
+    .dump_state          = gicv3_dump_state,
+    .gicv_setup          = gicv_v3_init,
+    .gic_host_irq_type   = &gicv3_host_irq_type,
+    .gic_guest_irq_type  = &gicv3_guest_irq_type,
+    .eoi_irq             = gicv3_eoi_irq,
+    .deactivate_irq      = gicv3_dir_irq,
+    .read_irq            = gicv3_read_irq,
+    .set_irq_properties  = gicv3_set_irq_properties,
+    .send_SGI            = gicv3_send_sgi,
+    .disable_interface   = gicv3_disable_interface,
+    .update_lr           = gicv3_update_lr,
+    .update_hcr_status   = gicv3_hcr_status,
+    .clear_lr            = gicv3_clear_lr,
+    .read_lr             = gicv3_read_lr,
+    .write_lr            = gicv3_write_lr,
+    .read_vmcr_priority  = gicv3_read_vmcr_priority,
+    .secondary_init      = gicv3_secondary_cpu_init,
+};
+
+/* Set up the GIC */
+static int __init gicv3_init(struct dt_device_node *node, const void *data)
+{
+    struct rdist_region *rdist_regs;
+    int res, i;
+    uint32_t reg;
+
+    dt_device_set_used_by(node, DOMID_XEN);
+
+    res = dt_device_get_address(node, 0, &gicv3.dbase, &gicv3.dbase_size);
+    if ( res || !gicv3.dbase )
+        panic("GICv3: Cannot find a valid distributor address");
+
+    if ( (gicv3.dbase & ~PAGE_MASK) || (gicv3.dbase_size & ~PAGE_MASK) )
+        panic("GICv3:  Found unaligned distributor address %"PRIpaddr"",
+              gicv3.dbase);
+
+    gicv3.map_dbase = ioremap_nocache(gicv3.dbase, gicv3.dbase_size);
+    if ( !gicv3.map_dbase )
+        panic("GICv3: Failed to ioremap for GIC distributor\n");
+
+    reg = readl_relaxed(GICD + GICD_PIDR2) & GICD_PIDR2_ARCH_MASK;
+    if ( reg  != (GICV3_GICD_PIDR2 & GICD_PIDR2_ARCH_MASK) )
+         panic("GICv3: no distributor detected\n");
+
+    if ( !dt_property_read_u32(node, "#redistributor-regions",
+                &gicv3.rdist_count) )
+        gicv3.rdist_count = 1;
+
+    if ( gicv3.rdist_count > MAX_RDIST_COUNT )
+        panic("GICv3: Number of redistributor regions is more than \
+               %d (Increase MAX_RDIST_COUNT!!)\n", MAX_RDIST_COUNT);
+
+    rdist_regs = xzalloc_array(struct rdist_region, gicv3.rdist_count);
+    if ( !rdist_regs )
+        panic("GICv3: Failed to allocate memory for rdist regions\n");
+
+    for ( i = 0; i < gicv3.rdist_count; i++ )
+    {
+        uint64_t rdist_base, rdist_size;
+
+        res = dt_device_get_address(node, 1 + i, &rdist_base, &rdist_size);
+        if ( res || !rdist_base )
+            panic("GICv3: No rdist base found for region %d\n", i);
+
+        rdist_regs[i].base = rdist_base;
+        rdist_regs[i].size = rdist_size;
+    }
+
+    /* If stride is not set in dt. Set default to 2 * SZ_64K */
+    if ( !dt_property_read_u32(node, "redistributor-stride", &gicv3.rdist_stride) )
+        gicv3.rdist_stride = 2 * SZ_64K;
+
+    gicv3.rdist_regions= rdist_regs;
+
+    res = platform_get_irq(node, 0);
+    if ( res < 0 )
+        panic("GICv3: Cannot find the maintenance IRQ");
+    gicv3_info.maintenance_irq = res;
+
+    /* Set the GIC as the primary interrupt controller */
+    dt_interrupt_controller = node;
+
+    for ( i = 0; i < gicv3.rdist_count; i++ )
+    {
+        /* map dbase & rdist regions */
+        gicv3.rdist_regions[i].map_base =
+                ioremap_nocache(gicv3.rdist_regions[i].base,
+                                gicv3.rdist_regions[i].size);
+
+        if ( !gicv3.rdist_regions[i].map_base )
+            panic("GICv3: Failed to ioremap rdist region for region %d\n", i);
+    }
+
+    printk("GICv3 initialization:\n"
+           "      gic_dist_addr=%"PRIpaddr"\n"
+           "      gic_dist_size=%"PRIpaddr"\n"
+           "      gic_dist_mapaddr=%p\n"
+           "      gic_rdist_regions=%d\n"
+           "      gic_rdist_stride=%x\n"
+           "      gic_rdist_base=%"PRIpaddr"\n"
+           "      gic_rdist_base_size=%"PRIpaddr"\n"
+           "      gic_rdist_base_mapaddr=%p\n"
+           "      gic_maintenance_irq=%u\n",
+           gicv3.dbase, gicv3.dbase_size, gicv3.map_dbase, gicv3.rdist_count,
+           gicv3.rdist_stride, gicv3.rdist_regions[0].base,
+           gicv3.rdist_regions[0].size, gicv3.rdist_regions[0].map_base,
+           gicv3_info.maintenance_irq);
+
+    spin_lock_init(&gicv3.lock);
+
+    spin_lock(&gicv3.lock);
+
+    gicv3_dist_init();
+    res = gicv3_cpu_init();
+    gicv3_hyp_init();
+
+    gicv3_info.hw_version = GIC_V3;
+    /* Register hw ops*/
+    register_gic_ops(&gicv3_ops);
+
+    spin_unlock(&gicv3.lock);
+
+    return res;
+}
+
+static const char * const gicv3_dt_compat[] __initconst =
+{
+    DT_MATCH_GIC_V3_STRING1,
+    NULL
+};
+
+DT_DEVICE_START(gicv3, "GICv3", DEVICE_GIC)
+        .compatible = gicv3_dt_compat,
+        .init = gicv3_init,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 8a00aa1..66e246e 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -154,6 +154,14 @@ struct arch_domain
         /* Base address for guest GIC */
         paddr_t dbase; /* Distributor base address */
         paddr_t cbase; /* CPU base address */
+#ifdef CONFIG_ARM_64
+        /* GIC V3 addressing */
+        paddr_t dbase_size; /* Distributor base size */
+        paddr_t rbase[MAX_RDIST_COUNT];      /* Re-Distributor base address */
+        paddr_t rbase_size[MAX_RDIST_COUNT]; /* Re-Distributor size */
+        uint32_t rdist_stride;               /* Re-Distributor stride */
+        int rdist_count;                     /* No. of Re-Distributors */
+#endif
     } vgic;
 
     struct vuart {
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 2aadcb6..b40beb5 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -18,6 +18,10 @@
 #ifndef __ASM_ARM_GIC_H__
 #define __ASM_ARM_GIC_H__
 
+#define NR_GIC_LOCAL_IRQS  NR_LOCAL_IRQS
+#define NR_GIC_SGI         16
+#define MAX_RDIST_COUNT    4
+
 #define GICD_CTLR       (0x000)
 #define GICD_TYPER      (0x004)
 #define GICD_IIDR       (0x008)
@@ -150,6 +154,20 @@
 #define DT_MATCH_GIC_V2 DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V2_STRING_1), \
                         DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V2_STRING_2)
 
+#define DT_MATCH_GIC_V3_STRING1      "arm,gic-v3"
+
+#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V3_STRING1);
+
+/*
+ * GICv3 registers that needs to be saved/restored
+ */
+struct gic_v3 {
+    uint32_t hcr, vmcr;
+    uint32_t apr0[4];
+    uint32_t apr1[4];
+    uint64_t lr[16];
+};
+
 /*
  * GICv2 register that needs to be saved/restored
  * on VCPU context switch
@@ -166,6 +184,7 @@ struct gic_v2 {
  */ 
 union gic_state_data {
     struct gic_v2 v2;
+    struct gic_v3 v3;
 };
 
 /*
@@ -251,6 +270,7 @@ void gic_clear_lrs(struct vcpu *v);
 
 enum gic_version {
     GIC_V2 = 2,
+    GIC_V3 = 3,
 };
 
 struct gic_info {
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
new file mode 100644
index 0000000..6f393aa
--- /dev/null
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -0,0 +1,157 @@
+/*
+ * ARM Generic Interrupt Controller v3 definitions
+ *
+ * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>
+ * Copyright (c) 2014 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ASM_ARM_GIC_V3_DEFS_H__
+#define __ASM_ARM_GIC_V3_DEFS_H__
+
+/*
+ * Additional registers defined in GIC v3.
+ * Common GICD registers are defined in gic.h
+ */
+ 
+#define GICD_STATUSR                 (0x010)
+#define GICD_SETSPI_NSR              (0x040)
+#define GICD_CLRSPI_NSR              (0x048)
+#define GICD_SETSPI_SR               (0x050)
+#define GICD_CLRSPI_SR               (0x058)
+#define GICD_IROUTER                 (0x6000)
+#define GICD_IROUTER31               (0x60F8)
+#define GICD_IROUTER32               (0x6100)
+#define GICD_IROUTERN                (0x7FF8)
+#define GICD_PIDR0                   (0xFFE0)
+#define GICD_PIDR1                   (0xFFE4)
+#define GICD_PIDR2                   (0xFFE8)
+#define GICD_PIDR3                   (0xFFEC)
+#define GICD_PIDR4                   (0xFFD0)
+#define GICD_PIDR5                   (0xFFD4)
+#define GICD_PIDR7                   (0xFFDC)
+
+#define GICC_SRE_EL2_SRE             (1UL << 0)
+#define GICC_SRE_EL2_DFB             (1UL << 1)
+#define GICC_SRE_EL2_DIB             (1UL << 2)
+#define GICC_SRE_EL2_ENEL1           (1UL << 3)
+
+#define GICD_CTLR_RWP                (1UL << 31)
+#define GICD_CTLR_ARE_NS             (1U << 4)
+#define GICD_CTLR_ENABLE_G1A         (1U << 1)
+#define GICD_CTLR_ENABLE_G1          (1U << 0)
+#define GICD_IROUTER_SPI_MODE_ANY    (1UL << 31)
+
+#define GICC_CTLR_EL1_EOImode_drop   (1U << 1)
+
+#define GICR_WAKER_ProcessorSleep    (1U << 1)
+#define GICR_WAKER_ChildrenAsleep    (1U << 2)
+
+#define GICV3_GICD_PIDR0             (0x92)
+#define GICV3_GICD_PIDR1             (0xb4)
+#define GICV3_GICD_PIDR2             (0x3b)
+#define GICV3_GICD_PIDR4             (0x04)
+#define GICD_PIDR2_ARCH_MASK         (0xf0)
+
+#define GICV3_GICR_PIDR0             (0x93)
+#define GICV3_GICR_PIDR1             GICV3_GICD_PIDR1
+#define GICV3_GICR_PIDR2             GICV3_GICD_PIDR2
+#define GICV3_GICR_PIDR4             GICV3_GICD_PIDR4
+#define GICR_PIDR2_ARCH_MASK         GICD_PIDR2_ARCH_MASK
+#define GICR_SYNCR_NOT_BUSY          1
+/* 
+ * Implementation defined value JEP106?
+ * use physical hw value for now
+ */
+#define GICV3_GICD_IIDR_VAL          0x34c
+#define GICV3_GICR_IIDR_VAL          GICV3_GICD_IIDR_VAL
+
+#define GICR_CTLR                    (0x0000)
+#define GICR_IIDR                    (0x0004)
+#define GICR_TYPER                   (0x0008)
+#define GICR_STATUSR                 (0x0010)
+#define GICR_WAKER                   (0x0014)
+#define GICR_SETLPIR                 (0x0040)
+#define GICR_CLRLPIR                 (0x0048)
+#define GICR_PROPBASER               (0x0070)
+#define GICR_PENDBASER               (0x0078)
+#define GICR_INVLPIR                 (0x00A0)
+#define GICR_INVALLR                 (0x00B0)
+#define GICR_SYNCR                   (0x00C0)
+#define GICR_MOVLPIR                 (0x100)
+#define GICR_MOVALLR                 (0x0110)
+#define GICR_PIDR0                   GICD_PIDR0
+#define GICR_PIDR1                   GICD_PIDR1
+#define GICR_PIDR2                   GICD_PIDR2
+#define GICR_PIDR3                   GICD_PIDR3
+#define GICR_PIDR4                   GICD_PIDR4
+#define GICR_PIDR5                   GICD_PIDR5
+#define GICR_PIDR7                   GICD_PIDR7
+
+/* GICR for SGI's & PPI's */
+
+#define GICR_IGROUPR0                (0x0080)
+#define GICR_IGRPMODR0               (0x0F80)
+#define GICR_ISENABLER0              (0x0100)
+#define GICR_ICENABLER0              (0x0180)
+#define GICR_ISPENDR0                (0x0200)
+#define GICR_ICPENDR0                (0x0280)
+#define GICR_ISACTIVER0              (0x0300)
+#define GICR_ICACTIVER0              (0x0380)
+#define GICR_IPRIORITYR0             (0x0400)
+#define GICR_IPRIORITYR7             (0x041C)
+#define GICR_ICFGR0                  (0x0C00)
+#define GICR_ICFGR1                  (0x0C04)
+#define GICR_NSACR                   (0x0E00)
+
+#define GICR_TYPER_PLPIS             (1U << 0)
+#define GICR_TYPER_VLPIS             (1U << 1)
+#define GICR_TYPER_LAST              (1U << 4)
+
+#define DEFAULT_PMR_VALUE            0xff
+
+#define GICH_VMCR_EOI                (1 << 9)
+#define GICH_VMCR_VENG1              (1 << 1)
+
+#define GICH_LR_VIRTUAL_MASK         0xffff
+#define GICH_LR_VIRTUAL_SHIFT        0
+#define GICH_LR_PHYSICAL_MASK        0x3ff
+#define GICH_LR_PHYSICAL_SHIFT       32
+#define GICH_LR_STATE_MASK           0x3
+#define GICH_LR_STATE_SHIFT          62
+#define GICH_LR_PRIORITY_MASK        0xff
+#define GICH_LR_PRIORITY_SHIFT       48
+#define GICH_LR_HW_MASK              0x1
+#define GICH_LR_HW_SHIFT             61
+#define GICH_LR_GRP_MASK             0x1
+#define GICH_LR_GRP_SHIFT            60
+#define GICH_LR_MAINTENANCE_IRQ      (1UL<<41)
+#define GICH_LR_GRP1                 (1UL<<60)
+#define GICH_LR_HW                   (1UL<<61)
+
+#define GICH_VTR_NRLRGS              0x3f
+#define GICH_VTR_PRIBITS_MASK        0x7
+#define GICH_VTR_PRIBITS_SHIFT       29
+
+#define GICH_VMCR_PRIORITY_MASK      0xff
+#define GICH_VMCR_PRIORITY_SHIFT     24
+
+#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 3662749..5978b8a 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -17,6 +17,20 @@
 #define MPIDR_HWID_MASK     _AC(0xffffff,U)
 #define MPIDR_INVALID       (~MPIDR_HWID_MASK)
 
+/*
+ * Macros to extract affinity level. picked from kernel
+ */
+
+#define MPIDR_LEVEL_BITS_SHIFT  3
+#define MPIDR_LEVEL_BITS        (1 << MPIDR_LEVEL_BITS_SHIFT)
+#define MPIDR_LEVEL_MASK        ((1 << MPIDR_LEVEL_BITS) - 1)
+
+#define MPIDR_LEVEL_SHIFT(level) \
+         (((1 << level) >> 1) << MPIDR_LEVEL_BITS_SHIFT)
+
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+         ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
+
 /* TTBCR Translation Table Base Control Register */
 #define TTBCR_EAE    _AC(0x80000000,U)
 #define TTBCR_N_MASK _AC(0x07,U)
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 1369b2b..499a798 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -67,6 +67,8 @@ do {                                                            \
 
 #define reserve_bootmem(_p,_l) ((void)0)
 
+#define SZ_64K  0x00010000
+
 struct domain;
 
 void cmdline_parse(const char *cmdline);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (12 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 13/16] xen/arm: Add support for GIC v3 vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-06-02 15:50   ` Stefano Stabellini
  2014-06-02 16:10   ` Julien Grall
  2014-05-26 10:26 ` [PATCH v4 15/16] xen/arm: Update Dom0 GIC dt node with GICv3 information vijay.kilari
                   ` (2 subsequent siblings)
  16 siblings, 2 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Add virtual GICv3 driver support

This patch adds only basic v3 support.
Does not support Interrupt Translation support (ITS)

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/Makefile      |    1 +
 xen/arch/arm/vgic-v2.c     |    8 +-
 xen/arch/arm/vgic-v3.c     |  895 ++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic.c        |   13 +-
 xen/include/asm-arm/vgic.h |   13 +-
 5 files changed, 923 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 1684c09..ba61e58 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -28,6 +28,7 @@ obj-y += smp.o
 obj-y += shutdown.o
 obj-y += traps.o
 obj-y += vgic.o vgic-v2.o
+obj-$(CONFIG_ARM_64) += vgic-v3.o
 obj-y += vtimer.o
 obj-y += vuart.o
 obj-y += hvm.o
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index b146efe..e05d821 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -133,7 +133,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
         if ( rank == NULL) goto read_as_zero;
 
         vgic_lock_rank(v, rank);
-        *r = rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+        *r = rank->v2.itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
                                            DABT_WORD)];
         if ( dabt.size == DABT_BYTE )
             *r = byte_read(*r, dabt.sign, gicd_reg);
@@ -344,10 +344,10 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
         if ( rank == NULL) goto write_ignore;
         vgic_lock_rank(v, rank);
         if ( dabt.size == DABT_WORD )
-            rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
+            rank->v2.itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR,
                                           DABT_WORD)] = *r;
         else
-            byte_write(&rank->itargets[REG_RANK_INDEX(8,
+            byte_write(&rank->v2.itargets[REG_RANK_INDEX(8,
                        gicd_reg - GICD_ITARGETSR, DABT_WORD)], *r, gicd_reg);
         vgic_unlock_rank(v, rank);
         return 1;
@@ -449,7 +449,7 @@ static int vgic_v2_vcpu_init(struct vcpu *v)
 
     /* For SGI and PPI the target is always this CPU */
     for ( i = 0 ; i < 8 ; i++ )
-        v->arch.vgic.private_irqs->itargets[i] =
+        v->arch.vgic.private_irqs->v2.itargets[i] =
               (1<<(v->vcpu_id+0))
             | (1<<(v->vcpu_id+8))
             | (1<<(v->vcpu_id+16))
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
new file mode 100644
index 0000000..d80683d
--- /dev/null
+++ b/xen/arch/arm/vgic-v3.c
@@ -0,0 +1,895 @@
+/*
+ * xen/arch/arm/vgic-v3.c
+ *
+ * ARM Virtual Generic Interrupt Controller v3 support
+ * based on xen/arch/arm/vgic.c
+ *
+ * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>
+ * Copyright (c) 2014 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/bitops.h>
+#include <xen/config.h>
+#include <xen/lib.h>
+#include <xen/init.h>
+#include <xen/softirq.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+
+#include <asm/current.h>
+#include <asm/device.h>
+
+#include <asm/mmio.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/vgic.h>
+
+static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
+                                        uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    uint64_t mpidr;
+    uint64_t aff;
+
+    switch ( gicr_reg )
+    {
+    case GICR_CTLR:
+        /* We have not implemented LPI's, read zero */
+        goto read_as_zero;
+    case GICR_IIDR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_IIDR_VAL;
+        return 1;
+    case GICR_TYPER:
+        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+        /* TBD: Update processor id in [23:8] when ITS support is added */
+        mpidr = cpu_logical_map(v->vcpu_id);
+        aff = (MPIDR_AFFINITY_LEVEL(mpidr, 3) << 56 |
+               MPIDR_AFFINITY_LEVEL(mpidr, 2) << 48 |
+               MPIDR_AFFINITY_LEVEL(mpidr, 1) << 40 |
+               MPIDR_AFFINITY_LEVEL(mpidr, 0) << 32);
+        *r = aff;
+        return 1;
+    case GICR_STATUSR:
+        /* Not implemented */
+        goto read_as_zero;
+    case GICR_WAKER:
+        /* Power management is not implemented */
+        goto read_as_zero;
+    case GICR_SETLPIR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+    case GICR_CLRLPIR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+    case GICR_PROPBASER:
+        /* LPI's not implemented */
+        goto read_as_zero_64;
+    case GICR_PENDBASER:
+        /* LPI's not implemented */
+        goto read_as_zero_64;
+    case GICR_INVLPIR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+    case GICR_INVALLR:
+        /* WO. Read as zero */
+        goto read_as_zero_64;
+        return 0;
+    case GICR_SYNCR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* RO . But when read it always returns busy bito bit[0] */
+        *r = GICR_SYNCR_NOT_BUSY;
+        return 1;
+    case GICR_MOVLPIR:
+        /* WO Read as zero */
+        goto read_as_zero_64;
+    case GICR_MOVALLR:
+        /* WO Read as zero */
+        goto read_as_zero_64;
+    case GICR_PIDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR0;
+         return 1;
+    case GICR_PIDR1:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR1;
+         return 1;
+    case GICR_PIDR2:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR2;
+         return 1;
+    case GICR_PIDR3:
+        /* Manufacture/customer defined */
+        goto read_as_zero;
+    case GICR_PIDR4:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICR_PIDR4;
+         return 1;
+    case GICR_PIDR5 ... GICR_PIDR7:
+        /* Reserved0 */
+        goto read_as_zero;
+    default:
+        printk("vGICv3: vGICR: read r%d offset %#08x\n not found",
+               dabt.reg, gicr_reg);
+        return 0;
+    }
+bad_width:
+    printk("vGICv3: vGICR: bad read width %d r%d offset %#08x\n",
+           dabt.size, dabt.reg, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
+                                    uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+
+    switch ( gicr_reg )
+    {
+    case GICR_CTLR:
+        /* LPI's not implemented */
+        goto write_ignore;
+    case GICR_IIDR:
+        /* RO */
+        goto write_ignore;
+    case GICR_TYPER:
+        /* RO */
+        goto write_ignore_64;
+    case GICR_STATUSR:
+        /* Not implemented */
+        goto write_ignore;
+    case GICR_WAKER:
+        /* Power mgmt not implemented */
+        goto write_ignore;
+    case GICR_SETLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_CLRLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_PROPBASER:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_PENDBASER:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_INVLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_INVALLR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_SYNCR:
+        /* RO */
+        goto write_ignore;
+    case GICR_MOVLPIR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_MOVALLR:
+        /* LPI is not implemented */
+        goto write_ignore_64;
+    case GICR_PIDR7... GICR_PIDR0:
+        /* RO */
+        goto write_ignore;
+    default:
+        printk("vGICR: write r%d offset %#08x\n not found", dabt.reg, gicr_reg);
+        return 0;
+    }
+bad_width:
+    printk("vGICR: bad write width %d r%d=%"PRIregister" offset %#08x\n",
+           dabt.size, dabt.reg, *r, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    return 1;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+}
+
+static int __vgic_v3_distr_common_mmio_read(struct vcpu *v, mmio_info_t *info,
+                                           uint32_t reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+
+    switch ( reg )
+    {
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ISENABLER, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->ienable;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ICENABLER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->ienable;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ISPENDR, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = byte_read(rank->ipend, dabt.sign, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ICPENDR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = byte_read(rank->ipend, dabt.sign, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ISACTIVER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->iactive;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ICACTIVER, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->iactive;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, reg - GICD_IPRIORITYR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+
+        vgic_lock_rank(v, rank);
+        *r = rank->ipriority[REG_RANK_INDEX(8, reg - GICD_IPRIORITYR,
+                                            DABT_WORD)];
+        if ( dabt.size == DABT_BYTE )
+            *r = byte_read(*r, dabt.sign, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICFGR ... GICD_ICFGRN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 2, reg - GICD_ICFGR, DABT_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGR, DABT_WORD)];
+        vgic_unlock_rank(v, rank);
+        return 1;
+    default:
+        printk("GICv3: vGICD/vGICR: unhandled read r%d offset %#08x\n",
+               dabt.reg, reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR, "vGICv3: vGICD/vGICR: bad read width %d r%d \
+            offset %#08x\n", dabt.size, dabt.reg, reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int __vgic_v3_distr_common_mmio_write(struct vcpu *v, mmio_info_t *info,
+                                            uint32_t reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    uint32_t tr;
+
+    switch ( reg )
+    {
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ISENABLER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        tr = rank->ienable;
+        rank->ienable |= *r;
+        vgic_unlock_rank(v, rank);
+        /* The irq number is extracted from offset. so shift by register size */
+        vgic_enable_irqs(v, (*r) & (~tr), (reg - GICD_ISENABLER) >> DABT_WORD);
+        return 1;
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ICENABLER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        tr = rank->ienable;
+        rank->ienable &= ~*r;
+        vgic_unlock_rank(v, rank);
+        /* The irq number is extracted from offset. so shift by register size */
+        vgic_disable_irqs(v, (*r) & tr, (reg - GICD_ICENABLER) >> DABT_WORD);
+        return 1;
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ISPENDR, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->ipend = *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ICPENDR, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->ipend &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ISACTIVER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->iactive &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, reg - GICD_ICACTIVER, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->iactive &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 8, reg - GICD_IPRIORITYR, DABT_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        if ( dabt.size == DABT_WORD )
+            rank->ipriority[REG_RANK_INDEX(8, reg - GICD_IPRIORITYR,
+                                           DABT_WORD)] = *r;
+        else
+            byte_write(&rank->ipriority[REG_RANK_INDEX(8,
+                       reg - GICD_IPRIORITYR, DABT_WORD)], *r, reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_ICFGR: /* Restricted to configure SGIs */
+        goto write_ignore;
+    case GICD_ICFGR + 4 ... GICD_ICFGRN: /* PPI + SPIs */
+        /* ICFGR1 for PPI's, which is implementation defined
+           if ICFGR1 is programmable or not. We chose to program */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 2, reg - GICD_ICFGR, DABT_WORD);
+        vgic_lock_rank(v, rank);
+        if ( rank == NULL) goto write_ignore;
+        rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGR, DABT_WORD)] = *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    default:
+        printk("vGICv3: vGICD/vGICR: unhandled write r%d \
+                =%"PRIregister" offset %#08x\n", dabt.reg, *r, reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR, "vGICv3: vGICD/vGICR: bad write width %d \
+            r%d=%"PRIregister" offset %#08x\n", dabt.size, dabt.reg, *r, reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+}
+
+static int vgic_v3_rdistr_sgi_mmio_read(struct vcpu *v, mmio_info_t *info,
+                                     uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+
+    switch ( gicr_reg )
+    {
+    case GICR_IGRPMODR0:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+    case GICR_IGROUPR0:
+    case GICR_ISENABLER0:
+    case GICR_ICENABLER0:
+    case GICR_ISACTIVER0:
+    case GICR_ICACTIVER0:
+    case GICR_IPRIORITYR0...GICR_IPRIORITYR7:
+    case GICR_ICFGR0... GICR_ICFGR1:
+         /*
+          * Above registers offset are common with GICD.
+          * So handle in common with GICD handling
+          */
+        return __vgic_v3_distr_common_mmio_read(v, info, gicr_reg);
+    case GICR_ISPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicr_reg - GICR_ISPENDR0, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->pendsgi;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_ICPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicr_reg - GICR_ICPENDR0, DABT_WORD);
+        if ( rank == NULL ) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        *r = rank->pendsgi;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_NSACR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        return 1;
+    default:
+        printk("vGICv3: vGICR: read r%d offset %#08x\n not found",
+               dabt.reg, gicr_reg);
+        return 0;
+    }
+bad_width:
+    printk("vGICv3: vGICR: bad read width %d r%d offset %#08x\n",
+           dabt.size, dabt.reg, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int vgic_v3_rdistr_sgi_mmio_write(struct vcpu *v, mmio_info_t *info,
+                                      uint32_t gicr_reg)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+
+    switch ( gicr_reg )
+    {
+    case GICR_IGRPMODR0:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    case GICR_IGROUPR0:
+    case GICR_ISENABLER0:
+    case GICR_ICENABLER0:
+    case GICR_ISACTIVER0:
+    case GICR_ICACTIVER0:
+    case GICR_ICFGR1:
+    case GICR_IPRIORITYR0...GICR_IPRIORITYR7:
+         /*
+          * Above registers offset are common with GICD.
+          * So handle common with GICD handling
+          */
+        return __vgic_v3_distr_common_mmio_write(v, info, gicr_reg);
+    case GICR_ISPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicr_reg - GICR_ISACTIVER0, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->pendsgi |= *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_ICPENDR0:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 1, gicr_reg - GICR_ISACTIVER0, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->pendsgi &= ~*r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICR_NSACR:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    default:
+        printk("vGICv3: vGICR SGI: write r%d offset %#08x\n not found",
+               dabt.reg, gicr_reg);
+        return 0;
+    }
+
+bad_width:
+    printk("vGICR SGI: bad write width %d r%d=%"PRIregister" offset %#08x\n",
+           dabt.size, dabt.reg, *r, gicr_reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+}
+
+static int vgic_v3_rdistr_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+    uint32_t offset;
+
+    offset = info->gpa & (v->domain->arch.vgic.rdist_stride - 1);
+
+    if ( offset < SZ_64K )
+        return __vgic_v3_rdistr_rd_mmio_read(v, info, offset);
+    else  if ( (offset >= SZ_64K) && (offset < 2 * SZ_64K) )
+        return vgic_v3_rdistr_sgi_mmio_read(v, info, (offset - SZ_64K));
+    else
+        gdprintk(XENLOG_WARNING, "vGICv3: vGICR: unknown gpa read address \
+                  %"PRIpaddr"\n", info->gpa);
+
+    return 0;
+}
+
+static int vgic_v3_rdistr_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+    uint32_t offset;
+
+    offset = info->gpa & (v->domain->arch.vgic.rdist_stride - 1);
+    if ( offset < SZ_64K )
+        return __vgic_v3_rdistr_rd_mmio_write(v, info, offset);
+    else  if ( (offset >= SZ_64K) && (offset < 2 * SZ_64K) )
+        return vgic_v3_rdistr_sgi_mmio_write(v, info, (offset - SZ_64K));
+    else
+        gdprintk(XENLOG_WARNING, "vGICV3: vGICR: unknown gpa write address \
+                 %"PRIpaddr"\n", info->gpa);
+
+    return 0;
+}
+
+static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
+
+    switch ( gicd_reg )
+    {
+    case GICD_CTLR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        vgic_lock(v);
+        *r = v->domain->arch.vgic.ctlr;
+        vgic_unlock(v);
+        return 1;
+    case GICD_TYPER:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* No secure world support for guests. */
+        vgic_lock(v);
+        *r = (((v->domain->max_vcpus << 5) & GICD_TYPE_CPUS ) |
+              ((v->domain->arch.vgic.nr_lines / 32) & GICD_TYPE_LINES));
+        vgic_unlock(v);
+        return 1;
+    case GICD_STATUSR:
+        /*
+         *  Optional, Not implemented for now.
+         *  Update to support guest debugging.
+         */
+        goto read_as_zero;
+    case GICD_IIDR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_IIDR_VAL;
+        return 1;
+    case 0x020 ... 0x03c:
+    case 0xc000 ... 0xffcc:
+        /* Implementation defined -- read as zero */
+        goto read_as_zero;
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+    case GICD_ICFGR ... GICD_ICFGRN:
+        /*
+         * Above all register are common with GICR and GICD
+         * Manage in common
+         */
+        return __vgic_v3_distr_common_mmio_read(v, info, gicd_reg);
+    case GICD_IROUTER ... GICD_IROUTER31:
+        /* SGI/PPI is RES0 */
+        goto read_as_zero_64;
+    case GICD_IROUTER32 ... GICD_IROUTERN:
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_DOUBLE_WORD )
+            goto bad_width;
+        rank = vgic_irq_rank(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
+        if ( rank == NULL) goto read_as_zero;
+        vgic_lock_rank(v, rank);
+        /* IROUTER is 64 bit so, to make it byte size right shift by 3.
+           Here once. macro REG_RANK_INDEX will do it twice */
+        *r = rank->v3.irouter[REG_RANK_INDEX(64,
+                           (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)];
+        if ( dabt.size == DABT_BYTE )
+            *r = byte_read(*r, dabt.sign, gicd_reg);
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_NSACR ... GICD_NSACRN:
+        /* We do not implement security extensions for guests, read zero */
+        goto read_as_zero;
+    case GICD_SGIR:
+        /* Read as ICH_SGIR system register with SRE set. So ignore */
+        goto read_as_zero;
+    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
+        /* Replaced with GICR_ICPENDR0. So ignore write */
+        goto read_as_zero;
+    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
+        /* Replaced with GICR_ISPENDR0. So ignore write */
+        goto read_as_zero;
+    case GICD_PIDR0:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR0;
+        return 1;
+    case GICD_PIDR1:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR1;
+        return 1;
+    case GICD_PIDR2:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR2;
+        return 1;
+    case GICD_PIDR3:
+        /* GICv3 identification value. Manufacturer/Customer defined */
+        goto read_as_zero;
+    case GICD_PIDR4:
+        /* GICv3 identification value */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        *r = GICV3_GICD_PIDR4;
+        return 1;
+    case GICD_PIDR5 ... GICD_PIDR7:
+        /* Reserved0 */
+        goto read_as_zero;
+    case 0x00c:
+    case 0x044:
+    case 0x04c:
+    case 0x05c ... 0x07c:
+    case 0xf30 ... 0x5fcc:
+    case 0x8000 ... 0xbfcc:
+        /* These are reserved register addresses */
+        printk("vGICv3: vGICD: read unknown 0x00c .. 0xfcc r%d offset %#08x\n",
+               dabt.reg, gicd_reg);
+        goto read_as_zero;
+    default:
+        printk("vGICv3: vGICD: unhandled read r%d offset %#08x\n",
+               dabt.reg, gicd_reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR, "vGICv3: vGICD: bad read width %d r%d offset %#08x\n",
+           dabt.size, dabt.reg, gicd_reg);
+    domain_crash_synchronous();
+    return 0;
+
+read_as_zero_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+
+read_as_zero:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    *r = 0;
+    return 1;
+}
+
+static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+    struct hsr_dabt dabt = info->dabt;
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    register_t *r = select_user_reg(regs, dabt.reg);
+    struct vgic_irq_rank *rank;
+    int gicd_reg = (int)(info->gpa - v->domain->arch.vgic.dbase);
+
+    switch ( gicd_reg )
+    {
+    case GICD_CTLR:
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        /* Ignore all but the enable bit */
+        v->domain->arch.vgic.ctlr = (*r) & GICD_CTL_ENABLE;
+        return 1;
+    case GICD_TYPER:
+        /* RO -- write ignored */
+        goto write_ignore;
+    case GICD_IIDR:
+        /* RO -- write ignored */
+        goto write_ignore;
+    case GICD_STATUSR:
+        /* RO -- write ignored */
+        goto write_ignore;
+    case GICD_SETSPI_NSR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case GICD_CLRSPI_NSR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case GICD_SETSPI_SR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case GICD_CLRSPI_SR:
+        /* Message based SPI is not implemented */
+        goto write_ignore;
+    case 0x020 ... 0x03c:
+    case 0xc000 ... 0xffcc:
+        /* Implementation defined -- write ignored */
+        printk("vGICD: write unknown 0x020 - 0x03c r%d offset %#08x\n",
+               dabt.reg, gicd_reg);
+        goto write_ignore;
+    case GICD_IGROUPR ... GICD_IGROUPRN:
+    case GICD_ISENABLER ... GICD_ISENABLERN:
+    case GICD_ICENABLER ... GICD_ICENABLERN:
+    case GICD_ISPENDR ... GICD_ISPENDRN:
+    case GICD_ICPENDR ... GICD_ICPENDRN:
+    case GICD_ISACTIVER ... GICD_ISACTIVERN:
+    case GICD_ICACTIVER ... GICD_ICACTIVERN:
+    case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+    case GICD_ICFGR ... GICD_ICFGRN:
+        /* Above registers are common with GICR and GICD
+         * Manage in common */
+        return __vgic_v3_distr_common_mmio_write(v, info, gicd_reg);
+    case GICD_IROUTER ... GICD_IROUTER31:
+        /* SGI/PPI is RES0 */
+        goto write_ignore_64;
+    case GICD_IROUTER32 ... GICD_IROUTERN:
+        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+        rank = vgic_irq_rank(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
+        if ( rank == NULL) goto write_ignore;
+        vgic_lock_rank(v, rank);
+        rank->v3.irouter[REG_RANK_INDEX(64,
+                      (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)] = *r;
+        vgic_unlock_rank(v, rank);
+        return 1;
+    case GICD_NSACR ... GICD_NSACRN:
+        /* We do not implement security extensions for guests, write ignore */
+        goto write_ignore;
+    case GICD_SGIR:
+        /* it is accessed as system register in GICv3 */
+        goto write_ignore;
+    case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
+        /* Replaced with GICR_ICPENDR0. So ignore write */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        return 0;
+    case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
+        /* Replaced with GICR_ISPENDR0. So ignore write */
+        if ( dabt.size != DABT_WORD ) goto bad_width;
+        return 0;
+    case GICD_PIDR7... GICD_PIDR0:
+        /* RO -- write ignore */
+        goto write_ignore;
+    case 0x00c:
+    case 0x044:
+    case 0x04c:
+    case 0x05c ... 0x07c:
+    case 0xf30 ... 0x5fcc:
+    case 0x8000 ... 0xbfcc:
+        /* Reserved register addresses */
+        printk("vGICv3: vGICD: write unknown 0x00c 0xfcc  r%d offset %#08x\n",
+                dabt.reg, gicd_reg);
+        goto write_ignore;
+    default:
+        printk("vGICv3: vGICD: unhandled write r%d=%"PRIregister" \
+                 offset %#08x\n", dabt.reg, *r, gicd_reg);
+        return 0;
+    }
+
+bad_width:
+    dprintk(XENLOG_ERR, "VGICv3: vGICD: bad write width %d r%d=%"PRIregister" \
+             offset %#08x\n", dabt.size, dabt.reg, *r, gicd_reg);
+    domain_crash_synchronous();
+    return 0;
+
+write_ignore:
+    if ( dabt.size != DABT_WORD ) goto bad_width;
+    return 1;
+
+write_ignore_64:
+    if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+    return 1;
+}
+
+const static struct mmio_handler_ops vgic_rdistr_mmio_handler = {
+    .read_handler  = vgic_v3_rdistr_mmio_read,
+    .write_handler = vgic_v3_rdistr_mmio_write,
+};
+
+const static struct mmio_handler_ops vgic_distr_mmio_handler = {
+    .read_handler  = vgic_v3_distr_mmio_read,
+    .write_handler = vgic_v3_distr_mmio_write,
+};
+
+static int vgicv3_vcpu_init(struct vcpu *v)
+{
+    int i;
+    uint64_t affinity;
+
+    /* For SGI and PPI the target is always this CPU */
+    affinity = cpu_logical_map(smp_processor_id());
+    for ( i = 0 ; i < 32 ; i++ )
+        v->arch.vgic.private_irqs->v3.irouter[i] = affinity;
+
+    return 0;
+}
+
+static int vgicv3_domain_init(struct domain *d)
+{
+    int i;
+
+    register_mmio_handler(d, &vgic_distr_mmio_handler, d->arch.vgic.dbase,
+                          d->arch.vgic.dbase_size);
+
+    /*
+     * Register mmio handler per redistributor region but not for
+     * every sgi rdist region which is per core.
+     * The redistributor region encompasses per core sgi region.
+     */
+    for ( i = 0; i < d->arch.vgic.rdist_count; i++ )
+        register_mmio_handler(d, &vgic_rdistr_mmio_handler,
+            d->arch.vgic.rbase[i], d->arch.vgic.rbase_size[i]);
+
+    return 0;
+}
+
+const static struct vgic_ops v3_ops = {
+    .vcpu_init   = vgicv3_vcpu_init,
+    .domain_init = vgicv3_domain_init,
+};
+
+int vgic_v3_init(struct domain *d)
+{
+    register_vgic_ops(d, &v3_ops);
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 3fa0857..787c547 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -264,10 +264,19 @@ int domain_vgic_init(struct domain *d)
     else
         d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
 
-    if ( gic_hw_version() == GIC_V2 )
+    switch ( gic_hw_version() )
+    {
+#ifdef CONFIG_ARM_64
+    case GIC_V3:
+        vgic_v3_init(d);
+        break;
+#endif
+    case GIC_V2:
         vgic_v2_init(d);
-    else
+        break;
+    default:
         panic("No VGIC found\n");
+    }
 
     d->arch.vgic.shared_irqs =
         xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index aa4e94d..754d521 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -26,7 +26,14 @@ struct vgic_irq_rank {
     uint32_t ienable, iactive, ipend, pendsgi;
     uint32_t icfg[2];
     uint32_t ipriority[8];
-    uint32_t itargets[8];
+    union {
+        struct {
+            uint32_t itargets[8];
+        }v2;
+        struct {
+            uint64_t irouter[32];
+        }v3;
+    };
 };
 
 struct vgic_ops {
@@ -53,6 +60,9 @@ static inline int REG_RANK_NR(int b, uint32_t n)
 {
     switch ( b )
     {
+    case 64: return n >> 6;
+    case 32: return n >> 5;
+    case 16: return n >> 4;
     case 8: return n >> 3;
     case 4: return n >> 2;
     case 2: return n >> 1;
@@ -120,6 +130,7 @@ extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
 int vgic_v2_init(struct domain *d);
+int vgic_v3_init(struct domain *d);
 
 extern int vcpu_vgic_free(struct vcpu *v);
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 15/16] xen/arm: Update Dom0 GIC dt node with GICv3 information
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (13 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 14/16] xen/arm: Add virtual GICv3 support vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-05-26 10:26 ` [PATCH v4 16/16] xen/arm: add SGI handling for GICv3 vijay.kilari
  2014-05-28 10:26 ` [PATCH v4 00/16] xen/arm: Add GICv3 support Ian Campbell
  16 siblings, 0 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Update GIC device tree node for DOM0 with GICv3
information. GIC hw specfic device tree information
is moved to respective GIC driver.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/domain_build.c |   47 ++------------------------
 xen/arch/arm/gic-v2.c       |   51 ++++++++++++++++++++++++++++
 xen/arch/arm/gic-v3.c       |   77 +++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/gic.c          |    6 ++++
 xen/include/asm-arm/gic.h   |    7 +++-
 5 files changed, 142 insertions(+), 46 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 15ed97b..c5905af 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -524,9 +524,6 @@ static int make_gic_node(const struct domain *d, void *fdt,
                          const struct dt_device_node *node)
 {
     const struct dt_device_node *gic = dt_interrupt_controller;
-    const void *compatible = NULL;
-    u32 len;
-    __be32 *new_cells, *tmp;
     int res = 0;
 
     /*
@@ -541,48 +538,7 @@ static int make_gic_node(const struct domain *d, void *fdt,
 
     DPRINT("Create gic node\n");
 
-    compatible = dt_get_property(gic, "compatible", &len);
-    if ( !compatible )
-    {
-        dprintk(XENLOG_ERR, "Can't find compatible property for the gic node\n");
-        return -FDT_ERR_XEN(ENOENT);
-    }
-
-    res = fdt_begin_node(fdt, "interrupt-controller");
-    if ( res )
-        return res;
-
-    res = fdt_property(fdt, "compatible", compatible, len);
-    if ( res )
-        return res;
-
-    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
-    if ( res )
-        return res;
-
-    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
-
-    if ( res )
-        return res;
-
-    len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
-    len *= 2; /* GIC has two memory regions: Distributor + CPU interface */
-    new_cells = xzalloc_bytes(len);
-    if ( new_cells == NULL )
-        return -FDT_ERR_XEN(ENOMEM);
-
-    tmp = new_cells;
-    DPRINT("  Set Distributor Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
-           d->arch.vgic.dbase, d->arch.vgic.dbase + PAGE_SIZE - 1);
-    dt_set_range(&tmp, node, d->arch.vgic.dbase, PAGE_SIZE);
-
-    DPRINT("  Set Cpu Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
-           d->arch.vgic.cbase, d->arch.vgic.cbase + (PAGE_SIZE * 2) - 1);
-    dt_set_range(&tmp, node, d->arch.vgic.cbase, PAGE_SIZE * 2);
-
-    res = fdt_property(fdt, "reg", new_cells, len);
-    xfree(new_cells);
-
+    res = gic_make_node(d, node, fdt);
     if ( res )
         return res;
 
@@ -781,6 +737,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
     static const struct dt_device_match gic_matches[] __initconst =
     {
         DT_MATCH_GIC_V2,
+        DT_MATCH_GIC_V3,
         { /* sentinel */ },
     };
     static const struct dt_device_match timer_matches[] __initconst =
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 8060de3..a1f65c3 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -27,6 +27,7 @@
 #include <xen/softirq.h>
 #include <xen/list.h>
 #include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
 #include <asm/p2m.h>
 #include <asm/domain.h>
 #include <asm/platform.h>
@@ -538,6 +539,55 @@ static void gicv2_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
     BUG();
 }
 
+static int gicv2_make_dt_node(const struct domain *d,
+                              const struct dt_device_node *node, void *fdt)
+{
+    const struct dt_device_node *gic = dt_interrupt_controller;
+    const void *compatible = NULL;
+    u32 len;
+    __be32 *new_cells, *tmp;
+    int res = 0;
+
+    compatible = dt_get_property(gic, "compatible", &len);
+    if ( !compatible )
+    {
+        dprintk(XENLOG_ERR, "Can't find compatible property for the gic node\n");
+        return -FDT_ERR_XEN(ENOENT);
+    }
+
+    res = fdt_begin_node(fdt, "interrupt-controller");
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "compatible", compatible, len);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+
+    if ( res )
+        return res;
+
+    len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+    len *= 2; /* GIC has two memory regions: Distributor + CPU interface */
+    new_cells = xzalloc_bytes(len);
+    if ( new_cells == NULL )
+        return -FDT_ERR_XEN(ENOMEM);
+
+    tmp = new_cells;
+    dt_set_range(&tmp, node, d->arch.vgic.dbase, PAGE_SIZE);
+    dt_set_range(&tmp, node, d->arch.vgic.cbase, PAGE_SIZE * 2);
+
+    res = fdt_property(fdt, "reg", new_cells, len);
+    xfree(new_cells);
+
+    return res;
+}
+
 /* XXX different for level vs edge */
 static hw_irq_controller gicv2_host_irq_type = {
     .typename     = "gic-v2",
@@ -582,6 +632,7 @@ const static struct gic_hw_operations gicv2_ops = {
     .read_lr             = gicv2_read_lr,
     .write_lr            = gicv2_write_lr,
     .read_vmcr_priority  = gicv2_read_vmcr_priority,
+    .make_dt_node        = gicv2_make_dt_node,
 };
 
 /* Set up the GIC */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 7543fb4..654234f 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -953,6 +953,82 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
     BUG();
 }
 
+static int gicv3_make_dt_node(const struct domain *d,
+                              const struct dt_device_node *node, void *fdt)
+{
+    const struct dt_device_node *gic = dt_interrupt_controller;
+    const void *compatible = NULL;
+    uint32_t len;
+    __be32 *new_cells, *tmp;
+    uint32_t rd_stride = 0;
+    uint32_t rd_count = 0;
+
+    int i, res = 0;
+
+    compatible = dt_get_property(gic, "compatible", &len);
+    if ( !compatible )
+    {
+        dprintk(XENLOG_ERR, "Can't find compatible property for the gic node\n");
+        return -FDT_ERR_XEN(ENOENT);
+    }
+
+    res = fdt_begin_node(fdt, "interrupt-controller");
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "compatible", compatible, len);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    res = dt_property_read_u32(gic, "redistributor-stride", &rd_stride);
+    if ( !res )
+        rd_stride = 0;
+
+    res = dt_property_read_u32(gic, "#redistributor-regions", &rd_count);
+    if ( !res )
+        rd_count = 1;
+
+    res = fdt_property_cell(fdt, "redistributor-stride", rd_stride);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#redistributor-regions", rd_count);
+    if ( res )
+        return res;
+
+    len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+    /*
+     * GIC has two memory regions: Distributor + rdist regions
+     * CPU interface and virtual cpu interfaces accessesed as System registers
+     * So cells are created only for Distributor and rdist regions
+     */
+    len = len * (d->arch.vgic.rdist_count + 1);
+    new_cells = xzalloc_bytes(len);
+    if ( new_cells == NULL )
+        return -FDT_ERR_XEN(ENOMEM);
+
+    tmp = new_cells;
+
+    dt_set_range(&tmp, node, d->arch.vgic.dbase, d->arch.vgic.dbase_size);
+
+    for ( i = 0; i < d->arch.vgic.rdist_count; i++ )
+        dt_set_range(&tmp, node, d->arch.vgic.rbase[i],
+                     d->arch.vgic.rbase_size[i]);
+
+    res = fdt_property(fdt, "reg", new_cells, len);
+    xfree(new_cells);
+
+    return res;
+}
+
 const static hw_irq_controller gicv3_host_irq_type = {
     .typename     = "gic-v3",
     .startup      = gicv3_irq_startup,
@@ -996,6 +1072,7 @@ const static struct gic_hw_operations gicv3_ops = {
     .write_lr            = gicv3_write_lr,
     .read_vmcr_priority  = gicv3_read_vmcr_priority,
     .secondary_init      = gicv3_secondary_cpu_init,
+    .make_dt_node        = gicv3_make_dt_node,
 };
 
 /* Set up the GIC */
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 9331216..2c50c72 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -622,6 +622,12 @@ void __cpuinit init_maintenance_interrupt(void)
                 "irq-maintenance", NULL);
 }
 
+int gic_make_node(const struct domain *d,const struct dt_device_node *node,
+                   void *fdt)
+{
+    return gic_hw_ops->make_dt_node(d, node, fdt);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index b40beb5..acf1304 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -156,7 +156,7 @@
 
 #define DT_MATCH_GIC_V3_STRING1      "arm,gic-v3"
 
-#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V3_STRING1);
+#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V3_STRING1)
 
 /*
  * GICv3 registers that needs to be saved/restored
@@ -332,10 +332,15 @@ struct gic_hw_operations {
     unsigned int (*read_vmcr_priority)(void);
     /* Secondary CPU init */
     int (*secondary_init)(void);
+    int (*make_dt_node)(const struct domain *d,
+                        const struct dt_device_node *node, void *fdt);
 };
 
 void register_gic_ops(const struct gic_hw_operations *ops);
 extern void update_cpu_lr_mask(void);
+int gic_make_node(const struct domain *d,const struct dt_device_node *node,
+                  void *fdt);
+
 
 #endif /* __ASSEMBLY__ */
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (14 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 15/16] xen/arm: Update Dom0 GIC dt node with GICv3 information vijay.kilari
@ 2014-05-26 10:26 ` vijay.kilari
  2014-06-02 16:05   ` Stefano Stabellini
  2014-06-02 16:17   ` Julien Grall
  2014-05-28 10:26 ` [PATCH v4 00/16] xen/arm: Add GICv3 support Ian Campbell
  16 siblings, 2 replies; 78+ messages in thread
From: vijay.kilari @ 2014-05-26 10:26 UTC (permalink / raw)
  To: Ian.Campbell, julien.grall, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, Vijaya Kumar K, vijay.kilari

From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

In ARMv8, write to ICC_SGI1R_EL1 register raises trap to EL2.
Handle the trap and inject SGI to vcpu.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
 xen/arch/arm/traps.c              |   30 ++++++++++++++++++++++++++++++
 xen/arch/arm/vgic-v3.c            |   37 +++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/gic_v3_defs.h |    7 +++++++
 xen/include/asm-arm/sysregs.h     |    3 +++
 xen/include/asm-arm/vgic.h        |    2 +-
 5 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 9348147..1ac01ee 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -41,6 +41,7 @@
 #include "decode.h"
 #include "vtimer.h"
 #include <asm/gic.h>
+#include <asm/vgic.h>
 
 /* The base of the stack must always be double-word aligned, which means
  * that both the kernel half of struct cpu_user_regs (which is pushed in
@@ -496,6 +497,18 @@ static void inject_dabt_exception(struct cpu_user_regs *regs,
 #endif
 }
 
+static void inject_undef_exception(struct cpu_user_regs *regs,
+                                   register_t addr,
+                                   int instr_len)
+{
+    if ( is_32bit_domain(current->domain) )
+        inject_undef32_exception(regs);
+#ifdef CONFIG_ARM_64
+    else
+        inject_undef64_exception(regs, instr_len);
+#endif
+}
+
 struct reg_ctxt {
     /* Guest-side state */
     uint32_t sctlr_el1;
@@ -1467,6 +1480,7 @@ static void do_sysreg(struct cpu_user_regs *regs,
                       union hsr hsr)
 {
     register_t *x = select_user_reg(regs, hsr.sysreg.reg);
+    register_t addr;
 
     switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
     {
@@ -1515,6 +1529,22 @@ static void do_sysreg(struct cpu_user_regs *regs,
             domain_crash_synchronous();
         }
         break;
+    case HSR_SYSREG_ICC_SGI1R_EL1:
+        if ( !vgic_emulate(regs, hsr) )
+        {
+            addr = READ_SYSREG64(FAR_EL2);
+            dprintk(XENLOG_WARNING,
+                    "failed emulation of sysreg ICC_SGI1R_EL1 access\n");
+            inject_undef_exception(regs, addr, hsr.len);
+        }
+        break;
+    case HSR_SYSREG_ICC_SGI0R_EL1:
+    case HSR_SYSREG_ICC_ASGI1R_EL1:
+        /* TBD: Implement to support secure grp0/1 SGI forwarding */
+        dprintk(XENLOG_WARNING,
+                "Emulation of sysreg ICC_SGI0R_EL1/ASGI1R_EL1 not supported\n");
+        addr = READ_SYSREG64(FAR_EL2);
+        inject_undef_exception(regs, addr, hsr.len);
     default:
  bad_sysreg:
         {
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index d80683d..99d0d46 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -832,6 +832,43 @@ write_ignore_64:
     return 1;
 }
 
+static int vgicv3_to_sgi(struct vcpu *v, register_t sgir)
+{
+    int virq;
+    int irqmode;
+    unsigned long vcpu_mask = 0;
+
+    irqmode = (sgir >> ICH_SGI_IRQMODE_SHIFT) & ICH_SGI_IRQMODE_MASK;
+    virq = (sgir >> ICH_SGI_IRQ_SHIFT ) & ICH_SGI_IRQ_MASK;
+    vcpu_mask = sgir & ICH_SGI_TARGETLIST_MASK;
+
+    return vgic_to_sgi(v, sgir, irqmode, virq, vcpu_mask);
+}
+
+int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
+{
+    struct vcpu *v = current;
+    struct hsr_sysreg sysreg = hsr.sysreg;
+    register_t *r = select_user_reg(regs, sysreg.reg);
+
+    ASSERT (hsr.ec == HSR_EC_SYSREG);
+
+    switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
+    {
+    case HSR_SYSREG_ICC_SGI1R_EL1:
+        /* WO */
+        if ( !sysreg.read )
+            return vgicv3_to_sgi(v, *r);
+        else
+        {
+            gdprintk(XENLOG_WARNING, "Reading SGI1R_EL1 - WO register\n");
+            return 0;
+        }
+    default:
+        return 0;
+    }
+}
+
 const static struct mmio_handler_ops vgic_rdistr_mmio_handler = {
     .read_handler  = vgic_v3_rdistr_mmio_read,
     .write_handler = vgic_v3_rdistr_mmio_write,
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 6f393aa..5e75632 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -145,6 +145,13 @@
 #define GICH_VMCR_PRIORITY_MASK      0xff
 #define GICH_VMCR_PRIORITY_SHIFT     24
 
+#define ICH_SGI_IRQMODE_SHIFT        40
+#define ICH_SGI_IRQMODE_MASK         0x1
+#define ICH_SGI_TARGET_OTHERS        1
+#define ICH_SGI_TARGET_LIST          0
+#define ICH_SGI_IRQ_SHIFT            24
+#define ICH_SGI_IRQ_MASK             0xf
+#define ICH_SGI_TARGETLIST_MASK      0xffff
 #endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
 
 /*
diff --git a/xen/include/asm-arm/sysregs.h b/xen/include/asm-arm/sysregs.h
index 4a4de34..5029851 100644
--- a/xen/include/asm-arm/sysregs.h
+++ b/xen/include/asm-arm/sysregs.h
@@ -77,6 +77,9 @@
 #define HSR_SYSREG_PMINTENCLR_EL1 HSR_SYSREG(3,0,c9,c14,2)
 #define HSR_SYSREG_MAIR_EL1       HSR_SYSREG(3,0,c10,c2,0)
 #define HSR_SYSREG_AMAIR_EL1      HSR_SYSREG(3,0,c10,c3,0)
+#define HSR_SYSREG_ICC_SGI1R_EL1  HSR_SYSREG(3,0,c12,c11,5)
+#define HSR_SYSREG_ICC_ASGI1R_EL1 HSR_SYSREG(3,1,c12,c11,6)
+#define HSR_SYSREG_ICC_SGI0R_EL1  HSR_SYSREG(3,2,c12,c11,7)
 #define HSR_SYSREG_CONTEXTIDR_EL1 HSR_SYSREG(3,0,c13,c0,1)
 
 #define HSR_SYSREG_PMCR_EL0       HSR_SYSREG(3,3,c9,c12,0)
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 754d521..4258840 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -125,7 +125,7 @@ extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
 extern void vgic_clear_pending_irqs(struct vcpu *v);
 extern int vgic_to_sgi(struct vcpu *v, register_t sgir, int irqmode, int virq,
                        unsigned long vcpu_mask);
-
+extern int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr);
 extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder
  2014-05-26 10:26 ` [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder vijay.kilari
@ 2014-05-26 11:28   ` Julien Grall
  2014-05-28 13:55   ` Stefano Stabellini
  1 sibling, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-26 11:28 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> io.h is local to arch/arm folder. move this file as mmio.h
> file to include/asm-arm folder as it might be
> required for inclusion in other header files in future.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Acked-by: Julien Grall <julien.grall@linaro.org>

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 02/16] xen/arm: make mmio handlers domain specific
  2014-05-26 10:26 ` [PATCH v4 02/16] xen/arm: make mmio handlers domain specific vijay.kilari
@ 2014-05-26 12:33   ` Julien Grall
  2014-05-28 14:05     ` Stefano Stabellini
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-26 12:33 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
>   int handle_mmio(mmio_info_t *info)
>   {
>       struct vcpu *v = current;
>       int i;
> +    struct mmio_handler *mmio_handler;
> +    struct io_handler *io_handlers = &v->domain->arch.io_handlers;

NIT: I think mmio_handler and io_handlers can be const.

> +
> +void register_mmio_handler(struct domain *d,
> +                           const struct mmio_handler_ops *handle,
> +                           paddr_t addr, paddr_t size)
> +{
> +    struct io_handler *handler = &d->arch.io_handlers;
> +
> +    BUG_ON(handler->num_entries >= MAX_IO_HANDLER);
> +
> +    spin_lock(&handler->lock);
> +
> +    handler->mmio_handlers[handler->num_entries].mmio_handler_ops = handle;
> +    handler->mmio_handlers[handler->num_entries].addr = addr;
> +    handler->mmio_handlers[handler->num_entries].size = size;
> +    handler->num_entries++;
> +    dsb(sy);

This is wrong. As I said on the previous version, the dsb needs to be 
called before incrementing the num_entries.

This is because as you don't use spinlock in handle_mmio, you have to 
make sure the array modification has reached the memory before update 
num_entries.

At the same time dsb(is) is enough.

> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 4962e70..151ec3e 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -73,43 +73,6 @@ static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
>           return NULL;
>   }
>
> -int domain_vgic_init(struct domain *d)
> -{
> -    int i;
> -
> -    d->arch.vgic.ctlr = 0;
> -
> -    /* Currently nr_lines in vgic and gic doesn't have the same meanings
> -     * Here nr_lines = number of SPIs
> -     */
> -    if ( is_hardware_domain(d) )
> -        d->arch.vgic.nr_lines = gic_number_lines() - 32;
> -    else
> -        d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
> -
> -    d->arch.vgic.shared_irqs =
> -        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
> -    if ( d->arch.vgic.shared_irqs == NULL )
> -        return -ENOMEM;
> -
> -    d->arch.vgic.pending_irqs =
> -        xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
> -    if ( d->arch.vgic.pending_irqs == NULL )
> -    {
> -        xfree(d->arch.vgic.shared_irqs);
> -        return -ENOMEM;
> -    }
> -
> -    for (i=0; i<d->arch.vgic.nr_lines; i++)
> -    {
> -        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
> -        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
> -    }
> -    for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> -        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
> -    return 0;
> -}
> -

Rather than moving a whole chunk of code, why can't you add forward 
declaration for vgic_disk_mmio_{read,write}?

>   void domain_vgic_free(struct domain *d)
>   {
>       xfree(d->arch.vgic.shared_irqs);
> @@ -676,15 +639,7 @@ write_ignore:
>       return 1;
>   }
>
> -static int vgic_distr_mmio_check(struct vcpu *v, paddr_t addr)
> -{
> -    struct domain *d = v->domain;
> -
> -    return (addr >= (d->arch.vgic.dbase)) && (addr < (d->arch.vgic.dbase + PAGE_SIZE));
> -}
> -
> -const struct mmio_handler vgic_distr_mmio_handler = {
> -    .check_handler = vgic_distr_mmio_check,
> +const struct mmio_handler_ops vgic_distr_mmio_handler = {
>       .read_handler  = vgic_distr_mmio_read,
>       .write_handler = vgic_distr_mmio_write,
>   };
> @@ -766,6 +721,38 @@ out:
>           smp_send_event_check_mask(cpumask_of(v->processor));
>   }
>
> +int domain_vgic_init(struct domain *d)
> +{
> +    int i;
> +
> +    d->arch.vgic.ctlr = 0;
> +
> +    /* Currently nr_lines in vgic and gic doesn't have the same meanings
> +     * Here nr_lines = number of SPIs
> +     */
> +    if ( d->domain_id == 0 )
> +        d->arch.vgic.nr_lines = gic_number_lines() - 32;
> +    else
> +        d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
> +
> +    d->arch.vgic.shared_irqs =
> +        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
> +    d->arch.vgic.pending_irqs =
> +        xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
> +    for (i=0; i<d->arch.vgic.nr_lines; i++)
> +    {
> +        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
> +        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
> +    }
> +    for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> +        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
> +
> +    register_mmio_handler(d, &vgic_distr_mmio_handler,
> +                          d->arch.vgic.dbase, PAGE_SIZE);
> +

Sounds like a bit strange to call register_mmio_handler here and let 
gicv_setup set dbase. Can you add a comment saying to smth like "We rely 
on gicv_setup to initialize dbase"?

> +    return 0;
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/arch/arm/vuart.c b/xen/arch/arm/vuart.c
> index 953cd46..52f3259 100644
> --- a/xen/arch/arm/vuart.c
> +++ b/xen/arch/arm/vuart.c
> @@ -44,24 +44,6 @@
>
>   #define domain_has_vuart(d) ((d)->arch.vuart.info != NULL)
>
> -int domain_vuart_init(struct domain *d)
> -{
> -    ASSERT( is_hardware_domain(d) );
> -
> -    d->arch.vuart.info = serial_vuart_info(SERHND_DTUART);
> -    if ( !d->arch.vuart.info )
> -        return 0;
> -
> -    spin_lock_init(&d->arch.vuart.lock);
> -    d->arch.vuart.idx = 0;
> -
> -    d->arch.vuart.buf = xzalloc_array(char, VUART_BUF_SIZE);
> -    if ( !d->arch.vuart.buf )
> -        return -ENOMEM;
> -
> -    return 0;
> -}
> -

Same remark here about forward declaration.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 03/16] xen/arm: make sgi handling generic
  2014-05-26 10:26 ` [PATCH v4 03/16] xen/arm: make sgi handling generic vijay.kilari
@ 2014-05-26 12:41   ` Julien Grall
  2014-05-26 12:45     ` Julien Grall
  2014-05-28 14:10   ` Stefano Stabellini
  1 sibling, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-26 12:41 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> move all the hw specific sgi handling functionality
> to one function and use it.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
>   xen/arch/arm/gic.c        |   37 ++++++++++++++++++++++++++++---------
>   xen/include/asm-arm/gic.h |    8 ++++++++
>   2 files changed, 36 insertions(+), 9 deletions(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index efcd785..f8e49df 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -483,21 +483,40 @@ void __init gic_init(void)
>       spin_unlock(&gic.lock);
>   }
>
> -void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
> +static void send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
> +                     uint8_t irqmode)

You define the enum gic_sgi_mode below, you should use it here rather 
than uint8_t.

>   {
>       unsigned int mask = 0;
> +
> +    switch ( irqmode )
> +    {
> +    case SGI_TARGET_OTHERS:
> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS | sgi;
> +        break;
> +    case SGI_TARGET_SELF:
> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF | sgi;
> +        break;
> +    case SGI_TARGET_LIST:
> +        mask = gic_cpu_mask(cpu_mask);
> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST |
> +                          (mask<<GICD_SGI_TARGET_SHIFT) | sgi;
> +        break;
> +    default:
> +        BUG_ON(1);

You can directly use BUG() here.

Assuming this 2 changes:

Acked-by: Julien Grall <julien.grall@linaro.org>

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 03/16] xen/arm: make sgi handling generic
  2014-05-26 12:41   ` Julien Grall
@ 2014-05-26 12:45     ` Julien Grall
  0 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-26 12:45 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar



On 26/05/14 13:41, Julien Grall wrote:
> Hi Vijay,
>
> On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> move all the hw specific sgi handling functionality
>> to one function and use it.
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>>   xen/arch/arm/gic.c        |   37 ++++++++++++++++++++++++++++---------
>>   xen/include/asm-arm/gic.h |    8 ++++++++
>>   2 files changed, 36 insertions(+), 9 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index efcd785..f8e49df 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -483,21 +483,40 @@ void __init gic_init(void)
>>       spin_unlock(&gic.lock);
>>   }
>>
>> -void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
>> +static void send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
>> +                     uint8_t irqmode)
>
> You define the enum gic_sgi_mode below, you should use it here rather
> than uint8_t.
>
>>   {
>>       unsigned int mask = 0;
>> +
>> +    switch ( irqmode )
>> +    {
>> +    case SGI_TARGET_OTHERS:
>> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS | sgi;
>> +        break;
>> +    case SGI_TARGET_SELF:
>> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF | sgi;
>> +        break;
>> +    case SGI_TARGET_LIST:
>> +        mask = gic_cpu_mask(cpu_mask);
>> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST |
>> +                          (mask<<GICD_SGI_TARGET_SHIFT) | sgi;
>> +        break;
>> +    default:
>> +        BUG_ON(1);
>
> You can directly use BUG() here.

Just noticed the BUG_ON was asked by me before. Sorry to mislead you.

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 04/16] xen/arm: remove unused parameter in do_sgi call
  2014-05-26 10:26 ` [PATCH v4 04/16] xen/arm: remove unused parameter in do_sgi call vijay.kilari
@ 2014-05-26 12:48   ` Julien Grall
  0 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-26 12:48 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> othercpu parameter in do_sgi is unused
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Ian and myself have acked the patch on v3. Unless if you heavily 
modified the patch, please retain the acks across the version.

It would avoid us to waste time to read again acked patch in a such big 
patch series.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers
  2014-05-26 10:26 ` [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers vijay.kilari
@ 2014-05-26 13:10   ` Julien Grall
  2014-05-30 12:54     ` Vijay Kilari
  2014-05-28 14:26   ` Stefano Stabellini
  1 sibling, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-26 13:10 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

Thank you for the patch. Did you verify that GICv2 is correctly working 
with theses changes?

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> gic-v2 driver uses fixmap to map the registers.
> Instead use ioremap to access mmio registers.
>
> With this patch, gic-v2 register definitions are updated
> to use obsolute offset address instead of dividing the
> register offset by 4.
>
> Update vgic driver logic to compute using obsolute register

s/obsolute/obsolete/

>       /* Local settings: interface controller */
> -    GICC[GICC_PMR] = 0xff;                /* Don't mask by priority */
> -    GICC[GICC_BPR] = 0;                   /* Finest granularity of priority */
> -    GICC[GICC_CTLR] = GICC_CTL_ENABLE|GICC_CTL_EOI;    /* Turn on delivery */
> +    writel_relaxed(0xff, GICC + GICC_PMR);                /* Don't mask by priority */
> +    writel_relaxed(0x0, GICC + GICC_BPR);                   /* Finest granularity of priority */
> +    writel_relaxed(GICC_CTL_ENABLE|GICC_CTL_EOI, GICC + GICC_CTLR);    /* Turn on delivery */

The coding style requests 80 characters per line.

> @@ -700,7 +713,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>                       irq, v->domain->domain_id, v->vcpu_id, i);
>   #endif
>       } else {
> -        GICH[GICH_LR + i] = 0;
> +        writel_relaxed(0, GICH + GICH_LR + i * 4);
>           clear_bit(i, &this_cpu(lr_mask));
>
>           if ( p->desc != NULL )
> @@ -808,7 +821,7 @@ int gic_events_need_delivery(void)
>       struct pending_irq *p;
>       unsigned long flags;
>
> -    mask_priority = (GICH[GICH_VMCR] >> GICH_VMCR_PRIORITY_SHIFT) & GICH_VMCR_PRIORITY_MASK;
> +    mask_priority = (readl_relaxed(GICH + GICH_VMCR) >> GICH_VMCR_PRIORITY_SHIFT) & GICH_VMCR_PRIORITY_MASK;

Same remark here.

>   /* Number of ranks of interrupt registers for a domain */
>   #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
> @@ -55,7 +55,7 @@ static inline int REG_RANK_NR(int b, uint32_t n)
>    * Offset of GICD_<FOO><n> with its rank, for GICD_<FOO> with
>    * <b>-bits-per-interrupt.
>    */
> -#define REG_RANK_INDEX(b, n) ((n) & ((b)-1))
> +#define REG_RANK_INDEX(b, n) (((n) >> 2) & ((b)-1))
>
>   /*
>    * Returns rank corresponding to a GICD_<FOO><n> register for
> @@ -63,7 +63,9 @@ static inline int REG_RANK_NR(int b, uint32_t n)
>    */
>   static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
>   {
> -    int rank = REG_RANK_NR(b, n);
> +    int rank;
> +    n = n >> 2;
> +    rank = REG_RANK_NR(b, n);

As n is only used for REG_RANK_NR, I would do:

int rank = REG_RANK_NR(b, n >> 2)

Otherwise than the few changes here, the patch looks good to me, 
assuming you did some tests on GICv2.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality
  2014-05-26 10:26 ` [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality vijay.kilari
@ 2014-05-26 14:09   ` Julien Grall
  2014-05-27 19:13   ` Julien Grall
  2014-05-28 14:43   ` Stefano Stabellini
  2 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-26 14:09 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> GIC driver contains both generic and hardware specific low
> level functionality in gic.c file.
>
> With this patch, low level functionality is moved to separate
> file gic-v2.c and generic code is kept in gic.c file
>
> Callbacks are registered by low level driver with generic driver
> and are called whereever required.

whereever? Did you intend to mean "when it's" ?

[..]

> +static void gicv2_enable_irq(struct irq_desc *irqd)
> +{
> +    int irq = irqd->irq;
> +    /* Enable routing */
> +    writel_relaxed((1u << (irq % 32)), GICD + GICD_ISENABLER + (irq / 32) * 4);
> +}
> +
> +static void gicv2_disable_irq(struct irq_desc *irqd)
> +{
> +    int irq = irqd->irq;
> +    /* Disable routing */
> +    writel_relaxed(1u << (irq % 32), GICD + GICD_ICENABLER + (irq / 32) * 4);
> +}

Why do you create gicv2_{enable,disable}_irq ? They are only used in 
gicv2_irq_{enable,disable}. Hence you've create odd names which may 
confuse the developer...

Please inline these 2 functions as the shouldn't be used directly.

[..]

> +static void gicv2_update_lr(int lr, const struct pending_irq *p,
> +                            unsigned int state)
> +{
> +    uint32_t lr_reg;
> +
> +    BUG_ON(lr >= gicv2_info.nr_lrs);
> +    BUG_ON(lr < 0);
> +    BUG_ON(state & ~(GICH_V2_LR_STATE_MASK << GICH_V2_LR_STATE_SHIFT));

Hmmm... why do you validate a value given by the common code with an 
architectural value?

[..]

> -static unsigned int gic_cpu_mask(const cpumask_t *cpumask)
> +static const struct gic_hw_operations *gic_hw_ops;
> +
> +void register_gic_ops(const struct gic_hw_operations *ops)
>   {
> -    unsigned int cpu;
> -    unsigned int mask = 0;
> -    cpumask_t possible_mask;
> +    gic_hw_ops = ops;
> +}
>
> -    cpumask_and(&possible_mask, cpumask, &cpu_possible_map);
> -    for_each_cpu(cpu, &possible_mask)
> -    {
> -        ASSERT(cpu < NR_GIC_CPU_IF);
> -        mask |= per_cpu(gic_cpu_id, cpu);
> -    }
> +void update_cpu_lr_mask(void)
> +{
> +    this_cpu(lr_mask) = 0ULL;
> +}

The name of this function doesn't match what it does. Hence, it seems 
you only call it during CPU initialization. Why can't you directly call 
in the common function gic_init_secondary_cpu?

>
> -    return mask;
> +int gic_hw_version(void)
> +{
> +   return gic_hw_ops->info->hw_version;

hw_version is an enum gic_version. Why don't you return a enum gic_version?

>   }
>
>   unsigned int gic_number_lines(void)
>   {
> -    return gic.lines;
> +    return gic_hw_ops->info->nr_lines;
>   }
>
>   void gic_save_state(struct vcpu *v)
>   {
> -    int i;
>       ASSERT(!local_irq_is_enabled());
>
> +    if ( is_idle_vcpu(v) )
> +        return;

I though this patch was only code movement... This change doesn't seem 
to be one.

[..]

> @@ -684,28 +317,31 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
>   static void gic_update_one_lr(struct vcpu *v, int i)
>   {

[..]

> -    } else if ( lr & GICH_LR_PENDING ) {
> +    } else if ( lr_val.state & GICH_LR_PENDING ) {

Please respect the coding style...

[..]

>   static void gic_restore_pending_irqs(struct vcpu *v)
>   {
> -    int lr = 0, lrs = nr_lrs;
> +    int lr = 0, lrs;
>       struct pending_irq *p, *t, *p_r;
>       struct list_head *inflight_r;
>       unsigned long flags;
> +    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
>
> +    lrs = nr_lrs;

Hmmm ... why did you create a temporary variable nr_lrs to set lrs just 
after???

>       spin_lock_irqsave(&v->arch.vgic.lock, flags);
>
>       if ( list_empty(&v->arch.vgic.lr_pending) )
> @@ -815,13 +454,15 @@ void gic_clear_pending_irqs(struct vcpu *v)
>
>   int gic_events_need_delivery(void)
>   {
> -    int mask_priority, lrs = nr_lrs;
> +    int mask_priority, lrs;
>       int max_priority = 0xff, active_priority = 0xff;
>       struct vcpu *v = current;
>       struct pending_irq *p;
>       unsigned long flags;
> +    unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
> +    lrs = nr_lrs;

Same question here.

>   static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
>   {
>       /* Lower the priority */
> -    writel_relaxed(sgi, GICC + GICC_EOIR);
> +    struct irq_desc *desc = irq_to_desc(sgi);

Missing newline here.

Also the comment "/* Lower the priority */" should be just before eoi_irq...

> +    gic_hw_ops->eoi_irq(desc);
>
>       switch (sgi)
>       {
> @@ -890,19 +531,17 @@ static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
>       }
>
>       /* Deactivate */
> -    writel_relaxed(sgi, GICC + GICC_DIR);
> +    gic_hw_ops->deactivate_irq(sgi);

I will try to send a patch to use desc->handler for SGI. It will avoid 
use to export {eoi,deactivate}_irq callback.

I'm not sure I will have time to do it before this serie will be upstreamed.

>   }
>
>   /* Accept an interrupt from the GIC and dispatch its handler */
>   void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
>   {
> -    uint32_t intack;
>       unsigned int irq;
>
> -
>       do  {
> -        intack = readl_relaxed(GICC + GICC_IAR);
> -        irq = intack & GICC_IA_IRQ;
> +        /* Reading IRQ will ACK it */
> +        irq = gic_hw_ops->read_irq();

[..]

> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 7fa3b95..f119bcd 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h

[..]


> +#define GICH_LR_PENDING         1
> +#define GICH_LR_ACTIVE          2

Ok that what I though.... you didn't test this code on GICv2 otherwise 
you will hit a BUG_ON in gicv2_update_lr. You are checking the value 
against the architectural value which will always fail.

Please either test this code (with the fastmodel if you don't have real 
hardware) or ask someone to test for you.

[..]

> @@ -213,9 +215,9 @@ enum gic_sgi {
>
>   /* SGI irq mode types */
>   enum gic_sgi_mode {
> -    SGI_TARGET_LIST,
> -    SGI_TARGET_OTHERS,
> -    SGI_TARGET_SELF,
> +    SGI_TARGET_LIST   = 0,
> +    SGI_TARGET_OTHERS = 1,
> +    SGI_TARGET_SELF   = 2,

Why did you introduce the enum in patch #3 and change it here? Hence, I 
don't think it's useful to assign a value for each item.

[..]

> +enum gic_version {
> +    GIC_V2 = 2,

Do you really need to assign a value here?

> +};
> +
> +struct gic_info {
> +    /* GIC version */
> +    enum gic_version hw_version;
> +    /* Number of GIC lines supported */
> +    unsigned int nr_lines;
> +    /* Number of LR registers */
> +    unsigned int nr_lrs;

nr_lrs as been changes to uin8_t with Stefano's series.

> +    /* Maintenance irq number */
> +    unsigned int maintenance_irq;
> +};
> +
> +struct gic_hw_operations {
> +    /* Hold GIC HW information */
> +    struct gic_info *info;

const gic_info *info ?

> +    /* hw_irq_controller to enable/disable/eoi host irq */
> +    hw_irq_controller *gic_host_irq_type;
> +
> +    /* hw_irq_controller to enable/disable/eoi guest irq */
> +    hw_irq_controller *gic_guest_irq_type;

It's a shame that we can't constify hw_irq_controller. I will look at it 
later.

> +    /* End of Interrupt */
> +    void (*eoi_irq)(struct irq_desc *irqd);
> +    /* Deactivate/reduce priority of irq */
> +    void (*deactivate_irq)(int);
> +    /* Read IRQ id and Ack */
> +    unsigned int (*read_irq)(void);
> +    /* Set IRQ property */
> +    void (*set_irq_properties)(struct irq_desc *desc,
> +                               const cpumask_t *cpu_mask,
> +                               unsigned int priority);
> +    /* Send SGI */
> +    void (*send_SGI)(const cpumask_t *online_mask,
> +                     enum gic_sgi sgi, uint8_t irqmode);

see patch #3 for my remark on the type of irqmode.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 07/16] arm/xen: move GIC context data structure to gic driver
  2014-05-26 10:26 ` [PATCH v4 07/16] arm/xen: move GIC context data structure to gic driver vijay.kilari
@ 2014-05-26 14:32   ` Julien Grall
  2014-05-28 14:49     ` Stefano Stabellini
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-26 14:32 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 61a498f..f46b631 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -8,6 +8,7 @@
>   #include <asm/p2m.h>
>   #include <asm/vfp.h>
>   #include <asm/mmio.h>
> +#include <asm/gic.h>
>   #include <public/hvm/params.h>
>   #include <xen/serial.h>
>
> @@ -261,8 +262,10 @@ struct arch_vcpu
>       uint32_t csselr;
>       register_t vmpidr;
>
> -    uint32_t gic_hcr, gic_vmcr, gic_apr;
> -    uint32_t gic_lr[64];
> +    /* Holds gic context data */
> +    union gic_state_data gic;
> +
> +    uint64_t event_mask;

Why do you add back this field? It has been dropped on upstream...

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 08/16] xen/arm: use device api to detect GIC version
  2014-05-26 10:26 ` [PATCH v4 08/16] xen/arm: use device api to detect GIC version vijay.kilari
@ 2014-05-26 14:39   ` Julien Grall
  2014-05-28 14:52     ` Stefano Stabellini
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-26 14:39 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
>   enum device_type
>   {
> -    DEVICE_SERIAL
> +    DEVICE_SERIAL,
> +    DEVICE_GIC,

Sounds like you are using an old version of Xen. This enum has been 
modified a couple of weeks ago. Please rebase you tree on the latest Xen.

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file
  2014-05-26 10:26 ` [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file vijay.kilari
@ 2014-05-27 11:32   ` Julien Grall
  2014-05-28 14:54   ` Stefano Stabellini
  1 sibling, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-27 11:32 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> new file mode 100644
> index 0000000..104a87d
> --- /dev/null
> +++ b/xen/include/asm-arm/vgic.h
> @@ -0,0 +1,40 @@
> +/*
> + * ARM Virtual Generic Interrupt Controller support
> + *
> + * Ian Campbell <ian.campbell@citrix.com>
> + * Copyright (c) 2011 Citrix Systems.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __ASM_ARM_VGIC_H__
> +#define __ASM_ARM_VGIC_H__
> +
> +/* Represents state corresponding to a block of 32 interrupts */
> +struct vgic_irq_rank {
> +    spinlock_t lock; /* Covers access to all other members of this struct */
> +    uint32_t ienable, iactive, ipend, pendsgi;
> +    uint32_t icfg[2];
> +    uint32_t ipriority[8];
> +    uint32_t itargets[8];
> +};
> +
> +extern int vcpu_vgic_free(struct vcpu *v);
> +#endif

NIT: this should be #endif /* __ASM_ARM_VGIC_H__ */

Acked-by: Julien Grall <julien.grall@linaro.org>

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 10/16] xen/arm: move vgic defines to vgic header file
  2014-05-26 10:26 ` [PATCH v4 10/16] xen/arm: move vgic defines to vgic " vijay.kilari
@ 2014-05-27 11:49   ` Julien Grall
  2014-06-10  8:30     ` Vijay Kilari
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-27 11:49 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Keir (Xen.org),
	Prasun.Kapoor, Ian Jackson, vijaya.kumar, Tim Deegan,
	Jan Beulich

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index f4c467a..2aadcb6 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -185,13 +185,6 @@ struct gic_lr {
>  
>  extern int gic_hw_version(void);
>  
> -extern int domain_vgic_init(struct domain *d);
> -extern void domain_vgic_free(struct domain *d);
> -
> -extern int vcpu_vgic_init(struct vcpu *v);
> -
> -extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
> -extern void vgic_clear_pending_irqs(struct vcpu *v);
>  extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);

irq_to_pending is also a VGIC function. Can you move it to vgic.h?

[..]

> +static inline uint32_t byte_read(uint32_t val, int sign, int offset)

[..]

> +static inline void byte_write(uint32_t *reg, uint32_t var, int offset)

On V3, I told you that these 2 names are too generic too be exported
here. Please add vgic_* before, so developer won't use them any where...

> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 44851ae..03416b5 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -826,6 +826,22 @@ void watchdog_domain_destroy(struct domain *d);
>  #define need_iommu(d)    (0)
>  #endif
>  
> +static inline int is_vcpu_running(struct domain *d, int vcpuid)
> +{

Please add the relevant maintainers when you add code in other part than
ARM. This time, I've cced them.

I think this code movement should be in another patch and justify why
this function should go in common code.

> +    struct vcpu *v;
> +
> +    if ( vcpuid >= d->max_vcpus )
> +        return 0;
> +
> +    v = d->vcpu[vcpuid];
> +    if ( v == NULL )
> +        return 0;
> +    if (test_bit(_VPF_down, &v->pause_flags) )
> +        return 0;
> +
> +    return 1;
> +}
> +
>  void set_vcpu_migration_delay(unsigned int delay);
>  unsigned int get_vcpu_migration_delay(void);
>  
> 

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-26 10:26 ` [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size vijay.kilari
@ 2014-05-27 11:56   ` Julien Grall
  2014-05-30  8:59     ` Vijay Kilari
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-27 11:56 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> -        *r = ( (v->domain->max_vcpus<<5) & GICD_TYPE_CPUS )
> -            |( ((v->domain->arch.vgic.nr_lines/32)) & GICD_TYPE_LINES );
> +        *r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
> +            |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );

Why these changes?

>          vgic_unlock(v);
>          return 1;
>      case GICD_IIDR:
> -        if ( dabt.size != 2 ) goto bad_width;
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>          /*
>           * XXX Do we need a JEP106 manufacturer ID?
>           * Just use the physical h/w value for now
> @@ -119,8 +119,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
>          goto read_as_zero;
>  
>      case GICD_ISENABLER ... GICD_ISENABLERN:
> -        if ( dabt.size != 2 ) goto bad_width;
> -        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);

In your commit message you explicitly say that use DABT_* will help you
to get the register offset but... you still hardcode the size.

Why can't you use dabt.size here? And all the other places.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 12/16] xen/arm: split vgic driver into generic and vgic-v2 driver
  2014-05-26 10:26 ` [PATCH v4 12/16] xen/arm: split vgic driver into generic and vgic-v2 driver vijay.kilari
@ 2014-05-27 16:50   ` Julien Grall
  0 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-27 16:50 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> +static int vgic_v2_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{

I didn't review closer this function. I guess it's a simple copy from
vgic_distr_mmio_read?

[..]

> +    /* Reserved -- read as zero */
> +    case 0x00c ... 0x01c:
> +    case 0x040 ... 0x07c:
> +    case 0x7fc:
> +    case 0xbfc:
> +    case 0xf04 ... 0xf0c:
> +    case 0xf30 ... 0xfcc:
> +        goto read_as_zero;

Hrrmm ... actually you dropped REG( ). Please specify every big changes
you made in the commit message.

[..]

> +static int vgic_v2_to_sgi(struct vcpu *v, register_t sgir)
> +{
> +
> +    int virq;
> +    int irqmode;
> +    unsigned long vcpu_mask = 0;
> +
> +    irqmode = (sgir  >> GICD_SGI_TARGET_LIST_SHIFT) & GICD_SGI_TARGET_LIST_MASK;

irqmode is an enum gic_sgi_mode, right? If so the type should be irqmode.

Futhermore in vgic_to_sgi you are using SGI_TARGET_* which as no
assigned value (see your enum). It doesn't sound right to blindly store
the architectural value in this enum...

> +    virq = (sgir & GICD_SGI_INTID_MASK);
> +    vcpu_mask = (sgir & GICD_SGI_TARGET_MASK) >> GICD_SGI_TARGET_SHIFT;
> +
> +    return vgic_to_sgi(v, sgir, irqmode, virq, vcpu_mask);
> +}
> +
> +static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{

Same remark vgic_v2_distr_mmio_write. I didn't review closer this function.

[..]

> +    case GICD_ICFGR: /* SGIs */
> +        goto write_ignore;
> +    case GICD_ICFGR + 1: /* PPIs */
> +        /* It is implementation defined if these are writeable. We chose not */
> +        goto write_ignore;
> +    case GICD_ICFGR + 2 ... GICD_ICFGRN: /* SPIs */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 2, gicd_reg - GICD_ICFGR, DABT_WORD);
> +        vgic_lock_rank(v, rank);
> +        if ( rank == NULL) goto write_ignore;

You've reintroduce the XSA-94 here (see bf70db7 vgic: Check rank in
GICD_ICFGR* emulation before locking). When you send a new version of a
serie, please check there is no update on this code which may fix error.

I saw you shared a part of the emulation between the distributor and the
redistributor in GICv3. I think you can also share with GICv2, this
could avoid fix in 2 places the same bug (or worst only fixing in 1 place).

[..]

> -static int vgic_to_sgi(struct vcpu *v, register_t sgir)
> +int vgic_to_sgi(struct vcpu *v, register_t sgir, int irqmode, int virq,

irqmode should be enum gic_sgi_mode.

> +                unsigned long vcpu_mask)

You can't assume that all the VCPU bits will fit in an unsigned long. We
will have to use cpumask_t at some point.

I'm fine if you don't handle it for now, but you need to write down
somewhere the limitation of this function.

[..]

> +    case SGI_TARGET_OTHERS:

> +    case SGI_TARGET_SELF:

For this 2 case, you can't assume that vcpu_mask will be equal to 0...
It comes from the GICD_SGIR...

> +    default:
> +        gdprintk(XENLOG_WARNING, "vGICD: unhandled GICD_SGIR write %"PRIregister" with wrong mode\n",
> +                 sgir);
> +       return 0;
>      }
>  
>      for_each_set_bit( vcpuid, &vcpu_mask, d->max_vcpus )
>      {
>          if ( !is_vcpu_running(d, vcpuid) )
>          {
> -            gdprintk(XENLOG_WARNING, "vGICD: GICD_SGIR write r=%"PRIregister" vcpu_mask=%lx, wrong CPUTargetList\n",
> +            gdprintk(XENLOG_WARNING, " write r=%"PRIregister" vcpu_mask=%lx, wrong CPUTargetList\n",

For clarity, I would keep at least vGIC in the warning message.

[..]

> @@ -654,7 +254,6 @@ out:
>  int domain_vgic_init(struct domain *d)
>  {
>      int i;
> -

Spurious change? And also invalid following the coding style.

>      d->arch.vgic.ctlr = 0;
>  
>      /* Currently nr_lines in vgic and gic doesn't have the same meanings
> @@ -665,20 +264,34 @@ int domain_vgic_init(struct domain *d)
>      else
>          d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
>  
> +    if ( gic_hw_version() == GIC_V2 )
> +        vgic_v2_init(d);
> +    else
> +        panic("No VGIC found\n");

As said on V3, panic is not the right solution in a domain creation code
path. You should return here.

Futhermore, gic_hw_version is returning an enum. I would use switch/case
there.

[..]

> +    if ( d->arch.vgic.pending_irqs == NULL )
> +    {
> +        xfree(d->arch.vgic.shared_irqs);
> +        return -ENOMEM;
> +    }
> +

Hrrrmmm... this change has been pused more than one month ago on master.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality
  2014-05-26 10:26 ` [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality vijay.kilari
  2014-05-26 14:09   ` Julien Grall
@ 2014-05-27 19:13   ` Julien Grall
  2014-05-28 14:43   ` Stefano Stabellini
  2 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-27 19:13 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> +    /*
> +     * Domain 0 gets the hardware address.
> +     * Guests get the virtual platform layout.
> +     */
> +    if ( d->domain_id == 0 )
> +    {
> +        d->arch.vgic.dbase = gicv2.dbase;
> +        d->arch.vgic.cbase = gicv2.cbase;
> +    }

I guess you had conflict when you rebase your patch... You should avoid
to ignore them.

Some of this code (here is one of the example) is modified while you are
sending new version of your patch series.

The original code (i.e in gic.c) was:

if ( is_hardware_domain(d) )
{
	d->arch.vgic.dbase = gicv2.dbase;
	d->arch.vgic.cbase = gicv2.cbase;
}

Now, you've moved the code and we end up to:

if ( d->domain_id == 0 )
{
	d->arch.vgic.dbase = gicv2.dbase;
	d->arch.vgic.cbase = gicv2.cbase;
}

I didn't check every place closely, but this can be a serious problem
with Stefano's patch series. His serie still baking and he made some
important changes in the v8 serie. We don't want to loose them.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-05-26 10:26 ` [PATCH v4 13/16] xen/arm: Add support for GIC v3 vijay.kilari
@ 2014-05-27 19:47   ` Julien Grall
  2014-06-02 17:33   ` Stefano Stabellini
  1 sibling, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-27 19:47 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Keir (Xen.org),
	Prasun.Kapoor, Ian Jackson, vijaya.kumar, Tim Deegan,
	Jan Beulich

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> Add support for GIC v3 specification System register access(SRE)
> is enabled to access cpu and virtual interface regiseters based

s/regiseters/register/

[..]

> +#include <xen/config.h>
> +#include <xen/lib.h>
> +#include <xen/init.h>
> +#include <xen/cpu.h>
> +#include <xen/mm.h>
> +#include <xen/irq.h>
> +#include <xen/sched.h>
> +#include <xen/errno.h>
> +#include <xen/delay.h>
> +#include <xen/device_tree.h>
> +#include <xen/libfdt/libfdt.h>

NIT: I don't think you need this include...

> +#include <asm/p2m.h>
> +#include <asm/domain.h>
> +#include <asm/platform.h>

Same here.

> +#include <asm/io.h>
> +#include <asm/device.h>
> +#include <asm/gic.h>
> +#include <asm/gic_v3_defs.h>
> +
> +struct rdist_region {
> +    paddr_t base;
> +    paddr_t size;
> +    void __iomem *map_base;
> +};
> +
> +/* Global state */
> +static struct {
> +    paddr_t dbase;            /* Address of distributor registers */
> +    paddr_t dbase_size;
> +    void __iomem *map_dbase;  /* Mapped address of distributor registers */
> +    struct rdist_region *rdist_regions;
> +    uint32_t  rdist_stride;
> +    unsigned int rdist_count; /* Number of rdist regions count */
> +    unsigned int nr_priorities;
> +    spinlock_t lock;
> +}gicv3;

Missing after the brace.

> +
> +static struct gic_info gicv3_info;


> +/*
> + * System Register Enable (SRE). Enable to access CPU & Virtual
> + * interface registers as system registers in EL2
> + */
> +static void gicv3_enable_sre(void)
> +{
> +    uint32_t val;
> +
> +    val = READ_SYSREG32(ICC_SRE_EL2);
> +    val |= GICC_SRE_EL2_SRE | GICC_SRE_EL2_ENEL1 |
> +           GICC_SRE_EL2_DFB | GICC_SRE_EL2_DIB;

Please add a bit of comment explaining why you choose to enable those bits.

It's clear for SRE and ENEL1 but not DFB and DIB...

[..]

> +static void gicv3_save_state(struct vcpu *v)
> +{
> +
> +    /* No need for spinlocks here because interrupts are disabled around
> +     * this call and it only accesses struct vcpu fields that cannot be
> +     * accessed simultaneously by another pCPU.
> +     */
> +    dsb(sy);

Why a dsb here? I don't think it's useful. Hence it was not there on v3.

> +    gicv3_save_lr(gicv3_info.nr_lrs, v);

You are saving multiple LRs, so gicv3_save_lrs sounds a better name.

[..]

> +static void gicv3_disable_irq(struct irq_desc *irqd)
> +{
> +    /* Disable routing */
> +    if ( irqd->irq < NR_GIC_LOCAL_IRQS )
> +        writel_relaxed((1u << irqd->irq),
> +                       GICD_RDIST_SGI_BASE + GICR_ICENABLER0);
> +    else
> +        writel_relaxed(1u << (irqd->irq % 32),
> +                       GICD + GICD_ICENABLER + ((irqd->irq / 32) * 4));

Don't you forget gicv3_wait_for_rwp here?

The code of gicv3_disable_irq is very similar to gicv3_enable_irq. I
would create an helper here (see gic_poke_irq) on GICv3 for Linux.

> +}
> +
> +static void gicv3_eoi_irq(struct irq_desc *irqd)
> +{
> +    /* Lower the priority */
> +    WRITE_SYSREG32(irqd->irq, ICC_EOIR1_EL1);

Write to a sysreg register requires an isb after.

> +}
> +
> +static void gicv3_dir_irq(int irq)
> +{
> +    /* Deactivate */
> +    WRITE_SYSREG32(irq, ICC_DIR_EL1);

isb();

> +}
> +
> +static unsigned int gicv3_read_irq(void)
> +{
> +    return (READ_SYSREG32(ICC_IAR1_EL1) & GICC_IA_IRQ);
> +}
> +
> +static inline uint64_t gicv3_mpidr_to_affinity(uint64_t mpidr)

As said on V3, you are using
gicv3_mpidr_to_affinity(cpu_logical_map(foo)) every where.
You can rework this function to get a cpu in parameter and return the
affinity.

It will avoid you duplicate code.

> +{
> +     return (MPIDR_AFFINITY_LEVEL(mpidr, 3) << 32 |
> +             MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16 |
> +             MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8  |
> +             MPIDR_AFFINITY_LEVEL(mpidr, 0));
> +}
> +
> +static void gicv3_set_irq_properties(struct irq_desc *desc,
> +                                     const cpumask_t *cpu_mask,
> +                                     unsigned int priority)
> +{
> +    unsigned long flags;
> +    uint32_t cfg, edgebit;
> +    uint64_t affinity;
> +    void __iomem *base;
> +    unsigned int cpu = gicv3_get_cpu_from_mask(cpu_mask);
> +    unsigned int irq = desc->irq;
> +    unsigned int type = desc->arch.type;
> +
> +    spin_lock_irqsave(&gicv3.lock, flags);
> +
> +    /* SGI's are always edge-triggered not need to call GICD_ICFGR0 */
> +    if ( irq >= NR_GIC_SGI )
> +    {

gicv3_set_irq_properties is never called on SGI. Except adding another
indentation this is not useful.

> +        if ( irq < NR_GIC_LOCAL_IRQS)
> +            base = GICD + GICD_ICFGR + (irq / 16) * 4;
> +        else
> +            base = GICD_RDIST_SGI_BASE + GICR_ICFGR1;
> +
> +        cfg = readl_relaxed(base);
> +
> +        edgebit = 2u << (2 * (irq % 16));
> +        if ( type & DT_IRQ_TYPE_LEVEL_MASK )
> +           cfg &= ~edgebit;
> +        else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
> +           cfg |= edgebit;
> +
> +        writel_relaxed(cfg, base);
> +    }
> +
> +    affinity = gicv3_mpidr_to_affinity(cpu_logical_map(cpu));
> +    /* Make sure we don't broadcast the interrupt */
> +    affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
> +
> +    if ( irq >= NR_GIC_LOCAL_IRQS )

Sometimes you use this define and sometime the hardcode 32... Please be
consistent.

[..]

> +static int gicv3_enable_redist(void)
> +{
> +    uint32_t val;
> +    bool_t timeout = 0;
> +    s_time_t deadline = NOW() + MILLISECS(1000);
> +
> +    /* Wake up this CPU redistributor */
> +    val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
> +    val &= ~GICR_WAKER_ProcessorSleep;
> +    writel_relaxed(val, GICD_RDIST_BASE + GICR_WAKER);
> +
> +    do {
> +        val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
> +        if ( !(val & GICR_WAKER_ChildrenAsleep) )
> +            break;
> +        if ( NOW() > deadline )
> +        {
> +            timeout = 1;
> +            break;
> +        }
> +        cpu_relax();
> +        udelay(1);
> +    } while ( timeout );
> +
> +    if ( timeout )
> +        dprintk(XENLOG_ERR, "GICv3: Redist enable RWP timeout\n");
> +
> +    return 0;

Shouldn't you return an error if we timeout?

> +}
> +
> +static int __init gicv3_populate_rdist(void)
> +{
> +    int i;
> +    uint32_t aff;
> +    uint32_t reg;
> +    uint64_t typer;
> +    uint64_t mpidr = cpu_logical_map(smp_processor_id());
> +
> +    aff = gicv3_mpidr_to_affinity(mpidr);

This function is returning an uint64_t but you are using a uint32_t to
store the value. Is it normal?

> +
> +    for ( i = 0; i < gicv3.rdist_count; i++ )
> +    {
> +        void __iomem *ptr = gicv3.rdist_regions[i].map_base;
> +
> +        reg = readl_relaxed(ptr + GICR_PIDR2) & GICR_PIDR2_ARCH_MASK;
> +        if ( reg  != (GICV3_GICR_PIDR2 & GICR_PIDR2_ARCH_MASK) )

GICV3_GICR_PIDR2 is used for the emulation. You can't use this value to
test hardware stuff. Please use 0x30 (GICv3) which is clearer.

> +        {
> +            dprintk(XENLOG_ERR,
> +                    "GICv3: No redistributor present @%"PRIpaddr"\n",
> +                    gicv3.rdist_regions[i].base);
> +            break;
> +        }
> +
> +        do {
> +            typer = readq_relaxed(ptr + GICR_TYPER);
> +
> +            if ( (typer >> 32) == aff )
> +            {

Hrrmmm... now I understand why aff is 32 bits. Computing manually
(rather than using gicv3_mpidr_to_affinity function) was right on v3.
This was because TYPER doesn't exactly match the affinity field.

While you add again the correct affinity, please also add a comment such
as: "Convert affinity to a 32bit value that can be matched to GICR_TYPER
bits [63:32]."

> +                this_cpu(rbase) = ptr;
> +                printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
> +                        smp_processor_id(), i, ptr);
> +                return 0;
> +            }
> +
> +            ptr += gicv3.rdist_stride;
> +            if ( typer & GICR_TYPER_VLPIS )
> +                    ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */

You differ from Linux code here (even though VLPIS is RAZ on GIcv3). The
Linux driver is only check VLPIS when rdist_stride is equal to 0.

Actually, you set rdist_stride to SZ_64K * 2 if the property is not
existent. I guess you have to check with Marc Zyngier what's the correct
solution.

[..]

> +    /* No priority grouping */
> +    WRITE_SYSREG32(0, ICC_BPR1_EL1);
> +
> +    /* Set priority mask register */
> +    WRITE_SYSREG32(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
> +
> +    /* EOI drops priority too (mode 0) */
> +    WRITE_SYSREG32(GICC_CTLR_EL1_EOImode_drop, ICC_CTLR_EL1);

isb() ?

> +
> +    /* Enable Group1 interrupts */
> +    WRITE_SYSREG32(1, ICC_IGRPEN1_EL1);

isb() ?

> +
> +    return 0;
> +}
> +
> +static void gicv3_cpu_disable(void)
> +{
> +    WRITE_SYSREG32(0, ICC_CTLR_EL1);

isb() ?

> +}
> +
> +static void __cpuinit gicv3_hyp_init(void)
> +{
> +    uint32_t vtr;
> +
> +    vtr = READ_SYSREG32(ICH_VTR_EL2);
> +    gicv3_info.nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
> +    gicv3.nr_priorities = ((vtr >> GICH_VTR_PRIBITS_SHIFT) &
> +                          GICH_VTR_PRIBITS_MASK) + 1;
> +
> +    ASSERT((gicv3.nr_priorities > 4 && gicv3.nr_priorities < 8));

For hardware "feature" checking, it's better to print a correct error
message than ASSERT (which is not enabled on non-debug build).

> +
> +    WRITE_SYSREG32(GICH_VMCR_EOI | GICH_VMCR_VENG1, ICH_VMCR_EL2);
> +    WRITE_SYSREG32(GICH_HCR_EN, ICH_HCR_EL2);

isb()?

[..]

> +static void __cpuinit gicv3_hyp_disable(void)
> +{
> +    uint32_t vtr;
> +
> +    vtr = READ_SYSREG32(ICH_HCR_EL2);
> +    vtr &= ~0x1;
> +    WRITE_SYSREG32( vtr, ICH_HCR_EL2);

isb() ?

[..]

> +static void gicv3_send_sgi(const cpumask_t *cpumask, enum gic_sgi sgi,
> +                           uint8_t mode)
> +{
> +    int cpu = 0;
> +    uint64_t val;
> +
> +    dsb(sy);

Why a dsb there?

[..]

> +static void gicv3_update_lr(int lr, const struct pending_irq *p,
> +                            unsigned int state)
> +{
> +    uint64_t grp = GICH_LR_GRP1;
> +    uint64_t val = 0;
> +
> +    BUG_ON(lr >= gicv3_info.nr_lrs);
> +    BUG_ON(lr < 0);
> +
> +    val =  (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
> +    val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
> +    val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
> +
> +   if ( p->desc != NULL )
> +       val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
> +                           << GICH_LR_PHYSICAL_SHIFT);
> +
> +    gicv3_ich_write_lr(lr, val);

isb() ? Might be better directly in gicv3_ich_write_lr.

> +}
> +
> +static void gicv3_clear_lr(int lr)
> +{
> +    gicv3_ich_write_lr(lr, 0);

Same question here and ...

[..]

> +static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
> +{
> +    uint64_t lrv = 0;
> +
> +    lrv = ( ((u64)(lr->pirq & GICH_LR_PHYSICAL_MASK) << GICH_LR_PHYSICAL_SHIFT)|
> +        ((u64)(lr->virq & GICH_LR_VIRTUAL_MASK)  << GICH_LR_VIRTUAL_SHIFT) |
> +        ((u64)(lr->priority & GICH_LR_PRIORITY_MASK) << GICH_LR_PRIORITY_SHIFT)|
> +        ((u64)(lr->state & GICH_LR_STATE_MASK) << GICH_LR_STATE_SHIFT) |
> +        ((u64)(lr->hw_status & GICH_LR_HW_MASK) << GICH_LR_HW_SHIFT)  |
> +        ((u64)(lr->grp & GICH_LR_GRP_MASK) << GICH_LR_GRP_SHIFT) );
> +
> +    gicv3_ich_write_lr(lr_reg, lrv);

here.

> +}
> +
> +static int gicv_v3_init(struct domain *d)
> +{
> +    int i;
> +
> +    /*
> +     * Domain 0 gets the hardware address.
> +     * Guests get the virtual platform layout.
> +     */
> +    if ( d->domain_id == 0 )

is_hardware_domain(d)

[..]

> +static void gicv3_hcr_status(uint32_t flag, bool_t status)
> +{
> +    uint32_t hcr;
> +
> +    hcr = READ_SYSREG32(ICH_HCR_EL2);
> +    if ( status )
> +        WRITE_SYSREG32(hcr | flag, ICH_HCR_EL2);
> +    else
> +        WRITE_SYSREG32(hcr & (~flag), ICH_HCR_EL2);

isb()?

> +static void gicv3_host_irq_end(struct irq_desc *desc)
> +{
> +    /* Lower the priority */
> +    gicv3_eoi_irq(desc);
> +    /* Deactivate */
> +    gicv3_dir_irq(desc->irq);

You are not consistent between the 2 function. One is taking an irq_desc
the other an unsigned int.

[..]

> +const static hw_irq_controller gicv3_host_irq_type = {

[..]

> +const static hw_irq_controller gicv3_guest_irq_type = {

[..]

> +const static struct gic_hw_operations gicv3_ops = {

We tend to use static const rather than "const static".

> +
> +/* Set up the GIC */
> +static int __init gicv3_init(struct dt_device_node *node, const void *data)
> +{
> +    struct rdist_region *rdist_regs;
> +    int res, i;
> +    uint32_t reg;
> +
> +    dt_device_set_used_by(node, DOMID_XEN);
> +
> +    res = dt_device_get_address(node, 0, &gicv3.dbase, &gicv3.dbase_size);
> +    if ( res || !gicv3.dbase )
> +        panic("GICv3: Cannot find a valid distributor address");
> +
> +    if ( (gicv3.dbase & ~PAGE_MASK) || (gicv3.dbase_size & ~PAGE_MASK) )
> +        panic("GICv3:  Found unaligned distributor address %"PRIpaddr"",
> +              gicv3.dbase);
> +
> +    gicv3.map_dbase = ioremap_nocache(gicv3.dbase, gicv3.dbase_size);
> +    if ( !gicv3.map_dbase )
> +        panic("GICv3: Failed to ioremap for GIC distributor\n");
> +
> +    reg = readl_relaxed(GICD + GICD_PIDR2) & GICD_PIDR2_ARCH_MASK;
> +    if ( reg  != (GICV3_GICD_PIDR2 & GICD_PIDR2_ARCH_MASK) )
> +         panic("GICv3: no distributor detected\n");
> +
> +    if ( !dt_property_read_u32(node, "#redistributor-regions",
> +                &gicv3.rdist_count) )
> +        gicv3.rdist_count = 1;
> +
> +    if ( gicv3.rdist_count > MAX_RDIST_COUNT )
> +        panic("GICv3: Number of redistributor regions is more than \

You have to end the line with a " then beginning the new one with a ".
With your solution the compiler will drop append the second line from
the beginning, i.e with all the space...

> +               %d (Increase MAX_RDIST_COUNT!!)\n", MAX_RDIST_COUNT);
> +
> +    rdist_regs = xzalloc_array(struct rdist_region, gicv3.rdist_count);
> +    if ( !rdist_regs )
> +        panic("GICv3: Failed to allocate memory for rdist regions\n");
> +
> +    for ( i = 0; i < gicv3.rdist_count; i++ )
> +    {
> +        uint64_t rdist_base, rdist_size;
> +
> +        res = dt_device_get_address(node, 1 + i, &rdist_base, &rdist_size);
> +        if ( res || !rdist_base )
> +            panic("GICv3: No rdist base found for region %d\n", i);
> +
> +        rdist_regs[i].base = rdist_base;
> +        rdist_regs[i].size = rdist_size;
> +    }
> +
> +    /* If stride is not set in dt. Set default to 2 * SZ_64K */
> +    if ( !dt_property_read_u32(node, "redistributor-stride", &gicv3.rdist_stride) )
> +        gicv3.rdist_stride = 2 * SZ_64K;

I don't think this is right, see my comment a bit above for the function
gicv3_populate_rdist.

[..]

> diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
> index 1369b2b..499a798 100644
> --- a/xen/include/xen/lib.h
> +++ b/xen/include/xen/lib.h
> @@ -67,6 +67,8 @@ do {                                                            \
>  
>  #define reserve_bootmem(_p,_l) ((void)0)
>  
> +#define SZ_64K  0x00010000
> +
>  struct domain;
>  
>  void cmdline_parse(const char *cmdline);
> 

Same remark as on patch #10, you have to cc the relevant maintenairs for
every file you've modified (cced them).

You can use script/get_maintainers.pl for this purpose.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 00/16] xen/arm: Add GICv3 support
  2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
                   ` (15 preceding siblings ...)
  2014-05-26 10:26 ` [PATCH v4 16/16] xen/arm: add SGI handling for GICv3 vijay.kilari
@ 2014-05-28 10:26 ` Ian Campbell
  2014-05-28 12:34   ` Ian Campbell
  16 siblings, 1 reply; 78+ messages in thread
From: Ian Campbell @ 2014-05-28 10:26 UTC (permalink / raw)
  To: vijay.kilari
  Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, julien.grall,
	xen-devel, stefano.stabellini

On Mon, 2014-05-26 at 15:56 +0530, vijay.kilari@gmail.com wrote:
>  - Rebased on remotes/origin/no_maintenance_interrupts-v8

I assume you mean from Stefano's tree? "remote/origin" doesn't tell me
much.

>  + Julien's
>    patch set

What is this? Julien has posted lots of patch sets.

It's good that you say what trees you are basing things on, thanks, but
for it to be any use you need to be more specific (ideally a URL and a
branch name).

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 00/16] xen/arm: Add GICv3 support
  2014-05-28 10:26 ` [PATCH v4 00/16] xen/arm: Add GICv3 support Ian Campbell
@ 2014-05-28 12:34   ` Ian Campbell
  0 siblings, 0 replies; 78+ messages in thread
From: Ian Campbell @ 2014-05-28 12:34 UTC (permalink / raw)
  To: vijay.kilari
  Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, julien.grall,
	xen-devel, stefano.stabellini

On Wed, 2014-05-28 at 11:26 +0100, Ian Campbell wrote:
> On Mon, 2014-05-26 at 15:56 +0530, vijay.kilari@gmail.com wrote:
> >  - Rebased on remotes/origin/no_maintenance_interrupts-v8
> 
> I assume you mean from Stefano's tree? "remote/origin" doesn't tell me
> much.
> 
> >  + Julien's
> >    patch set
> 
> What is this? Julien has posted lots of patch sets.

> 
> It's good that you say what trees you are basing things on, thanks, but
> for it to be any use you need to be more specific (ideally a URL and a
> branch name).

Case in point, I've tried applying this to Stefano's
no_maintenance_interrutps-v8 + v4,v5,v6 and v7 of Julien's "interrupt
management reworking" series and it didn't apply on top of any of those
combinations.

Can you register for gitorious or something and push a tree there?

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder
  2014-05-26 10:26 ` [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder vijay.kilari
  2014-05-26 11:28   ` Julien Grall
@ 2014-05-28 13:55   ` Stefano Stabellini
  1 sibling, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 13:55 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> io.h is local to arch/arm folder. move this file as mmio.h
> file to include/asm-arm folder as it might be
> required for inclusion in other header files in future.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>


>  xen/arch/arm/io.c                             |    3 +--
>  xen/arch/arm/traps.c                          |    2 +-
>  xen/arch/arm/vgic.c                           |    2 +-
>  xen/arch/arm/vuart.c                          |    2 +-
>  xen/{arch/arm/io.h => include/asm-arm/mmio.h} |    8 ++++----
>  5 files changed, 8 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
> index a6db00b..ada1918 100644
> --- a/xen/arch/arm/io.c
> +++ b/xen/arch/arm/io.c
> @@ -19,8 +19,7 @@
>  #include <xen/config.h>
>  #include <xen/lib.h>
>  #include <asm/current.h>
> -
> -#include "io.h"
> +#include <asm/mmio.h>
>  
>  static const struct mmio_handler *const mmio_handlers[] =
>  {
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index a4bdaaa..9348147 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -36,9 +36,9 @@
>  #include <asm/regs.h>
>  #include <asm/cpregs.h>
>  #include <asm/psci.h>
> +#include <asm/mmio.h>
>  
>  #include "decode.h"
> -#include "io.h"
>  #include "vtimer.h"
>  #include <asm/gic.h>
>  
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 4869b87..4962e70 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -27,7 +27,7 @@
>  
>  #include <asm/current.h>
>  
> -#include "io.h"
> +#include <asm/mmio.h>
>  #include <asm/gic.h>
>  
>  #define REG(n) (n/4)
> diff --git a/xen/arch/arm/vuart.c b/xen/arch/arm/vuart.c
> index c02a8a9..953cd46 100644
> --- a/xen/arch/arm/vuart.c
> +++ b/xen/arch/arm/vuart.c
> @@ -38,9 +38,9 @@
>  #include <xen/errno.h>
>  #include <xen/ctype.h>
>  #include <xen/serial.h>
> +#include <asm/mmio.h>
>  
>  #include "vuart.h"
> -#include "io.h"
>  
>  #define domain_has_vuart(d) ((d)->arch.vuart.info != NULL)
>  
> diff --git a/xen/arch/arm/io.h b/xen/include/asm-arm/mmio.h
> similarity index 91%
> rename from xen/arch/arm/io.h
> rename to xen/include/asm-arm/mmio.h
> index 8d252c0..5870985 100644
> --- a/xen/arch/arm/io.h
> +++ b/xen/include/asm-arm/mmio.h
> @@ -1,5 +1,5 @@
>  /*
> - * xen/arch/arm/io.h
> + * xen/include/asm-arm/mmio.h
>   *
>   * ARM I/O handlers
>   *
> @@ -16,8 +16,8 @@
>   * GNU General Public License for more details.
>   */
>  
> -#ifndef __ARCH_ARM_IO_H__
> -#define __ARCH_ARM_IO_H__
> +#ifndef __ASM_ARM_MMIO_H__
> +#define __ASM_ARM_MMIO_H__
>  
>  #include <xen/lib.h>
>  #include <asm/processor.h>
> @@ -45,7 +45,7 @@ extern const struct mmio_handler vuart_mmio_handler;
>  
>  extern int handle_mmio(mmio_info_t *info);
>  
> -#endif
> +#endif  /* __ASM_ARM_MMIO_H__ */
>  
>  /*
>   * Local variables:
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 02/16] xen/arm: make mmio handlers domain specific
  2014-05-26 12:33   ` Julien Grall
@ 2014-05-28 14:05     ` Stefano Stabellini
  2014-05-28 14:11       ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 14:05 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian.Campbell, vijay.kilari, stefano.stabellini, Prasun.Kapoor,
	vijaya.kumar, xen-devel, stefano.stabellini

On Mon, 26 May 2014, Julien Grall wrote:
> Hi Vijay,
> 
> On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> >   int handle_mmio(mmio_info_t *info)
> >   {
> >       struct vcpu *v = current;
> >       int i;
> > +    struct mmio_handler *mmio_handler;
> > +    struct io_handler *io_handlers = &v->domain->arch.io_handlers;
> 
> NIT: I think mmio_handler and io_handlers can be const.
> 
> > +
> > +void register_mmio_handler(struct domain *d,
> > +                           const struct mmio_handler_ops *handle,
> > +                           paddr_t addr, paddr_t size)
> > +{
> > +    struct io_handler *handler = &d->arch.io_handlers;
> > +
> > +    BUG_ON(handler->num_entries >= MAX_IO_HANDLER);
> > +
> > +    spin_lock(&handler->lock);
> > +
> > +    handler->mmio_handlers[handler->num_entries].mmio_handler_ops = handle;
> > +    handler->mmio_handlers[handler->num_entries].addr = addr;
> > +    handler->mmio_handlers[handler->num_entries].size = size;
> > +    handler->num_entries++;
> > +    dsb(sy);
> 
> This is wrong. As I said on the previous version, the dsb needs to be called
> before incrementing the num_entries.
> 
> This is because as you don't use spinlock in handle_mmio, you have to make
> sure the array modification has reached the memory before update num_entries.
> 
> At the same time dsb(is) is enough.

yeah, this needs to be fixed


> > diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> > index 4962e70..151ec3e 100644
> > --- a/xen/arch/arm/vgic.c
> > +++ b/xen/arch/arm/vgic.c
> > @@ -73,43 +73,6 @@ static struct vgic_irq_rank *vgic_irq_rank(struct vcpu
> > *v, int b, int n)
> >           return NULL;
> >   }
> > 
> > -int domain_vgic_init(struct domain *d)
> > -{
> > -    int i;
> > -
> > -    d->arch.vgic.ctlr = 0;
> > -
> > -    /* Currently nr_lines in vgic and gic doesn't have the same meanings
> > -     * Here nr_lines = number of SPIs
> > -     */
> > -    if ( is_hardware_domain(d) )
> > -        d->arch.vgic.nr_lines = gic_number_lines() - 32;
> > -    else
> > -        d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
> > -
> > -    d->arch.vgic.shared_irqs =
> > -        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
> > -    if ( d->arch.vgic.shared_irqs == NULL )
> > -        return -ENOMEM;
> > -
> > -    d->arch.vgic.pending_irqs =
> > -        xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
> > -    if ( d->arch.vgic.pending_irqs == NULL )
> > -    {
> > -        xfree(d->arch.vgic.shared_irqs);
> > -        return -ENOMEM;
> > -    }
> > -
> > -    for (i=0; i<d->arch.vgic.nr_lines; i++)
> > -    {
> > -        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
> > -        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
> > -    }
> > -    for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> > -        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
> > -    return 0;
> > -}
> > -
> 
> Rather than moving a whole chunk of code, why can't you add forward
> declaration for vgic_disk_mmio_{read,write}?

I think it would be OK either way


> >   void domain_vgic_free(struct domain *d)
> >   {
> >       xfree(d->arch.vgic.shared_irqs);
> > @@ -676,15 +639,7 @@ write_ignore:
> >       return 1;
> >   }
> > 
> > -static int vgic_distr_mmio_check(struct vcpu *v, paddr_t addr)
> > -{
> > -    struct domain *d = v->domain;
> > -
> > -    return (addr >= (d->arch.vgic.dbase)) && (addr < (d->arch.vgic.dbase +
> > PAGE_SIZE));
> > -}
> > -
> > -const struct mmio_handler vgic_distr_mmio_handler = {
> > -    .check_handler = vgic_distr_mmio_check,
> > +const struct mmio_handler_ops vgic_distr_mmio_handler = {
> >       .read_handler  = vgic_distr_mmio_read,
> >       .write_handler = vgic_distr_mmio_write,
> >   };
> > @@ -766,6 +721,38 @@ out:
> >           smp_send_event_check_mask(cpumask_of(v->processor));
> >   }
> > 
> > +int domain_vgic_init(struct domain *d)
> > +{
> > +    int i;
> > +
> > +    d->arch.vgic.ctlr = 0;
> > +
> > +    /* Currently nr_lines in vgic and gic doesn't have the same meanings
> > +     * Here nr_lines = number of SPIs
> > +     */
> > +    if ( d->domain_id == 0 )
> > +        d->arch.vgic.nr_lines = gic_number_lines() - 32;
> > +    else
> > +        d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
> > +
> > +    d->arch.vgic.shared_irqs =
> > +        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
> > +    d->arch.vgic.pending_irqs =
> > +        xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
> > +    for (i=0; i<d->arch.vgic.nr_lines; i++)
> > +    {
> > +        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
> > +        INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
> > +    }
> > +    for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> > +        spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
> > +
> > +    register_mmio_handler(d, &vgic_distr_mmio_handler,
> > +                          d->arch.vgic.dbase, PAGE_SIZE);
> > +
> 
> Sounds like a bit strange to call register_mmio_handler here and let
> gicv_setup set dbase. Can you add a comment saying to smth like "We rely on
> gicv_setup to initialize dbase"?
> 
> > +    return 0;
> > +}
> > +
> >   /*
> >    * Local variables:
> >    * mode: C
> > diff --git a/xen/arch/arm/vuart.c b/xen/arch/arm/vuart.c
> > index 953cd46..52f3259 100644
> > --- a/xen/arch/arm/vuart.c
> > +++ b/xen/arch/arm/vuart.c
> > @@ -44,24 +44,6 @@
> > 
> >   #define domain_has_vuart(d) ((d)->arch.vuart.info != NULL)
> > 
> > -int domain_vuart_init(struct domain *d)
> > -{
> > -    ASSERT( is_hardware_domain(d) );
> > -
> > -    d->arch.vuart.info = serial_vuart_info(SERHND_DTUART);
> > -    if ( !d->arch.vuart.info )
> > -        return 0;
> > -
> > -    spin_lock_init(&d->arch.vuart.lock);
> > -    d->arch.vuart.idx = 0;
> > -
> > -    d->arch.vuart.buf = xzalloc_array(char, VUART_BUF_SIZE);
> > -    if ( !d->arch.vuart.buf )
> > -        return -ENOMEM;
> > -
> > -    return 0;
> > -}
> > -
> 
> Same remark here about forward declaration.
> 
> Regards,
> 
> -- 
> Julien Grall
> 

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 03/16] xen/arm: make sgi handling generic
  2014-05-26 10:26 ` [PATCH v4 03/16] xen/arm: make sgi handling generic vijay.kilari
  2014-05-26 12:41   ` Julien Grall
@ 2014-05-28 14:10   ` Stefano Stabellini
  2014-06-09  9:58     ` Vijay Kilari
  1 sibling, 1 reply; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 14:10 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> move all the hw specific sgi handling functionality
> to one function and use it.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
>  xen/arch/arm/gic.c        |   37 ++++++++++++++++++++++++++++---------
>  xen/include/asm-arm/gic.h |    8 ++++++++
>  2 files changed, 36 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index efcd785..f8e49df 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -483,21 +483,40 @@ void __init gic_init(void)
>      spin_unlock(&gic.lock);
>  }
>  
> -void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
> +static void send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
> +                     uint8_t irqmode)
>  {
>      unsigned int mask = 0;
> +
> +    switch ( irqmode )
> +    {
> +    case SGI_TARGET_OTHERS:
> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS | sgi;
> +        break;
> +    case SGI_TARGET_SELF:
> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF | sgi;
> +        break;
> +    case SGI_TARGET_LIST:
> +        mask = gic_cpu_mask(cpu_mask);
> +        GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST |
> +                          (mask<<GICD_SGI_TARGET_SHIFT) | sgi;
> +        break;
> +    default:
> +        BUG_ON(1);
> +    }
> +}
> +
> +void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi)
> +{
>      cpumask_t online_mask;
>  
>      ASSERT(sgi < 16); /* There are only 16 SGIs */
>  
>      cpumask_and(&online_mask, cpumask, &cpu_online_map);
> -    mask = gic_cpu_mask(&online_mask);
>  
>      dsb(sy);
>  
> -    GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST
> -        | (mask<<GICD_SGI_TARGET_SHIFT)
> -        | sgi;
> +    send_SGI(&online_mask, sgi, SGI_TARGET_LIST);
>  }
>  
>  void send_SGI_one(unsigned int cpu, enum gic_sgi sgi)
> @@ -512,18 +531,18 @@ void send_SGI_self(enum gic_sgi sgi)
>  
>      dsb(sy);
>  
> -    GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF
> -        | sgi;
> +    send_SGI(cpumask_of(smp_processor_id()), sgi, SGI_TARGET_SELF);
>  }

Given that the mask is unused in this case, I think it would be
acceptable to pass NULL.


>  void send_SGI_allbutself(enum gic_sgi sgi)
>  {
> +   cpumask_t all_others_mask;
>     ASSERT(sgi < 16); /* There are only 16 SGIs */
>  
> +   cpumask_andnot(&all_others_mask, &cpu_possible_map, cpumask_of(smp_processor_id()));
>     dsb(sy);
>  
> -   GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS
> -       | sgi;
> +   send_SGI(&all_others_mask, sgi, SGI_TARGET_OTHERS);

Same here. Especially in this case you would avoid the useless
cpumask_andnot call.

In any case

Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>


>  }
>  
>  void smp_send_state_dump(unsigned int cpu)
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 8e37ccf..c7b7368 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -210,6 +210,14 @@ enum gic_sgi {
>      GIC_SGI_DUMP_STATE  = 1,
>      GIC_SGI_CALL_FUNCTION = 2,
>  };
> +
> +/* SGI irq mode types */
> +enum gic_sgi_mode {
> +    SGI_TARGET_LIST,
> +    SGI_TARGET_OTHERS,
> +    SGI_TARGET_SELF,
> +};
> +
>  extern void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi);
>  extern void send_SGI_one(unsigned int cpu, enum gic_sgi sgi);
>  extern void send_SGI_self(enum gic_sgi sgi);
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 02/16] xen/arm: make mmio handlers domain specific
  2014-05-28 14:05     ` Stefano Stabellini
@ 2014-05-28 14:11       ` Julien Grall
  0 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-28 14:11 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Ian.Campbell, vijay.kilari, Prasun.Kapoor, vijaya.kumar,
	xen-devel, stefano.stabellini

On 05/28/2014 03:05 PM, Stefano Stabellini wrote:
>>
>> Rather than moving a whole chunk of code, why can't you add forward
>> declaration for vgic_disk_mmio_{read,write}?
> 
> I think it would be OK either way

IHMO, it's more difficult to see what was change with this solution.

This serie is quite difficult to review (because of the number of line
changes). If we can avoid few of them it's better.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers
  2014-05-26 10:26 ` [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers vijay.kilari
  2014-05-26 13:10   ` Julien Grall
@ 2014-05-28 14:26   ` Stefano Stabellini
  2014-06-09 10:29     ` Vijay Kilari
  1 sibling, 1 reply; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 14:26 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> gic-v2 driver uses fixmap to map the registers.
> Instead use ioremap to access mmio registers.
> 
> With this patch, gic-v2 register definitions are updated
> to use obsolute offset address instead of dividing the
> register offset by 4.
> 
> Update vgic driver logic to compute using obsolute register
> address offsets
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
>  xen/arch/arm/gic.c        |  150 ++++++++++++++++++++++++---------------------
>  xen/arch/arm/vgic.c       |   17 ++---
>  xen/include/asm-arm/gic.h |  104 +++++++++++++++----------------
>  3 files changed, 142 insertions(+), 129 deletions(-)
> 
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index ce2269f..470e6c0 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -31,20 +31,23 @@
>  #include <asm/p2m.h>
>  #include <asm/domain.h>
>  #include <asm/platform.h>
> -
> +#include <asm/io.h>
>  #include <asm/gic.h>
>  
> -/* Access to the GIC Distributor registers through the fixmap */
> -#define GICD ((volatile uint32_t *) FIXMAP_ADDR(FIXMAP_GICD))
> -#define GICC ((volatile uint32_t *) FIXMAP_ADDR(FIXMAP_GICC1))
> -#define GICH ((volatile uint32_t *) FIXMAP_ADDR(FIXMAP_GICH))
> +#define GICD (gic.map_dbase)
> +#define GICC (gic.map_cbase)
> +#define GICH (gic.map_hbase)
> +
>  static void gic_restore_pending_irqs(struct vcpu *v);
>  
>  /* Global state */
>  static struct {
>      paddr_t dbase;       /* Address of distributor registers */
> +    void __iomem * map_dbase;  /* IO mapped Address of distributor registers */
>      paddr_t cbase;       /* Address of CPU interface registers */
> +    void __iomem * map_cbase; /* IO mapped Address of CPU interface registers*/
>      paddr_t hbase;       /* Address of virtual interface registers */
> +    void __iomem * map_hbase; /* IO Address of virtual interface registers */
>      paddr_t vbase;       /* Address of virtual cpu interface registers */
>      unsigned int lines;  /* Number of interrupts (SPIs + PPIs + SGIs) */
>      unsigned int maintenance_irq; /* IRQ maintenance */
> @@ -101,12 +104,12 @@ void gic_save_state(struct vcpu *v)
>       * accessed simultaneously by another pCPU.
>       */
>      for ( i=0; i<nr_lrs; i++)
> -        v->arch.gic_lr[i] = GICH[GICH_LR + i];
> +        v->arch.gic_lr[i] = readl_relaxed(GICH + GICH_LR + i * 4);
>      v->arch.lr_mask = this_cpu(lr_mask);
> -    v->arch.gic_apr = GICH[GICH_APR];
> -    v->arch.gic_vmcr = GICH[GICH_VMCR];
> +    v->arch.gic_apr = readl_relaxed(GICH + GICH_APR);
> +    v->arch.gic_vmcr = readl_relaxed(GICH + GICH_VMCR);
>      /* Disable until next VCPU scheduled */
> -    GICH[GICH_HCR] = 0;
> +    writel_relaxed(0, GICH + GICH_HCR);
>      isb();
>  }
>  
> @@ -120,10 +123,10 @@ void gic_restore_state(struct vcpu *v)
>  
>      this_cpu(lr_mask) = v->arch.lr_mask;
>      for ( i=0; i<nr_lrs; i++)
> -        GICH[GICH_LR + i] = v->arch.gic_lr[i];
> -    GICH[GICH_APR] = v->arch.gic_apr;
> -    GICH[GICH_VMCR] = v->arch.gic_vmcr;
> -    GICH[GICH_HCR] = GICH_HCR_EN;
> +        writel_relaxed(v->arch.gic_lr[i], GICH + GICH_LR + i * 4);
> +    writel_relaxed(v->arch.gic_apr, GICH + GICH_APR);
> +    writel_relaxed(v->arch.gic_vmcr, GICH + GICH_VMCR);
> +    writel_relaxed(GICH_HCR_EN, GICH + GICH_HCR);
>      isb();
>  
>      gic_restore_pending_irqs(v);
> @@ -139,8 +142,7 @@ static void gic_irq_enable(struct irq_desc *desc)
>      spin_lock_irqsave(&gic.lock, flags);
>      desc->status &= ~IRQ_DISABLED;
>      dsb(sy);
> -    /* Enable routing */
> -    GICD[GICD_ISENABLER + irq / 32] = (1u << (irq % 32));
> +    writel_relaxed((1u << (irq % 32)), GICD + GICD_ISENABLER + (irq / 32) * 4);
>      spin_unlock_irqrestore(&gic.lock, flags);
>  }
>  
> @@ -153,7 +155,7 @@ static void gic_irq_disable(struct irq_desc *desc)
>  
>      spin_lock_irqsave(&gic.lock, flags);
>      /* Disable routing */
> -    GICD[GICD_ICENABLER + irq / 32] = (1u << (irq % 32));
> +    writel_relaxed(1u << (irq % 32), GICD + GICD_ICENABLER + (irq / 32) * 4);
>      desc->status |= IRQ_DISABLED;
>      spin_unlock_irqrestore(&gic.lock, flags);
>  }
> @@ -179,16 +181,16 @@ static void gic_host_irq_end(struct irq_desc *desc)
>  {
>      int irq = desc->irq;
>      /* Lower the priority */
> -    GICC[GICC_EOIR] = irq;
> +    writel_relaxed(irq, GICC + GICC_EOIR);
>      /* Deactivate */
> -    GICC[GICC_DIR] = irq;
> +    writel_relaxed(irq, GICC + GICC_DIR);
>  }
>  
>  static void gic_guest_irq_end(struct irq_desc *desc)
>  {
>      int irq = desc->irq;
>      /* Lower the priority of the IRQ */
> -    GICC[GICC_EOIR] = irq;
> +    writel_relaxed(irq, GICC + GICC_EOIR);
>      /* Deactivation happens in maintenance interrupt / via GICV */
>  }
>  
> @@ -243,13 +245,13 @@ static void gic_set_irq_properties(struct irq_desc *desc,
>      mask = gic_cpu_mask(cpu_mask);
>  
>      /* Set edge / level */
> -    cfg = GICD[GICD_ICFGR + irq / 16];
> +    cfg = readl_relaxed(GICD + GICD_ICFGR + (irq / 16) * 4);
>      edgebit = 2u << (2 * (irq % 16));
>      if ( type & DT_IRQ_TYPE_LEVEL_MASK )
>          cfg &= ~edgebit;
>      else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
>          cfg |= edgebit;
> -    GICD[GICD_ICFGR + irq / 16] = cfg;
> +    writel_relaxed(cfg, GICD + GICD_ICFGR + (irq / 16) * 4);
>  
>      /* Set target CPU mask (RAZ/WI on uniprocessor) */
>      bytereg = (unsigned char *) (GICD + GICD_ITARGETSR);
> @@ -303,87 +305,91 @@ static void __init gic_dist_init(void)
>      uint32_t cpumask;
>      int i;
>  
> -    cpumask = GICD[GICD_ITARGETSR] & 0xff;
> +    cpumask = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
>      cpumask |= cpumask << 8;
>      cpumask |= cpumask << 16;
>  
>      /* Disable the distributor */
> -    GICD[GICD_CTLR] = 0;
> +    writel_relaxed(0, GICD + GICD_CTLR);
>  
> -    type = GICD[GICD_TYPER];
> +    type = readl_relaxed(GICD + GICD_TYPER);
>      gic.lines = 32 * ((type & GICD_TYPE_LINES) + 1);
>      gic.cpus = 1 + ((type & GICD_TYPE_CPUS) >> 5);
>      printk("GIC: %d lines, %d cpu%s%s (IID %8.8x).\n",
>             gic.lines, gic.cpus, (gic.cpus == 1) ? "" : "s",
>             (type & GICD_TYPE_SEC) ? ", secure" : "",
> -           GICD[GICD_IIDR]);
> +           readl_relaxed(GICD + GICD_IIDR));
>  
>      /* Default all global IRQs to level, active low */
>      for ( i = 32; i < gic.lines; i += 16 )
> -        GICD[GICD_ICFGR + i / 16] = 0x0;
> +        writel_relaxed(0x0, GICD + GICD_ICFGR + (i / 16) * 4);
>  
>      /* Route all global IRQs to this CPU */
>      for ( i = 32; i < gic.lines; i += 4 )
> -        GICD[GICD_ITARGETSR + i / 4] = cpumask;
> +        writel_relaxed(cpumask, GICD + GICD_ITARGETSR + (i / 4) * 4);
>  
>      /* Default priority for global interrupts */
>      for ( i = 32; i < gic.lines; i += 4 )
> -        GICD[GICD_IPRIORITYR + i / 4] =
> -            GIC_PRI_IRQ<<24 | GIC_PRI_IRQ<<16 | GIC_PRI_IRQ<<8 | GIC_PRI_IRQ;
> +        writel_relaxed (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
> +                        GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
> +                        GICD + GICD_IPRIORITYR + (i / 4) * 4);
>  
>      /* Disable all global interrupts */
>      for ( i = 32; i < gic.lines; i += 32 )
> -        GICD[GICD_ICENABLER + i / 32] = (uint32_t)~0ul;
> +        writel_relaxed(~0x0, GICD + GICD_ICENABLER + (i / 32) * 4);
>  
>      /* Turn on the distributor */
> -    GICD[GICD_CTLR] = GICD_CTL_ENABLE;
> +    writel_relaxed(GICD_CTL_ENABLE, GICD + GICD_CTLR);
>  }
>  
>  static void __cpuinit gic_cpu_init(void)
>  {
>      int i;
>  
> -    this_cpu(gic_cpu_id) = GICD[GICD_ITARGETSR] & 0xff;
> +    this_cpu(gic_cpu_id) = readl_relaxed(GICD + GICD_ITARGETSR) & 0xff;
>  
>      /* The first 32 interrupts (PPI and SGI) are banked per-cpu, so
>       * even though they are controlled with GICD registers, they must
>       * be set up here with the other per-cpu state. */
> -    GICD[GICD_ICENABLER] = 0xffff0000; /* Disable all PPI */
> -    GICD[GICD_ISENABLER] = 0x0000ffff; /* Enable all SGI */
> +    writel_relaxed(0xffff0000, GICD + GICD_ICENABLER); /* Disable all PPI */
> +    writel_relaxed(0x0000ffff, GICD + GICD_ISENABLER); /* Enable all SGI */
> +
>      /* Set SGI priorities */
>      for (i = 0; i < 16; i += 4)
> -        GICD[GICD_IPRIORITYR + i / 4] =
> -            GIC_PRI_IPI<<24 | GIC_PRI_IPI<<16 | GIC_PRI_IPI<<8 | GIC_PRI_IPI;
> +        writel_relaxed(GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 |
> +                       GIC_PRI_IPI << 8 | GIC_PRI_IPI,
> +                       GICD + GICD_IPRIORITYR + (i / 4) * 4);
>      /* Set PPI priorities */
>      for (i = 16; i < 32; i += 4)
> -        GICD[GICD_IPRIORITYR + i / 4] =
> -            GIC_PRI_IRQ<<24 | GIC_PRI_IRQ<<16 | GIC_PRI_IRQ<<8 | GIC_PRI_IRQ;
> +        writel_relaxed(GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
> +                      GIC_PRI_IRQ << 8 | GIC_PRI_IRQ,
> +                      GICD + GICD_IPRIORITYR + (i / 4) * 4);
>  
>      /* Local settings: interface controller */
> -    GICC[GICC_PMR] = 0xff;                /* Don't mask by priority */
> -    GICC[GICC_BPR] = 0;                   /* Finest granularity of priority */
> -    GICC[GICC_CTLR] = GICC_CTL_ENABLE|GICC_CTL_EOI;    /* Turn on delivery */
> +    writel_relaxed(0xff, GICC + GICC_PMR);                /* Don't mask by priority */
> +    writel_relaxed(0x0, GICC + GICC_BPR);                   /* Finest granularity of priority */
> +    writel_relaxed(GICC_CTL_ENABLE|GICC_CTL_EOI, GICC + GICC_CTLR);    /* Turn on delivery */
>  }
>  
>  static void gic_cpu_disable(void)
>  {
> -    GICC[GICC_CTLR] = 0;
> +    writel_relaxed(0x0, GICC + GICC_CTLR);
>  }
>  
>  static void __cpuinit gic_hyp_init(void)
>  {
>      uint32_t vtr;
>  
> -    vtr = GICH[GICH_VTR];
> +    vtr = readl_relaxed(GICH + GICH_VTR);
>      nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
>  
> -    GICH[GICH_MISR] = GICH_MISR_EOI;
> +    writel_relaxed(GICH_MISR_EOI, GICH + GICH_MISR);
>      this_cpu(lr_mask) = 0ULL;
>  }
>  
>  static void __cpuinit gic_hyp_disable(void)
>  {
> -    GICH[GICH_HCR] = 0;
> +    writel_relaxed(0, GICH + GICH_HCR);
>  }
>  
>  int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
> @@ -462,15 +468,21 @@ void __init gic_init(void)
>           (gic.hbase & ~PAGE_MASK) || (gic.vbase & ~PAGE_MASK) )
>          panic("GIC interfaces not page aligned");
>  
> -    set_fixmap(FIXMAP_GICD, gic.dbase >> PAGE_SHIFT, DEV_SHARED);
> -    BUILD_BUG_ON(FIXMAP_ADDR(FIXMAP_GICC1) !=
> -                 FIXMAP_ADDR(FIXMAP_GICC2)-PAGE_SIZE);
> -    set_fixmap(FIXMAP_GICC1, gic.cbase >> PAGE_SHIFT, DEV_SHARED);
> +    gic.map_dbase = ioremap_nocache(gic.dbase, PAGE_SIZE);
> +    if ( !gic.map_dbase )
> +        panic("Failed to ioremap for GIC distributor\n");
> +
>      if ( platform_has_quirk(PLATFORM_QUIRK_GIC_64K_STRIDE) )
> -        set_fixmap(FIXMAP_GICC2, (gic.cbase >> PAGE_SHIFT) + 0x10, DEV_SHARED);
> +        gic.map_cbase = ioremap_nocache(gic.cbase, PAGE_SIZE * 0x10);
>      else
> -        set_fixmap(FIXMAP_GICC2, (gic.cbase >> PAGE_SHIFT) + 0x1, DEV_SHARED);
> -    set_fixmap(FIXMAP_GICH, gic.hbase >> PAGE_SHIFT, DEV_SHARED);
> +        gic.map_cbase = ioremap_nocache(gic.cbase, PAGE_SIZE * 2);
> +
> +    if ( !gic.map_cbase )
> +        panic("Failed to ioremap for GIC CPU interface\n");
> +
> +    gic.map_hbase = ioremap_nocache(gic.hbase, PAGE_SIZE);
> +    if ( !gic.map_hbase )
> +        panic("Failed to ioremap for GIC Virtual interface\n");
>  
>      /* Global settings: interrupt distributor */
>      spin_lock_init(&gic.lock);
> @@ -491,15 +503,16 @@ static void send_SGI(const cpumask_t *cpu_mask, enum gic_sgi sgi,
>      switch ( irqmode )
>      {
>      case SGI_TARGET_OTHERS:
> -        GICD[GICD_SGIR] = GICD_SGI_TARGET_OTHERS | sgi;
> +        writel_relaxed(GICD_SGI_TARGET_OTHERS | sgi, GICD + GICD_SGIR);
>          break;
>      case SGI_TARGET_SELF:
> -        GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF | sgi;
> +        writel_relaxed(GICD_SGI_TARGET_SELF | sgi, GICD + GICD_SGIR);
>          break;
>      case SGI_TARGET_LIST:
>          mask = gic_cpu_mask(cpu_mask);
> -        GICD[GICD_SGIR] = GICD_SGI_TARGET_LIST |
> -                          (mask<<GICD_SGI_TARGET_SHIFT) | sgi;
> +        writel_relaxed(GICD_SGI_TARGET_LIST |
> +                       (mask << GICD_SGI_TARGET_SHIFT) | sgi,
> +                       GICD + GICD_SGIR);
>          break;
>      default:
>          BUG_ON(1);
> @@ -585,7 +598,7 @@ static inline void gic_set_lr(int lr, struct pending_irq *p,
>      if ( p->desc != NULL )
>          lr_val |= GICH_LR_HW | (p->desc->irq << GICH_LR_PHYSICAL_SHIFT);
>  
> -    GICH[GICH_LR + lr] = lr_val;
> +    writel_relaxed(lr_reg, GICH + GICH_LR + lr * 4);
>  
>      set_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
>      clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
> @@ -677,7 +690,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>      ASSERT(spin_is_locked(&v->arch.vgic.lock));
>      ASSERT(!local_irq_is_enabled());
>  
> -    lr = GICH[GICH_LR + i];
> +    lr = readl_relaxed(GICH + GICH_LR + i * 4);
>      irq = (lr >> GICH_LR_VIRTUAL_SHIFT) & GICH_LR_VIRTUAL_MASK;
>      p = irq_to_pending(v, irq);
>      if ( lr & GICH_LR_ACTIVE )
> @@ -687,7 +700,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>               test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
>          {
>              if ( p->desc == NULL )
> -                GICH[GICH_LR + i] = lr | GICH_LR_PENDING;
> +                writel_relaxed(lr | GICH_LR_PENDING, GICH + GICH_LR + i * 4);
>              else
>                  gdprintk(XENLOG_WARNING, "unable to inject hw irq=%d into d%dv%d: already active in LR%d\n",
>                           irq, v->domain->domain_id, v->vcpu_id, i);
> @@ -700,7 +713,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>                      irq, v->domain->domain_id, v->vcpu_id, i);
>  #endif
>      } else {
> -        GICH[GICH_LR + i] = 0;
> +        writel_relaxed(0, GICH + GICH_LR + i * 4);
>          clear_bit(i, &this_cpu(lr_mask));
>  
>          if ( p->desc != NULL )
> @@ -808,7 +821,7 @@ int gic_events_need_delivery(void)
>      struct pending_irq *p;
>      unsigned long flags;
>  
> -    mask_priority = (GICH[GICH_VMCR] >> GICH_VMCR_PRIORITY_SHIFT) & GICH_VMCR_PRIORITY_MASK;
> +    mask_priority = (readl_relaxed(GICH + GICH_VMCR) >> GICH_VMCR_PRIORITY_SHIFT) & GICH_VMCR_PRIORITY_MASK;
>      mask_priority = mask_priority << 3;
>  
>      spin_lock_irqsave(&v->arch.vgic.lock, flags);
> @@ -844,22 +857,21 @@ int gic_events_need_delivery(void)
>  
>  void gic_inject(void)
>  {
> +    uint32_t hcr;
>      ASSERT(!local_irq_is_enabled());
>  
>      gic_restore_pending_irqs(current);
>  
> -
>      if ( !list_empty(&current->arch.vgic.lr_pending) && lr_all_full() )
> -        GICH[GICH_HCR] |= GICH_HCR_UIE;
> +        writel_relaxed(hcr | GICH_HCR_UIE, GICH + GICH_HCR);
>      else
> -        GICH[GICH_HCR] &= ~GICH_HCR_UIE;
> -
> +        writel_relaxed(hcr & ~GICH_HCR_UIE, GICH + GICH_HCR);
>  }
>  
>  static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
>  {
>      /* Lower the priority */
> -    GICC[GICC_EOIR] = sgi;
> +    writel_relaxed(sgi, GICC + GICC_EOIR);
>  
>      switch (sgi)
>      {
> @@ -878,7 +890,7 @@ static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
>      }
>  
>      /* Deactivate */
> -    GICC[GICC_DIR] = sgi;
> +    writel_relaxed(sgi, GICC + GICC_DIR);
>  }
>  
>  /* Accept an interrupt from the GIC and dispatch its handler */
> @@ -889,7 +901,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
>  
>  
>      do  {
> -        intack = GICC[GICC_IAR];
> +        intack = readl_relaxed(GICC + GICC_IAR);
>          irq = intack & GICC_IA_IRQ;
>  
>          if ( likely(irq >= 16 && irq < 1021) )
> @@ -976,7 +988,7 @@ void gic_dump_info(struct vcpu *v)
>      if ( v == current )
>      {
>          for ( i = 0; i < nr_lrs; i++ )
> -            printk("   HW_LR[%d]=%x\n", i, GICH[GICH_LR + i]);
> +            printk("   HW_LR[%d]=%x\n", i, readl_relaxed(GICH + GICH_LR + i * 4));
>      } else {
>          for ( i = 0; i < nr_lrs; i++ )
>              printk("   VCPU_LR[%d]=%x\n", i, v->arch.gic_lr[i]);

the changes to gic.c look fine


> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 151ec3e..b56f9d1 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -30,7 +30,7 @@
>  #include <asm/mmio.h>
>  #include <asm/gic.h>
>  
> -#define REG(n) (n/4)
> +#define REG(n) (n)

Is this macro actually needed still?
We might as well get rid of it.


>  /* Number of ranks of interrupt registers for a domain */
>  #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
> @@ -55,7 +55,7 @@ static inline int REG_RANK_NR(int b, uint32_t n)
>   * Offset of GICD_<FOO><n> with its rank, for GICD_<FOO> with
>   * <b>-bits-per-interrupt.
>   */
> -#define REG_RANK_INDEX(b, n) ((n) & ((b)-1))
> +#define REG_RANK_INDEX(b, n) (((n) >> 2) & ((b)-1))
>  
>  /*
>   * Returns rank corresponding to a GICD_<FOO><n> register for
> @@ -63,7 +63,9 @@ static inline int REG_RANK_NR(int b, uint32_t n)
>   */
>  static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
>  {
> -    int rank = REG_RANK_NR(b, n);
> +    int rank;
> +    n = n >> 2;
> +    rank = REG_RANK_NR(b, n);

I would be consistent and change REG_RANK_NR instead of vgic_irq_rank.


>      if ( rank == 0 )
>          return &v->arch.vgic.private_irqs;
> @@ -492,7 +494,7 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
>          tr = rank->ienable;
>          rank->ienable |= *r;
>          vgic_unlock_rank(v, rank);
> -        vgic_enable_irqs(v, (*r) & (~tr), gicd_reg - GICD_ISENABLER);
> +        vgic_enable_irqs(v, (*r) & (~tr), (gicd_reg - GICD_ISENABLER) >> 2);
>          return 1;
>  
>      case GICD_ICENABLER ... GICD_ICENABLERN:
> @@ -503,7 +505,7 @@ static int vgic_distr_mmio_write(struct vcpu *v, mmio_info_t *info)
>          tr = rank->ienable;
>          rank->ienable &= ~*r;
>          vgic_unlock_rank(v, rank);
> -        vgic_disable_irqs(v, (*r) & tr, gicd_reg - GICD_ICENABLER);
> +        vgic_disable_irqs(v, (*r) & tr, (gicd_reg - GICD_ICENABLER) >> 2);
>          return 1;
>  
>      case GICD_ISPENDR ... GICD_ISPENDRN:
> @@ -670,9 +672,8 @@ void vgic_clear_pending_irqs(struct vcpu *v)
>  
>  void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
>  {
> -    int idx = irq >> 2, byte = irq & 0x3;
>      uint8_t priority;
> -    struct vgic_irq_rank *rank = vgic_irq_rank(v, 8, idx);
> +    struct vgic_irq_rank *rank = vgic_irq_rank(v, 8, irq);
>      struct pending_irq *iter, *n = irq_to_pending(v, irq);
>      unsigned long flags;
>      bool_t running;
> @@ -693,7 +694,7 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
>          return;
>      }
>  
> -    priority = byte_read(rank->ipriority[REG_RANK_INDEX(8, idx)], 0, byte);
> +    priority = byte_read(rank->ipriority[REG_RANK_INDEX(8, irq)], 0, irq & 0x3);
>  
>      n->irq = irq;
>      set_bit(GIC_IRQ_GUEST_QUEUED, &n->status);
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index c7b7368..7fa3b95 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -18,37 +18,37 @@
>  #ifndef __ASM_ARM_GIC_H__
>  #define __ASM_ARM_GIC_H__
>  
> -#define GICD_CTLR       (0x000/4)
> -#define GICD_TYPER      (0x004/4)
> -#define GICD_IIDR       (0x008/4)
> -#define GICD_IGROUPR    (0x080/4)
> -#define GICD_IGROUPRN   (0x0FC/4)
> -#define GICD_ISENABLER  (0x100/4)
> -#define GICD_ISENABLERN (0x17C/4)
> -#define GICD_ICENABLER  (0x180/4)
> -#define GICD_ICENABLERN (0x1fC/4)
> -#define GICD_ISPENDR    (0x200/4)
> -#define GICD_ISPENDRN   (0x27C/4)
> -#define GICD_ICPENDR    (0x280/4)
> -#define GICD_ICPENDRN   (0x2FC/4)
> -#define GICD_ISACTIVER  (0x300/4)
> -#define GICD_ISACTIVERN (0x37C/4)
> -#define GICD_ICACTIVER  (0x380/4)
> -#define GICD_ICACTIVERN (0x3FC/4)
> -#define GICD_IPRIORITYR (0x400/4)
> -#define GICD_IPRIORITYRN (0x7F8/4)
> -#define GICD_ITARGETSR  (0x800/4)
> -#define GICD_ITARGETSRN (0xBF8/4)
> -#define GICD_ICFGR      (0xC00/4)
> -#define GICD_ICFGRN     (0xCFC/4)
> -#define GICD_NSACR      (0xE00/4)
> -#define GICD_NSACRN     (0xEFC/4)
> -#define GICD_SGIR       (0xF00/4)
> -#define GICD_CPENDSGIR  (0xF10/4)
> -#define GICD_CPENDSGIRN (0xF1C/4)
> -#define GICD_SPENDSGIR  (0xF20/4)
> -#define GICD_SPENDSGIRN (0xF2C/4)
> -#define GICD_ICPIDR2    (0xFE8/4)
> +#define GICD_CTLR       (0x000)
> +#define GICD_TYPER      (0x004)
> +#define GICD_IIDR       (0x008)
> +#define GICD_IGROUPR    (0x080)
> +#define GICD_IGROUPRN   (0x0FC)
> +#define GICD_ISENABLER  (0x100)
> +#define GICD_ISENABLERN (0x17C)
> +#define GICD_ICENABLER  (0x180)
> +#define GICD_ICENABLERN (0x1fC)
> +#define GICD_ISPENDR    (0x200)
> +#define GICD_ISPENDRN   (0x27C)
> +#define GICD_ICPENDR    (0x280)
> +#define GICD_ICPENDRN   (0x2FC)
> +#define GICD_ISACTIVER  (0x300)
> +#define GICD_ISACTIVERN (0x37C)
> +#define GICD_ICACTIVER  (0x380)
> +#define GICD_ICACTIVERN (0x3FC)
> +#define GICD_IPRIORITYR (0x400)
> +#define GICD_IPRIORITYRN (0x7F8)
> +#define GICD_ITARGETSR  (0x800)
> +#define GICD_ITARGETSRN (0xBF8)
> +#define GICD_ICFGR      (0xC00)
> +#define GICD_ICFGRN     (0xCFC)
> +#define GICD_NSACR      (0xE00)
> +#define GICD_NSACRN     (0xEFC)
> +#define GICD_SGIR       (0xF00)
> +#define GICD_CPENDSGIR  (0xF10)
> +#define GICD_CPENDSGIRN (0xF1C)
> +#define GICD_SPENDSGIR  (0xF20)
> +#define GICD_SPENDSGIRN (0xF2C)
> +#define GICD_ICPIDR2    (0xFE8)
>  
>  #define GICD_SGI_TARGET_LIST_SHIFT   (24)
>  #define GICD_SGI_TARGET_LIST_MASK    (0x3UL << GICD_SGI_TARGET_LIST_SHIFT)
> @@ -60,27 +60,27 @@
>  #define GICD_SGI_GROUP1              (1UL<<15)
>  #define GICD_SGI_INTID_MASK          (0xFUL)
>  
> -#define GICC_CTLR       (0x0000/4)
> -#define GICC_PMR        (0x0004/4)
> -#define GICC_BPR        (0x0008/4)
> -#define GICC_IAR        (0x000C/4)
> -#define GICC_EOIR       (0x0010/4)
> -#define GICC_RPR        (0x0014/4)
> -#define GICC_HPPIR      (0x0018/4)
> -#define GICC_APR        (0x00D0/4)
> -#define GICC_NSAPR      (0x00E0/4)
> -#define GICC_DIR        (0x1000/4)
> -
> -#define GICH_HCR        (0x00/4)
> -#define GICH_VTR        (0x04/4)
> -#define GICH_VMCR       (0x08/4)
> -#define GICH_MISR       (0x10/4)
> -#define GICH_EISR0      (0x20/4)
> -#define GICH_EISR1      (0x24/4)
> -#define GICH_ELSR0      (0x30/4)
> -#define GICH_ELSR1      (0x34/4)
> -#define GICH_APR        (0xF0/4)
> -#define GICH_LR         (0x100/4)
> +#define GICC_CTLR       (0x0000)
> +#define GICC_PMR        (0x0004)
> +#define GICC_BPR        (0x0008)
> +#define GICC_IAR        (0x000C)
> +#define GICC_EOIR       (0x0010)
> +#define GICC_RPR        (0x0014)
> +#define GICC_HPPIR      (0x0018)
> +#define GICC_APR        (0x00D0)
> +#define GICC_NSAPR      (0x00E0)
> +#define GICC_DIR        (0x1000)
> +
> +#define GICH_HCR        (0x00)
> +#define GICH_VTR        (0x04)
> +#define GICH_VMCR       (0x08)
> +#define GICH_MISR       (0x10)
> +#define GICH_EISR0      (0x20)
> +#define GICH_EISR1      (0x24)
> +#define GICH_ELSR0      (0x30)
> +#define GICH_ELSR1      (0x34)
> +#define GICH_APR        (0xF0)
> +#define GICH_LR         (0x100)
>  
>  /* Register bits */
>  #define GICD_CTL_ENABLE 0x1

this is fine

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality
  2014-05-26 10:26 ` [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality vijay.kilari
  2014-05-26 14:09   ` Julien Grall
  2014-05-27 19:13   ` Julien Grall
@ 2014-05-28 14:43   ` Stefano Stabellini
  2 siblings, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 14:43 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> GIC driver contains both generic and hardware specific low
> level functionality in gic.c file.
> 
> With this patch, low level functionality is moved to separate
> file gic-v2.c and generic code is kept in gic.c file
> 
> Callbacks are registered by low level driver with generic driver
> and are called whereever required.
> 
> The locking mechanism is not changed.
> 
> This helps to separate generic and hardware functionality
> and implement future hardware version drivers.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

[...]

> +static void gicv2_irq_enable(struct irq_desc *desc)
> +{
> +    unsigned long flags;
> +
> +    spin_lock_irqsave(&desc->lock, flags);
> +    spin_lock(&gicv2.lock);
> +    desc->status &= ~IRQ_DISABLED;
> +    dsb(sy);
> +    /* Enable routing */
> +    gicv2_enable_irq(desc);
> +    spin_unlock(&gicv2.lock);
> +    spin_unlock_irqrestore(&desc->lock, flags);
> +}

you say that you are not changing locking but the original function had:

ASSERT(spin_is_locked(&desc->lock));


> +static void gicv2_irq_disable(struct irq_desc *desc)
> +{
> +    unsigned long flags;
> +
> +    spin_lock_irqsave(&desc->lock, flags);
> +    spin_lock(&gicv2.lock);
> +    /* Disable routing */
> +    gicv2_disable_irq(desc);
> +    desc->status |= IRQ_DISABLED;
> +    spin_unlock(&gicv2.lock);
> +    spin_unlock_irqrestore(&desc->lock, flags);
> +}

same here

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 07/16] arm/xen: move GIC context data structure to gic driver
  2014-05-26 14:32   ` Julien Grall
@ 2014-05-28 14:49     ` Stefano Stabellini
  0 siblings, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 14:49 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian.Campbell, vijay.kilari, stefano.stabellini, Prasun.Kapoor,
	vijaya.kumar, xen-devel, stefano.stabellini

On Mon, 26 May 2014, Julien Grall wrote:
> Hi Vijay,
> 
> On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> > diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> > index 61a498f..f46b631 100644
> > --- a/xen/include/asm-arm/domain.h
> > +++ b/xen/include/asm-arm/domain.h
> > @@ -8,6 +8,7 @@
> >   #include <asm/p2m.h>
> >   #include <asm/vfp.h>
> >   #include <asm/mmio.h>
> > +#include <asm/gic.h>
> >   #include <public/hvm/params.h>
> >   #include <xen/serial.h>
> > 
> > @@ -261,8 +262,10 @@ struct arch_vcpu
> >       uint32_t csselr;
> >       register_t vmpidr;
> > 
> > -    uint32_t gic_hcr, gic_vmcr, gic_apr;
> > -    uint32_t gic_lr[64];
> > +    /* Holds gic context data */
> > +    union gic_state_data gic;
> > +
> > +    uint64_t event_mask;
> 
> Why do you add back this field? It has been dropped on upstream...

Yes, weird.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 08/16] xen/arm: use device api to detect GIC version
  2014-05-26 14:39   ` Julien Grall
@ 2014-05-28 14:52     ` Stefano Stabellini
  0 siblings, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 14:52 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian.Campbell, vijay.kilari, stefano.stabellini, Prasun.Kapoor,
	vijaya.kumar, xen-devel, stefano.stabellini

On Mon, 26 May 2014, Julien Grall wrote:
> Hi Vijay,
> 
> On 26/05/14 11:26, vijay.kilari@gmail.com wrote:
> >   enum device_type
> >   {
> > -    DEVICE_SERIAL
> > +    DEVICE_SERIAL,
> > +    DEVICE_GIC,
> 
> Sounds like you are using an old version of Xen. This enum has been modified a
> couple of weeks ago. Please rebase you tree on the latest Xen.

Aside from that I would have acked the patch

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file
  2014-05-26 10:26 ` [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file vijay.kilari
  2014-05-27 11:32   ` Julien Grall
@ 2014-05-28 14:54   ` Stefano Stabellini
  1 sibling, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-05-28 14:54 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> vgic_irq_rank structure contains gic specific data elements.
> Move this out of domain.h to new vgic header file vgic.h
> Allocate memory dynamically in vgic driver.
> 
> This patch reduces the size of domain struct and helps to
> keep domain struct within PAGE_SIZE when future GIC hw versions
> are added
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>

Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>


>  xen/arch/arm/domain.c        |    2 ++
>  xen/arch/arm/vgic.c          |   18 ++++++++++++++----
>  xen/include/asm-arm/domain.h |   11 +----------
>  xen/include/asm-arm/vgic.h   |   40 ++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 57 insertions(+), 14 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 40acfb3..bccbeda 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -31,6 +31,7 @@
>  #include <asm/procinfo.h>
>  
>  #include <asm/gic.h>
> +#include <asm/vgic.h>
>  #include <asm/platform.h>
>  #include "vtimer.h"
>  #include "vuart.h"
> @@ -481,6 +482,7 @@ int vcpu_initialise(struct vcpu *v)
>  void vcpu_destroy(struct vcpu *v)
>  {
>      vcpu_timer_destroy(v);
> +    vcpu_vgic_free(v);
>      free_xenheap_pages(v->arch.stack, STACK_ORDER);
>  }
>  
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index b56f9d1..d2a9e34 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -29,6 +29,7 @@
>  
>  #include <asm/mmio.h>
>  #include <asm/gic.h>
> +#include <asm/vgic.h>
>  
>  #define REG(n) (n)
>  
> @@ -68,7 +69,7 @@ static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
>      rank = REG_RANK_NR(b, n);
>  
>      if ( rank == 0 )
> -        return &v->arch.vgic.private_irqs;
> +        return v->arch.vgic.private_irqs;
>      else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>          return &v->domain->arch.vgic.shared_irqs[rank - 1];
>      else
> @@ -84,9 +85,12 @@ void domain_vgic_free(struct domain *d)
>  int vcpu_vgic_init(struct vcpu *v)
>  {
>      int i;
> -    memset(&v->arch.vgic.private_irqs, 0, sizeof(v->arch.vgic.private_irqs));
>  
> -    spin_lock_init(&v->arch.vgic.private_irqs.lock);
> +    v->arch.vgic.private_irqs = xzalloc(struct vgic_irq_rank);
> +    if ( v->arch.vgic.private_irqs == NULL )
> +      return -ENOMEM;
> +
> +    spin_lock_init(&v->arch.vgic.private_irqs->lock);
>  
>      memset(&v->arch.vgic.pending_irqs, 0, sizeof(v->arch.vgic.pending_irqs));
>      for (i = 0; i < 32; i++)
> @@ -97,7 +101,7 @@ int vcpu_vgic_init(struct vcpu *v)
>  
>      /* For SGI and PPI the target is always this CPU */
>      for ( i = 0 ; i < 8 ; i++ )
> -        v->arch.vgic.private_irqs.itargets[i] =
> +        v->arch.vgic.private_irqs->itargets[i] =
>                (1<<(v->vcpu_id+0))
>              | (1<<(v->vcpu_id+8))
>              | (1<<(v->vcpu_id+16))
> @@ -109,6 +113,12 @@ int vcpu_vgic_init(struct vcpu *v)
>      return 0;
>  }
>  
> +int vcpu_vgic_free(struct vcpu *v)
> +{
> +    xfree(v->arch.vgic.private_irqs);
> +    return 0;
> +}
> +
>  #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
>  #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>  
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index f46b631..fe84ce5 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -12,15 +12,6 @@
>  #include <public/hvm/params.h>
>  #include <xen/serial.h>
>  
> -/* Represents state corresponding to a block of 32 interrupts */
> -struct vgic_irq_rank {
> -    spinlock_t lock; /* Covers access to all other members of this struct */
> -    uint32_t ienable, iactive, ipend, pendsgi;
> -    uint32_t icfg[2];
> -    uint32_t ipriority[8];
> -    uint32_t itargets[8];
> -};
> -
>  struct pending_irq
>  {
>      /*
> @@ -274,7 +265,7 @@ struct arch_vcpu
>           * struct arch_domain.
>           */
>          struct pending_irq pending_irqs[32];
> -        struct vgic_irq_rank private_irqs;
> +        struct vgic_irq_rank *private_irqs;
>  
>          /* This list is ordered by IRQ priority and it is used to keep
>           * track of the IRQs that the VGIC injected into the guest.
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> new file mode 100644
> index 0000000..104a87d
> --- /dev/null
> +++ b/xen/include/asm-arm/vgic.h
> @@ -0,0 +1,40 @@
> +/*
> + * ARM Virtual Generic Interrupt Controller support
> + *
> + * Ian Campbell <ian.campbell@citrix.com>
> + * Copyright (c) 2011 Citrix Systems.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __ASM_ARM_VGIC_H__
> +#define __ASM_ARM_VGIC_H__
> +
> +/* Represents state corresponding to a block of 32 interrupts */
> +struct vgic_irq_rank {
> +    spinlock_t lock; /* Covers access to all other members of this struct */
> +    uint32_t ienable, iactive, ipend, pendsgi;
> +    uint32_t icfg[2];
> +    uint32_t ipriority[8];
> +    uint32_t itargets[8];
> +};
> +
> +extern int vcpu_vgic_free(struct vcpu *v);
> +#endif
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-27 11:56   ` Julien Grall
@ 2014-05-30  8:59     ` Vijay Kilari
  2014-05-30  9:58       ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Vijay Kilari @ 2014-05-30  8:59 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On Tue, May 27, 2014 at 5:26 PM, Julien Grall <julien.grall@linaro.org> wrote:
> Hi Vijay,
>
> On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
>> -        *r = ( (v->domain->max_vcpus<<5) & GICD_TYPE_CPUS )
>> -            |( ((v->domain->arch.vgic.nr_lines/32)) & GICD_TYPE_LINES );
>> +        *r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
>> +            |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );
>
> Why these changes?

Legacy coding style fixes.

>
>>          vgic_unlock(v);
>>          return 1;
>>      case GICD_IIDR:
>> -        if ( dabt.size != 2 ) goto bad_width;
>> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>>          /*
>>           * XXX Do we need a JEP106 manufacturer ID?
>>           * Just use the physical h/w value for now
>> @@ -119,8 +119,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
>>          goto read_as_zero;
>>
>>      case GICD_ISENABLER ... GICD_ISENABLERN:
>> -        if ( dabt.size != 2 ) goto bad_width;
>> -        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
>> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>> +        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
>
> In your commit message you explicitly say that use DABT_* will help you
> to get the register offset but... you still hardcode the size.
>
> Why can't you use dabt.size here? And all the other places.

   dabt.size gives the current register access size but not the actual
register size.
>
> Regards,
>
> --
> Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-30  8:59     ` Vijay Kilari
@ 2014-05-30  9:58       ` Julien Grall
  2014-05-30 10:24         ` Vijay Kilari
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-30  9:58 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

Hi Vijay,

On 30/05/14 09:59, Vijay Kilari wrote:
> On Tue, May 27, 2014 at 5:26 PM, Julien Grall <julien.grall@linaro.org> wrote:
>> On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
>>> -        *r = ( (v->domain->max_vcpus<<5) & GICD_TYPE_CPUS )
>>> -            |( ((v->domain->arch.vgic.nr_lines/32)) & GICD_TYPE_LINES );
>>> +        *r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
>>> +            |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );
>>
>> Why these changes?
>
> Legacy coding style fixes.

If you don't touch that part of the code, please don't do random coding 
style fix without any mention in the commit message.

>
>>
>>>           vgic_unlock(v);
>>>           return 1;
>>>       case GICD_IIDR:
>>> -        if ( dabt.size != 2 ) goto bad_width;
>>> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>>>           /*
>>>            * XXX Do we need a JEP106 manufacturer ID?
>>>            * Just use the physical h/w value for now
>>> @@ -119,8 +119,8 @@ static int vgic_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
>>>           goto read_as_zero;
>>>
>>>       case GICD_ISENABLER ... GICD_ISENABLERN:
>>> -        if ( dabt.size != 2 ) goto bad_width;
>>> -        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
>>> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>>> +        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
>>
>> In your commit message you explicitly say that use DABT_* will help you
>> to get the register offset but... you still hardcode the size.
>>
>> Why can't you use dabt.size here? And all the other places.
>
>     dabt.size gives the current register access size but not the actual
> register size.

In this specific case, the register access size and the actual register 
size is the same...

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-30  9:58       ` Julien Grall
@ 2014-05-30 10:24         ` Vijay Kilari
  2014-05-30 10:36           ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Vijay Kilari @ 2014-05-30 10:24 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On Fri, May 30, 2014 at 3:28 PM, Julien Grall <julien.grall@linaro.org> wrote:
> Hi Vijay,
>
>>>>
>>>>       case GICD_ISENABLER ... GICD_ISENABLERN:
>>>> -        if ( dabt.size != 2 ) goto bad_width;
>>>> -        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
>>>> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>>>> +        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER,
>>>> DABT_WORD);
>>>
>>>
>>> In your commit message you explicitly say that use DABT_* will help you
>>> to get the register offset but... you still hardcode the size.
>>>
>>> Why can't you use dabt.size here? And all the other places.
>>
>>
>>     dabt.size gives the current register access size but not the actual
>> register size.
>
>
> In this specific case, the register access size and the actual register size
> is the same...

Yes, in most of the cases it is same. But there are some register
access that supports
both byte and word size access. In that case we have to choose always
the register size DABT_*

To be consistent I have not used dabt.size.  In case if byte access to
particular register
is added then one can go wrong.

>
> Regards,
>
> --
> Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-30 10:24         ` Vijay Kilari
@ 2014-05-30 10:36           ` Julien Grall
  2014-05-30 10:51             ` Vijay Kilari
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-05-30 10:36 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini



On 30/05/14 11:24, Vijay Kilari wrote:
> On Fri, May 30, 2014 at 3:28 PM, Julien Grall <julien.grall@linaro.org> wrote:
>> Hi Vijay,
>>
>>>>>
>>>>>        case GICD_ISENABLER ... GICD_ISENABLERN:
>>>>> -        if ( dabt.size != 2 ) goto bad_width;
>>>>> -        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
>>>>> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>>>>> +        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER,
>>>>> DABT_WORD);
>>>>
>>>>
>>>> In your commit message you explicitly say that use DABT_* will help you
>>>> to get the register offset but... you still hardcode the size.
>>>>
>>>> Why can't you use dabt.size here? And all the other places.
>>>
>>>
>>>      dabt.size gives the current register access size but not the actual
>>> register size.
>>
>>
>> In this specific case, the register access size and the actual register size
>> is the same...
>
> Yes, in most of the cases it is same. But there are some register
> access that supports
> both byte and word size access. In that case we have to choose always
> the register size DABT_*
>
> To be consistent I have not used dabt.size.  In case if byte access to
> particular register
> is added then one can go wrong.

With your explanation, I don't see any reason to replace all the 
dabt.size != number by dat.size != DABT_*.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-30 10:36           ` Julien Grall
@ 2014-05-30 10:51             ` Vijay Kilari
  2014-05-30 10:54               ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Vijay Kilari @ 2014-05-30 10:51 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On Fri, May 30, 2014 at 4:06 PM, Julien Grall <julien.grall@linaro.org> wrote:
>
>
> On 30/05/14 11:24, Vijay Kilari wrote:
>>
>> On Fri, May 30, 2014 at 3:28 PM, Julien Grall <julien.grall@linaro.org>
>> wrote:
>>>
>>> Hi Vijay,
>>>
>>>>>>
>>>>>>        case GICD_ISENABLER ... GICD_ISENABLERN:
>>>>>> -        if ( dabt.size != 2 ) goto bad_width;
>>>>>> -        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER);
>>>>>> +        if ( dabt.size != DABT_WORD ) goto bad_width;
>>>>>> +        rank = vgic_irq_rank(v, 1, gicd_reg - GICD_ISENABLER,
>>>>>> DABT_WORD);
>>>>>
>>>>>
>>>>>
>>>>> In your commit message you explicitly say that use DABT_* will help you
>>>>> to get the register offset but... you still hardcode the size.
>>>>>
>>>>> Why can't you use dabt.size here? And all the other places.
>>>>
>>>>
>>>>
>>>>      dabt.size gives the current register access size but not the actual
>>>> register size.
>>>
>>>
>>>
>>> In this specific case, the register access size and the actual register
>>> size
>>> is the same...
>>
>>
>> Yes, in most of the cases it is same. But there are some register
>> access that supports
>> both byte and word size access. In that case we have to choose always
>> the register size DABT_*
>>
>> To be consistent I have not used dabt.size.  In case if byte access to
>> particular register
>> is added then one can go wrong.
>
>
> With your explanation, I don't see any reason to replace all the dabt.size
> != number by dat.size != DABT_*.

  Two things are done here
    (1)  In the code 'dabt.size != number' this number is always
BYTE/HALF_WORD/WORD/DOUBLE
      defined by hsr registers.  Instead of checking for hard coded
values I used hsr defined values.
    (2) The vgic_irq_rank also depends on the same hsr defined values
to calculate irq rank.
         Also, this make vgic_irq_rank generic as it takes register
size as parameter to calculate irq rank instead
          of hard coding to value 2 in previous patches

  IMO, still we can retain (1) changes and may be for point(2) we can
define similar enum to avoid
  using hsr definitions

>
> Regards,
>
> --
> Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size
  2014-05-30 10:51             ` Vijay Kilari
@ 2014-05-30 10:54               ` Julien Grall
  0 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-05-30 10:54 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini



On 30/05/14 11:51, Vijay Kilari wrote:
> On Fri, May 30, 2014 at 4:06 PM, Julien Grall <julien.grall@linaro.org> wrote:
>> With your explanation, I don't see any reason to replace all the dabt.size
>> != number by dat.size != DABT_*.
>
>    Two things are done here
>      (1)  In the code 'dabt.size != number' this number is always
> BYTE/HALF_WORD/WORD/DOUBLE
>        defined by hsr registers.  Instead of checking for hard coded
> values I used hsr defined values.
>      (2) The vgic_irq_rank also depends on the same hsr defined values
> to calculate irq rank.
>           Also, this make vgic_irq_rank generic as it takes register
> size as parameter to calculate irq rank instead
>            of hard coding to value 2 in previous patches

Please add theses explanations in the commit message. It's easier to 
understand the purpose of this patch with that.

Regards

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers
  2014-05-26 13:10   ` Julien Grall
@ 2014-05-30 12:54     ` Vijay Kilari
  0 siblings, 0 replies; 78+ messages in thread
From: Vijay Kilari @ 2014-05-30 12:54 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

Hi Julien,

On Mon, May 26, 2014 at 6:40 PM, Julien Grall <julien.grall@linaro.org> wrote:

>
>
> As n is only used for REG_RANK_NR, I would do:
>
> int rank = REG_RANK_NR(b, n >> 2)
>
> Otherwise than the few changes here, the patch looks good to me, assuming
> you did some tests on GICv2.
>

Yes, I tested the complete series with Foundation model today.
With this I tested GICv. minor issues are seen which will be fixed
in next version

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-05-26 10:26 ` [PATCH v4 14/16] xen/arm: Add virtual GICv3 support vijay.kilari
@ 2014-06-02 15:50   ` Stefano Stabellini
  2014-06-11 11:36     ` Vijay Kilari
  2014-06-02 16:10   ` Julien Grall
  1 sibling, 1 reply; 78+ messages in thread
From: Stefano Stabellini @ 2014-06-02 15:50 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> +    case GICD_ISPENDR ... GICD_ISPENDRN:
> +        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 1, reg - GICD_ISPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = byte_read(rank->ipend, dabt.sign, reg);
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICPENDR ... GICD_ICPENDRN:
> +        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 1, reg - GICD_ICPENDR, DABT_WORD);
> +        if ( rank == NULL) goto read_as_zero;
> +        vgic_lock_rank(v, rank);
> +        *r = byte_read(rank->ipend, dabt.sign, reg);
> +        vgic_unlock_rank(v, rank);
> +        return 1;

[...]

> +    case GICD_ISPENDR ... GICD_ISPENDRN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 1, reg - GICD_ISPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->ipend = *r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;
> +    case GICD_ICPENDR ... GICD_ICPENDRN:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 1, reg - GICD_ICPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->ipend &= ~*r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;

I do realize that this is not a regression compared to our current GICv2
code, but pendsgi, GICR_ISPENDR0 and GICR_ICPENDR0 should do something.
GICR_ISPENDR0 reads should return which interrupts are pending and
GICR_ISPENDR0 writes should set the status of a particular irq as
pending.
Given that we currently don't do this on GICv2, I would be OK with this
code as is, as long as you add a TODO comment to remind us that we need
to implement it properly.


> +    case GICD_IROUTER ... GICD_IROUTER31:
> +        /* SGI/PPI is RES0 */
> +        goto write_ignore_64;
> +    case GICD_IROUTER32 ... GICD_IROUTERN:
> +        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
> +        if ( rank == NULL) goto write_ignore;
> +        vgic_lock_rank(v, rank);
> +        rank->v3.irouter[REG_RANK_INDEX(64,
> +                      (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)] = *r;
> +        vgic_unlock_rank(v, rank);
> +        return 1;

Similarly to

http://marc.info/?l=xen-devel&m=140104126710713

We should ignore any writes to GICD_IROUTER*.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-05-26 10:26 ` [PATCH v4 16/16] xen/arm: add SGI handling for GICv3 vijay.kilari
@ 2014-06-02 16:05   ` Stefano Stabellini
  2014-06-02 16:13     ` Ian Campbell
  2014-06-02 16:17   ` Julien Grall
  1 sibling, 1 reply; 78+ messages in thread
From: Stefano Stabellini @ 2014-06-02 16:05 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> In ARMv8, write to ICC_SGI1R_EL1 register raises trap to EL2.
> Handle the trap and inject SGI to vcpu.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
>  xen/arch/arm/traps.c              |   30 ++++++++++++++++++++++++++++++
>  xen/arch/arm/vgic-v3.c            |   37 +++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/gic_v3_defs.h |    7 +++++++
>  xen/include/asm-arm/sysregs.h     |    3 +++
>  xen/include/asm-arm/vgic.h        |    2 +-
>  5 files changed, 78 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 9348147..1ac01ee 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -41,6 +41,7 @@
>  #include "decode.h"
>  #include "vtimer.h"
>  #include <asm/gic.h>
> +#include <asm/vgic.h>
>  
>  /* The base of the stack must always be double-word aligned, which means
>   * that both the kernel half of struct cpu_user_regs (which is pushed in
> @@ -496,6 +497,18 @@ static void inject_dabt_exception(struct cpu_user_regs *regs,
>  #endif
>  }
>  
> +static void inject_undef_exception(struct cpu_user_regs *regs,
> +                                   register_t addr,
> +                                   int instr_len)
> +{
> +    if ( is_32bit_domain(current->domain) )
> +        inject_undef32_exception(regs);
> +#ifdef CONFIG_ARM_64
> +    else
> +        inject_undef64_exception(regs, instr_len);
> +#endif
> +}
> +
>  struct reg_ctxt {
>      /* Guest-side state */
>      uint32_t sctlr_el1;
> @@ -1467,6 +1480,7 @@ static void do_sysreg(struct cpu_user_regs *regs,
>                        union hsr hsr)
>  {
>      register_t *x = select_user_reg(regs, hsr.sysreg.reg);
> +    register_t addr;
>  
>      switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
>      {
> @@ -1515,6 +1529,22 @@ static void do_sysreg(struct cpu_user_regs *regs,
>              domain_crash_synchronous();
>          }
>          break;
> +    case HSR_SYSREG_ICC_SGI1R_EL1:
> +        if ( !vgic_emulate(regs, hsr) )
> +        {
> +            addr = READ_SYSREG64(FAR_EL2);
> +            dprintk(XENLOG_WARNING,
> +                    "failed emulation of sysreg ICC_SGI1R_EL1 access\n");
> +            inject_undef_exception(regs, addr, hsr.len);
> +        }
> +        break;
> +    case HSR_SYSREG_ICC_SGI0R_EL1:
> +    case HSR_SYSREG_ICC_ASGI1R_EL1:
> +        /* TBD: Implement to support secure grp0/1 SGI forwarding */
> +        dprintk(XENLOG_WARNING,
> +                "Emulation of sysreg ICC_SGI0R_EL1/ASGI1R_EL1 not supported\n");
> +        addr = READ_SYSREG64(FAR_EL2);
> +        inject_undef_exception(regs, addr, hsr.len);
>      default:
>   bad_sysreg:
>          {
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index d80683d..99d0d46 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -832,6 +832,43 @@ write_ignore_64:
>      return 1;
>  }
>  
> +static int vgicv3_to_sgi(struct vcpu *v, register_t sgir)
> +{
> +    int virq;
> +    int irqmode;
> +    unsigned long vcpu_mask = 0;
> +
> +    irqmode = (sgir >> ICH_SGI_IRQMODE_SHIFT) & ICH_SGI_IRQMODE_MASK;
> +    virq = (sgir >> ICH_SGI_IRQ_SHIFT ) & ICH_SGI_IRQ_MASK;
> +    vcpu_mask = sgir & ICH_SGI_TARGETLIST_MASK;
> +
> +    return vgic_to_sgi(v, sgir, irqmode, virq, vcpu_mask);
> +}

Given that we are not handling affinity 1, 2 and 3 settings, we should
add an assert to make sure that they are 0.


> +int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
> +{
> +    struct vcpu *v = current;
> +    struct hsr_sysreg sysreg = hsr.sysreg;
> +    register_t *r = select_user_reg(regs, sysreg.reg);
> +
> +    ASSERT (hsr.ec == HSR_EC_SYSREG);
> +
> +    switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
> +    {
> +    case HSR_SYSREG_ICC_SGI1R_EL1:
> +        /* WO */
> +        if ( !sysreg.read )
> +            return vgicv3_to_sgi(v, *r);
> +        else
> +        {
> +            gdprintk(XENLOG_WARNING, "Reading SGI1R_EL1 - WO register\n");
> +            return 0;
> +        }
> +    default:
> +        return 0;
> +    }
> +}
> +
>  const static struct mmio_handler_ops vgic_rdistr_mmio_handler = {
>      .read_handler  = vgic_v3_rdistr_mmio_read,
>      .write_handler = vgic_v3_rdistr_mmio_write,
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 6f393aa..5e75632 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -145,6 +145,13 @@
>  #define GICH_VMCR_PRIORITY_MASK      0xff
>  #define GICH_VMCR_PRIORITY_SHIFT     24
>  
> +#define ICH_SGI_IRQMODE_SHIFT        40
> +#define ICH_SGI_IRQMODE_MASK         0x1
> +#define ICH_SGI_TARGET_OTHERS        1
> +#define ICH_SGI_TARGET_LIST          0
> +#define ICH_SGI_IRQ_SHIFT            24
> +#define ICH_SGI_IRQ_MASK             0xf
> +#define ICH_SGI_TARGETLIST_MASK      0xffff
>  #endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
>  
>  /*
> diff --git a/xen/include/asm-arm/sysregs.h b/xen/include/asm-arm/sysregs.h
> index 4a4de34..5029851 100644
> --- a/xen/include/asm-arm/sysregs.h
> +++ b/xen/include/asm-arm/sysregs.h
> @@ -77,6 +77,9 @@
>  #define HSR_SYSREG_PMINTENCLR_EL1 HSR_SYSREG(3,0,c9,c14,2)
>  #define HSR_SYSREG_MAIR_EL1       HSR_SYSREG(3,0,c10,c2,0)
>  #define HSR_SYSREG_AMAIR_EL1      HSR_SYSREG(3,0,c10,c3,0)
> +#define HSR_SYSREG_ICC_SGI1R_EL1  HSR_SYSREG(3,0,c12,c11,5)
> +#define HSR_SYSREG_ICC_ASGI1R_EL1 HSR_SYSREG(3,1,c12,c11,6)
> +#define HSR_SYSREG_ICC_SGI0R_EL1  HSR_SYSREG(3,2,c12,c11,7)
>  #define HSR_SYSREG_CONTEXTIDR_EL1 HSR_SYSREG(3,0,c13,c0,1)
>  
>  #define HSR_SYSREG_PMCR_EL0       HSR_SYSREG(3,3,c9,c12,0)
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 754d521..4258840 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -125,7 +125,7 @@ extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
>  extern void vgic_clear_pending_irqs(struct vcpu *v);
>  extern int vgic_to_sgi(struct vcpu *v, register_t sgir, int irqmode, int virq,
>                         unsigned long vcpu_mask);
> -
> +extern int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr);
>  extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
>  extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-05-26 10:26 ` [PATCH v4 14/16] xen/arm: Add virtual GICv3 support vijay.kilari
  2014-06-02 15:50   ` Stefano Stabellini
@ 2014-06-02 16:10   ` Julien Grall
  2014-06-02 16:15     ` Ian Campbell
  1 sibling, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-06-02 16:10 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> Add virtual GICv3 driver support
> 
> This patch adds only basic v3 support.
> Does not support Interrupt Translation support (ITS)

You are also modify the vgic-v2 driver. Please update the commit message.

> +static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                                        uint32_t gicr_reg)
> +{
> +    struct hsr_dabt dabt = info->dabt;
> +    struct cpu_user_regs *regs = guest_cpu_user_regs();
> +    register_t *r = select_user_reg(regs, dabt.reg);
> +    uint64_t mpidr;
> +    uint64_t aff;
> +
> +    switch ( gicr_reg )
> +    {
> +    case GICR_CTLR:
> +        /* We have not implemented LPI's, read zero */
> +        goto read_as_zero;
> +    case GICR_IIDR:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        *r = GICV3_GICR_IIDR_VAL;
> +        return 1;
> +    case GICR_TYPER:
> +        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> +        /* TBD: Update processor id in [23:8] when ITS support is added */
> +        mpidr = cpu_logical_map(v->vcpu_id);

hu? cpu_logical_map contains the MPIDR for the physical CPUs not virtual
CPUs.

You should look at v->arch.vmpidr.

[..]

> +    case GICD_ICFGR: /* Restricted to configure SGIs */
> +        goto write_ignore;
> +    case GICD_ICFGR + 4 ... GICD_ICFGRN: /* PPI + SPIs */
> +        /* ICFGR1 for PPI's, which is implementation defined
> +           if ICFGR1 is programmable or not. We chose to program */
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_irq_rank(v, 2, reg - GICD_ICFGR, DABT_WORD);
> +        vgic_lock_rank(v, rank);
> +        if ( rank == NULL) goto write_ignore;

You've blindly copied the code from GICv2 and keep the security issue.
This should be:

if ( rank == NULL ) ...
vgic_lock_rank(v, rank);

[..]

> +static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)

[..]

> +    case GICD_TYPER:
> +        if ( dabt.size != DABT_WORD ) goto bad_width;
> +        /* No secure world support for guests. */
> +        vgic_lock(v);

This is a copy from GICv2. I'm not sure if we need to take the vgic lock
here.

[..]

> +const static struct mmio_handler_ops vgic_rdistr_mmio_handler = {

static const

> +    .read_handler  = vgic_v3_rdistr_mmio_read,
> +    .write_handler = vgic_v3_rdistr_mmio_write,
> +};
> +
> +const static struct mmio_handler_ops vgic_distr_mmio_handler = {

static const

> +    .read_handler  = vgic_v3_distr_mmio_read,
> +    .write_handler = vgic_v3_distr_mmio_write,
> +};
> +
> +static int vgicv3_vcpu_init(struct vcpu *v)
> +{
> +    int i;
> +    uint64_t affinity;
> +
> +    /* For SGI and PPI the target is always this CPU */
> +    affinity = cpu_logical_map(smp_processor_id());

The SGI and PPI should be redirect to the virtual VCPU. In this case you
have to use the virtual CPU ID *not* the physical CPU ID.

> +    for ( i = 0 ; i < 32 ; i++ )
> +        v->arch.vgic.private_irqs->v3.irouter[i] = affinity;
> +
> +    return 0;
> +}
> +
> +static int vgicv3_domain_init(struct domain *d)
> +{
> +    int i;
> +
> +    register_mmio_handler(d, &vgic_distr_mmio_handler, d->arch.vgic.dbase,
> +                          d->arch.vgic.dbase_size);
> +
> +    /*
> +     * Register mmio handler per redistributor region but not for
> +     * every sgi rdist region which is per core.
> +     * The redistributor region encompasses per core sgi region.
> +     */
> +    for ( i = 0; i < d->arch.vgic.rdist_count; i++ )
> +        register_mmio_handler(d, &vgic_rdistr_mmio_handler,
> +            d->arch.vgic.rbase[i], d->arch.vgic.rbase_size[i]);
> +
> +    return 0;
> +}
> +
> +const static struct vgic_ops v3_ops = {

static const

[..]

> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 3fa0857..787c547 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -264,10 +264,19 @@ int domain_vgic_init(struct domain *d)
>      else
>          d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
>  
> -    if ( gic_hw_version() == GIC_V2 )
> +    switch ( gic_hw_version() )
> +    {
> +#ifdef CONFIG_ARM_64
> +    case GIC_V3:
> +        vgic_v3_init(d);
> +        break;
> +#endif
> +    case GIC_V2:
>          vgic_v2_init(d);
> -    else
> +        break;
> +    default:
>          panic("No VGIC found\n");

I think I've already said in an earlier patch. Please avoid to use panic
when this function is called by a VM.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-02 16:05   ` Stefano Stabellini
@ 2014-06-02 16:13     ` Ian Campbell
  2014-06-11 12:34       ` Vijay Kilari
  0 siblings, 1 reply; 78+ messages in thread
From: Ian Campbell @ 2014-06-02 16:13 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: vijay.kilari, Prasun.Kapoor, Vijaya Kumar K, julien.grall,
	xen-devel, stefano.stabellini

On Mon, 2014-06-02 at 17:05 +0100, Stefano Stabellini wrote:
> > +static int vgicv3_to_sgi(struct vcpu *v, register_t sgir)
> > +{
> > +    int virq;
> > +    int irqmode;
> > +    unsigned long vcpu_mask = 0;
> > +
> > +    irqmode = (sgir >> ICH_SGI_IRQMODE_SHIFT) & ICH_SGI_IRQMODE_MASK;
> > +    virq = (sgir >> ICH_SGI_IRQ_SHIFT ) & ICH_SGI_IRQ_MASK;
> > +    vcpu_mask = sgir & ICH_SGI_TARGETLIST_MASK;
> > +
> > +    return vgic_to_sgi(v, sgir, irqmode, virq, vcpu_mask);
> > +}
> 
> Given that we are not handling affinity 1, 2 and 3 settings, we should
> add an assert to make sure that they are 0.

Not literally an assert since that would give guests a trivial way to
DoS the system....

Given that we create guest VCPUs with MPIDR's which do not use
AFF{1,2,3} I think we can simply ignore any attempt to SGI those
processors, since they can't exist.

Of course this will mean implementing a 16 VCPU limit somewhere on the
creation side, so we actually do not create VCPUs with higher AFF's
used...

There should of course be a suitable comment both here and there...

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-06-02 16:10   ` Julien Grall
@ 2014-06-02 16:15     ` Ian Campbell
  2014-06-02 16:18       ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Ian Campbell @ 2014-06-02 16:15 UTC (permalink / raw)
  To: Julien Grall
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	xen-devel, stefano.stabellini

On Mon, 2014-06-02 at 17:10 +0100, Julien Grall wrote:
> 
> > diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> > index 3fa0857..787c547 100644
> > --- a/xen/arch/arm/vgic.c
> > +++ b/xen/arch/arm/vgic.c
> > @@ -264,10 +264,19 @@ int domain_vgic_init(struct domain *d)
> >      else
> >          d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
> >  
> > -    if ( gic_hw_version() == GIC_V2 )
> > +    switch ( gic_hw_version() )
> > +    {
> > +#ifdef CONFIG_ARM_64
> > +    case GIC_V3:
> > +        vgic_v3_init(d);
> > +        break;
> > +#endif
> > +    case GIC_V2:
> >          vgic_v2_init(d);
> > -    else
> > +        break;
> > +    default:
> >          panic("No VGIC found\n");
> 
> I think I've already said in an earlier patch. Please avoid to use panic
> when this function is called by a VM.

Whether called from a guest or not, gic_hw_version's result is, I think,
completely under hypervisor control, if it returns a gic version which
we don't support then that is worthy of a panic() I think, or a
BUG_ON(), since that is a hypervisor coding issue.

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-05-26 10:26 ` [PATCH v4 16/16] xen/arm: add SGI handling for GICv3 vijay.kilari
  2014-06-02 16:05   ` Stefano Stabellini
@ 2014-06-02 16:17   ` Julien Grall
  2014-06-11 12:35     ` Vijay Kilari
  1 sibling, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-06-02 16:17 UTC (permalink / raw)
  To: vijay.kilari, Ian.Campbell, stefano.stabellini,
	stefano.stabellini, xen-devel
  Cc: Prasun.Kapoor, vijaya.kumar

Hi Vijay,

On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index d80683d..99d0d46 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c

[..]

> +int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
> +{

You are by-passing without any reason the vgic structure. Why didn't you
add a new callback there?

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-06-02 16:15     ` Ian Campbell
@ 2014-06-02 16:18       ` Julien Grall
  2014-06-02 16:38         ` Ian Campbell
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-06-02 16:18 UTC (permalink / raw)
  To: Ian Campbell
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	xen-devel, stefano.stabellini

On 06/02/2014 05:15 PM, Ian Campbell wrote:
> On Mon, 2014-06-02 at 17:10 +0100, Julien Grall wrote:
>>
>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>> index 3fa0857..787c547 100644
>>> --- a/xen/arch/arm/vgic.c
>>> +++ b/xen/arch/arm/vgic.c
>>> @@ -264,10 +264,19 @@ int domain_vgic_init(struct domain *d)
>>>      else
>>>          d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
>>>  
>>> -    if ( gic_hw_version() == GIC_V2 )
>>> +    switch ( gic_hw_version() )
>>> +    {
>>> +#ifdef CONFIG_ARM_64
>>> +    case GIC_V3:
>>> +        vgic_v3_init(d);
>>> +        break;
>>> +#endif
>>> +    case GIC_V2:
>>>          vgic_v2_init(d);
>>> -    else
>>> +        break;
>>> +    default:
>>>          panic("No VGIC found\n");
>>
>> I think I've already said in an earlier patch. Please avoid to use panic
>> when this function is called by a VM.
> 
> Whether called from a guest or not, gic_hw_version's result is, I think,
> completely under hypervisor control, if it returns a gic version which
> we don't support then that is worthy of a panic() I think, or a
> BUG_ON(), since that is a hypervisor coding issue.

It won't be under hypervisor control if we add an hypercall to choose
whether we want to use GICv2 or GICv3.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-06-02 16:18       ` Julien Grall
@ 2014-06-02 16:38         ` Ian Campbell
  2014-06-02 16:46           ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Ian Campbell @ 2014-06-02 16:38 UTC (permalink / raw)
  To: Julien Grall
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	xen-devel, stefano.stabellini

On Mon, 2014-06-02 at 17:18 +0100, Julien Grall wrote:
> On 06/02/2014 05:15 PM, Ian Campbell wrote:
> > On Mon, 2014-06-02 at 17:10 +0100, Julien Grall wrote:
> >>
> >>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> >>> index 3fa0857..787c547 100644
> >>> --- a/xen/arch/arm/vgic.c
> >>> +++ b/xen/arch/arm/vgic.c
> >>> @@ -264,10 +264,19 @@ int domain_vgic_init(struct domain *d)
> >>>      else
> >>>          d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
> >>>  
> >>> -    if ( gic_hw_version() == GIC_V2 )
> >>> +    switch ( gic_hw_version() )
> >>> +    {
> >>> +#ifdef CONFIG_ARM_64
> >>> +    case GIC_V3:
> >>> +        vgic_v3_init(d);
> >>> +        break;
> >>> +#endif
> >>> +    case GIC_V2:
> >>>          vgic_v2_init(d);
> >>> -    else
> >>> +        break;
> >>> +    default:
> >>>          panic("No VGIC found\n");
> >>
> >> I think I've already said in an earlier patch. Please avoid to use panic
> >> when this function is called by a VM.
> > 
> > Whether called from a guest or not, gic_hw_version's result is, I think,
> > completely under hypervisor control, if it returns a gic version which
> > we don't support then that is worthy of a panic() I think, or a
> > BUG_ON(), since that is a hypervisor coding issue.
> 
> It won't be under hypervisor control if we add an hypercall to choose
> whether we want to use GICv2 or GICv3.

That (tools only) hypercall should obviously validate that it knows
about the gic that is being asked for at the time.

If on retrieval it is not something we understand then that is still a
hypervisor bug.

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-06-02 16:38         ` Ian Campbell
@ 2014-06-02 16:46           ` Julien Grall
  0 siblings, 0 replies; 78+ messages in thread
From: Julien Grall @ 2014-06-02 16:46 UTC (permalink / raw)
  To: Ian Campbell
  Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
	xen-devel, stefano.stabellini

On 06/02/2014 05:38 PM, Ian Campbell wrote:
> On Mon, 2014-06-02 at 17:18 +0100, Julien Grall wrote:
>> On 06/02/2014 05:15 PM, Ian Campbell wrote:
>>> On Mon, 2014-06-02 at 17:10 +0100, Julien Grall wrote:
>>>>
>>>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>>>> index 3fa0857..787c547 100644
>>>>> --- a/xen/arch/arm/vgic.c
>>>>> +++ b/xen/arch/arm/vgic.c
>>>>> @@ -264,10 +264,19 @@ int domain_vgic_init(struct domain *d)
>>>>>      else
>>>>>          d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
>>>>>  
>>>>> -    if ( gic_hw_version() == GIC_V2 )
>>>>> +    switch ( gic_hw_version() )
>>>>> +    {
>>>>> +#ifdef CONFIG_ARM_64
>>>>> +    case GIC_V3:
>>>>> +        vgic_v3_init(d);
>>>>> +        break;
>>>>> +#endif
>>>>> +    case GIC_V2:
>>>>>          vgic_v2_init(d);
>>>>> -    else
>>>>> +        break;
>>>>> +    default:
>>>>>          panic("No VGIC found\n");
>>>>
>>>> I think I've already said in an earlier patch. Please avoid to use panic
>>>> when this function is called by a VM.
>>>
>>> Whether called from a guest or not, gic_hw_version's result is, I think,
>>> completely under hypervisor control, if it returns a gic version which
>>> we don't support then that is worthy of a panic() I think, or a
>>> BUG_ON(), since that is a hypervisor coding issue.
>>
>> It won't be under hypervisor control if we add an hypercall to choose
>> whether we want to use GICv2 or GICv3.
> 
> That (tools only) hypercall should obviously validate that it knows
> about the gic that is being asked for at the time.
> 
> If on retrieval it is not something we understand then that is still a
> hypervisor bug.

Assuming we correctly validated the version given by the toolstack. I'm
not very confident with a BUG_ON/panic in hypercall (this function is
called during domain creation) when we can just return an error code and
safely come back in a valid state (i.e nothing will leak...)

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-05-26 10:26 ` [PATCH v4 13/16] xen/arm: Add support for GIC v3 vijay.kilari
  2014-05-27 19:47   ` Julien Grall
@ 2014-06-02 17:33   ` Stefano Stabellini
  2014-06-03  8:54     ` Ian Campbell
  1 sibling, 1 reply; 78+ messages in thread
From: Stefano Stabellini @ 2014-06-02 17:33 UTC (permalink / raw)
  To: vijay.kilari
  Cc: Ian Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, marc.zyngier, stefano.stabellini

On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> 
> Add support for GIC v3 specification System register access(SRE)
> is enabled to access cpu and virtual interface regiseters based
> on kernel GICv3 driver.
> 
> This patch adds only basic v3 support.
> Does not support Interrupt Translation support (ITS)
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
>  xen/arch/arm/Makefile             |    1 +
>  xen/arch/arm/gic-v3.c             | 1125 +++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/domain.h      |    8 +
>  xen/include/asm-arm/gic.h         |   20 +
>  xen/include/asm-arm/gic_v3_defs.h |  157 ++++++
>  xen/include/asm-arm/processor.h   |   14 +
>  xen/include/xen/lib.h             |    2 +
>  7 files changed, 1327 insertions(+)
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 20f59f4..1684c09 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -11,6 +11,7 @@ obj-y += domctl.o
>  obj-y += sysctl.o
>  obj-y += domain_build.o
>  obj-y += gic.o gic-v2.o
> +obj-$(CONFIG_ARM_64) += gic-v3.o
>  obj-y += io.o
>  obj-y += irq.o
>  obj-y += kernel.o
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> new file mode 100644
> index 0000000..7543fb4
> --- /dev/null
> +++ b/xen/arch/arm/gic-v3.c
> @@ -0,0 +1,1125 @@
> +/*
> + * xen/arch/arm/gic-v3.c
> + *
> + * ARM Generic Interrupt Controller support v3 version
> + * based on xen/arch/arm/gic-v2.c and kernel GICv3 driver
> + *
> + * Copyright (C) 2012,2013 - ARM Ltd
> + * Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>, Cavium Inc
> + * ported to Xen
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <xen/config.h>
> +#include <xen/lib.h>
> +#include <xen/init.h>
> +#include <xen/cpu.h>
> +#include <xen/mm.h>
> +#include <xen/irq.h>
> +#include <xen/sched.h>
> +#include <xen/errno.h>
> +#include <xen/delay.h>
> +#include <xen/device_tree.h>
> +#include <xen/libfdt/libfdt.h>
> +#include <asm/p2m.h>
> +#include <asm/domain.h>
> +#include <asm/platform.h>
> +#include <asm/io.h>
> +#include <asm/device.h>
> +#include <asm/gic.h>
> +#include <asm/gic_v3_defs.h>
> +
> +struct rdist_region {
> +    paddr_t base;
> +    paddr_t size;
> +    void __iomem *map_base;
> +};
> +
> +/* Global state */
> +static struct {
> +    paddr_t dbase;            /* Address of distributor registers */
> +    paddr_t dbase_size;
> +    void __iomem *map_dbase;  /* Mapped address of distributor registers */
> +    struct rdist_region *rdist_regions;
> +    uint32_t  rdist_stride;
> +    unsigned int rdist_count; /* Number of rdist regions count */
> +    unsigned int nr_priorities;
> +    spinlock_t lock;
> +}gicv3;
> +
> +static struct gic_info gicv3_info;
> +
> +/* per-cpu re-distributor base */
> +static DEFINE_PER_CPU(void __iomem*, rbase);
> +
> +#define GICD                   (gicv3.map_dbase)
> +#define GICD_RDIST_BASE        (this_cpu(rbase))
> +#define GICD_RDIST_SGI_BASE    (GICD_RDIST_BASE + SZ_64K)
> +
> +/*
> + * Saves all 16(Max) LR registers. Though number of LRs implemented
> + * is implementation specific.
> + */
> +static inline void gicv3_save_lr(int nr_lrs, struct vcpu *v)
> +{
> +    /* Fall through for all the cases */
> +    switch ( nr_lrs )
> +    {
> +    case 16:
> +        v->arch.gic.v3.lr[15] = READ_SYSREG(ICH_LR15_EL2);
> +    case 15:
> +        v->arch.gic.v3.lr[14] = READ_SYSREG(ICH_LR14_EL2);
> +    case 14:
> +        v->arch.gic.v3.lr[13] = READ_SYSREG(ICH_LR13_EL2);
> +    case 13:
> +        v->arch.gic.v3.lr[12] = READ_SYSREG(ICH_LR12_EL2);
> +    case 12:
> +        v->arch.gic.v3.lr[11] = READ_SYSREG(ICH_LR11_EL2);
> +    case 11:
> +        v->arch.gic.v3.lr[10] = READ_SYSREG(ICH_LR10_EL2);
> +    case 10:
> +        v->arch.gic.v3.lr[9] = READ_SYSREG(ICH_LR9_EL2);
> +    case 9:
> +        v->arch.gic.v3.lr[8] = READ_SYSREG(ICH_LR8_EL2);
> +    case 8:
> +        v->arch.gic.v3.lr[7] = READ_SYSREG(ICH_LR7_EL2);
> +    case 7:
> +        v->arch.gic.v3.lr[6] = READ_SYSREG(ICH_LR6_EL2);
> +    case 6:
> +        v->arch.gic.v3.lr[5] = READ_SYSREG(ICH_LR5_EL2);
> +    case 5:
> +        v->arch.gic.v3.lr[4] = READ_SYSREG(ICH_LR4_EL2);
> +    case 4:
> +        v->arch.gic.v3.lr[3] = READ_SYSREG(ICH_LR3_EL2);
> +    case 3:
> +        v->arch.gic.v3.lr[2] = READ_SYSREG(ICH_LR2_EL2);
> +    case 2:
> +        v->arch.gic.v3.lr[1] = READ_SYSREG(ICH_LR1_EL2);
> +    case 1:
> +         v->arch.gic.v3.lr[0] = READ_SYSREG(ICH_LR0_EL2);
> +         break;
> +    default:
> +         BUG();
> +    }
> +}
> +
> +/*
> + * Restores all 16(Max) LR registers. Though number of LRs implemented
> + * is implementation specific.
> + */
> +static inline void gicv3_restore_lr(int nr_lrs, const struct vcpu *v)
> +{
> +    /* Fall through for all the cases */
> +    switch ( nr_lrs )
> +    {
> +    case 16:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[15], ICH_LR15_EL2);
> +    case 15:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[14], ICH_LR14_EL2);
> +    case 14:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[13], ICH_LR13_EL2);
> +    case 13:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[12], ICH_LR12_EL2);
> +    case 12:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[11], ICH_LR11_EL2);
> +    case 11:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[10], ICH_LR10_EL2);
> +    case 10:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[9], ICH_LR9_EL2);
> +    case 9:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[8], ICH_LR8_EL2);
> +    case 8:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[7], ICH_LR7_EL2);
> +    case 7:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[6], ICH_LR6_EL2);
> +    case 6:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[5], ICH_LR5_EL2);
> +    case 5:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[4], ICH_LR4_EL2);
> +    case 4:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[3], ICH_LR3_EL2);
> +    case 3:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[2], ICH_LR2_EL2);
> +    case 2:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[1], ICH_LR1_EL2);
> +    case 1:
> +        WRITE_SYSREG(v->arch.gic.v3.lr[0], ICH_LR0_EL2);
> +         break;
> +    default:
> +         BUG();
> +    }
> +}

Given that the number of LR registers has grown up quite a bit from
GICv2, we should optimize this code and only save and restore the
registers that are actually in used by checking on lr_mask.

It would also help performances if we restored them in ascending order.

> +static uint64_t gicv3_ich_read_lr(int lr)
> +{
> +    switch ( lr )
> +    {
> +    case 0: return READ_SYSREG(ICH_LR0_EL2);
> +    case 1: return READ_SYSREG(ICH_LR1_EL2);
> +    case 2: return READ_SYSREG(ICH_LR2_EL2);
> +    case 3: return READ_SYSREG(ICH_LR3_EL2);
> +    case 4: return READ_SYSREG(ICH_LR4_EL2);
> +    case 5: return READ_SYSREG(ICH_LR5_EL2);
> +    case 6: return READ_SYSREG(ICH_LR6_EL2);
> +    case 7: return READ_SYSREG(ICH_LR7_EL2);
> +    case 8: return READ_SYSREG(ICH_LR8_EL2);
> +    case 9: return READ_SYSREG(ICH_LR9_EL2);
> +    case 10: return READ_SYSREG(ICH_LR10_EL2);
> +    case 11: return READ_SYSREG(ICH_LR11_EL2);
> +    case 12: return READ_SYSREG(ICH_LR12_EL2);
> +    case 13: return READ_SYSREG(ICH_LR13_EL2);
> +    case 14: return READ_SYSREG(ICH_LR14_EL2);
> +    case 15: return READ_SYSREG(ICH_LR15_EL2);
> +    default:
> +        BUG();
> +    }
> +}
> +
> +static void gicv3_ich_write_lr(int lr, uint64_t val)
> +{
> +    switch ( lr )
> +    {
> +    case 0:
> +        WRITE_SYSREG(val, ICH_LR0_EL2);
> +        break;
> +    case 1:
> +        WRITE_SYSREG(val, ICH_LR1_EL2);
> +        break;
> +    case 2:
> +        WRITE_SYSREG(val, ICH_LR2_EL2);
> +        break;
> +    case 3:
> +        WRITE_SYSREG(val, ICH_LR3_EL2);
> +        break;
> +    case 4:
> +        WRITE_SYSREG(val, ICH_LR4_EL2);
> +        break;
> +    case 5:
> +        WRITE_SYSREG(val, ICH_LR5_EL2);
> +        break;
> +    case 6:
> +        WRITE_SYSREG(val, ICH_LR6_EL2);
> +        break;
> +    case 7:
> +        WRITE_SYSREG(val, ICH_LR7_EL2);
> +        break;
> +    case 8:
> +        WRITE_SYSREG(val, ICH_LR8_EL2);
> +        break;
> +    case 9:
> +        WRITE_SYSREG(val, ICH_LR9_EL2);
> +        break;
> +    case 10:
> +        WRITE_SYSREG(val, ICH_LR10_EL2);
> +        break;
> +    case 11:
> +        WRITE_SYSREG(val, ICH_LR11_EL2);
> +        break;
> +    case 12:
> +        WRITE_SYSREG(val, ICH_LR12_EL2);
> +        break;
> +    case 13:
> +        WRITE_SYSREG(val, ICH_LR13_EL2);
> +        break;
> +    case 14:
> +        WRITE_SYSREG(val, ICH_LR14_EL2);
> +        break;
> +    case 15:
> +        WRITE_SYSREG(val, ICH_LR15_EL2);
> +        break;
> +    default:
> +        return;
> +    }
> +}
> +
> +/*
> + * System Register Enable (SRE). Enable to access CPU & Virtual
> + * interface registers as system registers in EL2
> + */
> +static void gicv3_enable_sre(void)
> +{
> +    uint32_t val;
> +
> +    val = READ_SYSREG32(ICC_SRE_EL2);
> +    val |= GICC_SRE_EL2_SRE | GICC_SRE_EL2_ENEL1 |
> +           GICC_SRE_EL2_DFB | GICC_SRE_EL2_DIB;
> +
> +    WRITE_SYSREG32(val, ICC_SRE_EL2);
> +    isb();
> +}
> +
> +/* Wait for completion of a distributor change */
> +static void gicv3_do_wait_for_rwp(void __iomem *base)
> +{
> +    uint32_t val;
> +    bool_t timeout = 0;
> +    s_time_t deadline = NOW() + MILLISECS(1000);
> +
> +    do {
> +        val = readl_relaxed(base + GICD_CTLR);
> +        if ( !(val & GICD_CTLR_RWP) )
> +            break;
> +        if ( NOW() > deadline )
> +        {
> +            timeout = 1;
> +            break;
> +        }
> +        cpu_relax();
> +        udelay(1);
> +    } while ( 1 );
> +
> +    if ( timeout )
> +        dprintk(XENLOG_ERR, "RWP timeout\n");
> +}
> +
> +static void gicv3_dist_wait_for_rwp(void)
> +{
> +    gicv3_do_wait_for_rwp(GICD);
> +}
> +
> +static void gicv3_redist_wait_for_rwp(void)
> +{
> +    gicv3_do_wait_for_rwp(GICD_RDIST_BASE);
> +}
> +
> +static void gicv3_wait_for_rwp(int irq)
> +{
> +    if ( irq < NR_LOCAL_IRQS )
> +         gicv3_redist_wait_for_rwp();
> +    else
> +         gicv3_dist_wait_for_rwp();
> +}
> +
> +static unsigned int gicv3_get_cpu_from_mask(const cpumask_t *cpumask)
> +{
> +    unsigned int cpu;
> +    cpumask_t possible_mask;
> +
> +    cpumask_and(&possible_mask, cpumask, &cpu_possible_map);
> +    cpu = cpumask_any(&possible_mask);
> +
> +    return cpu;
> +}
> +
> +static void restore_aprn_regs(const union gic_state_data *d)
> +{
> +    /* Write APRn register based on number of priorities
> +       platform has implemented */
> +    switch ( gicv3.nr_priorities )
> +    {
> +    case 7:
> +        WRITE_SYSREG32(d->v3.apr0[2], ICH_AP0R2_EL2);
> +        WRITE_SYSREG32(d->v3.apr1[2], ICH_AP1R2_EL2);
> +        /* Fall through */
> +    case 6:
> +        WRITE_SYSREG32(d->v3.apr0[1], ICH_AP0R1_EL2);
> +        WRITE_SYSREG32(d->v3.apr1[1], ICH_AP1R1_EL2);
> +        /* Fall through */
> +    case 5:
> +        WRITE_SYSREG32(d->v3.apr0[0], ICH_AP0R0_EL2);
> +        WRITE_SYSREG32(d->v3.apr1[0], ICH_AP1R0_EL2);
> +        break;
> +    default:
> +        BUG();
> +    }
> +}
> +
> +static void save_aprn_regs(union gic_state_data *d)
> +{
> +    /* Read APRn register based on number of priorities
> +       platform has implemented */
> +    switch ( gicv3.nr_priorities )
> +    {
> +    case 7:
> +        d->v3.apr0[2] = READ_SYSREG32(ICH_AP0R2_EL2);
> +        d->v3.apr1[2] = READ_SYSREG32(ICH_AP1R2_EL2);
> +        /* Fall through */
> +    case 6:
> +        d->v3.apr0[1] = READ_SYSREG32(ICH_AP0R1_EL2);
> +        d->v3.apr1[1] = READ_SYSREG32(ICH_AP1R1_EL2);
> +        /* Fall through */
> +    case 5:
> +        d->v3.apr0[0] = READ_SYSREG32(ICH_AP0R0_EL2);
> +        d->v3.apr1[0] = READ_SYSREG32(ICH_AP1R0_EL2);
> +        break;
> +    default:
> +        BUG();
> +    }
> +}
> +
> +static void gicv3_save_state(struct vcpu *v)
> +{
> +
> +    /* No need for spinlocks here because interrupts are disabled around
> +     * this call and it only accesses struct vcpu fields that cannot be
> +     * accessed simultaneously by another pCPU.
> +     */
> +    dsb(sy);
> +    gicv3_save_lr(gicv3_info.nr_lrs, v);
> +    save_aprn_regs(&v->arch.gic);
> +    v->arch.gic.v3.vmcr = READ_SYSREG32(ICH_VMCR_EL2);
> +}
> +
> +static void gicv3_restore_state(const struct vcpu *v)
> +{
> +    gicv3_restore_lr(gicv3_info.nr_lrs, v);
> +    restore_aprn_regs(&v->arch.gic);
> +    WRITE_SYSREG32(v->arch.gic.v3.vmcr, ICH_VMCR_EL2);
> +}

Shouldn't we restore vmcr first? Before writing to the LRs?

Should we save/restore SRE_EL1 too?
Also looking at the Linux code, it seems that SRE_EL1 and VMCR_EL2 might
have strange interactions, in fact Linux disables SRE_EL1 before writing
to VMCR_EL2 (see restore_vgic_v3_state).


> +static void gicv3_dump_state(const struct vcpu *v)
> +{
> +    int i;
> +
> +    if ( v == current )
> +    {
> +        for ( i = 0; i < gicv3_info.nr_lrs; i++ )
> +            printk("   HW_LR[%d]=%lx\n", i, gicv3_ich_read_lr(i));
> +    }
> +    else
> +    {
> +        for ( i = 0; i < gicv3_info.nr_lrs; i++ )
> +            printk("   VCPU_LR[%d]=%lx\n", i, v->arch.gic.v3.lr[i]);
> +    }
> +}
> +
> +static void gicv3_enable_irq(struct irq_desc *irqd)
> +{
> +    int irq = irqd->irq;
> +
> +    /* Enable routing */
> +    if ( irq < NR_GIC_LOCAL_IRQS )
> +        writel_relaxed((1u << irq), GICD_RDIST_SGI_BASE + GICR_ISENABLER0);
> +    else
> +        writel_relaxed((1u << (irq % 32)),
> +                        GICD + GICD_ISENABLER + (irq / 32) * 4);
> +
> +    gicv3_wait_for_rwp(irq);
> +}
> +
> +static void gicv3_disable_irq(struct irq_desc *irqd)
> +{
> +    /* Disable routing */
> +    if ( irqd->irq < NR_GIC_LOCAL_IRQS )
> +        writel_relaxed((1u << irqd->irq),
> +                       GICD_RDIST_SGI_BASE + GICR_ICENABLER0);
> +    else
> +        writel_relaxed(1u << (irqd->irq % 32),
> +                       GICD + GICD_ICENABLER + ((irqd->irq / 32) * 4));
> +}

Strangely we have a wait in gicv3_enable_irq but not a wait in
gicv3_disable_irq.
Is that deliberate?


> +static void gicv3_eoi_irq(struct irq_desc *irqd)
> +{
> +    /* Lower the priority */
> +    WRITE_SYSREG32(irqd->irq, ICC_EOIR1_EL1);

We need an isb here.


> +}
> +
> +static void gicv3_dir_irq(int irq)
> +{
> +    /* Deactivate */
> +    WRITE_SYSREG32(irq, ICC_DIR_EL1);
> +}

I am not sure if this needs an isb or not.


> +static unsigned int gicv3_read_irq(void)
> +{
> +    return (READ_SYSREG32(ICC_IAR1_EL1) & GICC_IA_IRQ);
> +}
> +
> +static inline uint64_t gicv3_mpidr_to_affinity(uint64_t mpidr)
> +{
> +     return (MPIDR_AFFINITY_LEVEL(mpidr, 3) << 32 |
> +             MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16 |
> +             MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8  |
> +             MPIDR_AFFINITY_LEVEL(mpidr, 0));
> +}
> +
> +static void gicv3_set_irq_properties(struct irq_desc *desc,
> +                                     const cpumask_t *cpu_mask,
> +                                     unsigned int priority)
> +{
> +    unsigned long flags;
> +    uint32_t cfg, edgebit;
> +    uint64_t affinity;
> +    void __iomem *base;
> +    unsigned int cpu = gicv3_get_cpu_from_mask(cpu_mask);
> +    unsigned int irq = desc->irq;
> +    unsigned int type = desc->arch.type;
> +
> +    spin_lock_irqsave(&gicv3.lock, flags);
> +
> +    /* SGI's are always edge-triggered not need to call GICD_ICFGR0 */
> +    if ( irq >= NR_GIC_SGI )
> +    {
> +        if ( irq < NR_GIC_LOCAL_IRQS)
> +            base = GICD + GICD_ICFGR + (irq / 16) * 4;
> +        else
> +            base = GICD_RDIST_SGI_BASE + GICR_ICFGR1;
> +
> +        cfg = readl_relaxed(base);
> +
> +        edgebit = 2u << (2 * (irq % 16));
> +        if ( type & DT_IRQ_TYPE_LEVEL_MASK )
> +           cfg &= ~edgebit;
> +        else if ( type & DT_IRQ_TYPE_EDGE_BOTH )
> +           cfg |= edgebit;
> +
> +        writel_relaxed(cfg, base);
> +    }
> +
> +    affinity = gicv3_mpidr_to_affinity(cpu_logical_map(cpu));
> +    /* Make sure we don't broadcast the interrupt */
> +    affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
> +
> +    if ( irq >= NR_GIC_LOCAL_IRQS )
> +        writeq_relaxed(affinity, (GICD + GICD_IROUTER + irq * 8));
> +
> +    /* Set priority */
> +    if ( irq < NR_GIC_LOCAL_IRQS )
> +        writeb_relaxed(priority, GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irq);
> +    else
> +        writeb_relaxed(priority, GICD + GICD_IPRIORITYR + irq);
> +
> +    spin_unlock_irqrestore(&gicv3.lock, flags);
> +}
> +
> +static void __init gicv3_dist_init(void)
> +{
> +    uint32_t type;
> +    uint32_t priority;
> +    uint64_t affinity;
> +    int i;
> +
> +    /* Disable the distributor */
> +    writel_relaxed(0, GICD + GICD_CTLR);
> +
> +    type = readl_relaxed(GICD + GICD_TYPER);
> +    gicv3_info.nr_lines = 32 * ((type & GICD_TYPE_LINES) + 1);
> +
> +    printk("GICv3: %d lines, (IID %8.8x).\n",
> +           gicv3_info.nr_lines, readl_relaxed(GICD + GICD_IIDR));
> +
> +    /* Default all global IRQs to level, active low */
> +    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 16 )
> +        writel_relaxed(0, GICD + GICD_ICFGR + (i / 16) * 4);
> +
> +    /* Default priority for global interrupts */
> +    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 4 )
> +    {
> +        priority = (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 |
> +                    GIC_PRI_IRQ << 8 | GIC_PRI_IRQ);
> +        writel_relaxed(priority, GICD + GICD_IPRIORITYR + (i / 4) * 4);
> +    }
> +
> +    /* Disable all global interrupts */
> +    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i += 32 )
> +        writel_relaxed(0xffffffff, GICD + GICD_ICENABLER + (i / 32) * 4);
> +
> +    gicv3_dist_wait_for_rwp();
> +
> +    /* Turn on the distributor */
> +    writel_relaxed(GICD_CTL_ENABLE | GICD_CTLR_ARE_NS |
> +                GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1, GICD + GICD_CTLR);
> +
> +    /* Route all global IRQs to this CPU */
> +    affinity = gicv3_mpidr_to_affinity(cpu_logical_map(smp_processor_id()));
> +    /* Make sure we don't broadcast the interrupt */
> +    affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
> +
> +    for ( i = NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i++ )
> +        writeq_relaxed(affinity, GICD + GICD_IROUTER + i * 8);
> +}
> +
> +static int gicv3_enable_redist(void)
> +{
> +    uint32_t val;
> +    bool_t timeout = 0;
> +    s_time_t deadline = NOW() + MILLISECS(1000);
> +
> +    /* Wake up this CPU redistributor */
> +    val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
> +    val &= ~GICR_WAKER_ProcessorSleep;
> +    writel_relaxed(val, GICD_RDIST_BASE + GICR_WAKER);
> +
> +    do {
> +        val = readl_relaxed(GICD_RDIST_BASE + GICR_WAKER);
> +        if ( !(val & GICR_WAKER_ChildrenAsleep) )
> +            break;
> +        if ( NOW() > deadline )
> +        {
> +            timeout = 1;
> +            break;
> +        }
> +        cpu_relax();
> +        udelay(1);
> +    } while ( timeout );
> +
> +    if ( timeout )
> +        dprintk(XENLOG_ERR, "GICv3: Redist enable RWP timeout\n");
> +
> +    return 0;
> +}
> +
> +static int __init gicv3_populate_rdist(void)
> +{
> +    int i;
> +    uint32_t aff;
> +    uint32_t reg;
> +    uint64_t typer;
> +    uint64_t mpidr = cpu_logical_map(smp_processor_id());
> +
> +    aff = gicv3_mpidr_to_affinity(mpidr);
> +
> +    for ( i = 0; i < gicv3.rdist_count; i++ )
> +    {
> +        void __iomem *ptr = gicv3.rdist_regions[i].map_base;
> +
> +        reg = readl_relaxed(ptr + GICR_PIDR2) & GICR_PIDR2_ARCH_MASK;
> +        if ( reg  != (GICV3_GICR_PIDR2 & GICR_PIDR2_ARCH_MASK) )
> +        {
> +            dprintk(XENLOG_ERR,
> +                    "GICv3: No redistributor present @%"PRIpaddr"\n",
> +                    gicv3.rdist_regions[i].base);
> +            break;
> +        }
> +
> +        do {
> +            typer = readq_relaxed(ptr + GICR_TYPER);
> +
> +            if ( (typer >> 32) == aff )
> +            {
> +                this_cpu(rbase) = ptr;
> +                printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
> +                        smp_processor_id(), i, ptr);
> +                return 0;
> +            }
> +
> +            ptr += gicv3.rdist_stride;
> +            if ( typer & GICR_TYPER_VLPIS )
> +                    ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
> +
> +        } while ( !(typer & GICR_TYPER_LAST) );
> +    }
> +
> +    dprintk(XENLOG_ERR, "GICv3: CPU%d: mpidr %lx has no re-distributor!\n",
> +                  smp_processor_id(), mpidr);
> +    return -ENODEV;
> +}
> +
> +static int __cpuinit gicv3_cpu_init(void)
> +{
> +    int i;
> +    uint32_t priority;
> +
> +    /* Register ourselves with the rest of the world */
> +    if ( gicv3_populate_rdist() )
> +        return -ENODEV;
> +
> +    if ( gicv3_enable_redist() )
> +        return -ENODEV;
> +
> +    /* Set priority on PPI and SGI interrupts */
> +    priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
> +                GIC_PRI_IPI);
> +    for (i = 0; i < NR_GIC_SGI; i += 4)
> +        writel_relaxed(priority,
> +                GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
> +
> +    priority = (GIC_PRI_IRQ << 24 | GIC_PRI_IRQ << 16 | GIC_PRI_IRQ << 8 |
> +                GIC_PRI_IRQ);
> +    for (i = NR_GIC_SGI; i < NR_GIC_LOCAL_IRQS; i += 4)
> +        writel_relaxed(priority,
> +                GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
> +
> +    /*
> +     * Disable all PPI interrupts, ensure all SGI interrupts are
> +     * enabled.
> +     */
> +    writel_relaxed(0xffff0000, GICD_RDIST_SGI_BASE + GICR_ICENABLER0);
> +    writel_relaxed(0x0000ffff, GICD_RDIST_SGI_BASE + GICR_ISENABLER0);
> +
> +    gicv3_redist_wait_for_rwp();
> +
> +    /* Enable system registers */
> +    gicv3_enable_sre();
> +
> +    /* No priority grouping */
> +    WRITE_SYSREG32(0, ICC_BPR1_EL1);
> +
> +    /* Set priority mask register */
> +    WRITE_SYSREG32(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
> +
> +    /* EOI drops priority too (mode 0) */
> +    WRITE_SYSREG32(GICC_CTLR_EL1_EOImode_drop, ICC_CTLR_EL1);
> +
> +    /* Enable Group1 interrupts */
> +    WRITE_SYSREG32(1, ICC_IGRPEN1_EL1);
> +
> +    return 0;
> +}
> +
> +static void gicv3_cpu_disable(void)
> +{
> +    WRITE_SYSREG32(0, ICC_CTLR_EL1);
> +}
> +
> +static void __cpuinit gicv3_hyp_init(void)
> +{
> +    uint32_t vtr;
> +
> +    vtr = READ_SYSREG32(ICH_VTR_EL2);
> +    gicv3_info.nr_lrs  = (vtr & GICH_VTR_NRLRGS) + 1;
> +    gicv3.nr_priorities = ((vtr >> GICH_VTR_PRIBITS_SHIFT) &
> +                          GICH_VTR_PRIBITS_MASK) + 1;
> +
> +    ASSERT((gicv3.nr_priorities > 4 && gicv3.nr_priorities < 8));
> +
> +    WRITE_SYSREG32(GICH_VMCR_EOI | GICH_VMCR_VENG1, ICH_VMCR_EL2);
> +    WRITE_SYSREG32(GICH_HCR_EN, ICH_HCR_EL2);
> +
> +    update_cpu_lr_mask();
> +}
> +
> +/* Set up the per-CPU parts of the GIC for a secondary CPU */
> +static int gicv3_secondary_cpu_init(void)
> +{
> +    int res;
> +
> +    spin_lock(&gicv3.lock);
> +
> +    res = gicv3_cpu_init();
> +    gicv3_hyp_init();
> +
> +    spin_unlock(&gicv3.lock);
> +
> +    return res;
> +}
> +
> +static void __cpuinit gicv3_hyp_disable(void)
> +{
> +    uint32_t vtr;
> +
> +    vtr = READ_SYSREG32(ICH_HCR_EL2);
> +    vtr &= ~0x1;
> +    WRITE_SYSREG32( vtr, ICH_HCR_EL2);
> +}
> +
> +static u16 gicv3_compute_target_list(int *base_cpu, const struct cpumask *mask,
> +                                     uint64_t cluster_id)
> +{
> +    int cpu = *base_cpu;
> +    uint64_t mpidr = cpu_logical_map(cpu);
> +    u16 tlist = 0;
> +
> +    while ( cpu < nr_cpu_ids )
> +    {
> +        /*
> +         * If we ever get a cluster of more than 16 CPUs, just
> +         * scream and skip that CPU.
> +         */
> +        if ( (mpidr & 0xff) >= 16 )
> +        {
> +            dprintk(XENLOG_WARNING, "GICv3:Cluster with more than 16's cpus\n");
> +            goto out;
> +        }
> +        tlist |= 1 << (mpidr & 0xf);
> +
> +        cpu = cpumask_next(cpu, mask);
> +        if ( cpu == nr_cpu_ids )
> +        {
> +            cpu--;
> +            goto out;
> +        }
> +
> +        mpidr = cpu_logical_map(cpu);
> +        if ( cluster_id != (mpidr & ~0xffUL) ) {
> +            cpu--;
> +            goto out;
> +        }
> +    }
> +out:
> +    *base_cpu = cpu;
> +
> +    return tlist;
> +}
> +
> +static void gicv3_send_sgi(const cpumask_t *cpumask, enum gic_sgi sgi,
> +                           uint8_t mode)
> +{
> +    int cpu = 0;
> +    uint64_t val;
> +
> +    dsb(sy);
> +
> +    for_each_cpu(cpu, cpumask)
> +    {
> +        uint64_t cluster_id = cpu_logical_map(cpu) & ~0xffUL;
> +        u16 tlist;
> +
> +        /* Get targetlist for the cluster to send SGI */
> +        tlist = gicv3_compute_target_list(&cpu, cpumask, cluster_id);
> +
> +        /*
> +         * Prepare affinity path of the cluster for which SGI is generated
> +         * along with SGI number
> +         */
> +        val = (MPIDR_AFFINITY_LEVEL(cluster_id, 3) << 48  |
> +               MPIDR_AFFINITY_LEVEL(cluster_id, 2) << 32  |
> +               sgi << 24                                  |
> +               MPIDR_AFFINITY_LEVEL(cluster_id, 1) << 16  |
> +               tlist);
> +
> +        WRITE_SYSREG(val, ICC_SGI1R_EL1);
> +    }
> +}
> +
> +/* Shut down the per-CPU GIC interface */
> +static void gicv3_disable_interface(void)
> +{
> +    spin_lock(&gicv3.lock);
> +
> +    gicv3_cpu_disable();
> +    gicv3_hyp_disable();
> +
> +    spin_unlock(&gicv3.lock);
> +}
> +
> +static void gicv3_update_lr(int lr, const struct pending_irq *p,
> +                            unsigned int state)
> +{
> +    uint64_t grp = GICH_LR_GRP1;
> +    uint64_t val = 0;

Could you please add a TODO comment here to remind us that one day we
might need to handle GRP0 as well?


> +    BUG_ON(lr >= gicv3_info.nr_lrs);
> +    BUG_ON(lr < 0);
> +
> +    val =  (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
> +    val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
> +    val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
> +
> +   if ( p->desc != NULL )
> +       val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
> +                           << GICH_LR_PHYSICAL_SHIFT);
> +
> +    gicv3_ich_write_lr(lr, val);
> +}
> +
> +static void gicv3_clear_lr(int lr)
> +{
> +    gicv3_ich_write_lr(lr, 0);
> +}
> +
> +static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
> +{
> +    uint64_t lrv;
> +
> +    lrv = gicv3_ich_read_lr(lr);
> +
> +    lr_reg->pirq = (lrv >> GICH_LR_PHYSICAL_SHIFT) & GICH_LR_PHYSICAL_MASK;
> +    lr_reg->virq = (lrv >> GICH_LR_VIRTUAL_SHIFT) & GICH_LR_VIRTUAL_MASK;
> +
> +    lr_reg->priority  = (lrv >> GICH_LR_PRIORITY_SHIFT) & GICH_LR_PRIORITY_MASK;
> +    lr_reg->state     = (lrv >> GICH_LR_STATE_SHIFT) & GICH_LR_STATE_MASK;
> +    lr_reg->hw_status = (lrv >> GICH_LR_HW_SHIFT) & GICH_LR_HW_MASK;
> +    lr_reg->grp       = (lrv >> GICH_LR_GRP_SHIFT) & GICH_LR_GRP_MASK;
> +}
> +
> +static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
> +{
> +    uint64_t lrv = 0;
> +
> +    lrv = ( ((u64)(lr->pirq & GICH_LR_PHYSICAL_MASK) << GICH_LR_PHYSICAL_SHIFT)|
> +        ((u64)(lr->virq & GICH_LR_VIRTUAL_MASK)  << GICH_LR_VIRTUAL_SHIFT) |
> +        ((u64)(lr->priority & GICH_LR_PRIORITY_MASK) << GICH_LR_PRIORITY_SHIFT)|
> +        ((u64)(lr->state & GICH_LR_STATE_MASK) << GICH_LR_STATE_SHIFT) |
> +        ((u64)(lr->hw_status & GICH_LR_HW_MASK) << GICH_LR_HW_SHIFT)  |
> +        ((u64)(lr->grp & GICH_LR_GRP_MASK) << GICH_LR_GRP_SHIFT) );
> +
> +    gicv3_ich_write_lr(lr_reg, lrv);
> +}
> +
> +static int gicv_v3_init(struct domain *d)
> +{
> +    int i;
> +
> +    /*
> +     * Domain 0 gets the hardware address.
> +     * Guests get the virtual platform layout.
> +     */
> +    if ( d->domain_id == 0 )
> +    {
> +        d->arch.vgic.dbase = gicv3.dbase;
> +        d->arch.vgic.dbase_size = gicv3.dbase_size;
> +        for ( i = 0; i < gicv3.rdist_count; i++ )
> +        {
> +            d->arch.vgic.rbase[i] = gicv3.rdist_regions[i].base;
> +            d->arch.vgic.rbase_size[i] = gicv3.rdist_regions[i].size;
> +        }
> +        d->arch.vgic.rdist_stride = gicv3.rdist_stride;
> +        d->arch.vgic.rdist_count = gicv3.rdist_count;
> +    }
> +    else
> +        d->arch.vgic.dbase = GUEST_GICD_BASE;
> +
> +    d->arch.vgic.nr_lines = 0;
> +
> +    return 0;
> +}
> +
> +static void gicv3_hcr_status(uint32_t flag, bool_t status)
> +{
> +    uint32_t hcr;
> +
> +    hcr = READ_SYSREG32(ICH_HCR_EL2);
> +    if ( status )
> +        WRITE_SYSREG32(hcr | flag, ICH_HCR_EL2);
> +    else
> +        WRITE_SYSREG32(hcr & (~flag), ICH_HCR_EL2);
> +}
> +
> +static unsigned int gicv3_read_vmcr_priority(void)
> +{
> +   return ((READ_SYSREG32(ICH_VMCR_EL2) >> GICH_VMCR_PRIORITY_SHIFT) &
> +            GICH_VMCR_PRIORITY_MASK);
> +}
> +
> +static void gicv3_irq_enable(struct irq_desc *desc)
> +{
> +    unsigned long flags;
> +
> +    spin_lock_irqsave(&gicv3.lock, flags);
> +    desc->status &= ~IRQ_DISABLED;
> +    dsb(sy);
> +    /* Enable routing */
> +    gicv3_enable_irq(desc);
> +    spin_unlock_irqrestore(&gicv3.lock, flags);
> +}
> +
> +static void gicv3_irq_disable(struct irq_desc *desc)
> +{
> +    unsigned long flags;
> +
> +    spin_lock_irqsave(&gicv3.lock, flags);
> +    /* Disable routing */
> +    gicv3_disable_irq(desc);
> +    desc->status |= IRQ_DISABLED;
> +    spin_unlock_irqrestore(&gicv3.lock, flags);
> +}
> +
> +static unsigned int gicv3_irq_startup(struct irq_desc *desc)
> +{
> +    gicv3_irq_enable(desc);
> +
> +    return 0;
> +}
> +
> +static void gicv3_irq_shutdown(struct irq_desc *desc)
> +{
> +    gicv3_irq_disable(desc);
> +}
> +
> +static void gicv3_irq_ack(struct irq_desc *desc)
> +{
> +    /* No ACK -- reading IAR has done this for us */
> +}
> +
> +static void gicv3_host_irq_end(struct irq_desc *desc)
> +{
> +    /* Lower the priority */
> +    gicv3_eoi_irq(desc);
> +    /* Deactivate */
> +    gicv3_dir_irq(desc->irq);
> +}
> +
> +static void gicv3_guest_irq_end(struct irq_desc *desc)
> +{
> +    /* Lower the priority of the IRQ */
> +    gicv3_eoi_irq(desc);
> +    /* Deactivation happens in maintenance interrupt / via GICV */
> +}
> +
> +static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
> +{
> +    BUG();
> +}
> +
> +const static hw_irq_controller gicv3_host_irq_type = {
> +    .typename     = "gic-v3",
> +    .startup      = gicv3_irq_startup,
> +    .shutdown     = gicv3_irq_shutdown,
> +    .enable       = gicv3_irq_enable,
> +    .disable      = gicv3_irq_disable,
> +    .ack          = gicv3_irq_ack,
> +    .end          = gicv3_host_irq_end,
> +    .set_affinity = gicv3_irq_set_affinity,
> +};
> +
> +const static hw_irq_controller gicv3_guest_irq_type = {
> +    .typename     = "gic-v3",
> +    .startup      = gicv3_irq_startup,
> +    .shutdown     = gicv3_irq_shutdown,
> +    .enable       = gicv3_irq_enable,
> +    .disable      = gicv3_irq_disable,
> +    .ack          = gicv3_irq_ack,
> +    .end          = gicv3_guest_irq_end,
> +    .set_affinity = gicv3_irq_set_affinity,
> +};
> +
> +const static struct gic_hw_operations gicv3_ops = {
> +    .info                = &gicv3_info,
> +    .save_state          = gicv3_save_state,
> +    .restore_state       = gicv3_restore_state,
> +    .dump_state          = gicv3_dump_state,
> +    .gicv_setup          = gicv_v3_init,
> +    .gic_host_irq_type   = &gicv3_host_irq_type,
> +    .gic_guest_irq_type  = &gicv3_guest_irq_type,
> +    .eoi_irq             = gicv3_eoi_irq,
> +    .deactivate_irq      = gicv3_dir_irq,
> +    .read_irq            = gicv3_read_irq,
> +    .set_irq_properties  = gicv3_set_irq_properties,
> +    .send_SGI            = gicv3_send_sgi,
> +    .disable_interface   = gicv3_disable_interface,
> +    .update_lr           = gicv3_update_lr,
> +    .update_hcr_status   = gicv3_hcr_status,
> +    .clear_lr            = gicv3_clear_lr,
> +    .read_lr             = gicv3_read_lr,
> +    .write_lr            = gicv3_write_lr,
> +    .read_vmcr_priority  = gicv3_read_vmcr_priority,
> +    .secondary_init      = gicv3_secondary_cpu_init,
> +};
> +
> +/* Set up the GIC */
> +static int __init gicv3_init(struct dt_device_node *node, const void *data)
> +{
> +    struct rdist_region *rdist_regs;
> +    int res, i;
> +    uint32_t reg;
> +
> +    dt_device_set_used_by(node, DOMID_XEN);
> +
> +    res = dt_device_get_address(node, 0, &gicv3.dbase, &gicv3.dbase_size);
> +    if ( res || !gicv3.dbase )
> +        panic("GICv3: Cannot find a valid distributor address");
> +
> +    if ( (gicv3.dbase & ~PAGE_MASK) || (gicv3.dbase_size & ~PAGE_MASK) )
> +        panic("GICv3:  Found unaligned distributor address %"PRIpaddr"",
> +              gicv3.dbase);
> +
> +    gicv3.map_dbase = ioremap_nocache(gicv3.dbase, gicv3.dbase_size);
> +    if ( !gicv3.map_dbase )
> +        panic("GICv3: Failed to ioremap for GIC distributor\n");
> +
> +    reg = readl_relaxed(GICD + GICD_PIDR2) & GICD_PIDR2_ARCH_MASK;
> +    if ( reg  != (GICV3_GICD_PIDR2 & GICD_PIDR2_ARCH_MASK) )
> +         panic("GICv3: no distributor detected\n");
> +
> +    if ( !dt_property_read_u32(node, "#redistributor-regions",
> +                &gicv3.rdist_count) )
> +        gicv3.rdist_count = 1;
> +
> +    if ( gicv3.rdist_count > MAX_RDIST_COUNT )
> +        panic("GICv3: Number of redistributor regions is more than \
> +               %d (Increase MAX_RDIST_COUNT!!)\n", MAX_RDIST_COUNT);
> +
> +    rdist_regs = xzalloc_array(struct rdist_region, gicv3.rdist_count);
> +    if ( !rdist_regs )
> +        panic("GICv3: Failed to allocate memory for rdist regions\n");
> +
> +    for ( i = 0; i < gicv3.rdist_count; i++ )
> +    {
> +        uint64_t rdist_base, rdist_size;
> +
> +        res = dt_device_get_address(node, 1 + i, &rdist_base, &rdist_size);
> +        if ( res || !rdist_base )
> +            panic("GICv3: No rdist base found for region %d\n", i);
> +
> +        rdist_regs[i].base = rdist_base;
> +        rdist_regs[i].size = rdist_size;
> +    }
> +
> +    /* If stride is not set in dt. Set default to 2 * SZ_64K */
> +    if ( !dt_property_read_u32(node, "redistributor-stride", &gicv3.rdist_stride) )
> +        gicv3.rdist_stride = 2 * SZ_64K;
> +
> +    gicv3.rdist_regions= rdist_regs;
> +
> +    res = platform_get_irq(node, 0);
> +    if ( res < 0 )
> +        panic("GICv3: Cannot find the maintenance IRQ");
> +    gicv3_info.maintenance_irq = res;
> +
> +    /* Set the GIC as the primary interrupt controller */
> +    dt_interrupt_controller = node;
> +
> +    for ( i = 0; i < gicv3.rdist_count; i++ )
> +    {
> +        /* map dbase & rdist regions */
> +        gicv3.rdist_regions[i].map_base =
> +                ioremap_nocache(gicv3.rdist_regions[i].base,
> +                                gicv3.rdist_regions[i].size);
> +
> +        if ( !gicv3.rdist_regions[i].map_base )
> +            panic("GICv3: Failed to ioremap rdist region for region %d\n", i);
> +    }
> +
> +    printk("GICv3 initialization:\n"
> +           "      gic_dist_addr=%"PRIpaddr"\n"
> +           "      gic_dist_size=%"PRIpaddr"\n"
> +           "      gic_dist_mapaddr=%p\n"
> +           "      gic_rdist_regions=%d\n"
> +           "      gic_rdist_stride=%x\n"
> +           "      gic_rdist_base=%"PRIpaddr"\n"
> +           "      gic_rdist_base_size=%"PRIpaddr"\n"
> +           "      gic_rdist_base_mapaddr=%p\n"
> +           "      gic_maintenance_irq=%u\n",
> +           gicv3.dbase, gicv3.dbase_size, gicv3.map_dbase, gicv3.rdist_count,
> +           gicv3.rdist_stride, gicv3.rdist_regions[0].base,
> +           gicv3.rdist_regions[0].size, gicv3.rdist_regions[0].map_base,
> +           gicv3_info.maintenance_irq);
> +
> +    spin_lock_init(&gicv3.lock);
> +
> +    spin_lock(&gicv3.lock);
> +
> +    gicv3_dist_init();
> +    res = gicv3_cpu_init();
> +    gicv3_hyp_init();
> +
> +    gicv3_info.hw_version = GIC_V3;
> +    /* Register hw ops*/
> +    register_gic_ops(&gicv3_ops);
> +
> +    spin_unlock(&gicv3.lock);
> +
> +    return res;
> +}
> +
> +static const char * const gicv3_dt_compat[] __initconst =
> +{
> +    DT_MATCH_GIC_V3_STRING1,
> +    NULL
> +};
> +
> +DT_DEVICE_START(gicv3, "GICv3", DEVICE_GIC)
> +        .compatible = gicv3_dt_compat,
> +        .init = gicv3_init,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 8a00aa1..66e246e 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -154,6 +154,14 @@ struct arch_domain
>          /* Base address for guest GIC */
>          paddr_t dbase; /* Distributor base address */
>          paddr_t cbase; /* CPU base address */
> +#ifdef CONFIG_ARM_64
> +        /* GIC V3 addressing */
> +        paddr_t dbase_size; /* Distributor base size */
> +        paddr_t rbase[MAX_RDIST_COUNT];      /* Re-Distributor base address */
> +        paddr_t rbase_size[MAX_RDIST_COUNT]; /* Re-Distributor size */
> +        uint32_t rdist_stride;               /* Re-Distributor stride */
> +        int rdist_count;                     /* No. of Re-Distributors */
> +#endif
>      } vgic;
>  
>      struct vuart {
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 2aadcb6..b40beb5 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -18,6 +18,10 @@
>  #ifndef __ASM_ARM_GIC_H__
>  #define __ASM_ARM_GIC_H__
>  
> +#define NR_GIC_LOCAL_IRQS  NR_LOCAL_IRQS
> +#define NR_GIC_SGI         16
> +#define MAX_RDIST_COUNT    4
> +
>  #define GICD_CTLR       (0x000)
>  #define GICD_TYPER      (0x004)
>  #define GICD_IIDR       (0x008)
> @@ -150,6 +154,20 @@
>  #define DT_MATCH_GIC_V2 DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V2_STRING_1), \
>                          DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V2_STRING_2)
>  
> +#define DT_MATCH_GIC_V3_STRING1      "arm,gic-v3"
> +
> +#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE(DT_MATCH_GIC_V3_STRING1);
> +
> +/*
> + * GICv3 registers that needs to be saved/restored
> + */
> +struct gic_v3 {
> +    uint32_t hcr, vmcr;
> +    uint32_t apr0[4];
> +    uint32_t apr1[4];
> +    uint64_t lr[16];
> +};
> +
>  /*
>   * GICv2 register that needs to be saved/restored
>   * on VCPU context switch
> @@ -166,6 +184,7 @@ struct gic_v2 {
>   */ 
>  union gic_state_data {
>      struct gic_v2 v2;
> +    struct gic_v3 v3;
>  };
>  
>  /*
> @@ -251,6 +270,7 @@ void gic_clear_lrs(struct vcpu *v);
>  
>  enum gic_version {
>      GIC_V2 = 2,
> +    GIC_V3 = 3,
>  };
>  
>  struct gic_info {
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> new file mode 100644
> index 0000000..6f393aa
> --- /dev/null
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -0,0 +1,157 @@
> +/*
> + * ARM Generic Interrupt Controller v3 definitions
> + *
> + * Vijaya Kumar K <vijaya.kumar@caviumnetworks.com>
> + * Copyright (c) 2014 Cavium Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __ASM_ARM_GIC_V3_DEFS_H__
> +#define __ASM_ARM_GIC_V3_DEFS_H__
> +
> +/*
> + * Additional registers defined in GIC v3.
> + * Common GICD registers are defined in gic.h
> + */
> + 
> +#define GICD_STATUSR                 (0x010)
> +#define GICD_SETSPI_NSR              (0x040)
> +#define GICD_CLRSPI_NSR              (0x048)
> +#define GICD_SETSPI_SR               (0x050)
> +#define GICD_CLRSPI_SR               (0x058)
> +#define GICD_IROUTER                 (0x6000)
> +#define GICD_IROUTER31               (0x60F8)
> +#define GICD_IROUTER32               (0x6100)
> +#define GICD_IROUTERN                (0x7FF8)
> +#define GICD_PIDR0                   (0xFFE0)
> +#define GICD_PIDR1                   (0xFFE4)
> +#define GICD_PIDR2                   (0xFFE8)
> +#define GICD_PIDR3                   (0xFFEC)
> +#define GICD_PIDR4                   (0xFFD0)
> +#define GICD_PIDR5                   (0xFFD4)
> +#define GICD_PIDR7                   (0xFFDC)
> +
> +#define GICC_SRE_EL2_SRE             (1UL << 0)
> +#define GICC_SRE_EL2_DFB             (1UL << 1)
> +#define GICC_SRE_EL2_DIB             (1UL << 2)
> +#define GICC_SRE_EL2_ENEL1           (1UL << 3)
> +
> +#define GICD_CTLR_RWP                (1UL << 31)
> +#define GICD_CTLR_ARE_NS             (1U << 4)
> +#define GICD_CTLR_ENABLE_G1A         (1U << 1)
> +#define GICD_CTLR_ENABLE_G1          (1U << 0)
> +#define GICD_IROUTER_SPI_MODE_ANY    (1UL << 31)
> +
> +#define GICC_CTLR_EL1_EOImode_drop   (1U << 1)
> +
> +#define GICR_WAKER_ProcessorSleep    (1U << 1)
> +#define GICR_WAKER_ChildrenAsleep    (1U << 2)
> +
> +#define GICV3_GICD_PIDR0             (0x92)
> +#define GICV3_GICD_PIDR1             (0xb4)
> +#define GICV3_GICD_PIDR2             (0x3b)
> +#define GICV3_GICD_PIDR4             (0x04)
> +#define GICD_PIDR2_ARCH_MASK         (0xf0)
> +
> +#define GICV3_GICR_PIDR0             (0x93)
> +#define GICV3_GICR_PIDR1             GICV3_GICD_PIDR1
> +#define GICV3_GICR_PIDR2             GICV3_GICD_PIDR2
> +#define GICV3_GICR_PIDR4             GICV3_GICD_PIDR4
> +#define GICR_PIDR2_ARCH_MASK         GICD_PIDR2_ARCH_MASK
> +#define GICR_SYNCR_NOT_BUSY          1
> +/* 
> + * Implementation defined value JEP106?
> + * use physical hw value for now
> + */
> +#define GICV3_GICD_IIDR_VAL          0x34c
> +#define GICV3_GICR_IIDR_VAL          GICV3_GICD_IIDR_VAL
> +
> +#define GICR_CTLR                    (0x0000)
> +#define GICR_IIDR                    (0x0004)
> +#define GICR_TYPER                   (0x0008)
> +#define GICR_STATUSR                 (0x0010)
> +#define GICR_WAKER                   (0x0014)
> +#define GICR_SETLPIR                 (0x0040)
> +#define GICR_CLRLPIR                 (0x0048)
> +#define GICR_PROPBASER               (0x0070)
> +#define GICR_PENDBASER               (0x0078)
> +#define GICR_INVLPIR                 (0x00A0)
> +#define GICR_INVALLR                 (0x00B0)
> +#define GICR_SYNCR                   (0x00C0)
> +#define GICR_MOVLPIR                 (0x100)
> +#define GICR_MOVALLR                 (0x0110)
> +#define GICR_PIDR0                   GICD_PIDR0
> +#define GICR_PIDR1                   GICD_PIDR1
> +#define GICR_PIDR2                   GICD_PIDR2
> +#define GICR_PIDR3                   GICD_PIDR3
> +#define GICR_PIDR4                   GICD_PIDR4
> +#define GICR_PIDR5                   GICD_PIDR5
> +#define GICR_PIDR7                   GICD_PIDR7
> +
> +/* GICR for SGI's & PPI's */
> +
> +#define GICR_IGROUPR0                (0x0080)
> +#define GICR_IGRPMODR0               (0x0F80)
> +#define GICR_ISENABLER0              (0x0100)
> +#define GICR_ICENABLER0              (0x0180)
> +#define GICR_ISPENDR0                (0x0200)
> +#define GICR_ICPENDR0                (0x0280)
> +#define GICR_ISACTIVER0              (0x0300)
> +#define GICR_ICACTIVER0              (0x0380)
> +#define GICR_IPRIORITYR0             (0x0400)
> +#define GICR_IPRIORITYR7             (0x041C)
> +#define GICR_ICFGR0                  (0x0C00)
> +#define GICR_ICFGR1                  (0x0C04)
> +#define GICR_NSACR                   (0x0E00)
> +
> +#define GICR_TYPER_PLPIS             (1U << 0)
> +#define GICR_TYPER_VLPIS             (1U << 1)
> +#define GICR_TYPER_LAST              (1U << 4)
> +
> +#define DEFAULT_PMR_VALUE            0xff
> +
> +#define GICH_VMCR_EOI                (1 << 9)
> +#define GICH_VMCR_VENG1              (1 << 1)
> +
> +#define GICH_LR_VIRTUAL_MASK         0xffff
> +#define GICH_LR_VIRTUAL_SHIFT        0
> +#define GICH_LR_PHYSICAL_MASK        0x3ff
> +#define GICH_LR_PHYSICAL_SHIFT       32
> +#define GICH_LR_STATE_MASK           0x3
> +#define GICH_LR_STATE_SHIFT          62
> +#define GICH_LR_PRIORITY_MASK        0xff
> +#define GICH_LR_PRIORITY_SHIFT       48
> +#define GICH_LR_HW_MASK              0x1
> +#define GICH_LR_HW_SHIFT             61
> +#define GICH_LR_GRP_MASK             0x1
> +#define GICH_LR_GRP_SHIFT            60
> +#define GICH_LR_MAINTENANCE_IRQ      (1UL<<41)
> +#define GICH_LR_GRP1                 (1UL<<60)
> +#define GICH_LR_HW                   (1UL<<61)
> +
> +#define GICH_VTR_NRLRGS              0x3f
> +#define GICH_VTR_PRIBITS_MASK        0x7
> +#define GICH_VTR_PRIBITS_SHIFT       29
> +
> +#define GICH_VMCR_PRIORITY_MASK      0xff
> +#define GICH_VMCR_PRIORITY_SHIFT     24
> +
> +#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
> index 3662749..5978b8a 100644
> --- a/xen/include/asm-arm/processor.h
> +++ b/xen/include/asm-arm/processor.h
> @@ -17,6 +17,20 @@
>  #define MPIDR_HWID_MASK     _AC(0xffffff,U)
>  #define MPIDR_INVALID       (~MPIDR_HWID_MASK)
>  
> +/*
> + * Macros to extract affinity level. picked from kernel
> + */
> +
> +#define MPIDR_LEVEL_BITS_SHIFT  3
> +#define MPIDR_LEVEL_BITS        (1 << MPIDR_LEVEL_BITS_SHIFT)
> +#define MPIDR_LEVEL_MASK        ((1 << MPIDR_LEVEL_BITS) - 1)
> +
> +#define MPIDR_LEVEL_SHIFT(level) \
> +         (((1 << level) >> 1) << MPIDR_LEVEL_BITS_SHIFT)
> +
> +#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> +         ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
> +
>  /* TTBCR Translation Table Base Control Register */
>  #define TTBCR_EAE    _AC(0x80000000,U)
>  #define TTBCR_N_MASK _AC(0x07,U)
> diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
> index 1369b2b..499a798 100644
> --- a/xen/include/xen/lib.h
> +++ b/xen/include/xen/lib.h
> @@ -67,6 +67,8 @@ do {                                                            \
>  
>  #define reserve_bootmem(_p,_l) ((void)0)
>  
> +#define SZ_64K  0x00010000
> +
>  struct domain;
>  
>  void cmdline_parse(const char *cmdline);
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-06-02 17:33   ` Stefano Stabellini
@ 2014-06-03  8:54     ` Ian Campbell
  2014-06-03  9:05       ` Julien Grall
  2014-06-03 10:46       ` Stefano Stabellini
  0 siblings, 2 replies; 78+ messages in thread
From: Ian Campbell @ 2014-06-03  8:54 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: vijay.kilari, Prasun.Kapoor, Vijaya Kumar K, julien.grall,
	xen-devel, marc.zyngier, stefano.stabellini

On Mon, 2014-06-02 at 18:33 +0100, Stefano Stabellini wrote:
> > +static inline void gicv3_restore_lr(int nr_lrs, const struct vcpu *v)
> > +{
> > +    /* Fall through for all the cases */
> > +    switch ( nr_lrs )
> > +    {
> > +    case 16:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[15], ICH_LR15_EL2);
> > +    case 15:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[14], ICH_LR14_EL2);
> > +    case 14:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[13], ICH_LR13_EL2);
> > +    case 13:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[12], ICH_LR12_EL2);
> > +    case 12:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[11], ICH_LR11_EL2);
> > +    case 11:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[10], ICH_LR10_EL2);
> > +    case 10:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[9], ICH_LR9_EL2);
> > +    case 9:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[8], ICH_LR8_EL2);
> > +    case 8:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[7], ICH_LR7_EL2);
> > +    case 7:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[6], ICH_LR6_EL2);
> > +    case 6:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[5], ICH_LR5_EL2);
> > +    case 5:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[4], ICH_LR4_EL2);
> > +    case 4:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[3], ICH_LR3_EL2);
> > +    case 3:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[2], ICH_LR2_EL2);
> > +    case 2:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[1], ICH_LR1_EL2);
> > +    case 1:
> > +        WRITE_SYSREG(v->arch.gic.v3.lr[0], ICH_LR0_EL2);
> > +         break;
> > +    default:
> > +         BUG();
> > +    }
> > +}
> 
> Given that the number of LR registers has grown up quite a bit from
> GICv2, we should optimize this code and only save and restore the
> registers that are actually in used by checking on lr_mask.

I'm not convinced that will be faster, the code above involves a
straight line with no branches, but potentially some unnecessary stores.
Looping or lr_mask involves a loop, a test_bit and most critically a
switch over the lr number (to select the correct sysreg, which is a
static register, not an opcode argument), but has no redundant stores. I
suspect the straight line version will actually be quicker.

But in any case choosing one or the other would require someone to run
the numbers, I'm happy with the variant above in the first instance.

> It would also help performances if we restored them in ascending order.

This might help with memory prefetching, yes, but makes this switch
fall-thru trick harder.

It was suggested that perhaps the lr array should be backwards -- e.g.
ICH_LR0_EL2 is stored in v->arch.gic.v3.lr[15] and ICH_LR15_EL2 is
stored in v->arch.git.v3.lr[0], of course this should be hidden behind
an accessor macro e.g.

#define VCPU_GICV3_LR(v, n) v->arch.git.v3.lrs_descending[MAX_NR_LRS-n-1]

> > +static void gicv3_restore_state(const struct vcpu *v)
> > +{
> > +    gicv3_restore_lr(gicv3_info.nr_lrs, v);
> > +    restore_aprn_regs(&v->arch.gic);
> > +    WRITE_SYSREG32(v->arch.gic.v3.vmcr, ICH_VMCR_EL2);
> > +}
> 
> Shouldn't we restore vmcr first? Before writing to the LRs?

Can you remind me of the dependency?

> Should we save/restore SRE_EL1 too?

We certainly should IMHO.

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-06-03  8:54     ` Ian Campbell
@ 2014-06-03  9:05       ` Julien Grall
  2014-06-03  9:07         ` Ian Campbell
  2014-06-03 10:46       ` Stefano Stabellini
  1 sibling, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-06-03  9:05 UTC (permalink / raw)
  To: Ian Campbell, Stefano Stabellini
  Cc: vijay.kilari, Prasun.Kapoor, Vijaya Kumar K, xen-devel,
	marc.zyngier, stefano.stabellini



On 03/06/14 09:54, Ian Campbell wrote:
> On Mon, 2014-06-02 at 18:33 +0100, Stefano Stabellini wrote:
>>> +static inline void gicv3_restore_lr(int nr_lrs, const struct vcpu *v)
>>> +{
>>> +    /* Fall through for all the cases */
>>> +    switch ( nr_lrs )
>>> +    {
>>> +    case 16:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[15], ICH_LR15_EL2);
>>> +    case 15:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[14], ICH_LR14_EL2);
>>> +    case 14:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[13], ICH_LR13_EL2);
>>> +    case 13:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[12], ICH_LR12_EL2);
>>> +    case 12:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[11], ICH_LR11_EL2);
>>> +    case 11:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[10], ICH_LR10_EL2);
>>> +    case 10:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[9], ICH_LR9_EL2);
>>> +    case 9:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[8], ICH_LR8_EL2);
>>> +    case 8:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[7], ICH_LR7_EL2);
>>> +    case 7:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[6], ICH_LR6_EL2);
>>> +    case 6:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[5], ICH_LR5_EL2);
>>> +    case 5:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[4], ICH_LR4_EL2);
>>> +    case 4:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[3], ICH_LR3_EL2);
>>> +    case 3:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[2], ICH_LR2_EL2);
>>> +    case 2:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[1], ICH_LR1_EL2);
>>> +    case 1:
>>> +        WRITE_SYSREG(v->arch.gic.v3.lr[0], ICH_LR0_EL2);
>>> +         break;
>>> +    default:
>>> +         BUG();
>>> +    }
>>> +}
>>
>> Given that the number of LR registers has grown up quite a bit from
>> GICv2, we should optimize this code and only save and restore the
>> registers that are actually in used by checking on lr_mask.
>
> I'm not convinced that will be faster, the code above involves a
> straight line with no branches, but potentially some unnecessary stores.
> Looping or lr_mask involves a loop, a test_bit and most critically a
> switch over the lr number (to select the correct sysreg, which is a
> static register, not an opcode argument), but has no redundant stores. I
> suspect the straight line version will actually be quicker.

The solution suggested by Stefano doesn't work as it is. We only 
save/restore the LR register marked as used in the lr_mask (which is 
per-VCPU). We may end up with LR from the previous running VCPU because 
Xen will restore only the necessary LRs.

To fix the solution would be to clear the LRs when we save all LRs. But 
I'm not convince it will be faster than the current solution.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-06-03  9:05       ` Julien Grall
@ 2014-06-03  9:07         ` Ian Campbell
  2014-06-03 10:43           ` Stefano Stabellini
  0 siblings, 1 reply; 78+ messages in thread
From: Ian Campbell @ 2014-06-03  9:07 UTC (permalink / raw)
  To: Julien Grall
  Cc: vijay.kilari, Stefano Stabellini, Prasun.Kapoor, Vijaya Kumar K,
	xen-devel, marc.zyngier, stefano.stabellini

On Tue, 2014-06-03 at 10:05 +0100, Julien Grall wrote:
> 
> On 03/06/14 09:54, Ian Campbell wrote:
> > On Mon, 2014-06-02 at 18:33 +0100, Stefano Stabellini wrote:
> >>> +static inline void gicv3_restore_lr(int nr_lrs, const struct vcpu *v)
> >>> +{
> >>> +    /* Fall through for all the cases */
> >>> +    switch ( nr_lrs )
> >>> +    {
> >>> +    case 16:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[15], ICH_LR15_EL2);
> >>> +    case 15:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[14], ICH_LR14_EL2);
> >>> +    case 14:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[13], ICH_LR13_EL2);
> >>> +    case 13:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[12], ICH_LR12_EL2);
> >>> +    case 12:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[11], ICH_LR11_EL2);
> >>> +    case 11:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[10], ICH_LR10_EL2);
> >>> +    case 10:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[9], ICH_LR9_EL2);
> >>> +    case 9:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[8], ICH_LR8_EL2);
> >>> +    case 8:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[7], ICH_LR7_EL2);
> >>> +    case 7:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[6], ICH_LR6_EL2);
> >>> +    case 6:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[5], ICH_LR5_EL2);
> >>> +    case 5:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[4], ICH_LR4_EL2);
> >>> +    case 4:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[3], ICH_LR3_EL2);
> >>> +    case 3:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[2], ICH_LR2_EL2);
> >>> +    case 2:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[1], ICH_LR1_EL2);
> >>> +    case 1:
> >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[0], ICH_LR0_EL2);
> >>> +         break;
> >>> +    default:
> >>> +         BUG();
> >>> +    }
> >>> +}
> >>
> >> Given that the number of LR registers has grown up quite a bit from
> >> GICv2, we should optimize this code and only save and restore the
> >> registers that are actually in used by checking on lr_mask.
> >
> > I'm not convinced that will be faster, the code above involves a
> > straight line with no branches, but potentially some unnecessary stores.
> > Looping or lr_mask involves a loop, a test_bit and most critically a
> > switch over the lr number (to select the correct sysreg, which is a
> > static register, not an opcode argument), but has no redundant stores. I
> > suspect the straight line version will actually be quicker.
> 
> The solution suggested by Stefano doesn't work as it is. We only 
> save/restore the LR register marked as used in the lr_mask (which is 
> per-VCPU). We may end up with LR from the previous running VCPU because 
> Xen will restore only the necessary LRs.
> 
> To fix the solution would be to clear the LRs when we save all LRs. But 
> I'm not convince it will be faster than the current solution.

yes, that's a second nail in that idea's coffin I think.

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-06-03  9:07         ` Ian Campbell
@ 2014-06-03 10:43           ` Stefano Stabellini
  0 siblings, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-06-03 10:43 UTC (permalink / raw)
  To: Ian Campbell
  Cc: vijay.kilari, Stefano Stabellini, Prasun.Kapoor, Vijaya Kumar K,
	Julien Grall, xen-devel, marc.zyngier, stefano.stabellini

On Tue, 3 Jun 2014, Ian Campbell wrote:
> On Tue, 2014-06-03 at 10:05 +0100, Julien Grall wrote:
> > 
> > On 03/06/14 09:54, Ian Campbell wrote:
> > > On Mon, 2014-06-02 at 18:33 +0100, Stefano Stabellini wrote:
> > >>> +static inline void gicv3_restore_lr(int nr_lrs, const struct vcpu *v)
> > >>> +{
> > >>> +    /* Fall through for all the cases */
> > >>> +    switch ( nr_lrs )
> > >>> +    {
> > >>> +    case 16:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[15], ICH_LR15_EL2);
> > >>> +    case 15:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[14], ICH_LR14_EL2);
> > >>> +    case 14:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[13], ICH_LR13_EL2);
> > >>> +    case 13:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[12], ICH_LR12_EL2);
> > >>> +    case 12:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[11], ICH_LR11_EL2);
> > >>> +    case 11:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[10], ICH_LR10_EL2);
> > >>> +    case 10:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[9], ICH_LR9_EL2);
> > >>> +    case 9:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[8], ICH_LR8_EL2);
> > >>> +    case 8:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[7], ICH_LR7_EL2);
> > >>> +    case 7:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[6], ICH_LR6_EL2);
> > >>> +    case 6:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[5], ICH_LR5_EL2);
> > >>> +    case 5:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[4], ICH_LR4_EL2);
> > >>> +    case 4:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[3], ICH_LR3_EL2);
> > >>> +    case 3:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[2], ICH_LR2_EL2);
> > >>> +    case 2:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[1], ICH_LR1_EL2);
> > >>> +    case 1:
> > >>> +        WRITE_SYSREG(v->arch.gic.v3.lr[0], ICH_LR0_EL2);
> > >>> +         break;
> > >>> +    default:
> > >>> +         BUG();
> > >>> +    }
> > >>> +}
> > >>
> > >> Given that the number of LR registers has grown up quite a bit from
> > >> GICv2, we should optimize this code and only save and restore the
> > >> registers that are actually in used by checking on lr_mask.
> > >
> > > I'm not convinced that will be faster, the code above involves a
> > > straight line with no branches, but potentially some unnecessary stores.
> > > Looping or lr_mask involves a loop, a test_bit and most critically a
> > > switch over the lr number (to select the correct sysreg, which is a
> > > static register, not an opcode argument), but has no redundant stores. I
> > > suspect the straight line version will actually be quicker.
> > 
> > The solution suggested by Stefano doesn't work as it is. We only 
> > save/restore the LR register marked as used in the lr_mask (which is 
> > per-VCPU). We may end up with LR from the previous running VCPU because 
> > Xen will restore only the necessary LRs.
> > 
> > To fix the solution would be to clear the LRs when we save all LRs. But 
> > I'm not convince it will be faster than the current solution.
> 
> yes, that's a second nail in that idea's coffin I think.
 
I agree that we would need to run a few benchmarks to be sure and I
agree that this is OK in first instance.

However keep in mind that last time I checked on GICv2 reading or
writing to one LR costs about 120-140 nanoseconds. I dropped the idea of
using lr_mask on GICv2 because there are usually only very few LRs (4),
so it wouldn't make sense there but I bet it makes sense here.

16*130 ns = 2080 ns

That would be about 4000 cycles on a 2Ghz processor. We can certainly
loop through a bitmask in far less than that.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 13/16] xen/arm: Add support for GIC v3
  2014-06-03  8:54     ` Ian Campbell
  2014-06-03  9:05       ` Julien Grall
@ 2014-06-03 10:46       ` Stefano Stabellini
  1 sibling, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-06-03 10:46 UTC (permalink / raw)
  To: Ian Campbell
  Cc: vijay.kilari, Stefano Stabellini, Prasun.Kapoor, Vijaya Kumar K,
	julien.grall, xen-devel, marc.zyngier, stefano.stabellini

On Tue, 3 Jun 2014, Ian Campbell wrote:
> > > +static void gicv3_restore_state(const struct vcpu *v)
> > > +{
> > > +    gicv3_restore_lr(gicv3_info.nr_lrs, v);
> > > +    restore_aprn_regs(&v->arch.gic);
> > > +    WRITE_SYSREG32(v->arch.gic.v3.vmcr, ICH_VMCR_EL2);
> > > +}
> > 
> > Shouldn't we restore vmcr first? Before writing to the LRs?
> 
> Can you remind me of the dependency?

The ICH_VMCR_EL2 settings affect the LRs. For example VEOIM (bit 9),
VFIQEn (bit 3), VAckCtl (bit 2), VENG1 (bit 1) and VENG0 (bit 0).

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 03/16] xen/arm: make sgi handling generic
  2014-05-28 14:10   ` Stefano Stabellini
@ 2014-06-09  9:58     ` Vijay Kilari
  0 siblings, 0 replies; 78+ messages in thread
From: Vijay Kilari @ 2014-06-09  9:58 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Ian Campbell, Prasun Kapoor, Vijaya Kumar K, Julien Grall,
	xen-devel, Stefano Stabellini

On Wed, May 28, 2014 at 7:40 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> move all the hw specific sgi handling functionality
>> to one function and use it.
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>  void send_SGI_one(unsigned int cpu, enum gic_sgi sgi)
>> @@ -512,18 +531,18 @@ void send_SGI_self(enum gic_sgi sgi)
>>
>>      dsb(sy);
>>
>> -    GICD[GICD_SGIR] = GICD_SGI_TARGET_SELF
>> -        | sgi;
>> +    send_SGI(cpumask_of(smp_processor_id()), sgi, SGI_TARGET_SELF);
>>  }
>
> Given that the mask is unused in this case, I think it would be
> acceptable to pass NULL.

In GICv2 the mask is not used in some cases. However mask value is used
in GICv3. To make it generic mask value is always computed.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers
  2014-05-28 14:26   ` Stefano Stabellini
@ 2014-06-09 10:29     ` Vijay Kilari
  0 siblings, 0 replies; 78+ messages in thread
From: Vijay Kilari @ 2014-06-09 10:29 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Ian Campbell, Prasun Kapoor, Vijaya Kumar K, Julien Grall,
	xen-devel, Stefano Stabellini

On Wed, May 28, 2014 at 7:56 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>>  #include <asm/mmio.h>
>>  #include <asm/gic.h>
>>
>> -#define REG(n) (n/4)
>> +#define REG(n) (n)
>
> Is this macro actually needed still?
> We might as well get rid of it.

This is remove in another relevant patch

>>  /*
>>   * Returns rank corresponding to a GICD_<FOO><n> register for
>> @@ -63,7 +63,9 @@ static inline int REG_RANK_NR(int b, uint32_t n)
>>   */
>>  static struct vgic_irq_rank *vgic_irq_rank(struct vcpu *v, int b, int n)
>>  {
>> -    int rank = REG_RANK_NR(b, n);
>> +    int rank;
>> +    n = n >> 2;
>> +    rank = REG_RANK_NR(b, n);
>
> I would be consistent and change REG_RANK_NR instead of vgic_irq_rank.

 REG_RANK_NR is generic, instead parameter is changed

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 10/16] xen/arm: move vgic defines to vgic header file
  2014-05-27 11:49   ` Julien Grall
@ 2014-06-10  8:30     ` Vijay Kilari
  0 siblings, 0 replies; 78+ messages in thread
From: Vijay Kilari @ 2014-06-10  8:30 UTC (permalink / raw)
  To: Julien Grall
  Cc: Keir (Xen.org),
	Ian Campbell, Stefano Stabellini, Prasun Kapoor, Tim Deegan,
	Vijaya Kumar K, Ian Jackson, xen-devel, Stefano Stabellini,
	Jan Beulich

Hi Julien,

On Tue, May 27, 2014 at 5:19 PM, Julien Grall <julien.grall@linaro.org> wrote:
> Hi Vijay,
>
> On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>> index f4c467a..2aadcb6 100644
>> --- a/xen/include/asm-arm/gic.h
>> +++ b/xen/include/asm-arm/gic.h
>> @@ -185,13 +185,6 @@ struct gic_lr {
>>
>>  extern int gic_hw_version(void);
>>
>> -extern int domain_vgic_init(struct domain *d);
>> -extern void domain_vgic_free(struct domain *d);
>> -
>> -extern int vcpu_vgic_init(struct vcpu *v);
>> -
>> -extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq);
>> -extern void vgic_clear_pending_irqs(struct vcpu *v);
>>  extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
>
> irq_to_pending is also a VGIC function. Can you move it to vgic.h?

   OK. I missed it. I can move it.
   But when I  move this to vgic.h file, there are compilation errors.
   The issue is pending_irq structure is defined in asm/domain.h file
   so domain.h should be included in gic.h. However domain.h needs gic.h
   for gic_state definition so domain.h depends on gic.h and gic.h depends on
   domain.h.

   I propose to move pending_irq structure to gic.h or vgic.h from domain.h

Regards
Vijay

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-06-02 15:50   ` Stefano Stabellini
@ 2014-06-11 11:36     ` Vijay Kilari
  2014-06-11 12:44       ` Stefano Stabellini
  0 siblings, 1 reply; 78+ messages in thread
From: Vijay Kilari @ 2014-06-11 11:36 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Ian Campbell, Prasun Kapoor, Vijaya Kumar K, Julien Grall,
	xen-devel, Stefano Stabellini

Hi Stefano,

On Mon, Jun 2, 2014 at 9:20 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
>> +    case GICD_IROUTER ... GICD_IROUTER31:
>> +        /* SGI/PPI is RES0 */
>> +        goto write_ignore_64;
>> +    case GICD_IROUTER32 ... GICD_IROUTERN:
>> +        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
>> +        rank = vgic_irq_rank(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
>> +        if ( rank == NULL) goto write_ignore;
>> +        vgic_lock_rank(v, rank);
>> +        rank->v3.irouter[REG_RANK_INDEX(64,
>> +                      (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)] = *r;
>> +        vgic_unlock_rank(v, rank);
>> +        return 1;
>
> Similarly to
>
> http://marc.info/?l=xen-devel&m=140104126710713

Shall we handle this patch separately on top of this series instead of
including this changes in the first version of vgic-v3.c?

>
> We should ignore any writes to GICD_IROUTER*.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-02 16:13     ` Ian Campbell
@ 2014-06-11 12:34       ` Vijay Kilari
  0 siblings, 0 replies; 78+ messages in thread
From: Vijay Kilari @ 2014-06-11 12:34 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Julien Grall,
	xen-devel, Stefano Stabellini

On Mon, Jun 2, 2014 at 9:43 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2014-06-02 at 17:05 +0100, Stefano Stabellini wrote:
>> > +static int vgicv3_to_sgi(struct vcpu *v, register_t sgir)
>> > +{
>> > +    int virq;
>> > +    int irqmode;
>> > +    unsigned long vcpu_mask = 0;
>> > +
>> > +    irqmode = (sgir >> ICH_SGI_IRQMODE_SHIFT) & ICH_SGI_IRQMODE_MASK;
>> > +    virq = (sgir >> ICH_SGI_IRQ_SHIFT ) & ICH_SGI_IRQ_MASK;
>> > +    vcpu_mask = sgir & ICH_SGI_TARGETLIST_MASK;
>> > +
>> > +    return vgic_to_sgi(v, sgir, irqmode, virq, vcpu_mask);
>> > +}
>>
>> Given that we are not handling affinity 1, 2 and 3 settings, we should
>> add an assert to make sure that they are 0.
>
> Not literally an assert since that would give guests a trivial way to
> DoS the system....

IMHO, Since we are ignoring AFF{1,2,3} instead of assert, a warning
should be enough

>
> Given that we create guest VCPUs with MPIDR's which do not use
> AFF{1,2,3} I think we can simply ignore any attempt to SGI those
> processors, since they can't exist.
>
> Of course this will mean implementing a 16 VCPU limit somewhere on the
> creation side, so we actually do not create VCPUs with higher AFF's
> used...

Also from the spec it says that invalid AFF's will be discarded

"Note: if software specifies Affinity 3 (if appropriate), Affinity 2
and Affinity 1 values that do not correspond to a
valid set of target processors the Distributor must acknowledge and
discard the Generate SGI packet. It may
optionally generate an SEI."

>
> There should of course be a suitable comment both here and there...
>
> Ian.
>

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-02 16:17   ` Julien Grall
@ 2014-06-11 12:35     ` Vijay Kilari
  2014-06-11 12:38       ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Vijay Kilari @ 2014-06-11 12:35 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On Mon, Jun 2, 2014 at 9:47 PM, Julien Grall <julien.grall@linaro.org> wrote:
> Hi Vijay,
>
> On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index d80683d..99d0d46 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>
> [..]
>
>> +int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
>> +{
>
> You are by-passing without any reason the vgic structure. Why didn't you
> add a new callback there?

Sorry, I could not get you. Can you please be more clear?

>
> Regards,
>
> --
> Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-11 12:35     ` Vijay Kilari
@ 2014-06-11 12:38       ` Julien Grall
  2014-06-12  6:53         ` Vijay Kilari
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-06-11 12:38 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On 06/11/2014 01:35 PM, Vijay Kilari wrote:
> On Mon, Jun 2, 2014 at 9:47 PM, Julien Grall <julien.grall@linaro.org> wrote:
>> Hi Vijay,
>>
>> On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
>>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>>> index d80683d..99d0d46 100644
>>> --- a/xen/arch/arm/vgic-v3.c
>>> +++ b/xen/arch/arm/vgic-v3.c
>>
>> [..]
>>
>>> +int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
>>> +{
>>
>> You are by-passing without any reason the vgic structure. Why didn't you
>> add a new callback there?
> 
> Sorry, I could not get you. Can you please be more clear?

Why didn't you add a callback in the vgic structure?

The vgic structure is per-domain, so it's perfectly valid (even though
it's not you use case) to run a GICv2 guest and a GICv3 guest at the
same time.

On the former, you don't want to let the guest send an SGI via this
solution.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 14/16] xen/arm: Add virtual GICv3 support
  2014-06-11 11:36     ` Vijay Kilari
@ 2014-06-11 12:44       ` Stefano Stabellini
  0 siblings, 0 replies; 78+ messages in thread
From: Stefano Stabellini @ 2014-06-11 12:44 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	Julien Grall, xen-devel, Stefano Stabellini

On Wed, 11 Jun 2014, Vijay Kilari wrote:
> Hi Stefano,
> 
> On Mon, Jun 2, 2014 at 9:20 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Mon, 26 May 2014, vijay.kilari@gmail.com wrote:
> >> +    case GICD_IROUTER ... GICD_IROUTER31:
> >> +        /* SGI/PPI is RES0 */
> >> +        goto write_ignore_64;
> >> +    case GICD_IROUTER32 ... GICD_IROUTERN:
> >> +        if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> >> +        rank = vgic_irq_rank(v, 64, gicd_reg - GICD_IROUTER, DABT_DOUBLE_WORD);
> >> +        if ( rank == NULL) goto write_ignore;
> >> +        vgic_lock_rank(v, rank);
> >> +        rank->v3.irouter[REG_RANK_INDEX(64,
> >> +                      (gicd_reg - GICD_IROUTER), DABT_DOUBLE_WORD)] = *r;
> >> +        vgic_unlock_rank(v, rank);
> >> +        return 1;
> >
> > Similarly to
> >
> > http://marc.info/?l=xen-devel&m=140104126710713
> 
> Shall we handle this patch separately on top of this series instead of
> including this changes in the first version of vgic-v3.c?

I am fine with not having a complete implementation of GICD_IROUTER* in
the first version of the series.

In that case you might want to write_ignore any writes to GICD_IROUTER32
... GICD_IROUTERN.


> > We should ignore any writes to GICD_IROUTER*.
> 

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-11 12:38       ` Julien Grall
@ 2014-06-12  6:53         ` Vijay Kilari
  2014-06-12 21:56           ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Vijay Kilari @ 2014-06-12  6:53 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On Wed, Jun 11, 2014 at 6:08 PM, Julien Grall <julien.grall@linaro.org> wrote:
> On 06/11/2014 01:35 PM, Vijay Kilari wrote:
>> On Mon, Jun 2, 2014 at 9:47 PM, Julien Grall <julien.grall@linaro.org> wrote:
>>> Hi Vijay,
>>>
>>> On 05/26/2014 11:26 AM, vijay.kilari@gmail.com wrote:
>>>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>>>> index d80683d..99d0d46 100644
>>>> --- a/xen/arch/arm/vgic-v3.c
>>>> +++ b/xen/arch/arm/vgic-v3.c
>>>
>>> [..]
>>>
>>>> +int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
>>>> +{
>>>
>>> You are by-passing without any reason the vgic structure. Why didn't you
>>> add a new callback there?
>>
>> Sorry, I could not get you. Can you please be more clear?
>
> Why didn't you add a callback in the vgic structure?
>
> The vgic structure is per-domain, so it's perfectly valid (even though
> it's not you use case) to run a GICv2 guest and a GICv3 guest at the
> same time.
>
   In GICv3 case the sending SGI by guest raises sysreg trap where
as in GICv2 it raises mmio write trap. So these traps lands in respective
vgic driver. ( mmio write trap => vgic-v2.c and sysreg => vgic-v3.c)
These vgic-v{2,3}.c driver calls generic vgic driver to inject SGI to VCPU.

If I understand correctly, you mean creating callback in vgic, which is
common function in vgic driver and from there it should call
respective vgic-v{2,3}.c driver.

> On the former, you don't want to let the guest send an SGI via this
> solution.
>
> Regards,
>
> --
> Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-12  6:53         ` Vijay Kilari
@ 2014-06-12 21:56           ` Julien Grall
  2014-06-13  8:34             ` Ian Campbell
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-06-12 21:56 UTC (permalink / raw)
  To: Vijay Kilari
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini



On 12/06/14 07:53, Vijay Kilari wrote:
>     In GICv3 case the sending SGI by guest raises sysreg trap where
> as in GICv2 it raises mmio write trap. So these traps lands in respective
> vgic driver. ( mmio write trap => vgic-v2.c and sysreg => vgic-v3.c)
> These vgic-v{2,3}.c driver calls generic vgic driver to inject SGI to VCPU.

What does prevent a malicious buggy gicv2 guest too call the sysreg 
version? AFAIU nothing, so you will end up to interpret GICv2 data as 
GICv3 data which will result to a possible security issue.

> If I understand correctly, you mean creating callback in vgic, which is
> common function in vgic driver and from there it should call
> respective vgic-v{2,3}.c driver.

Yes, I see that on your new version you didn't use a callback. Please 
use it.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-12 21:56           ` Julien Grall
@ 2014-06-13  8:34             ` Ian Campbell
  2014-06-15 18:44               ` Julien Grall
  0 siblings, 1 reply; 78+ messages in thread
From: Ian Campbell @ 2014-06-13  8:34 UTC (permalink / raw)
  To: Julien Grall
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On Thu, 2014-06-12 at 22:56 +0100, Julien Grall wrote:
> 
> On 12/06/14 07:53, Vijay Kilari wrote:
> >     In GICv3 case the sending SGI by guest raises sysreg trap where
> > as in GICv2 it raises mmio write trap. So these traps lands in respective
> > vgic driver. ( mmio write trap => vgic-v2.c and sysreg => vgic-v3.c)
> > These vgic-v{2,3}.c driver calls generic vgic driver to inject SGI to VCPU.
> 
> What does prevent a malicious buggy gicv2 guest too call the sysreg 
> version? AFAIU nothing, so you will end up to interpret GICv2 data as 
> GICv3 data which will result to a possible security issue.

ICC_SRE_EL2.Enable should gate EL1's access to this register. We should
certainly set this to the appropriate value corresponding to what we
expose to the guest. (I haven't looked if this series does not).

Ian.

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-13  8:34             ` Ian Campbell
@ 2014-06-15 18:44               ` Julien Grall
  2014-06-20  8:48                 ` Vijay Kilari
  0 siblings, 1 reply; 78+ messages in thread
From: Julien Grall @ 2014-06-15 18:44 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

Hi Ian,

On 13 June 2014 09:34, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Thu, 2014-06-12 at 22:56 +0100, Julien Grall wrote:
>>
>> On 12/06/14 07:53, Vijay Kilari wrote:
>> >     In GICv3 case the sending SGI by guest raises sysreg trap where
>> > as in GICv2 it raises mmio write trap. So these traps lands in respective
>> > vgic driver. ( mmio write trap => vgic-v2.c and sysreg => vgic-v3.c)
>> > These vgic-v{2,3}.c driver calls generic vgic driver to inject SGI to VCPU.
>>
>> What does prevent a malicious buggy gicv2 guest too call the sysreg
>> version? AFAIU nothing, so you will end up to interpret GICv2 data as
>> GICv3 data which will result to a possible security issue.
>
> ICC_SRE_EL2.Enable should gate EL1's access to this register. We should
> certainly set this to the appropriate value corresponding to what we
> expose to the guest. (I haven't looked if this series does not).

Thanks for information. I suspect the current GICv3 driver is not
ready to support
GICv2 guest. Actually this bit is not context switch.

I'm not sure if it's useful to be there in this current series.

In any case, I think we should add a callback in the vgic structure rather than
exposing a function from the GICv3 driver in the ARM common code.

It might be useful to support other GIC driver... and break, without
good reason,
the idea to have a common layer for the vgic implementation.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [PATCH v4 16/16] xen/arm: add SGI handling for GICv3
  2014-06-15 18:44               ` Julien Grall
@ 2014-06-20  8:48                 ` Vijay Kilari
  0 siblings, 0 replies; 78+ messages in thread
From: Vijay Kilari @ 2014-06-20  8:48 UTC (permalink / raw)
  To: Julien Grall
  Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
	xen-devel, Stefano Stabellini

On Mon, Jun 16, 2014 at 12:14 AM, Julien Grall <julien.grall@linaro.org> wrote:
> Hi Ian,
>
> On 13 June 2014 09:34, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> On Thu, 2014-06-12 at 22:56 +0100, Julien Grall wrote:
>>>
>>> On 12/06/14 07:53, Vijay Kilari wrote:
>>> >     In GICv3 case the sending SGI by guest raises sysreg trap where
>>> > as in GICv2 it raises mmio write trap. So these traps lands in respective
>>> > vgic driver. ( mmio write trap => vgic-v2.c and sysreg => vgic-v3.c)
>>> > These vgic-v{2,3}.c driver calls generic vgic driver to inject SGI to VCPU.
>>>
>>> What does prevent a malicious buggy gicv2 guest too call the sysreg
>>> version? AFAIU nothing, so you will end up to interpret GICv2 data as
>>> GICv3 data which will result to a possible security issue.
>>
>> ICC_SRE_EL2.Enable should gate EL1's access to this register. We should
>> certainly set this to the appropriate value corresponding to what we
>> expose to the guest. (I haven't looked if this series does not).
>
> Thanks for information. I suspect the current GICv3 driver is not
> ready to support
> GICv2 guest. Actually this bit is not context switch.

As per GICv3 spec 4.8.17 it recommends to save and restore ICC_SRE_EL1
register on VM switch. I will added this in next version

^ permalink raw reply	[flat|nested] 78+ messages in thread

end of thread, other threads:[~2014-06-20  8:48 UTC | newest]

Thread overview: 78+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-05-26 10:26 [PATCH v4 00/16] xen/arm: Add GICv3 support vijay.kilari
2014-05-26 10:26 ` [PATCH v4 01/16] xen/arm: move io.h as mmio.h to include folder vijay.kilari
2014-05-26 11:28   ` Julien Grall
2014-05-28 13:55   ` Stefano Stabellini
2014-05-26 10:26 ` [PATCH v4 02/16] xen/arm: make mmio handlers domain specific vijay.kilari
2014-05-26 12:33   ` Julien Grall
2014-05-28 14:05     ` Stefano Stabellini
2014-05-28 14:11       ` Julien Grall
2014-05-26 10:26 ` [PATCH v4 03/16] xen/arm: make sgi handling generic vijay.kilari
2014-05-26 12:41   ` Julien Grall
2014-05-26 12:45     ` Julien Grall
2014-05-28 14:10   ` Stefano Stabellini
2014-06-09  9:58     ` Vijay Kilari
2014-05-26 10:26 ` [PATCH v4 04/16] xen/arm: remove unused parameter in do_sgi call vijay.kilari
2014-05-26 12:48   ` Julien Grall
2014-05-26 10:26 ` [PATCH v4 05/16] xen/arm: use ioremap to map gic-v2 registers vijay.kilari
2014-05-26 13:10   ` Julien Grall
2014-05-30 12:54     ` Vijay Kilari
2014-05-28 14:26   ` Stefano Stabellini
2014-06-09 10:29     ` Vijay Kilari
2014-05-26 10:26 ` [PATCH v4 06/16] xen/arm: segregate and split GIC low level functionality vijay.kilari
2014-05-26 14:09   ` Julien Grall
2014-05-27 19:13   ` Julien Grall
2014-05-28 14:43   ` Stefano Stabellini
2014-05-26 10:26 ` [PATCH v4 07/16] arm/xen: move GIC context data structure to gic driver vijay.kilari
2014-05-26 14:32   ` Julien Grall
2014-05-28 14:49     ` Stefano Stabellini
2014-05-26 10:26 ` [PATCH v4 08/16] xen/arm: use device api to detect GIC version vijay.kilari
2014-05-26 14:39   ` Julien Grall
2014-05-28 14:52     ` Stefano Stabellini
2014-05-26 10:26 ` [PATCH v4 09/16] xen/arm: move vgic rank data to gic header file vijay.kilari
2014-05-27 11:32   ` Julien Grall
2014-05-28 14:54   ` Stefano Stabellini
2014-05-26 10:26 ` [PATCH v4 10/16] xen/arm: move vgic defines to vgic " vijay.kilari
2014-05-27 11:49   ` Julien Grall
2014-06-10  8:30     ` Vijay Kilari
2014-05-26 10:26 ` [PATCH v4 11/16] xen/arm: calculate vgic irq rank based on register size vijay.kilari
2014-05-27 11:56   ` Julien Grall
2014-05-30  8:59     ` Vijay Kilari
2014-05-30  9:58       ` Julien Grall
2014-05-30 10:24         ` Vijay Kilari
2014-05-30 10:36           ` Julien Grall
2014-05-30 10:51             ` Vijay Kilari
2014-05-30 10:54               ` Julien Grall
2014-05-26 10:26 ` [PATCH v4 12/16] xen/arm: split vgic driver into generic and vgic-v2 driver vijay.kilari
2014-05-27 16:50   ` Julien Grall
2014-05-26 10:26 ` [PATCH v4 13/16] xen/arm: Add support for GIC v3 vijay.kilari
2014-05-27 19:47   ` Julien Grall
2014-06-02 17:33   ` Stefano Stabellini
2014-06-03  8:54     ` Ian Campbell
2014-06-03  9:05       ` Julien Grall
2014-06-03  9:07         ` Ian Campbell
2014-06-03 10:43           ` Stefano Stabellini
2014-06-03 10:46       ` Stefano Stabellini
2014-05-26 10:26 ` [PATCH v4 14/16] xen/arm: Add virtual GICv3 support vijay.kilari
2014-06-02 15:50   ` Stefano Stabellini
2014-06-11 11:36     ` Vijay Kilari
2014-06-11 12:44       ` Stefano Stabellini
2014-06-02 16:10   ` Julien Grall
2014-06-02 16:15     ` Ian Campbell
2014-06-02 16:18       ` Julien Grall
2014-06-02 16:38         ` Ian Campbell
2014-06-02 16:46           ` Julien Grall
2014-05-26 10:26 ` [PATCH v4 15/16] xen/arm: Update Dom0 GIC dt node with GICv3 information vijay.kilari
2014-05-26 10:26 ` [PATCH v4 16/16] xen/arm: add SGI handling for GICv3 vijay.kilari
2014-06-02 16:05   ` Stefano Stabellini
2014-06-02 16:13     ` Ian Campbell
2014-06-11 12:34       ` Vijay Kilari
2014-06-02 16:17   ` Julien Grall
2014-06-11 12:35     ` Vijay Kilari
2014-06-11 12:38       ` Julien Grall
2014-06-12  6:53         ` Vijay Kilari
2014-06-12 21:56           ` Julien Grall
2014-06-13  8:34             ` Ian Campbell
2014-06-15 18:44               ` Julien Grall
2014-06-20  8:48                 ` Vijay Kilari
2014-05-28 10:26 ` [PATCH v4 00/16] xen/arm: Add GICv3 support Ian Campbell
2014-05-28 12:34   ` Ian Campbell

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.