[PATCH v2 00/45] New VGIC(-v2) implementation

[PATCH v2 00/45] New VGIC(-v2) implementation

[Andre Przywara]

tl;dr: Coarse changelog below, individual patches have changelogs as
well. git branch:
http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/vgic-new/v2
git://linux-arm.org/xen-ap.git branch vgic-new/v2

Another update, addressing the review comments. Nothing too outstanding this
time, the most interesting changes include:
- removing the split-out preparatory patches, which are already merged
- changing the setting and clearing of _IRQ_INPROGRESS
- including Julien's LR access rework series
- restricting new level IRQ handling to the new VGIC
- fix multiple SGI handling (mimicing the recent Linux/KVM patch)
- ASSERTing that h/w IRQs stay connected to their virtual IRQs
- directly update h/w affinity, without taking the desc lock
- restrict 8K struct vcpu to new VGIC and ARM64
- use separate Makefile for vgic/ directory
- many minor changes to address whitespace issues and usage of unsigned,
  also extending comments

A summarising changelog can be found below, each individual patch has
its own changelog as well.

There are some things that have (still) not been covered yet:
- struct VCPU still allocates two pages on ARM64 when using the new VGIC now.
We could try to look if we can allocate some parts of struct vcpu instead of
embedding sub-structures into it.
- vGICv3 support is not implemented, but should be fairly straight-forward to
add, as the design incorporated this already. Will look at this next.
- There is a possible DOS vector on the VCPU ap_list, which holds pending
vIRQs. A guest can make this list rather long, which forces the hypervisor
to hold the list lock when iterating the list. This should be bounded by
the number of emulated vIRQs though, and there are ideas how to mitigate
this issue. Those fixes would be posted on top as fixes later.
- There is no ITS support, though the VGIC code itself is more ready for that
than the old VGIC ever was. However due to differences between the Xen
and KVM architecture the ITS bits are not easy to port over to Xen.
- Do we need to call vgic_evtchn_irq_pending() in
local_event_needs_delivery_nomask()? The event channel IRQ should be covered
by the VGIC already.

Cheers,
Andre

=====================
During development of the Dom0 ITS MSI support last year we realised
that the existing GIC interrupt controller emulation has some shortcomings.
After some tries to fix those in the existing code, it was agreed upon
that the problems are fundamental and a new implementation based on the
"new VGIC" in KVM is the best choice.
This is this new VGIC implementation, based on the (heavily modified)
KVM version. It lives in the xen/arch/arm/vgic/ directory and is written
to be a compile time option, so people can choose whether to use the new
VGIC or the existing implementation. This is just for a transitional period,
the old VGIC is expected to be removed after confidence in the new
implementation has grown.

This series starts with some cleanup and refactoring patches for the
existing VGIC/GIC code, this includes preparations to properly support
level triggered interrupts. This is one of the biggest problems in the
existing VGIC, which only correctly emulates edge triggered IRQs. This
affects both arch code and some users like the timer and the event channel.

Starting with patch 14 we plumb in the new VGIC then. This is done in a
new directory, with all the files actually not wired into the build system
until the very last patch. The idea is to split the series into reviewable
chunks without resorting to nasty hacks to keep bisectability.
The code was forked from Linux' virt/kvm/arm/vgic/, as of 4.14-rc7, plus
some recent changes to improve support for level triggered and hardware
mapped interrupts, which is what we use heavily in Dom0. The code was
heavily adapted to fit into Xen, starting with using the Xen coding style
and using Xen structure and variable names (struct domain instead of
struct kvm, for instance). Where interfacing functions were similar enough,
they were changed over to the existing Xen name and prototypes (for instance
kvm_vgic_create() was renamed to domain_vgic_register()). As far as possible
the code layout and split was re-used from KVM, so patches in Linux should
be relatively easy to port into Xen. Due to the mentioned changes this can
not be done easily in an automatic way, but it should be not too complicated
to extract the gist of the patch and re-apply this to our code base.

The actual VGIC code splits into several parts:
- The core is the struct vgic_irq, which holds every information about a
virtual IRQ, including a per-IRQ lock. Also there is on (ordered) per-VCPU
list (ap_list), which links the interrupts to be considered by a VCPU.
There are functions to deal with queuing and removing IRQs from those lists
safely, obeying the locking order. (patches 14-18)
- There are functions to push vIRQs on a VCPU list to the list registers,
and handle their state changes. (patches 19-21)
- The distributor MMIO emulation is using separate functions per register,
also having read and write split. (patches 22-32)
- There are functions to deal with Xen specialities. (patches 33-39)
- The data structures and the wiring of the emulation into the hypervisor
  and the guests are done in vgic-init.c. (patches 40-43)
- Finally patch 45 enables the build of the new VGIC. This requires to
  increase the size limit for struct vcpu in patch 44.

Andre

Changelog v1 ... v2:
- add VCPU parameter to renamed gic_event_needs_delivery()
- use vcpu_kick, using existing x86 prototype
- include Julien's struct gic_lr rework series
- extend setting of _IRQ_INPROGRESS when tweaking active/pending state
- restrict level IRQ device handling to new VGIC
- cleanup vgic.h
- make vgic_inject_irq() and sync_{to,from}_lr() functions return void
- add dropped code to properly handle 
- split off introduction of Linux' list_sort() into separate patch
- fix handling of multiple-source-SGIs, as done in Linux recently
- use KVM IIDR identifier, but use different variant for Xen
- ASSERT that association between hardware and virtual IRQs do not change
- print warning on every IRQ failing to set/clear active bit
- avoid unneeded calls to vgic_sync_hardware_irq(), avoiding desc lock
- fixup wrong number of SPIs (not a multiple of 32)
- move vgic_v2_enable patch around
- confine two 4K pages for struct vcpu to new VGIC and ARM64
- use separate Makefile for new VGIC
- enhance Kconfig help text
- many whitespace and indentation fixes
- using more unsigned ints
- adding and extending comments

Changelog RFC ... v1:
- observe review comments on GICv3 redistributor patches
- implement physical-follows-virtual IRQ affinity
- actually implement arch_move_irq()
- move max_domain_vcpus() into vgic.c, to make it VGIC specific
- improved many commit messages
- add ACKs so far
- added and extended many comments
- use C99 data types (uint32_t)
- use unsigned data types
- use symbolic names for constants
- white space fixes (indentation mostly)
- adapt later patches to changes earlier in the series (renames etc.)
- use 32 bit data types where sufficient
- add helper functions as requested (for instance gicv2/3_peek/poke_irq)
- use struct irq_desc * in interface of hardware facing functions
- rename some existing Xen function names to be more readable
- rename new header file from arm_vgic.h to new_vgic.h
- drop code or variables dealing with unimplemented features (ITS, CPU i/f)
- reorder struct vgic_irq and use bitfield to shrink data structure size
- remove not needed functions (gic_clear_lrs(), save/restore_state())
- add ASSERTS as requested
- add locking where missing (dump_vgic_info, read pending state, enabling GIC)
- keep Linux coding style for list_sort.c
- add set_pending_state() GIC abstraction function
- factor out and use kick_vcpu()
- use frame number instead of physical address
- use existing LR accessor functions, drop GICH_ accesses from vgic-v2.c
- skip already disabled/enabled IRQs and setting enabled state
- use PRODUCT_ID_XEN
- simplify and clarify on ACTIVE bit MMIO accesses
- use interface for HCR bit changes
- iterate over set CPU bits in SGI injection handler

Andre Przywara (39):
  ARM: VGIC: rename gic_event_needs_delivery()
  ARM: Implement vcpu_kick()
  ARM: GIC: Allow tweaking the active and pending state of an IRQ
  ARM: GIC: Allow reading pending state of a hardware IRQ
  ARM: timer: Handle level triggered IRQs correctly
  ARM: evtchn: Handle level triggered IRQs correctly
  ARM: vPL011: Use the VGIC's level triggered IRQs handling if available
  ARM: new VGIC: Add data structure definitions
  ARM: new VGIC: Add acccessor to new struct vgic_irq instance
  ARM: new VGIC: Implement virtual IRQ injection
  Add list_sort() routine from Linux
  ARM: new VGIC: Add IRQ sorting
  ARM: new VGIC: Add IRQ sync/flush framework
  ARM: new VGIC: Add GICv2 world switch backend
  ARM: new VGIC: Implement vgic_vcpu_pending_irq
  ARM: new VGIC: Add MMIO handling framework
  ARM: new VGIC: Add GICv2 MMIO handling framework
  ARM: new VGIC: Add CTLR, TYPER and IIDR handlers
  ARM: new VGIC: Add ENABLE registers handlers
  ARM: new VGIC: Add PENDING registers handlers
  ARM: new VGIC: Add ACTIVE registers handlers
  ARM: new VGIC: Add PRIORITY registers handlers
  ARM: new VGIC: Add CONFIG registers handlers
  ARM: new VGIC: Add TARGET registers handlers
  ARM: new VGIC: Add SGIR register handler
  ARM: new VGIC: Add SGIPENDR register handlers
  ARM: new VGIC: Handle hardware mapped IRQs
  ARM: new VGIC: Add event channel IRQ handling
  ARM: new VGIC: Handle virtual IRQ allocation/reservation
  ARM: new VGIC: Dump virtual IRQ info
  ARM: new VGIC: Provide system register emulation stub
  ARM: new VGIC: Implement arch_move_irqs()
  ARM: new VGIC: Add preliminary stub implementation
  ARM: new VGIC: vgic-init: register VGIC
  ARM: new VGIC: Add vgic_v2_enable
  ARM: new VGIC: vgic-init: implement vgic_init
  ARM: new VGIC: vgic-init: implement map_resources
  ARM: new VGIC: Allocate two pages for struct vcpu
  ARM: VGIC: wire new VGIC(-v2) files into Xen build system

Julien Grall (6):
  xen/arm: gic: Fix indentation in gic_update_one_lr
  xen/arm: vgic: Override the group in lr everytime
  xen/arm: gic: Use bool instead of uint8_t for the hw_status in gic_lr
  xen/arm: gic: Split the field state in gic_lr in 2 fields active and
    pending
  xen/arm: GIC: Only set pirq in the LR when hw_status is set
  ARM: GIC: extend LR read/write functions to cover EOI and source

 xen/arch/arm/Kconfig              |  17 +-
 xen/arch/arm/Makefile             |   5 +-
 xen/arch/arm/domain.c             |  34 ++
 xen/arch/arm/gic-v2.c             | 111 ++++-
 xen/arch/arm/gic-v3.c             | 123 ++++-
 xen/arch/arm/gic-vgic.c           |  24 +-
 xen/arch/arm/time.c               |  36 ++
 xen/arch/arm/traps.c              |  12 +
 xen/arch/arm/vgic.c               |  11 +-
 xen/arch/arm/vgic/Makefile        |   5 +
 xen/arch/arm/vgic/vgic-init.c     | 259 ++++++++++
 xen/arch/arm/vgic/vgic-mmio-v2.c  | 321 +++++++++++++
 xen/arch/arm/vgic/vgic-mmio.c     | 639 ++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h     | 138 ++++++
 xen/arch/arm/vgic/vgic-v2.c       | 301 ++++++++++++
 xen/arch/arm/vgic/vgic.c          | 988 ++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.h          |  79 +++
 xen/arch/arm/vpl011.c             |   4 +
 xen/common/Makefile               |   1 +
 xen/common/list_sort.c            | 157 ++++++
 xen/include/asm-arm/event.h       |   3 +-
 xen/include/asm-arm/gic.h         |  59 ++-
 xen/include/asm-arm/gic_v3_defs.h |   4 +
 xen/include/asm-arm/new_vgic.h    | 198 ++++++++
 xen/include/asm-arm/perfc_defn.h  |   3 +-
 xen/include/asm-arm/vgic.h        |   6 +
 xen/include/xen/list_sort.h       |  11 +
 xen/include/xen/timer.h           |   2 +
 28 files changed, 3505 insertions(+), 46 deletions(-)
 create mode 100644 xen/arch/arm/vgic/Makefile
 create mode 100644 xen/arch/arm/vgic/vgic-init.c
 create mode 100644 xen/arch/arm/vgic/vgic-mmio-v2.c
 create mode 100644 xen/arch/arm/vgic/vgic-mmio.c
 create mode 100644 xen/arch/arm/vgic/vgic-mmio.h
 create mode 100644 xen/arch/arm/vgic/vgic-v2.c
 create mode 100644 xen/arch/arm/vgic/vgic.c
 create mode 100644 xen/arch/arm/vgic/vgic.h
 create mode 100644 xen/common/list_sort.c
 create mode 100644 xen/include/asm-arm/new_vgic.h
 create mode 100644 xen/include/xen/list_sort.h

-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 01/45] ARM: VGIC: rename gic_event_needs_delivery()

[Andre Przywara]

gic_event_needs_delivery() is not named very intuitively, especially
the gic_ prefix is somewhat misleading.
Rename it to vgic_vcpu_pending_irq(), which makes it clear that this
relates to the virtual GIC and is about interrupts.
Also add a VCPU parameter, which makes the code more flexible in the
future. The current VGIC expect this to be the current VCPU, so add
an assert to spot any regressions.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 .. v2:
- rename to vgic_vcpu_pending_irq()
- add VCPU parameter                                                            

 xen/arch/arm/gic-vgic.c     | 16 ++++++++++++++--
 xen/include/asm-arm/event.h |  2 +-
 xen/include/asm-arm/gic.h   |  2 +-
 3 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index ecb07ceb40..61f093db50 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -339,9 +339,18 @@ void gic_clear_pending_irqs(struct vcpu *v)
         gic_remove_from_lr_pending(v, p);
 }
 
-int gic_events_need_delivery(void)
+/**
+ * vgic_vcpu_pending_irq() - determine if interrupts need to be injected
+ * @vcpu: The vCPU on which to check for interrupts.
+ *
+ * Checks whether there is an interrupt on the given VCPU which needs
+ * handling in the guest. This requires at least one IRQ to be pending
+ * and enabled.
+ *
+ * Returns: 1 if the guest should run to handle interrupts, 0 otherwise.
+ */
+int vgic_vcpu_pending_irq(struct vcpu *v)
 {
-    struct vcpu *v = current;
     struct pending_irq *p;
     unsigned long flags;
     const unsigned long apr = gic_hw_ops->read_apr(0);
@@ -349,6 +358,9 @@ int gic_events_need_delivery(void)
     int active_priority;
     int rc = 0;
 
+    /* We rely on reading the VMCR, which is only accessible locally. */
+    ASSERT(v == current);
+
     mask_priority = gic_hw_ops->read_vmcr_priority();
     active_priority = find_next_bit(&apr, 32, 0);
 
diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
index e8c2a6cb44..c7a415ef57 100644
--- a/xen/include/asm-arm/event.h
+++ b/xen/include/asm-arm/event.h
@@ -24,7 +24,7 @@ static inline int local_events_need_delivery_nomask(void)
      * interrupts disabled so this shouldn't be a problem in the general
      * case.
      */
-    if ( gic_events_need_delivery() )
+    if ( vgic_vcpu_pending_irq(current) )
         return 1;
 
     if ( !vcpu_info(current, evtchn_upcall_pending) )
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index d568957dd1..49cb94f792 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -238,7 +238,7 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
 
 extern void vgic_sync_to_lrs(void);
 extern void gic_clear_pending_irqs(struct vcpu *v);
-extern int gic_events_need_delivery(void);
+extern int vgic_vcpu_pending_irq(struct vcpu *v);
 
 extern void init_maintenance_interrupt(void);
 extern void gic_raise_guest_irq(struct vcpu *v, unsigned int irq,
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 02/45] ARM: Implement vcpu_kick()

[Andre Przywara]

If we change something in a vCPU that affects its runnability or
otherwise needs the vCPU's attention, we might need to tell the scheduler
about it.
We are using this in one place (vIRQ injection) at the moment, but will
need this at more places soon.
So let's factor out this functionality, using the already existing
vcpu_kick() prototype (used in x86 only so far), to make this available
to the rest of the Xen code.
Also adjust the perfcounter name to reflect the new usage.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- use vcpu_kick() name from x86 and existing prototype
- rename performance counter

 xen/arch/arm/domain.c            | 12 ++++++++++++
 xen/arch/arm/vgic.c              | 11 +++--------
 xen/include/asm-arm/perfc_defn.h |  3 ++-
 3 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index bc10f412ba..4462e62599 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -967,6 +967,18 @@ void vcpu_block_unless_event_pending(struct vcpu *v)
         vcpu_unblock(current);
 }
 
+void vcpu_kick(struct vcpu *vcpu)
+{
+    bool running = vcpu->is_running;
+
+    vcpu_unblock(vcpu);
+    if ( running && vcpu != current )
+    {
+        perfc_incr(vcpu_kick);
+        smp_send_event_check_mask(cpumask_of(vcpu->processor));
+    }
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index eb09d9ca54..3fafdd0b66 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -26,6 +26,7 @@
 #include <xen/sched.h>
 #include <xen/perfc.h>
 
+#include <asm/event.h>
 #include <asm/current.h>
 
 #include <asm/mmio.h>
@@ -530,7 +531,6 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
     uint8_t priority;
     struct pending_irq *iter, *n;
     unsigned long flags;
-    bool running;
 
     /*
      * For edge triggered interrupts we always ignore a "falling edge".
@@ -590,14 +590,9 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
     list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
 out:
     spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+
     /* we have a new higher priority irq, inject it into the guest */
-    running = v->is_running;
-    vcpu_unblock(v);
-    if ( running && v != current )
-    {
-        perfc_incr(vgic_cross_cpu_intr_inject);
-        smp_send_event_check_mask(cpumask_of(v->processor));
-    }
+    vcpu_kick(v);
 
     return;
 }
diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
index 87866264ca..8922e9525a 100644
--- a/xen/include/asm-arm/perfc_defn.h
+++ b/xen/include/asm-arm/perfc_defn.h
@@ -33,6 +33,8 @@ PERFCOUNTER(vpsci_cpu_suspend,         "vpsci: cpu_suspend")
 PERFCOUNTER(vpsci_cpu_affinity_info,   "vpsci: cpu_affinity_info")
 PERFCOUNTER(vpsci_features,            "vpsci: features")
 
+PERFCOUNTER(vcpu_kick,                 "vcpu: notify other vcpu")
+
 PERFCOUNTER(vgicd_reads,                "vgicd: read")
 PERFCOUNTER(vgicd_writes,               "vgicd: write")
 PERFCOUNTER(vgicr_reads,                "vgicr: read")
@@ -44,7 +46,6 @@ PERFCOUNTER(vgic_sysreg_writes,         "vgic: sysreg write")
 PERFCOUNTER(vgic_sgi_list  ,            "vgic: SGI send to list")
 PERFCOUNTER(vgic_sgi_others,            "vgic: SGI send to others")
 PERFCOUNTER(vgic_sgi_self,              "vgic: SGI send to self")
-PERFCOUNTER(vgic_cross_cpu_intr_inject, "vgic: cross-CPU irq inject")
 PERFCOUNTER(vgic_irq_migrates,          "vgic: irq migration")
 
 PERFCOUNTER(vuart_reads,  "vuart: read")
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 03/45] xen/arm: gic: Fix indentation in gic_update_one_lr

[Andre Przywara]

From: Julien Grall <julien.grall@arm.com>

Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
Changelog v1 ... v2:
- Add Andre's reviewed-by

 xen/arch/arm/gic-vgic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index 61f093db50..e3cb47e80e 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -197,8 +197,8 @@ static void gic_update_one_lr(struct vcpu *v, int i)
         {
             if ( p->desc == NULL )
             {
-                 lr_val.state |= GICH_LR_PENDING;
-                 gic_hw_ops->write_lr(i, &lr_val);
+                lr_val.state |= GICH_LR_PENDING;
+                gic_hw_ops->write_lr(i, &lr_val);
             }
             else
                 gdprintk(XENLOG_WARNING, "unable to inject hw irq=%d into d%dv%d: already active in LR%d\n",
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 04/45] xen/arm: vgic: Override the group in lr everytime

[Andre Przywara]

From: Julien Grall <julien.grall@arm.com>

At the moment, write_lr is assuming the caller will set correctly the
group. However the group should always be 0 when the guest is using
vGICv2 and 1 for vGICv3. As the caller should not care about the group,
override it directly.

With that change, write_lr is now behaving like update_lr for the group.

Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
Changes:
- Add Andre's reviewed-by

 xen/arch/arm/gic-v2.c     |  4 +---
 xen/arch/arm/gic-v3.c     | 11 ++++++++---
 xen/include/asm-arm/gic.h |  1 -
 3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index f16e17c1a3..fc105c08b8 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -469,7 +469,6 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
     lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
     lr_reg->state     = (lrv >> GICH_V2_LR_STATE_SHIFT) & GICH_V2_LR_STATE_MASK;
     lr_reg->hw_status = (lrv >> GICH_V2_LR_HW_SHIFT) & GICH_V2_LR_HW_MASK;
-    lr_reg->grp       = (lrv >> GICH_V2_LR_GRP_SHIFT) & GICH_V2_LR_GRP_MASK;
 }
 
 static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
@@ -483,8 +482,7 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
           ((uint32_t)(lr_reg->state & GICH_V2_LR_STATE_MASK)
                                    << GICH_V2_LR_STATE_SHIFT) |
           ((uint32_t)(lr_reg->hw_status & GICH_V2_LR_HW_MASK)
-                                       << GICH_V2_LR_HW_SHIFT)  |
-          ((uint32_t)(lr_reg->grp & GICH_V2_LR_GRP_MASK) << GICH_V2_LR_GRP_SHIFT) );
+                                       << GICH_V2_LR_HW_SHIFT));
 
     writel_gich(lrv, GICH_LR + lr * 4);
 }
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 09b49a07d5..0dfa1a1e08 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1012,7 +1012,6 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
     lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
     lr_reg->state     = (lrv >> ICH_LR_STATE_SHIFT) & ICH_LR_STATE_MASK;
     lr_reg->hw_status = (lrv >> ICH_LR_HW_SHIFT) & ICH_LR_HW_MASK;
-    lr_reg->grp       = (lrv >> ICH_LR_GRP_SHIFT) & ICH_LR_GRP_MASK;
 }
 
 static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
@@ -1023,8 +1022,14 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
         ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
         ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT)|
         ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) |
-        ((u64)(lr->hw_status & ICH_LR_HW_MASK) << ICH_LR_HW_SHIFT)  |
-        ((u64)(lr->grp & ICH_LR_GRP_MASK) << ICH_LR_GRP_SHIFT) );
+        ((u64)(lr->hw_status & ICH_LR_HW_MASK) << ICH_LR_HW_SHIFT) );
+
+    /*
+     * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
+     * would result in a FIQ, which will not be expected by the guest OS.
+     */
+    if ( current->domain->arch.vgic.version == GIC_V3 )
+        lrv |= ICH_LR_GRP1;
 
     gicv3_ich_write_lr(lr_reg, lrv);
 }
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 49cb94f792..1eb08b856e 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -211,7 +211,6 @@ struct gic_lr {
    uint8_t priority;
    uint8_t state;
    uint8_t hw_status;
-   uint8_t grp;
 };
 
 enum gic_version {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 05/45] xen/arm: gic: Use bool instead of uint8_t for the hw_status in gic_lr

[Andre Przywara]

From: Julien Grall <julien.grall@arm.com>

hw_status can only be 1 or 0. So convert to a bool.

Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
Changes:
- Remove == *LR_HW as it is pointless
- Add Andre's reviewed-by

 xen/arch/arm/gic-v2.c     | 9 +++++----
 xen/arch/arm/gic-v3.c     | 8 +++++---
 xen/include/asm-arm/gic.h | 2 +-
 3 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index fc105c08b8..9d589115bd 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -468,7 +468,7 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
     lr_reg->virq = (lrv >> GICH_V2_LR_VIRTUAL_SHIFT) & GICH_V2_LR_VIRTUAL_MASK;
     lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
     lr_reg->state     = (lrv >> GICH_V2_LR_STATE_SHIFT) & GICH_V2_LR_STATE_MASK;
-    lr_reg->hw_status = (lrv >> GICH_V2_LR_HW_SHIFT) & GICH_V2_LR_HW_MASK;
+    lr_reg->hw_status = lrv & GICH_V2_LR_HW;
 }
 
 static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
@@ -480,9 +480,10 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
           ((uint32_t)(lr_reg->priority & GICH_V2_LR_PRIORITY_MASK)
                                       << GICH_V2_LR_PRIORITY_SHIFT) |
           ((uint32_t)(lr_reg->state & GICH_V2_LR_STATE_MASK)
-                                   << GICH_V2_LR_STATE_SHIFT) |
-          ((uint32_t)(lr_reg->hw_status & GICH_V2_LR_HW_MASK)
-                                       << GICH_V2_LR_HW_SHIFT));
+                                   << GICH_V2_LR_STATE_SHIFT) );
+
+    if ( lr_reg->hw_status )
+        lrv |= GICH_V2_LR_HW;
 
     writel_gich(lrv, GICH_LR + lr * 4);
 }
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 0dfa1a1e08..f761ae60d6 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1011,7 +1011,7 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
 
     lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
     lr_reg->state     = (lrv >> ICH_LR_STATE_SHIFT) & ICH_LR_STATE_MASK;
-    lr_reg->hw_status = (lrv >> ICH_LR_HW_SHIFT) & ICH_LR_HW_MASK;
+    lr_reg->hw_status = lrv & ICH_LR_HW;
 }
 
 static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
@@ -1021,8 +1021,10 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
     lrv = ( ((u64)(lr->pirq & ICH_LR_PHYSICAL_MASK) << ICH_LR_PHYSICAL_SHIFT)|
         ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
         ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT)|
-        ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) |
-        ((u64)(lr->hw_status & ICH_LR_HW_MASK) << ICH_LR_HW_SHIFT) );
+        ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) );
+
+    if ( lr->hw_status )
+        lrv |= ICH_LR_HW;
 
     /*
      * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 1eb08b856e..daec51499c 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -210,7 +210,7 @@ struct gic_lr {
    uint32_t virq;
    uint8_t priority;
    uint8_t state;
-   uint8_t hw_status;
+   bool hw_status;
 };
 
 enum gic_version {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 06/45] xen/arm: gic: Split the field state in gic_lr in 2 fields active and pending

[Andre Przywara]

From: Julien Grall <julien.grall@arm.com>

Mostly making the code nicer to read.

Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
Changes:
- Use 1ULL
- Remove pointless == *_STATE_*

 xen/arch/arm/gic-v2.c             | 15 +++++++++++----
 xen/arch/arm/gic-v3.c             | 12 +++++++++---
 xen/arch/arm/gic-vgic.c           |  6 +++---
 xen/include/asm-arm/gic.h         |  3 ++-
 xen/include/asm-arm/gic_v3_defs.h |  2 ++
 5 files changed, 27 insertions(+), 11 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 9d589115bd..6dae5c1e55 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -51,6 +51,8 @@
 #define GICH_V2_LR_PHYSICAL_SHIFT  10
 #define GICH_V2_LR_STATE_MASK      0x3
 #define GICH_V2_LR_STATE_SHIFT     28
+#define GICH_V2_LR_PENDING         (1U << 28)
+#define GICH_V2_LR_ACTIVE          (1U << 29)
 #define GICH_V2_LR_PRIORITY_SHIFT  23
 #define GICH_V2_LR_PRIORITY_MASK   0x1f
 #define GICH_V2_LR_HW_SHIFT        31
@@ -467,7 +469,8 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
     lr_reg->pirq = (lrv >> GICH_V2_LR_PHYSICAL_SHIFT) & GICH_V2_LR_PHYSICAL_MASK;
     lr_reg->virq = (lrv >> GICH_V2_LR_VIRTUAL_SHIFT) & GICH_V2_LR_VIRTUAL_MASK;
     lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
-    lr_reg->state     = (lrv >> GICH_V2_LR_STATE_SHIFT) & GICH_V2_LR_STATE_MASK;
+    lr_reg->pending = lrv & GICH_V2_LR_PENDING;
+    lr_reg->active = lrv & GICH_V2_LR_ACTIVE;
     lr_reg->hw_status = lrv & GICH_V2_LR_HW;
 }
 
@@ -478,9 +481,13 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
     lrv = ( ((lr_reg->pirq & GICH_V2_LR_PHYSICAL_MASK) << GICH_V2_LR_PHYSICAL_SHIFT) |
           ((lr_reg->virq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT)   |
           ((uint32_t)(lr_reg->priority & GICH_V2_LR_PRIORITY_MASK)
-                                      << GICH_V2_LR_PRIORITY_SHIFT) |
-          ((uint32_t)(lr_reg->state & GICH_V2_LR_STATE_MASK)
-                                   << GICH_V2_LR_STATE_SHIFT) );
+                                      << GICH_V2_LR_PRIORITY_SHIFT) );
+
+    if ( lr_reg->active )
+        lrv |= GICH_V2_LR_ACTIVE;
+
+    if ( lr_reg->pending )
+        lrv |= GICH_V2_LR_PENDING;
 
     if ( lr_reg->hw_status )
         lrv |= GICH_V2_LR_HW;
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index f761ae60d6..6547b5eb0d 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1010,7 +1010,8 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
     lr_reg->virq = (lrv >> ICH_LR_VIRTUAL_SHIFT) & ICH_LR_VIRTUAL_MASK;
 
     lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
-    lr_reg->state     = (lrv >> ICH_LR_STATE_SHIFT) & ICH_LR_STATE_MASK;
+    lr_reg->pending   = lrv & ICH_LR_STATE_PENDING;
+    lr_reg->active    = lrv & ICH_LR_STATE_ACTIVE;
     lr_reg->hw_status = lrv & ICH_LR_HW;
 }
 
@@ -1020,8 +1021,13 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
 
     lrv = ( ((u64)(lr->pirq & ICH_LR_PHYSICAL_MASK) << ICH_LR_PHYSICAL_SHIFT)|
         ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
-        ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT)|
-        ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) );
+        ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT) );
+
+    if ( lr->active )
+        lrv |= ICH_LR_STATE_ACTIVE;
+
+    if ( lr->pending )
+        lrv |= ICH_LR_STATE_PENDING;
 
     if ( lr->hw_status )
         lrv |= ICH_LR_HW;
diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index e3cb47e80e..d831b35525 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -189,7 +189,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
         return;
     }
 
-    if ( lr_val.state & GICH_LR_ACTIVE )
+    if ( lr_val.active )
     {
         set_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
         if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
@@ -197,7 +197,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
         {
             if ( p->desc == NULL )
             {
-                lr_val.state |= GICH_LR_PENDING;
+                lr_val.pending = true;
                 gic_hw_ops->write_lr(i, &lr_val);
             }
             else
@@ -205,7 +205,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
                          irq, v->domain->domain_id, v->vcpu_id, i);
         }
     }
-    else if ( lr_val.state & GICH_LR_PENDING )
+    else if ( lr_val.pending )
     {
         int q __attribute__ ((unused)) = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
 #ifdef GIC_DEBUG
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index daec51499c..c32861d4fa 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -209,7 +209,8 @@ struct gic_lr {
    /* Virtual IRQ */
    uint32_t virq;
    uint8_t priority;
-   uint8_t state;
+   bool active;
+   bool pending;
    bool hw_status;
 };
 
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index ccb72cf0f1..d9827bd84c 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -171,6 +171,8 @@
 #define ICH_LR_PHYSICAL_SHIFT        32
 #define ICH_LR_STATE_MASK            0x3
 #define ICH_LR_STATE_SHIFT           62
+#define ICH_LR_STATE_PENDING         (1ULL << 62)
+#define ICH_LR_STATE_ACTIVE          (1ULL << 63)
 #define ICH_LR_PRIORITY_MASK         0xff
 #define ICH_LR_PRIORITY_SHIFT        48
 #define ICH_LR_HW_MASK               0x1
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 07/45] xen/arm: GIC: Only set pirq in the LR when hw_status is set

[Andre Przywara]

From: Julien Grall <julien.grall@arm.com>

The field pirq should only be valid when the virtual interrupt
is associated to a physical interrupt.

This change will help to extend gic_lr for supporting specific virtual
interrupt field (e.g eoi, source) that clashes with the PIRQ field.

Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
Changes:
- Add Andre's reviewed-by

 xen/arch/arm/gic-v2.c     | 13 ++++++++++---
 xen/arch/arm/gic-v3.c     | 10 +++++++---
 xen/include/asm-arm/gic.h |  2 +-
 3 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 6dae5c1e55..2f012692e0 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -466,20 +466,24 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
     uint32_t lrv;
 
     lrv          = readl_gich(GICH_LR + lr * 4);
-    lr_reg->pirq = (lrv >> GICH_V2_LR_PHYSICAL_SHIFT) & GICH_V2_LR_PHYSICAL_MASK;
     lr_reg->virq = (lrv >> GICH_V2_LR_VIRTUAL_SHIFT) & GICH_V2_LR_VIRTUAL_MASK;
     lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
     lr_reg->pending = lrv & GICH_V2_LR_PENDING;
     lr_reg->active = lrv & GICH_V2_LR_ACTIVE;
     lr_reg->hw_status = lrv & GICH_V2_LR_HW;
+
+    if ( lr_reg->hw_status )
+    {
+        lr_reg->pirq = lrv >> GICH_V2_LR_PHYSICAL_SHIFT;
+        lr_reg->pirq &= GICH_V2_LR_PHYSICAL_MASK;
+    }
 }
 
 static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
 {
     uint32_t lrv = 0;
 
-    lrv = ( ((lr_reg->pirq & GICH_V2_LR_PHYSICAL_MASK) << GICH_V2_LR_PHYSICAL_SHIFT) |
-          ((lr_reg->virq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT)   |
+    lrv = (((lr_reg->virq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT)   |
           ((uint32_t)(lr_reg->priority & GICH_V2_LR_PRIORITY_MASK)
                                       << GICH_V2_LR_PRIORITY_SHIFT) );
 
@@ -490,7 +494,10 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
         lrv |= GICH_V2_LR_PENDING;
 
     if ( lr_reg->hw_status )
+    {
         lrv |= GICH_V2_LR_HW;
+        lrv |= lr_reg->pirq << GICH_V2_LR_PHYSICAL_SHIFT;
+    }
 
     writel_gich(lrv, GICH_LR + lr * 4);
 }
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 6547b5eb0d..e901210b78 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1006,21 +1006,22 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
 
     lrv = gicv3_ich_read_lr(lr);
 
-    lr_reg->pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
     lr_reg->virq = (lrv >> ICH_LR_VIRTUAL_SHIFT) & ICH_LR_VIRTUAL_MASK;
 
     lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
     lr_reg->pending   = lrv & ICH_LR_STATE_PENDING;
     lr_reg->active    = lrv & ICH_LR_STATE_ACTIVE;
     lr_reg->hw_status = lrv & ICH_LR_HW;
+
+    if ( lr_reg->hw_status )
+        lr_reg->pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
 }
 
 static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
 {
     uint64_t lrv = 0;
 
-    lrv = ( ((u64)(lr->pirq & ICH_LR_PHYSICAL_MASK) << ICH_LR_PHYSICAL_SHIFT)|
-        ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
+    lrv = ( ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
         ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT) );
 
     if ( lr->active )
@@ -1030,7 +1031,10 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
         lrv |= ICH_LR_STATE_PENDING;
 
     if ( lr->hw_status )
+    {
         lrv |= ICH_LR_HW;
+        lrv |= (uint64_t)lr->pirq << ICH_LR_PHYSICAL_SHIFT;
+    }
 
     /*
      * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index c32861d4fa..545901b120 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -204,7 +204,7 @@ union gic_state_data {
  * The LR register format is different for GIC HW version
  */
 struct gic_lr {
-   /* Physical IRQ */
+   /* Physical IRQ -> Only set when hw_status is set. */
    uint32_t pirq;
    /* Virtual IRQ */
    uint32_t virq;
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 08/45] ARM: GIC: extend LR read/write functions to cover EOI and source

[Andre Przywara]

From: Julien Grall <julien.grall@arm.com>

So far our LR read/write functions do not handle the EOI bit and the
source CPUID bits in an LR, because the current VGIC implementation does
not use them.
Extend the gic_lr data structure to hold these bits of information by
using a union to differentiate field used depending on whether the vIRQ
has a corresponding pIRQ.

This allows the new VGIC to use this information.

This is based on the original patch sent by Andre Przywara [1].

[1] https://lists.xenproject.org/archives/html/xen-devel/2018-03/msg00435.html

Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
Changes:
- Reset authorship
- Rework source
- Add source support on GICv3

 xen/arch/arm/gic-v2.c             | 26 +++++++++++++++++++++++---
 xen/arch/arm/gic-v3.c             | 34 +++++++++++++++++++++++++++++++---
 xen/include/asm-arm/gic.h         | 16 ++++++++++++++--
 xen/include/asm-arm/gic_v3_defs.h |  2 ++
 4 files changed, 70 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 2f012692e0..7dfe6fc68d 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -474,8 +474,17 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
 
     if ( lr_reg->hw_status )
     {
-        lr_reg->pirq = lrv >> GICH_V2_LR_PHYSICAL_SHIFT;
-        lr_reg->pirq &= GICH_V2_LR_PHYSICAL_MASK;
+        lr_reg->hw.pirq = lrv >> GICH_V2_LR_PHYSICAL_SHIFT;
+        lr_reg->hw.pirq &= GICH_V2_LR_PHYSICAL_MASK;
+    }
+    else
+    {
+        lr_reg->virt.eoi = (lrv & GICH_V2_LR_MAINTENANCE_IRQ);
+        if ( lr_reg->virq < NR_GIC_SGI )
+        {
+            lr_reg->virt.source = (lrv >> GICH_V2_LR_CPUID_SHIFT)
+                & GICH_V2_LR_CPUID_MASK;
+        }
     }
 }
 
@@ -496,7 +505,18 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
     if ( lr_reg->hw_status )
     {
         lrv |= GICH_V2_LR_HW;
-        lrv |= lr_reg->pirq << GICH_V2_LR_PHYSICAL_SHIFT;
+        lrv |= lr_reg->hw.pirq << GICH_V2_LR_PHYSICAL_SHIFT;
+    }
+    else
+    {
+        if ( lr_reg->virt.eoi )
+            lrv |= GICH_V2_LR_MAINTENANCE_IRQ;
+        /*
+         * This is only valid for SGI, but it does not matter to always
+         * read it as it should be 0 by default.
+         */
+        ASSERT(!lr_reg->virt.source || lr_reg->virq < NR_GIC_SGI);
+        lrv |= (uint32_t)lr_reg->virt.source << GICH_V2_LR_CPUID_SHIFT;
     }
 
     writel_gich(lrv, GICH_LR + lr * 4);
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index e901210b78..392cf91b58 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1014,12 +1014,25 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
     lr_reg->hw_status = lrv & ICH_LR_HW;
 
     if ( lr_reg->hw_status )
-        lr_reg->pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
+        lr_reg->hw.pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
+    else
+    {
+        lr_reg->virt.eoi = (lrv & ICH_LR_MAINTENANCE_IRQ);
+        /* Source only exists for SGI and in GICv2 compatible mode */
+        if ( lr_reg->virq < NR_GIC_SGI &&
+             current->domain->arch.vgic.version == GIC_V2 )
+        {
+            lr_reg->virt.source = (lrv >> ICH_LR_CPUID_SHIFT)
+                & ICH_LR_CPUID_MASK;
+        }
+    }
 }
 
 static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
 {
     uint64_t lrv = 0;
+    const enum gic_version vgic_version = current->domain->arch.vgic.version;
+
 
     lrv = ( ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
         ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT) );
@@ -1033,14 +1046,29 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
     if ( lr->hw_status )
     {
         lrv |= ICH_LR_HW;
-        lrv |= (uint64_t)lr->pirq << ICH_LR_PHYSICAL_SHIFT;
+        lrv |= (uint64_t)lr->hw.pirq << ICH_LR_PHYSICAL_SHIFT;
+    }
+    else
+    {
+        if ( lr->virt.eoi )
+            lrv |= ICH_LR_MAINTENANCE_IRQ;
+        /* Source is only set in GICv2 compatible mode */
+        if ( vgic_version == GIC_V2 )
+        {
+            /*
+             * This is only valid for SGI, but it does not matter to always
+             * read it as it should be 0 by default.
+             */
+            ASSERT(!lr->virt.source || lr->virq < NR_GIC_SGI);
+            lrv |= (uint64_t)lr->virt.source << ICH_LR_CPUID_SHIFT;
+        }
     }
 
     /*
      * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
      * would result in a FIQ, which will not be expected by the guest OS.
      */
-    if ( current->domain->arch.vgic.version == GIC_V3 )
+    if ( vgic_version == GIC_V3 )
         lrv |= ICH_LR_GRP1;
 
     gicv3_ich_write_lr(lr_reg, lrv);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 545901b120..565b0875ca 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -204,14 +204,26 @@ union gic_state_data {
  * The LR register format is different for GIC HW version
  */
 struct gic_lr {
-   /* Physical IRQ -> Only set when hw_status is set. */
-   uint32_t pirq;
    /* Virtual IRQ */
    uint32_t virq;
    uint8_t priority;
    bool active;
    bool pending;
    bool hw_status;
+   union
+   {
+       /* Only filled when there are a corresponding pIRQ (hw_state = true) */
+       struct
+       {
+           uint32_t pirq;
+       } hw;
+       /* Only filled when there are no corresponding pIRQ (hw_state = false) */
+       struct
+       {
+           bool eoi;
+           uint8_t source;      /* GICv2 only */
+       } virt;
+   };
 };
 
 enum gic_version {
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index d9827bd84c..10a2aeea93 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -167,6 +167,8 @@
 
 #define ICH_LR_VIRTUAL_MASK          0xffff
 #define ICH_LR_VIRTUAL_SHIFT         0
+#define ICH_LR_CPUID_MASK            0x7
+#define ICH_LR_CPUID_SHIFT           10
 #define ICH_LR_PHYSICAL_MASK         0x3ff
 #define ICH_LR_PHYSICAL_SHIFT        32
 #define ICH_LR_STATE_MASK            0x3
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 09/45] ARM: GIC: Allow tweaking the active and pending state of an IRQ

[Andre Przywara]

When playing around with hardware mapped, level triggered virtual IRQs,
there is the need to explicitly set the active or pending state of an
interrupt at some point.
To prepare the GIC for that, we introduce a set_active_state() and a
set_pending_state() function to let the VGIC manipulate the state of
an associated hardware IRQ.
This takes care of properly setting the _IRQ_INPROGRESS bit.
For this it adds gicv2/3_peek_irq() helper functions, to read a bit in a
bitmap spread over several MMIO registers.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- properly set _IRQ_INPROGRESS bit
- add gicv[23]_peek_irq() (pulled in from later patch)
- move wrappers functions into gic.h

 xen/arch/arm/gic-v2.c     | 48 +++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/gic-v3.c     | 52 +++++++++++++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/gic.h | 24 ++++++++++++++++++++++
 3 files changed, 124 insertions(+)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 7dfe6fc68d..c6fcbf59d0 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -243,6 +243,52 @@ static void gicv2_poke_irq(struct irq_desc *irqd, uint32_t offset)
     writel_gicd(1U << (irqd->irq % 32), offset + (irqd->irq / 32) * 4);
 }
 
+static bool gicv2_peek_irq(struct irq_desc *irqd, uint32_t offset)
+{
+    uint32_t reg;
+
+    reg = readl_gicd(offset + (irqd->irq / 32) * 4) & (1U << (irqd->irq % 32));
+
+    return reg;
+}
+
+static void gicv2_set_active_state(struct irq_desc *irqd, bool active)
+{
+    ASSERT(spin_is_locked(&irqd->lock));
+
+    if ( active )
+    {
+        if ( test_bit(_IRQ_GUEST, &irqd->status) )
+            set_bit(_IRQ_INPROGRESS, &irqd->status);
+        gicv2_poke_irq(irqd, GICD_ISACTIVER);
+    }
+    else
+    {
+        gicv2_poke_irq(irqd, GICD_ICACTIVER);
+        if ( !gicv2_peek_irq(irqd, GICD_ISPENDR) &&
+             test_bit(_IRQ_GUEST, &irqd->status) )
+            clear_bit(_IRQ_INPROGRESS, &irqd->status);
+    }
+}
+
+static void gicv2_set_pending_state(struct irq_desc *irqd, bool pending)
+{
+    ASSERT(spin_is_locked(&irqd->lock));
+
+    if ( pending )
+    {
+        /* The INPROGRESS bit will be set when the interrupt fires. */
+        gicv2_poke_irq(irqd, GICD_ISPENDR);
+    }
+    else
+    {
+        gicv2_poke_irq(irqd, GICD_ICPENDR);
+        if ( !gicv2_peek_irq(irqd, GICD_ISACTIVER) &&
+             test_bit(_IRQ_GUEST, &irqd->status) )
+            clear_bit(_IRQ_INPROGRESS, &irqd->status);
+    }
+}
+
 static void gicv2_set_irq_type(struct irq_desc *desc, unsigned int type)
 {
     uint32_t cfg, actual, edgebit;
@@ -1277,6 +1323,8 @@ const static struct gic_hw_operations gicv2_ops = {
     .eoi_irq             = gicv2_eoi_irq,
     .deactivate_irq      = gicv2_dir_irq,
     .read_irq            = gicv2_read_irq,
+    .set_active_state    = gicv2_set_active_state,
+    .set_pending_state   = gicv2_set_pending_state,
     .set_irq_type        = gicv2_set_irq_type,
     .set_irq_priority    = gicv2_set_irq_priority,
     .send_SGI            = gicv2_send_SGI,
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 392cf91b58..316f2c4142 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -444,6 +444,19 @@ static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset, bool wait_for_rwp)
         gicv3_wait_for_rwp(irqd->irq);
 }
 
+static bool gicv3_peek_irq(struct irq_desc *irqd, u32 offset)
+{
+    void __iomem *base;
+    unsigned int irq = irqd->irq;
+
+    if ( irq >= NR_GIC_LOCAL_IRQS)
+        base = GICD + (irq / 32) * 4;
+    else
+        base = GICD_RDIST_SGI_BASE;
+
+    return !!(readl(base + offset) & (1U << (irq % 32)));
+}
+
 static void gicv3_unmask_irq(struct irq_desc *irqd)
 {
     gicv3_poke_irq(irqd, GICD_ISENABLER, false);
@@ -477,6 +490,43 @@ static unsigned int gicv3_read_irq(void)
     return irq;
 }
 
+static void gicv3_set_active_state(struct irq_desc *irqd, bool active)
+{
+    ASSERT(spin_is_locked(&irqd->lock));
+
+    if ( active )
+    {
+        if ( test_bit(_IRQ_GUEST, &irqd->status) )
+            set_bit(_IRQ_INPROGRESS, &irqd->status);
+        gicv3_poke_irq(irqd, GICD_ISACTIVER, false);
+    }
+    else
+    {
+        gicv3_poke_irq(irqd, GICD_ICACTIVER, false);
+        if ( !gicv3_peek_irq(irqd, GICD_ISPENDR) &&
+             test_bit(_IRQ_GUEST, &irqd->status) )
+            clear_bit(_IRQ_INPROGRESS, &irqd->status);
+    }
+}
+
+static void gicv3_set_pending_state(struct irq_desc *irqd, bool pending)
+{
+    ASSERT(spin_is_locked(&irqd->lock));
+
+    if ( pending )
+    {
+        /* The INPROGRESS bit will be set when the interrupt fires. */
+        gicv3_poke_irq(irqd, GICD_ISPENDR, false);
+    }
+    else
+    {
+        gicv3_poke_irq(irqd, GICD_ICPENDR, false);
+        if ( !gicv3_peek_irq(irqd, GICD_ISACTIVER) &&
+             test_bit(_IRQ_GUEST, &irqd->status) )
+            clear_bit(_IRQ_INPROGRESS, &irqd->status);
+    }
+}
+
 static inline uint64_t gicv3_mpidr_to_affinity(int cpu)
 {
      uint64_t mpidr = cpu_logical_map(cpu);
@@ -1766,6 +1816,8 @@ static const struct gic_hw_operations gicv3_ops = {
     .eoi_irq             = gicv3_eoi_irq,
     .deactivate_irq      = gicv3_dir_irq,
     .read_irq            = gicv3_read_irq,
+    .set_active_state    = gicv3_set_active_state,
+    .set_pending_state   = gicv3_set_pending_state,
     .set_irq_type        = gicv3_set_irq_type,
     .set_irq_priority    = gicv3_set_irq_priority,
     .send_SGI            = gicv3_send_sgi,
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 565b0875ca..21cf35f106 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -344,6 +344,10 @@ struct gic_hw_operations {
     void (*deactivate_irq)(struct irq_desc *irqd);
     /* Read IRQ id and Ack */
     unsigned int (*read_irq)(void);
+    /* Force the active state of an IRQ by accessing the distributor */
+    void (*set_active_state)(struct irq_desc *irqd, bool state);
+    /* Force the pending state of an IRQ by accessing the distributor */
+    void (*set_pending_state)(struct irq_desc *irqd, bool state);
     /* Set IRQ type */
     void (*set_irq_type)(struct irq_desc *desc, unsigned int type);
     /* Set IRQ priority */
@@ -392,6 +396,26 @@ static inline unsigned int gic_get_nr_lrs(void)
     return gic_hw_ops->info->nr_lrs;
 }
 
+/*
+ * Set the active state of an IRQ. This should be used with care, as this
+ * directly forces the active bit, without considering the GIC state machine.
+ * For private IRQs this only works for those of the current CPU.
+ */
+static inline void gic_set_active_state(struct irq_desc *irqd, bool state)
+{
+    gic_hw_ops->set_active_state(irqd, state);
+}
+
+/*
+ * Set the pending state of an IRQ. This should be used with care, as this
+ * directly forces the pending bit, without considering the GIC state machine.
+ * For private IRQs this only works for those of the current CPU.
+ */
+static inline void gic_set_pending_state(struct irq_desc *irqd, bool state)
+{
+    gic_hw_ops->set_pending_state(irqd, state);
+}
+
 void register_gic_ops(const struct gic_hw_operations *ops);
 int gic_make_hwdom_dt_node(const struct domain *d,
                            const struct dt_device_node *gic,
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 10/45] ARM: GIC: Allow reading pending state of a hardware IRQ

[Andre Przywara]

To synchronize level triggered interrupts which are mapped into a guest,
we need to update the virtual line level at certain points in time.
For a hardware mapped interrupt the GIC is the only place where we can
easily access this information.
Implement a gic_hw_operations member to return the pending state of a
particular interrupt. Due to hardware limitations this only works for
private interrupts of the current CPU, so there is no CPU field in the
prototype.
This adds gicv2/3_peek_irq() helper functions, to read a bit in a bitmap
spread over several MMIO registers.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- remove introduction of gicv[23]_peek_irq() (now one patch earlier)
- move wrapper functions into gic.h

 xen/arch/arm/gic-v2.c     |  6 ++++++
 xen/arch/arm/gic-v3.c     |  6 ++++++
 xen/include/asm-arm/gic.h | 11 +++++++++++
 3 files changed, 23 insertions(+)

diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index c6fcbf59d0..52880d658c 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -591,6 +591,11 @@ static unsigned int gicv2_read_apr(int apr_reg)
    return readl_gich(GICH_APR);
 }
 
+static bool gicv2_read_pending_state(struct irq_desc *irqd)
+{
+    return gicv2_peek_irq(irqd, GICD_ISPENDR);
+}
+
 static void gicv2_irq_enable(struct irq_desc *desc)
 {
     unsigned long flags;
@@ -1336,6 +1341,7 @@ const static struct gic_hw_operations gicv2_ops = {
     .write_lr            = gicv2_write_lr,
     .read_vmcr_priority  = gicv2_read_vmcr_priority,
     .read_apr            = gicv2_read_apr,
+    .read_pending_state  = gicv2_read_pending_state,
     .make_hwdom_dt_node  = gicv2_make_hwdom_dt_node,
     .make_hwdom_madt     = gicv2_make_hwdom_madt,
     .get_hwdom_extra_madt_size = gicv2_get_hwdom_extra_madt_size,
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 316f2c4142..008be87572 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1161,6 +1161,11 @@ static unsigned int gicv3_read_apr(int apr_reg)
     }
 }
 
+static bool gicv3_read_pending_state(struct irq_desc *irqd)
+{
+    return gicv3_peek_irq(irqd, GICD_ISPENDR);
+}
+
 static void gicv3_irq_enable(struct irq_desc *desc)
 {
     unsigned long flags;
@@ -1829,6 +1834,7 @@ static const struct gic_hw_operations gicv3_ops = {
     .write_lr            = gicv3_write_lr,
     .read_vmcr_priority  = gicv3_read_vmcr_priority,
     .read_apr            = gicv3_read_apr,
+    .read_pending_state  = gicv3_read_pending_state,
     .secondary_init      = gicv3_secondary_cpu_init,
     .make_hwdom_dt_node  = gicv3_make_hwdom_dt_node,
     .make_hwdom_madt     = gicv3_make_hwdom_madt,
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 21cf35f106..bd7802667f 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -372,6 +372,8 @@ struct gic_hw_operations {
     unsigned int (*read_vmcr_priority)(void);
     /* Read APRn register */
     unsigned int (*read_apr)(int apr_reg);
+    /* Query the pending state of an interrupt at the distributor level. */
+    bool (*read_pending_state)(struct irq_desc *irqd);
     /* Secondary CPU init */
     int (*secondary_init)(void);
     /* Create GIC node for the hardware domain */
@@ -416,6 +418,15 @@ static inline void gic_set_pending_state(struct irq_desc *irqd, bool state)
     gic_hw_ops->set_pending_state(irqd, state);
 }
 
+/*
+ * Read the pending state of an interrupt from the distributor.
+ * For private IRQs this only works for those of the current CPU.
+ */
+static inline bool gic_read_pending_state(struct irq_desc *irqd)
+{
+    return gic_hw_ops->read_pending_state(irqd);
+}
+
 void register_gic_ops(const struct gic_hw_operations *ops);
 int gic_make_hwdom_dt_node(const struct domain *d,
                            const struct dt_device_node *gic,
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 11/45] ARM: timer: Handle level triggered IRQs correctly

[Andre Przywara]

The ARM Generic Timer uses a level-sensitive interrupt semantic. We
easily catch when the line goes high, as this triggers the hardware IRQ.
However we have to sync the state of the interrupt condition at certain
points to catch when the line goes low and we can remove the vtimer vIRQ
from the vGIC (and the LR).
The VGIC in Xen so far only implemented edge triggered vIRQs, really, so
we need to add new functionality to re-sample the interrupt state.
Do this only when the new VGIC is in use.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- restrict to new VGIC
- add TODO: comment

 xen/arch/arm/time.c     | 36 ++++++++++++++++++++++++++++++++++++
 xen/arch/arm/traps.c    | 11 +++++++++++
 xen/include/xen/timer.h |  2 ++
 3 files changed, 49 insertions(+)

diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c
index c11fcfeadd..c0ae781ecd 100644
--- a/xen/arch/arm/time.c
+++ b/xen/arch/arm/time.c
@@ -263,6 +263,42 @@ static void vtimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
     vgic_inject_irq(current->domain, current, current->arch.virt_timer.irq, true);
 }
 
+/**
+ * vtimer_sync() - update the state of the virtual timer after a guest run
+ * @vcpu: The VCPU to sync the arch timer state
+ *
+ * After returning from a guest, update the state of the virtual interrupt
+ * line, to model the level triggered interrupt correctly.
+ * If the guest has handled a timer interrupt, the virtual interrupt line
+ * needs to be lowered explicitly. vgic_inject_irq() takes care of that.
+ */
+void vtimer_sync(struct vcpu *vcpu)
+{
+    struct vtimer *vtimer = &vcpu->arch.virt_timer;
+    uint32_t vtimer_ctl = READ_SYSREG32(CNTV_CTL_EL0);
+    bool level;
+
+    /*
+     * Technically the mask should include the CNTx_CTL_MASK bit here,
+     * to catch if the timer interrupt is masked. However Xen always masks
+     * the timer upon entering the hypervisor, leaving it up to the guest
+     * to un-mask it. So we would always read a "low" level, despite the
+     * condition being actually "high".
+     * Ignoring the mask bit solves this (for now).
+     * Another possible check would be to compare the value of CNTVCT_EL0
+     * against vtimer->cval and derive the interrupt state from that.
+     */
+    vtimer_ctl &= (CNTx_CTL_ENABLE | CNTx_CTL_PENDING);
+    level = (vtimer_ctl == (CNTx_CTL_ENABLE | CNTx_CTL_PENDING));
+
+     /*
+      * TODO: The proper fix for this is to make vtimer vIRQ hardware mapped,
+      * but this requires reworking the arch timer to implement this.
+      */
+
+    vgic_inject_irq(vcpu->domain, vcpu, vtimer->irq, level);
+}
+
 /*
  * Arch timer interrupt really ought to be level triggered, since the
  * design of the timer/comparator mechanism is based around that
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 7411bff7a7..46464d7bb9 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2024,6 +2024,17 @@ static void enter_hypervisor_head(struct cpu_user_regs *regs)
         if ( current->arch.hcr_el2 & HCR_VA )
             current->arch.hcr_el2 = READ_SYSREG(HCR_EL2);
 
+#ifdef CONFIG_NEW_VGIC
+        /*
+         * We need to update the state of our emulated devices using level
+         * triggered interrupts before syncing back the VGIC state.
+         *
+         * TODO: Investigate whether this is necessary to do on every
+         * trap and how it can be optimised.
+         */
+        vtimer_sync(current);
+#endif
+
         vgic_sync_from_lrs(current);
     }
 }
diff --git a/xen/include/xen/timer.h b/xen/include/xen/timer.h
index 4513260b0d..eddbbf3903 100644
--- a/xen/include/xen/timer.h
+++ b/xen/include/xen/timer.h
@@ -94,6 +94,8 @@ DECLARE_PER_CPU(s_time_t, timer_deadline);
 /* Arch-defined function to reprogram timer hardware for new deadline. */
 int reprogram_timer(s_time_t timeout);
 
+void vtimer_sync(struct vcpu *vcpu);
+
 /* Calculate the aligned first tick time for a given periodic timer. */
 s_time_t align_timer(s_time_t firsttick, uint64_t period);
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 12/45] ARM: evtchn: Handle level triggered IRQs correctly

[Andre Przywara]

The event channel IRQ has level triggered semantics, however the current
VGIC treats everything as edge triggered.
To correctly process those IRQs, we have to lower the (virtual) IRQ line
at some point in time, depending on whether ther interrupt condition
still prevails.
Check the per-VCPU evtchn_upcall_pending variable to make the interrupt
line match its status, and call this function upon every hypervisor
entry.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
 xen/arch/arm/domain.c       | 7 +++++++
 xen/arch/arm/traps.c        | 1 +
 xen/include/asm-arm/event.h | 1 +
 3 files changed, 9 insertions(+)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 4462e62599..18b915d2e9 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -954,6 +954,13 @@ void vcpu_mark_events_pending(struct vcpu *v)
     vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
 }
 
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    bool pending = vcpu_info(v, evtchn_upcall_pending);
+
+    vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, pending);
+}
+
 /* The ARM spec declares that even if local irqs are masked in
  * the CPSR register, an irq should wake up a cpu from WFI anyway.
  * For this reason we need to check for irqs that need delivery,
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 46464d7bb9..c13223a69f 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2033,6 +2033,7 @@ static void enter_hypervisor_head(struct cpu_user_regs *regs)
          * trap and how it can be optimised.
          */
         vtimer_sync(current);
+        vcpu_update_evtchn_irq(current);
 #endif
 
         vgic_sync_from_lrs(current);
diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
index c7a415ef57..2f51864043 100644
--- a/xen/include/asm-arm/event.h
+++ b/xen/include/asm-arm/event.h
@@ -6,6 +6,7 @@
 
 void vcpu_kick(struct vcpu *v);
 void vcpu_mark_events_pending(struct vcpu *v);
+void vcpu_update_evtchn_irq(struct vcpu *v);
 void vcpu_block_unless_event_pending(struct vcpu *v);
 
 static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 13/45] ARM: vPL011: Use the VGIC's level triggered IRQs handling if available

[Andre Przywara]

The emulated ARM SBSA UART is using level triggered IRQ semantics,
however the current VGIC can only handle edge triggered IRQs, really.
Disable the existing workaround for this problem in case we have the
new VGIC in place, which can properly handle level triggered IRQs.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- remove unneeded "!!"

 xen/arch/arm/vpl011.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index 5dcf4bec18..a281eabd7e 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -54,6 +54,7 @@ static void vpl011_update_interrupt_status(struct domain *d)
      */
     ASSERT(spin_is_locked(&vpl011->lock));
 
+#ifndef CONFIG_NEW_VGIC
     /*
      * TODO: PL011 interrupts are level triggered which means
      * that interrupt needs to be set/clear instead of being
@@ -71,6 +72,9 @@ static void vpl011_update_interrupt_status(struct domain *d)
         vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, true);
 
     vpl011->shadow_uartmis = uartmis;
+#else
+    vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, uartmis);
+#endif
 }
 
 static uint8_t vpl011_read_data(struct domain *d)
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 14/45] ARM: new VGIC: Add data structure definitions

[Andre Przywara]

Add a new header file for the new and improved GIC implementation.
The big change is that we now have a struct vgic_irq per IRQ instead
of spreading all the information over various bitmaps in the ranks.

We include this new header conditionally from within the old header
file for the time being to avoid touching all the users.

This is based on Linux commit b18b57787f5e, written by Christoffer Dall.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- define CONFIG_EDGE to false instead of using enum
- remove not needed forward declaration
- remove unneeded macros
- add emacs footer

 xen/include/asm-arm/new_vgic.h | 198 +++++++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/vgic.h     |   6 ++
 2 files changed, 204 insertions(+)
 create mode 100644 xen/include/asm-arm/new_vgic.h

diff --git a/xen/include/asm-arm/new_vgic.h b/xen/include/asm-arm/new_vgic.h
new file mode 100644
index 0000000000..97d622bff6
--- /dev/null
+++ b/xen/include/asm-arm/new_vgic.h
@@ -0,0 +1,198 @@
+/*
+ * Copyright (C) 2015, 2016 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_ARM_NEW_VGIC_H
+#define __ASM_ARM_NEW_VGIC_H
+
+#include <asm/atomic.h>
+#include <asm/mmio.h>
+#include <xen/list.h>
+#include <xen/mm.h>
+#include <xen/spinlock.h>
+
+#define VGIC_V3_MAX_CPUS        255
+#define VGIC_V2_MAX_CPUS        8
+#define VGIC_NR_SGIS            16
+#define VGIC_NR_PPIS            16
+#define VGIC_NR_PRIVATE_IRQS    (VGIC_NR_SGIS + VGIC_NR_PPIS)
+#define VGIC_MAX_PRIVATE        (VGIC_NR_PRIVATE_IRQS - 1)
+#define VGIC_MAX_SPI            1019
+#define VGIC_MAX_RESERVED       1023
+#define VGIC_MIN_LPI            8192
+
+#define irq_is_ppi(irq) ((irq) >= VGIC_NR_SGIS && (irq) < VGIC_NR_PRIVATE_IRQS)
+#define irq_is_spi(irq) ((irq) >= VGIC_NR_PRIVATE_IRQS && \
+                         (irq) <= VGIC_MAX_SPI)
+
+enum vgic_type {
+    VGIC_V2,        /* Good ol' GICv2 */
+    VGIC_V3,        /* New fancy GICv3 */
+};
+
+#define VGIC_V2_MAX_LRS         (1 << 6)
+#define VGIC_V3_MAX_LRS         16
+#define VGIC_V3_LR_INDEX(lr)    (VGIC_V3_MAX_LRS - 1 - lr)
+
+#define VGIC_CONFIG_EDGE        false
+#define VGIC_CONFIG_LEVEL       true
+
+struct vgic_irq {
+    struct list_head ap_list;
+
+    struct vcpu *vcpu;          /*
+                                 * SGIs and PPIs: The VCPU
+                                 * SPIs and LPIs: The VCPU whose ap_list
+                                 * this is queued on.
+                                 */
+
+    struct vcpu *target_vcpu;   /*
+                                 * The VCPU that this interrupt should
+                                 * be sent to, as a result of the
+                                 * targets reg (v2) or the affinity reg (v3).
+                                 */
+
+    spinlock_t irq_lock;        /* Protects the content of the struct */
+    uint32_t intid;             /* Guest visible INTID */
+    atomic_t refcount;          /* Used for LPIs */
+    uint32_t hwintid;           /* HW INTID number */
+    union
+    {
+        struct {
+            uint8_t targets;    /* GICv2 target VCPUs mask */
+            uint8_t source;     /* GICv2 SGIs only */
+        };
+        uint32_t mpidr;         /* GICv3 target VCPU */
+    };
+    uint8_t priority;
+    bool line_level:1;          /* Level only */
+    bool pending_latch:1;       /*
+                                 * The pending latch state used to
+                                 * calculate the pending state for both
+                                 * level and edge triggered IRQs.
+                                 */
+    bool active:1;              /* not used for LPIs */
+    bool enabled:1;
+    bool hw:1;                  /* Tied to HW IRQ */
+    bool config:1;              /* Level or edge */
+    struct list_head lpi_list;  /* Used to link all LPIs together */
+};
+
+enum iodev_type {
+    IODEV_DIST,
+    IODEV_REDIST,
+};
+
+struct vgic_io_device {
+    gfn_t base_fn;
+    struct vcpu *redist_vcpu;
+    const struct vgic_register_region *regions;
+    enum iodev_type iodev_type;
+    unsigned int nr_regions;
+};
+
+struct vgic_dist {
+    bool                ready;
+    bool                initialized;
+
+    /* vGIC model the kernel emulates for the guest (GICv2 or GICv3) */
+    uint32_t            version;
+
+    /* Do injected MSIs require an additional device ID? */
+    bool                msis_require_devid;
+
+    unsigned int        nr_spis;
+
+    /* base addresses in guest physical address space: */
+    paddr_t             vgic_dist_base;     /* distributor */
+    union
+    {
+        /* either a GICv2 CPU interface */
+        paddr_t         vgic_cpu_base;
+        /* or a number of GICv3 redistributor regions */
+        struct
+        {
+            paddr_t     vgic_redist_base;
+            paddr_t     vgic_redist_free_offset;
+        };
+    };
+
+    /* distributor enabled */
+    bool                enabled;
+
+    struct vgic_irq     *spis;
+    unsigned long       *allocated_irqs; /* bitmap of IRQs allocated */
+
+    struct vgic_io_device   dist_iodev;
+
+    bool                has_its;
+
+    /*
+     * Contains the attributes and gpa of the LPI configuration table.
+     * Since we report GICR_TYPER.CommonLPIAff as 0b00, we can share
+     * one address across all redistributors.
+     * GICv3 spec: 6.1.2 "LPI Configuration tables"
+     */
+    uint64_t            propbaser;
+
+    /* Protects the lpi_list and the count value below. */
+    spinlock_t          lpi_list_lock;
+    struct list_head    lpi_list_head;
+    unsigned int        lpi_list_count;
+};
+
+struct vgic_cpu {
+    struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
+
+    struct list_head ap_list_head;
+    spinlock_t ap_list_lock;    /* Protects the ap_list */
+
+    unsigned int used_lrs;
+
+    /*
+     * List of IRQs that this VCPU should consider because they are either
+     * Active or Pending (hence the name; AP list), or because they recently
+     * were one of the two and need to be migrated off this list to another
+     * VCPU.
+     */
+
+    /*
+     * Members below are used with GICv3 emulation only and represent
+     * parts of the redistributor.
+     */
+    struct vgic_io_device   rd_iodev;
+    struct vgic_io_device   sgi_iodev;
+
+    /* Contains the attributes and gpa of the LPI pending tables. */
+    uint64_t pendbaser;
+
+    bool lpis_enabled;
+
+    /* Cache guest priority bits */
+    uint32_t num_pri_bits;
+
+    /* Cache guest interrupt ID bits */
+    uint32_t num_id_bits;
+};
+
+#endif /* __ASM_ARM_NEW_VGIC_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 0787ba9549..2a58ea30fe 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -18,6 +18,10 @@
 #ifndef __ASM_ARM_VGIC_H__
 #define __ASM_ARM_VGIC_H__
 
+#ifdef CONFIG_NEW_VGIC
+#include <asm/new_vgic.h>
+#else
+
 #include <xen/bitops.h>
 #include <xen/radix-tree.h>
 #include <xen/rbtree.h>
@@ -299,6 +303,8 @@ extern bool vgic_to_sgi(struct vcpu *v, register_t sgir,
                         const struct sgi_target *target);
 extern bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq);
 
+#endif /* !CONFIG_NEW_VGIC */
+
 /*** Common VGIC functions used by Xen arch code ****/
 
 /*
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 15/45] ARM: new VGIC: Add acccessor to new struct vgic_irq instance

[Andre Przywara]

The new VGIC implementation centers around a struct vgic_irq instance
per virtual IRQ.
Provide a function to retrieve the right instance for a given IRQ
number and (in case of private interrupts) the right VCPU.
This also includes the corresponding put function, which does nothing
for private interrupts and SPIs, but handles the ref-counting for LPIs.

This is based on Linux commit 64a959d66e47, written by Christoffer Dall.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- reorder header file inclusion

 xen/arch/arm/vgic/vgic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.h |  41 ++++++++++++++++
 2 files changed, 165 insertions(+)
 create mode 100644 xen/arch/arm/vgic/vgic.c
 create mode 100644 xen/arch/arm/vgic/vgic.h

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
new file mode 100644
index 0000000000..d9d285c361
--- /dev/null
+++ b/xen/arch/arm/vgic/vgic.c
@@ -0,0 +1,124 @@
+/*
+ * Copyright (C) 2015, 2016 ARM Ltd.
+ * Imported from Linux ("new" KVM VGIC) and heavily adapted to Xen.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/sched.h>
+#include <asm/bug.h>
+#include <asm/new_vgic.h>
+
+#include "vgic.h"
+
+/*
+ * Iterate over the VM's list of mapped LPIs to find the one with a
+ * matching interrupt ID and return a reference to the IRQ structure.
+ */
+static struct vgic_irq *vgic_get_lpi(struct domain *d, u32 intid)
+{
+    struct vgic_dist *dist = &d->arch.vgic;
+    struct vgic_irq *irq = NULL;
+
+    spin_lock(&dist->lpi_list_lock);
+
+    list_for_each_entry( irq, &dist->lpi_list_head, lpi_list )
+    {
+        if ( irq->intid != intid )
+            continue;
+
+        /*
+         * This increases the refcount, the caller is expected to
+         * call vgic_put_irq() later once it's finished with the IRQ.
+         */
+        vgic_get_irq_kref(irq);
+        goto out_unlock;
+    }
+    irq = NULL;
+
+out_unlock:
+    spin_unlock(&dist->lpi_list_lock);
+
+    return irq;
+}
+
+/**
+ * vgic_get_irq() - obtain a reference to a virtual IRQ
+ * @d:        The domain the virtual IRQ belongs to.
+ * @vcpu:     For private IRQs (SGIs, PPIs) the virtual CPU this IRQ
+ *            is associated with. Will be ignored for SPIs and LPIs.
+ * @intid:    The virtual IRQ number.
+ *
+ * This looks up the virtual interrupt ID to get the corresponding
+ * struct vgic_irq. It also increases the refcount, so any caller is expected
+ * to call vgic_put_irq() once it's finished with this IRQ.
+ *
+ * Return: The pointer to the requested struct vgic_irq.
+ */
+struct vgic_irq *vgic_get_irq(struct domain *d, struct vcpu *vcpu,
+                              u32 intid)
+{
+    /* SGIs and PPIs */
+    if ( intid <= VGIC_MAX_PRIVATE )
+        return &vcpu->arch.vgic.private_irqs[intid];
+
+    /* SPIs */
+    if ( intid <= VGIC_MAX_SPI )
+        return &d->arch.vgic.spis[intid - VGIC_NR_PRIVATE_IRQS];
+
+    /* LPIs */
+    if ( intid >= VGIC_MIN_LPI )
+        return vgic_get_lpi(d, intid);
+
+    ASSERT_UNREACHABLE();
+}
+
+/**
+ * vgic_put_irq() - drop the reference to a virtual IRQ
+ * @d:        The domain the virtual IRQ belongs to.
+ * @irq:      The pointer to struct vgic_irq, as obtained from vgic_get_irq().
+ *
+ * This drops the reference to a virtual IRQ. It decreases the refcount
+ * of the pointer, so dynamic IRQs can be freed when no longer needed.
+ * This should always be called after a vgic_get_irq(), though the reference
+ * can be deliberately held for longer periods, if needed.
+ */
+void vgic_put_irq(struct domain *d, struct vgic_irq *irq)
+{
+    struct vgic_dist *dist = &d->arch.vgic;
+
+    if ( irq->intid < VGIC_MIN_LPI )
+        return;
+
+    spin_lock(&dist->lpi_list_lock);
+    if ( !atomic_dec_and_test(&irq->refcount) )
+    {
+        spin_unlock(&dist->lpi_list_lock);
+        return;
+    };
+
+    list_del(&irq->lpi_list);
+    dist->lpi_list_count--;
+    spin_unlock(&dist->lpi_list_lock);
+
+    xfree(irq);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
new file mode 100644
index 0000000000..a3befd386b
--- /dev/null
+++ b/xen/arch/arm/vgic/vgic.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright (C) 2015, 2016 ARM Ltd.
+ * Imported from Linux ("new" KVM VGIC) and heavily adapted to Xen.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __XEN_ARM_VGIC_VGIC_H__
+#define __XEN_ARM_VGIC_VGIC_H__
+
+struct vgic_irq *vgic_get_irq(struct domain *d, struct vcpu *vcpu,
+                              u32 intid);
+void vgic_put_irq(struct domain *d, struct vgic_irq *irq);
+
+static inline void vgic_get_irq_kref(struct vgic_irq *irq)
+{
+    if ( irq->intid < VGIC_MIN_LPI )
+        return;
+
+    atomic_inc(&irq->refcount);
+}
+
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 16/45] ARM: new VGIC: Implement virtual IRQ injection

[Andre Przywara]

Provide a vgic_queue_irq_unlock() function which decides whether a
given IRQ needs to be queued to a VCPU's ap_list.
This should be called whenever an IRQ becomes pending or enabled,
either as a result of a hardware IRQ injection, from devices emulated by
Xen (like the architected timer) or from MMIO accesses to the distributor
emulation.
Also provides the necessary functions to allow to inject an IRQ to a guest.
Since this is the first code that starts using our locking mechanism,
we add some (hopefully) clear documentation of our locking strategy and
requirements along with this patch.

This is based on Linux commit 81eeb95ddbab, written by Christoffer Dall.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- rework validate_injection()
- add comments
- make vgic_inject_irq a void function
- fix comment

 xen/arch/arm/vgic/vgic.c | 226 +++++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.h |  10 +++
 2 files changed, 236 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index d9d285c361..20d48ac6f5 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -17,10 +17,36 @@
 
 #include <xen/sched.h>
 #include <asm/bug.h>
+#include <asm/event.h>
 #include <asm/new_vgic.h>
 
 #include "vgic.h"
 
+/*
+ * Locking order is always:
+ *   vgic->lock
+ *     vgic_cpu->ap_list_lock
+ *       vgic->lpi_list_lock
+ *         desc->lock
+ *           vgic_irq->irq_lock
+ *
+ * If you need to take multiple locks, always take the upper lock first,
+ * then the lower ones, e.g. first take the ap_list_lock, then the irq_lock.
+ * If you are already holding a lock and need to take a higher one, you
+ * have to drop the lower ranking lock first and re-acquire it after having
+ * taken the upper one.
+ *
+ * When taking more than one ap_list_lock at the same time, always take the
+ * lowest numbered VCPU's ap_list_lock first, so:
+ *   vcpuX->vcpu_id < vcpuY->vcpu_id:
+ *     spin_lock(vcpuX->arch.vgic.ap_list_lock);
+ *     spin_lock(vcpuY->arch.vgic.ap_list_lock);
+ *
+ * Since the VGIC must support injecting virtual interrupts from ISRs, we have
+ * to use the spin_lock_irqsave/spin_unlock_irqrestore versions of outer
+ * spinlocks for any lock that may be taken while injecting an interrupt.
+ */
+
 /*
  * Iterate over the VM's list of mapped LPIs to find the one with a
  * matching interrupt ID and return a reference to the IRQ structure.
@@ -114,6 +140,206 @@ void vgic_put_irq(struct domain *d, struct vgic_irq *irq)
     xfree(irq);
 }
 
+/**
+ * vgic_target_oracle() - compute the target vcpu for an irq
+ * @irq:    The irq to route. Must be already locked.
+ *
+ * Based on the current state of the interrupt (enabled, pending,
+ * active, vcpu and target_vcpu), compute the next vcpu this should be
+ * given to. Return NULL if this shouldn't be injected at all.
+ *
+ * Requires the IRQ lock to be held.
+ *
+ * Returns: The pointer to the virtual CPU this interrupt should be injected
+ *          to. Will be NULL if this IRQ does not need to be injected.
+ */
+static struct vcpu *vgic_target_oracle(struct vgic_irq *irq)
+{
+    ASSERT(spin_is_locked(&irq->irq_lock));
+
+    /* If the interrupt is active, it must stay on the current vcpu */
+    if ( irq->active )
+        return irq->vcpu ? : irq->target_vcpu;
+
+    /*
+     * If the IRQ is not active but enabled and pending, we should direct
+     * it to its configured target VCPU.
+     * If the distributor is disabled, pending interrupts shouldn't be
+     * forwarded.
+     */
+    if ( irq->enabled && irq_is_pending(irq) )
+    {
+        if ( unlikely(irq->target_vcpu &&
+                      !irq->target_vcpu->domain->arch.vgic.enabled) )
+            return NULL;
+
+        return irq->target_vcpu;
+    }
+
+    /*
+     * If neither active nor pending and enabled, then this IRQ should not
+     * be queued to any VCPU.
+     */
+    return NULL;
+}
+
+/*
+ * Only valid injection if changing level for level-triggered IRQs or for a
+ * rising edge.
+ */
+static bool vgic_validate_injection(struct vgic_irq *irq, bool level)
+{
+    /* For edge interrupts we only care about a rising edge. */
+    if ( irq->config == VGIC_CONFIG_EDGE )
+        return level;
+
+    /* For level interrupts we have to act when the line level changes. */
+    return irq->line_level != level;
+}
+
+/**
+ * vgic_queue_irq_unlock() - Queue an IRQ to a VCPU, to be injected to a guest.
+ * @d:        The domain the virtual IRQ belongs to.
+ * @irq:      A pointer to the vgic_irq of the virtual IRQ, with the lock held.
+ * @flags:    The flags used when having grabbed the IRQ lock.
+ *
+ * Check whether an IRQ needs to (and can) be queued to a VCPU's ap list.
+ * Do the queuing if necessary, taking the right locks in the right order.
+ *
+ * Needs to be entered with the IRQ lock already held, but will return
+ * with all locks dropped.
+ */
+void vgic_queue_irq_unlock(struct domain *d, struct vgic_irq *irq,
+                           unsigned long flags)
+{
+    struct vcpu *vcpu;
+
+    ASSERT(spin_is_locked(&irq->irq_lock));
+
+retry:
+    vcpu = vgic_target_oracle(irq);
+    if ( irq->vcpu || !vcpu )
+    {
+        /*
+         * If this IRQ is already on a VCPU's ap_list, then it
+         * cannot be moved or modified and there is no more work for
+         * us to do.
+         *
+         * Otherwise, if the irq is not pending and enabled, it does
+         * not need to be inserted into an ap_list and there is also
+         * no more work for us to do.
+         */
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+        /*
+         * We have to kick the VCPU here, because we could be
+         * queueing an edge-triggered interrupt for which we
+         * get no EOI maintenance interrupt. In that case,
+         * while the IRQ is already on the VCPU's AP list, the
+         * VCPU could have EOI'ed the original interrupt and
+         * won't see this one until it exits for some other
+         * reason.
+         */
+        if ( vcpu )
+            vcpu_kick(vcpu);
+
+        return;
+    }
+
+    /*
+     * We must unlock the irq lock to take the ap_list_lock where
+     * we are going to insert this new pending interrupt.
+     */
+    spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+    /* someone can do stuff here, which we re-check below */
+
+    spin_lock_irqsave(&vcpu->arch.vgic.ap_list_lock, flags);
+    spin_lock(&irq->irq_lock);
+
+    /*
+     * Did something change behind our backs?
+     *
+     * There are two cases:
+     * 1) The irq lost its pending state or was disabled behind our
+     *    backs and/or it was queued to another VCPU's ap_list.
+     * 2) Someone changed the affinity on this irq behind our
+     *    backs and we are now holding the wrong ap_list_lock.
+     *
+     * In both cases, drop the locks and retry.
+     */
+
+    if ( unlikely(irq->vcpu || vcpu != vgic_target_oracle(irq)) )
+    {
+        spin_unlock(&irq->irq_lock);
+        spin_unlock_irqrestore(&vcpu->arch.vgic.ap_list_lock, flags);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+        goto retry;
+    }
+
+    /*
+     * Grab a reference to the irq to reflect the fact that it is
+     * now in the ap_list.
+     */
+    vgic_get_irq_kref(irq);
+    list_add_tail(&irq->ap_list, &vcpu->arch.vgic.ap_list_head);
+    irq->vcpu = vcpu;
+
+    spin_unlock(&irq->irq_lock);
+    spin_unlock_irqrestore(&vcpu->arch.vgic.ap_list_lock, flags);
+
+    vcpu_kick(vcpu);
+
+    return;
+}
+
+/**
+ * vgic_inject_irq() - Inject an IRQ from a device to the vgic
+ * @d:       The domain pointer
+ * @vcpu:    The vCPU for private IRQs (PPIs, SGIs). Ignored for SPIs and LPIs.
+ * @intid:   The INTID to inject a new state to.
+ * @level:   Edge-triggered:  true:  to trigger the interrupt
+ *                            false: to ignore the call
+ *           Level-sensitive  true:  raise the input signal
+ *                            false: lower the input signal
+ *
+ * Injects an instance of the given virtual IRQ into a domain.
+ * The VGIC is not concerned with devices being active-LOW or active-HIGH for
+ * level-sensitive interrupts.  You can think of the level parameter as 1
+ * being HIGH and 0 being LOW and all devices being active-HIGH.
+ */
+void vgic_inject_irq(struct domain *d, struct vcpu *vcpu, unsigned int intid,
+                     bool level)
+{
+    struct vgic_irq *irq;
+    unsigned long flags;
+
+    irq = vgic_get_irq(d, vcpu, intid);
+    if ( !irq )
+        return;
+
+    spin_lock_irqsave(&irq->irq_lock, flags);
+
+    if ( !vgic_validate_injection(irq, level) )
+    {
+        /* Nothing to see here, move along... */
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+        vgic_put_irq(d, irq);
+        return;
+    }
+
+    if ( irq->config == VGIC_CONFIG_LEVEL )
+        irq->line_level = level;
+    else
+        irq->pending_latch = true;
+
+    vgic_queue_irq_unlock(d, irq, flags);
+    vgic_put_irq(d, irq);
+
+    return;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index a3befd386b..f9e2eeb2d6 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -17,9 +17,19 @@
 #ifndef __XEN_ARM_VGIC_VGIC_H__
 #define __XEN_ARM_VGIC_VGIC_H__
 
+static inline bool irq_is_pending(struct vgic_irq *irq)
+{
+    if ( irq->config == VGIC_CONFIG_EDGE )
+        return irq->pending_latch;
+    else
+        return irq->pending_latch || irq->line_level;
+}
+
 struct vgic_irq *vgic_get_irq(struct domain *d, struct vcpu *vcpu,
                               u32 intid);
 void vgic_put_irq(struct domain *d, struct vgic_irq *irq);
+void vgic_queue_irq_unlock(struct domain *d, struct vgic_irq *irq,
+                           unsigned long flags);
 
 static inline void vgic_get_irq_kref(struct vgic_irq *irq)
 {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 17/45] Add list_sort() routine from Linux

[Andre Przywara]

This pulls in Linux' list_sort.c, which is a merge sort implementation
for linked lists. Apart from adding a full featured license header and
adjusting the #include file, nothing has been changed in this code.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- split out to just contain Linux code dump

 xen/common/list_sort.c      | 157 ++++++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/list_sort.h |  11 ++++
 2 files changed, 168 insertions(+)
 create mode 100644 xen/common/list_sort.c
 create mode 100644 xen/include/xen/list_sort.h

diff --git a/xen/common/list_sort.c b/xen/common/list_sort.c
new file mode 100644
index 0000000000..af2b2f6519
--- /dev/null
+++ b/xen/common/list_sort.c
@@ -0,0 +1,157 @@
+/*
+ * list_sort.c: merge sort implementation for linked lists
+ * Copied from the Linux kernel (lib/list_sort.c)
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/lib.h>
+#include <xen/list.h>
+
+#define MAX_LIST_LENGTH_BITS 20
+
+/*
+ * Returns a list organized in an intermediate format suited
+ * to chaining of merge() calls: null-terminated, no reserved or
+ * sentinel head node, "prev" links not maintained.
+ */
+static struct list_head *merge(void *priv,
+				int (*cmp)(void *priv, struct list_head *a,
+					struct list_head *b),
+				struct list_head *a, struct list_head *b)
+{
+	struct list_head head, *tail = &head;
+
+	while (a && b) {
+		/* if equal, take 'a' -- important for sort stability */
+		if ((*cmp)(priv, a, b) <= 0) {
+			tail->next = a;
+			a = a->next;
+		} else {
+			tail->next = b;
+			b = b->next;
+		}
+		tail = tail->next;
+	}
+	tail->next = a?:b;
+	return head.next;
+}
+
+/*
+ * Combine final list merge with restoration of standard doubly-linked
+ * list structure.  This approach duplicates code from merge(), but
+ * runs faster than the tidier alternatives of either a separate final
+ * prev-link restoration pass, or maintaining the prev links
+ * throughout.
+ */
+static void merge_and_restore_back_links(void *priv,
+				int (*cmp)(void *priv, struct list_head *a,
+					struct list_head *b),
+				struct list_head *head,
+				struct list_head *a, struct list_head *b)
+{
+	struct list_head *tail = head;
+	u8 count = 0;
+
+	while (a && b) {
+		/* if equal, take 'a' -- important for sort stability */
+		if ((*cmp)(priv, a, b) <= 0) {
+			tail->next = a;
+			a->prev = tail;
+			a = a->next;
+		} else {
+			tail->next = b;
+			b->prev = tail;
+			b = b->next;
+		}
+		tail = tail->next;
+	}
+	tail->next = a ? : b;
+
+	do {
+		/*
+		 * In worst cases this loop may run many iterations.
+		 * Continue callbacks to the client even though no
+		 * element comparison is needed, so the client's cmp()
+		 * routine can invoke cond_resched() periodically.
+		 */
+		if (unlikely(!(++count)))
+			(*cmp)(priv, tail->next, tail->next);
+
+		tail->next->prev = tail;
+		tail = tail->next;
+	} while (tail->next);
+
+	tail->next = head;
+	head->prev = tail;
+}
+
+/**
+ * list_sort - sort a list
+ * @priv: private data, opaque to list_sort(), passed to @cmp
+ * @head: the list to sort
+ * @cmp: the elements comparison function
+ *
+ * This function implements "merge sort", which has O(nlog(n))
+ * complexity.
+ *
+ * The comparison function @cmp must return a negative value if @a
+ * should sort before @b, and a positive value if @a should sort after
+ * @b. If @a and @b are equivalent, and their original relative
+ * ordering is to be preserved, @cmp must return 0.
+ */
+void list_sort(void *priv, struct list_head *head,
+		int (*cmp)(void *priv, struct list_head *a,
+			struct list_head *b))
+{
+	struct list_head *part[MAX_LIST_LENGTH_BITS+1]; /* sorted partial lists
+						-- last slot is a sentinel */
+	int lev;  /* index into part[] */
+	int max_lev = 0;
+	struct list_head *list;
+
+	if (list_empty(head))
+		return;
+
+	memset(part, 0, sizeof(part));
+
+	head->prev->next = NULL;
+	list = head->next;
+
+	while (list) {
+		struct list_head *cur = list;
+		list = list->next;
+		cur->next = NULL;
+
+		for (lev = 0; part[lev]; lev++) {
+			cur = merge(priv, cmp, part[lev], cur);
+			part[lev] = NULL;
+		}
+		if (lev > max_lev) {
+			if (unlikely(lev >= ARRAY_SIZE(part)-1)) {
+				dprintk(XENLOG_DEBUG,
+					"list too long for efficiency\n");
+				lev--;
+			}
+			max_lev = lev;
+		}
+		part[lev] = cur;
+	}
+
+	for (lev = 0; lev < max_lev; lev++)
+		if (part[lev])
+			list = merge(priv, cmp, part[lev], list);
+
+	merge_and_restore_back_links(priv, cmp, head, part[max_lev], list);
+}
+EXPORT_SYMBOL(list_sort);
diff --git a/xen/include/xen/list_sort.h b/xen/include/xen/list_sort.h
new file mode 100644
index 0000000000..13ce0a55ec
--- /dev/null
+++ b/xen/include/xen/list_sort.h
@@ -0,0 +1,11 @@
+#ifndef _LINUX_LIST_SORT_H
+#define _LINUX_LIST_SORT_H
+
+#include <xen/types.h>
+
+struct list_head;
+
+void list_sort(void *priv, struct list_head *head,
+	       int (*cmp)(void *priv, struct list_head *a,
+			  struct list_head *b));
+#endif
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 18/45] ARM: new VGIC: Add IRQ sorting

[Andre Przywara]

Adds the sorting function to cover the case where you have more IRQs
to consider than you have LRs. We consider their priorities.
This uses the new sort_list() implementation imported from Linux.

This is based on Linux commit 8e4447457965, written by Christoffer Dall.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- split off Linux' list_sort() implementation

 xen/arch/arm/vgic/vgic.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 20d48ac6f5..7306a80dd3 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -15,6 +15,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <xen/list_sort.h>
 #include <xen/sched.h>
 #include <asm/bug.h>
 #include <asm/event.h>
@@ -183,6 +184,64 @@ static struct vcpu *vgic_target_oracle(struct vgic_irq *irq)
     return NULL;
 }
 
+/*
+ * The order of items in the ap_lists defines how we'll pack things in LRs as
+ * well, the first items in the list being the first things populated in the
+ * LRs.
+ *
+ * A hard rule is that active interrupts can never be pushed out of the LRs
+ * (and therefore take priority) since we cannot reliably trap on deactivation
+ * of IRQs and therefore they have to be present in the LRs.
+ *
+ * Otherwise things should be sorted by the priority field and the GIC
+ * hardware support will take care of preemption of priority groups etc.
+ *
+ * Return negative if "a" sorts before "b", 0 to preserve order, and positive
+ * to sort "b" before "a".
+ */
+static int vgic_irq_cmp(void *priv, struct list_head *a, struct list_head *b)
+{
+    struct vgic_irq *irqa = container_of(a, struct vgic_irq, ap_list);
+    struct vgic_irq *irqb = container_of(b, struct vgic_irq, ap_list);
+    bool penda, pendb;
+    int ret;
+
+    spin_lock(&irqa->irq_lock);
+    spin_lock(&irqb->irq_lock);
+
+    if ( irqa->active || irqb->active )
+    {
+        ret = (int)irqb->active - (int)irqa->active;
+        goto out;
+    }
+
+    penda = irqa->enabled && irq_is_pending(irqa);
+    pendb = irqb->enabled && irq_is_pending(irqb);
+
+    if ( !penda || !pendb )
+    {
+        ret = (int)pendb - (int)penda;
+        goto out;
+    }
+
+    /* Both pending and enabled, sort by priority */
+    ret = irqa->priority - irqb->priority;
+out:
+    spin_unlock(&irqb->irq_lock);
+    spin_unlock(&irqa->irq_lock);
+    return ret;
+}
+
+/* Must be called with the ap_list_lock held */
+static void vgic_sort_ap_list(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+
+    ASSERT(spin_is_locked(&vgic_cpu->ap_list_lock));
+
+    list_sort(NULL, &vgic_cpu->ap_list_head, vgic_irq_cmp);
+}
+
 /*
  * Only valid injection if changing level for level-triggered IRQs or for a
  * rising edge.
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 19/45] ARM: new VGIC: Add IRQ sync/flush framework

[Andre Przywara]

Implement the framework for syncing IRQs between our emulation and the
list registers, which represent the guest's view of IRQs.
This is done in vgic_sync_from_lrs() and vgic_sync_to_lrs(), which
get called on guest entry and exit, respectively.
The code talking to the actual GICv2/v3 hardware is added in the
following patches.

This is based on Linux commit 0919e84c0fc1, written by Marc Zyngier.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- make functions void
- do underflow setting directly (no v2/v3 indirection)
- fix multiple SGIs injections (as the late Linux bugfix)

 xen/arch/arm/vgic/vgic.c | 232 +++++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.h |   2 +
 2 files changed, 234 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 7306a80dd3..e82d498766 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -399,6 +399,238 @@ void vgic_inject_irq(struct domain *d, struct vcpu *vcpu, unsigned int intid,
     return;
 }
 
+/**
+ * vgic_prune_ap_list() - Remove non-relevant interrupts from the ap_list
+ *
+ * @vcpu:       The VCPU of which the ap_list should be pruned.
+ *
+ * Go over the list of interrupts on a VCPU's ap_list, and prune those that
+ * we won't have to consider in the near future.
+ * This removes interrupts that have been successfully handled by the guest,
+ * or that have otherwise became obsolete (not pending anymore).
+ * Also this moves interrupts between VCPUs, if their affinity has changed.
+ */
+static void vgic_prune_ap_list(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+    struct vgic_irq *irq, *tmp;
+    unsigned long flags;
+
+retry:
+    spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);
+
+    list_for_each_entry_safe( irq, tmp, &vgic_cpu->ap_list_head, ap_list )
+    {
+        struct vcpu *target_vcpu, *vcpuA, *vcpuB;
+
+        spin_lock(&irq->irq_lock);
+
+        BUG_ON(vcpu != irq->vcpu);
+
+        target_vcpu = vgic_target_oracle(irq);
+
+        if ( !target_vcpu )
+        {
+            /*
+             * We don't need to process this interrupt any
+             * further, move it off the list.
+             */
+            list_del(&irq->ap_list);
+            irq->vcpu = NULL;
+            spin_unlock(&irq->irq_lock);
+
+            /*
+             * This vgic_put_irq call matches the
+             * vgic_get_irq_kref in vgic_queue_irq_unlock,
+             * where we added the LPI to the ap_list. As
+             * we remove the irq from the list, we drop
+             * also drop the refcount.
+             */
+            vgic_put_irq(vcpu->domain, irq);
+            continue;
+        }
+
+        if ( target_vcpu == vcpu )
+        {
+            /* We're on the right CPU */
+            spin_unlock(&irq->irq_lock);
+            continue;
+        }
+
+        /* This interrupt looks like it has to be migrated. */
+
+        spin_unlock(&irq->irq_lock);
+        spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);
+
+        /*
+         * Ensure locking order by always locking the smallest
+         * ID first.
+         */
+        if ( vcpu->vcpu_id < target_vcpu->vcpu_id )
+        {
+            vcpuA = vcpu;
+            vcpuB = target_vcpu;
+        }
+        else
+        {
+            vcpuA = target_vcpu;
+            vcpuB = vcpu;
+        }
+
+        spin_lock_irqsave(&vcpuA->arch.vgic.ap_list_lock, flags);
+        spin_lock(&vcpuB->arch.vgic.ap_list_lock);
+        spin_lock(&irq->irq_lock);
+
+        /*
+         * If the affinity has been preserved, move the
+         * interrupt around. Otherwise, it means things have
+         * changed while the interrupt was unlocked, and we
+         * need to replay this.
+         *
+         * In all cases, we cannot trust the list not to have
+         * changed, so we restart from the beginning.
+         */
+        if ( target_vcpu == vgic_target_oracle(irq) )
+        {
+            struct vgic_cpu *new_cpu = &target_vcpu->arch.vgic;
+
+            list_del(&irq->ap_list);
+            irq->vcpu = target_vcpu;
+            list_add_tail(&irq->ap_list, &new_cpu->ap_list_head);
+        }
+
+        spin_unlock(&irq->irq_lock);
+        spin_unlock(&vcpuB->arch.vgic.ap_list_lock);
+        spin_unlock_irqrestore(&vcpuA->arch.vgic.ap_list_lock, flags);
+        goto retry;
+    }
+
+    spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);
+}
+
+static void vgic_fold_lr_state(struct vcpu *vcpu)
+{
+}
+
+/* Requires the irq_lock to be held. */
+static void vgic_populate_lr(struct vcpu *vcpu,
+                             struct vgic_irq *irq, int lr)
+{
+    ASSERT(spin_is_locked(&irq->irq_lock));
+}
+
+static void vgic_set_underflow(struct vcpu *vcpu)
+{
+    ASSERT(vcpu == current);
+
+    gic_hw_ops->update_hcr_status(GICH_HCR_UIE, 1);
+}
+
+/* Requires the ap_list_lock to be held. */
+static int compute_ap_list_depth(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+    struct vgic_irq *irq;
+    int count = 0;
+
+    ASSERT(spin_is_locked(&vgic_cpu->ap_list_lock));
+
+    list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list)
+    {
+        spin_lock(&irq->irq_lock);
+        /* GICv2 SGIs can count for more than one... */
+        if ( vgic_irq_is_sgi(irq->intid) && irq->source )
+            count += hweight8(irq->source);
+        else
+            count++;
+        spin_unlock(&irq->irq_lock);
+    }
+    return count;
+}
+
+/* Requires the VCPU's ap_list_lock to be held. */
+static void vgic_flush_lr_state(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+    struct vgic_irq *irq;
+    int count = 0;
+
+    ASSERT(spin_is_locked(&vgic_cpu->ap_list_lock));
+
+    if ( compute_ap_list_depth(vcpu) > gic_get_nr_lrs() )
+        vgic_sort_ap_list(vcpu);
+
+    list_for_each_entry( irq, &vgic_cpu->ap_list_head, ap_list )
+    {
+        spin_lock(&irq->irq_lock);
+
+        if ( likely(vgic_target_oracle(irq) == vcpu) )
+            vgic_populate_lr(vcpu, irq, count++);
+
+        spin_unlock(&irq->irq_lock);
+
+        if ( count == gic_get_nr_lrs() )
+        {
+            if ( !list_is_last(&irq->ap_list, &vgic_cpu->ap_list_head) )
+                vgic_set_underflow(vcpu);
+            break;
+        }
+    }
+
+    vcpu->arch.vgic.used_lrs = count;
+}
+
+/**
+ * vgic_sync_from_lrs() - Update VGIC state from hardware after a guest's run.
+ * @vcpu: the VCPU for which to transfer from the LRs to the IRQ list.
+ *
+ * Sync back the hardware VGIC state after the guest has run, into our
+ * VGIC emulation structures, It reads the LRs and updates the respective
+ * struct vgic_irq, taking level/edge into account.
+ * This is the high level function which takes care of the conditions,
+ * also bails out early if there were no interrupts queued.
+ * Was: kvm_vgic_sync_hwstate()
+ */
+void vgic_sync_from_lrs(struct vcpu *vcpu)
+{
+    /* An empty ap_list_head implies used_lrs == 0 */
+    if ( list_empty(&vcpu->arch.vgic.ap_list_head) )
+        return;
+
+    vgic_fold_lr_state(vcpu);
+
+    vgic_prune_ap_list(vcpu);
+}
+
+/**
+ * vgic_sync_to_lrs() - flush emulation state into the hardware on guest entry
+ *
+ * Before we enter a guest, we have to translate the virtual GIC state of a
+ * VCPU into the GIC virtualization hardware registers, namely the LRs.
+ * This is the high level function which takes care about the conditions
+ * and the locking, also bails out early if there are no interrupts queued.
+ * Was: kvm_vgic_flush_hwstate()
+ */
+void vgic_sync_to_lrs(void)
+{
+    /*
+     * If there are no virtual interrupts active or pending for this
+     * VCPU, then there is no work to do and we can bail out without
+     * taking any lock.  There is a potential race with someone injecting
+     * interrupts to the VCPU, but it is a benign race as the VCPU will
+     * either observe the new interrupt before or after doing this check,
+     * and introducing additional synchronization mechanism doesn't change
+     * this.
+     */
+    if ( list_empty(&current->arch.vgic.ap_list_head) )
+        return;
+
+    ASSERT(!local_irq_is_enabled());
+
+    spin_lock(&current->arch.vgic.ap_list_lock);
+    vgic_flush_lr_state(current);
+    spin_unlock(&current->arch.vgic.ap_list_lock);
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index f9e2eeb2d6..f530cfa078 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -17,6 +17,8 @@
 #ifndef __XEN_ARM_VGIC_VGIC_H__
 #define __XEN_ARM_VGIC_VGIC_H__
 
+#define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
+
 static inline bool irq_is_pending(struct vgic_irq *irq)
 {
     if ( irq->config == VGIC_CONFIG_EDGE )
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 20/45] ARM: new VGIC: Add GICv2 world switch backend

[Andre Przywara]

Processing maintenance interrupts and accessing the list registers
are dependent on the host's GIC version.
Introduce vgic-v2.c to contain GICv2 specific functions.
Implement the GICv2 specific code for syncing the emulation state
into the VGIC registers.
This also adds the hook to let Xen setup the host GIC addresses.

This is based on Linux commit 140b086dd197, written by Marc Zyngier.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- remove v2 specific underflow function (now generic)
- re-add Linux code to properly handle acked level IRQs

 xen/arch/arm/vgic/vgic-v2.c | 229 ++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.c    |   6 ++
 xen/arch/arm/vgic/vgic.h    |   9 ++
 3 files changed, 244 insertions(+)
 create mode 100644 xen/arch/arm/vgic/vgic-v2.c

diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
new file mode 100644
index 0000000000..bc5c7895f4
--- /dev/null
+++ b/xen/arch/arm/vgic/vgic-v2.c
@@ -0,0 +1,229 @@
+/*
+ * Copyright (C) 2015, 2016 ARM Ltd.
+ * Imported from Linux ("new" KVM VGIC) and heavily adapted to Xen.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/new_vgic.h>
+#include <asm/bug.h>
+#include <asm/gic.h>
+#include <asm/io.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+
+#include "vgic.h"
+
+static struct {
+    bool enabled;
+    paddr_t dbase;          /* Distributor interface address */
+    paddr_t cbase;          /* CPU interface address & size */
+    paddr_t csize;
+    paddr_t vbase;          /* Virtual CPU interface address */
+
+    /* Offset to add to get an 8kB contiguous region if GIC is aliased */
+    uint32_t aliased_offset;
+} gic_v2_hw_data;
+
+void vgic_v2_setup_hw(paddr_t dbase, paddr_t cbase, paddr_t csize,
+                      paddr_t vbase, uint32_t aliased_offset)
+{
+    gic_v2_hw_data.enabled = true;
+    gic_v2_hw_data.dbase = dbase;
+    gic_v2_hw_data.cbase = cbase;
+    gic_v2_hw_data.csize = csize;
+    gic_v2_hw_data.vbase = vbase;
+    gic_v2_hw_data.aliased_offset = aliased_offset;
+}
+
+/*
+ * transfer the content of the LRs back into the corresponding ap_list:
+ * - active bit is transferred as is
+ * - pending bit is
+ *   - transferred as is in case of edge sensitive IRQs
+ *   - set to the line-level (resample time) for level sensitive IRQs
+ */
+void vgic_v2_fold_lr_state(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+    unsigned int used_lrs = vcpu->arch.vgic.used_lrs;
+    unsigned long flags;
+    unsigned int lr;
+
+    if ( !used_lrs )    /* No LRs used, so nothing to sync back here. */
+        return;
+
+    gic_hw_ops->update_hcr_status(GICH_HCR_UIE, 0);
+
+    for ( lr = 0; lr < used_lrs; lr++ )
+    {
+        struct gic_lr lr_val;
+        uint32_t intid;
+        struct vgic_irq *irq;
+
+        gic_hw_ops->read_lr(lr, &lr_val);
+
+        /*
+         * TODO: Possible optimization to avoid reading LRs:
+         * Read the ELRSR to find out which of our LRs have been cleared
+         * by the guest. We just need to know the IRQ number for those, which
+         * we could save in an array when populating the LRs.
+         * This trades one MMIO access (ELRSR) for possibly more than one (LRs),
+         * but requires some more code to save the IRQ number and to handle
+         * those finished IRQs according to the algorithm below.
+         * We need some numbers to justify this: chances are that we don't
+         * have many LRs in use most of the time, so we might not save much.
+         */
+        gic_hw_ops->clear_lr(lr);
+
+        intid = lr_val.virq;
+        irq = vgic_get_irq(vcpu->domain, vcpu, intid);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        /* Always preserve the active bit */
+        irq->active = lr_val.active;
+
+        /* Edge is the only case where we preserve the pending bit */
+        if ( irq->config == VGIC_CONFIG_EDGE && lr_val.pending )
+        {
+            irq->pending_latch = true;
+
+            if ( vgic_irq_is_sgi(intid) )
+                irq->source |= (1U << lr_val.virt.source);
+        }
+
+        /* Clear soft pending state when level irqs have been acked. */
+        if ( irq->config == VGIC_CONFIG_LEVEL && !lr_val.pending )
+                irq->pending_latch = false;
+
+	/*
+	 * Level-triggered mapped IRQs are special because we only
+	 * observe rising edges as input to the VGIC.
+	 *
+	 * If the guest never acked the interrupt we have to sample
+	 * the physical line and set the line level, because the
+	 * device state could have changed or we simply need to
+	 * process the still pending interrupt later.
+	 *
+	 * If this causes us to lower the level, we have to also clear
+	 * the physical active state, since we will otherwise never be
+	 * told when the interrupt becomes asserted again.
+	 */
+        if ( vgic_irq_is_mapped_level(irq) && lr_val.pending )
+        {
+            struct irq_desc *irqd;
+
+            ASSERT(irq->hwintid >= VGIC_NR_PRIVATE_IRQS);
+
+            irqd = irq_to_desc(irq->hwintid);
+            irq->line_level = gic_read_pending_state(irqd);
+
+            if ( !irq->line_level )
+                gic_set_active_state(irqd, false);
+        }
+
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    gic_hw_ops->update_hcr_status(GICH_HCR_EN, 0);
+    vgic_cpu->used_lrs = 0;
+}
+
+/**
+ * vgic_v2_populate_lr() - Populates an LR with the state of a given IRQ.
+ * @vcpu: The VCPU which the given @irq belongs to.
+ * @irq:  The IRQ to convert into an LR. The irq_lock must be held already.
+ * @lr:   The LR number to transfer the state into.
+ *
+ * This moves a virtual IRQ, represented by its vgic_irq, into a list register.
+ * Apart from translating the logical state into the LR bitfields, it also
+ * changes some state in the vgic_irq.
+ * For an edge sensitive IRQ the pending state is cleared in struct vgic_irq,
+ * for a level sensitive IRQ the pending state value is unchanged, as it is
+ * dictated directly by the input line level.
+ *
+ * If @irq describes an SGI with multiple sources, we choose the
+ * lowest-numbered source VCPU and clear that bit in the source bitmap.
+ *
+ * The irq_lock must be held by the caller.
+ */
+void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr)
+{
+    struct gic_lr lr_val = {0};
+
+    lr_val.virq = irq->intid;
+
+    if ( irq_is_pending(irq) )
+    {
+        lr_val.pending = true;
+
+        if ( irq->config == VGIC_CONFIG_EDGE )
+            irq->pending_latch = false;
+
+        if ( vgic_irq_is_sgi(irq->intid) )
+        {
+            u32 src = ffs(irq->source);
+
+            BUG_ON(!src);
+            lr_val.virt.source = (src - 1);
+            irq->source &= ~(1 << (src - 1));
+            if ( irq->source )
+                irq->pending_latch = true;
+        }
+    }
+
+    lr_val.active = irq->active;
+
+    if ( irq->hw )
+    {
+        lr_val.hw_status = 1;
+        lr_val.hw.pirq = irq->hwintid;
+        /*
+         * Never set pending+active on a HW interrupt, as the
+         * pending state is kept at the physical distributor
+         * level.
+         */
+        if ( irq->active && irq_is_pending(irq) )
+            lr_val.pending = false;
+    }
+    else
+    {
+        if ( irq->config == VGIC_CONFIG_LEVEL )
+            lr_val.virt.eoi = 1;
+    }
+
+    /*
+     * Level-triggered mapped IRQs are special because we only observe
+     * rising edges as input to the VGIC.  We therefore lower the line
+     * level here, so that we can take new virtual IRQs.  See
+     * vgic_v2_fold_lr_state for more info.
+     */
+    if ( vgic_irq_is_mapped_level(irq) && lr_val.pending )
+        irq->line_level = false;
+
+    /* The GICv2 LR only holds five bits of priority. */
+    lr_val.priority = irq->priority >> 3;
+
+    gic_hw_ops->write_lr(lr, &lr_val);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index e82d498766..035c8c8f74 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -510,6 +510,7 @@ retry:
 
 static void vgic_fold_lr_state(struct vcpu *vcpu)
 {
+    vgic_v2_fold_lr_state(vcpu);
 }
 
 /* Requires the irq_lock to be held. */
@@ -517,6 +518,8 @@ static void vgic_populate_lr(struct vcpu *vcpu,
                              struct vgic_irq *irq, int lr)
 {
     ASSERT(spin_is_locked(&irq->irq_lock));
+
+    vgic_v2_populate_lr(vcpu, irq, lr);
 }
 
 static void vgic_set_underflow(struct vcpu *vcpu)
@@ -630,7 +633,10 @@ void vgic_sync_to_lrs(void)
     spin_lock(&current->arch.vgic.ap_list_lock);
     vgic_flush_lr_state(current);
     spin_unlock(&current->arch.vgic.ap_list_lock);
+
+    gic_hw_ops->update_hcr_status(GICH_HCR_EN, 1);
 }
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index f530cfa078..41cc0c5b54 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -27,6 +27,11 @@ static inline bool irq_is_pending(struct vgic_irq *irq)
         return irq->pending_latch || irq->line_level;
 }
 
+static inline bool vgic_irq_is_mapped_level(struct vgic_irq *irq)
+{
+    return irq->config == VGIC_CONFIG_LEVEL && irq->hw;
+}
+
 struct vgic_irq *vgic_get_irq(struct domain *d, struct vcpu *vcpu,
                               u32 intid);
 void vgic_put_irq(struct domain *d, struct vgic_irq *irq);
@@ -41,6 +46,10 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
     atomic_inc(&irq->refcount);
 }
 
+void vgic_v2_fold_lr_state(struct vcpu *vcpu);
+void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr);
+void vgic_v2_set_underflow(struct vcpu *vcpu);
+
 #endif
 
 /*
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 21/45] ARM: new VGIC: Implement vgic_vcpu_pending_irq

[Andre Przywara]

Tell Xen whether a particular VCPU has an IRQ that needs handling
in the guest. This is used to decide whether a VCPU is runnable or
if a hypercall should be preempted to let the guest handle the IRQ.

This is based on Linux commit 90eee56c5f90, written by Eric Auger.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- adjust to new vgic_vcpu_pending_irq() prototype, drop wrapper

 xen/arch/arm/vgic/vgic.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 035c8c8f74..a1556e53e8 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -637,6 +637,43 @@ void vgic_sync_to_lrs(void)
     gic_hw_ops->update_hcr_status(GICH_HCR_EN, 1);
 }
 
+/**
+ * vgic_vcpu_pending_irq() - determine if interrupts need to be injected
+ * @vcpu: The vCPU on which to check for interrupts.
+ *
+ * Checks whether there is an interrupt on the given VCPU which needs
+ * handling in the guest. This requires at least one IRQ to be pending
+ * and enabled.
+ *
+ * Returns: 1 if the guest should run to handle interrupts, 0 otherwise.
+ */
+int vgic_vcpu_pending_irq(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+    struct vgic_irq *irq;
+    bool pending = false;
+    unsigned long flags;
+
+    if ( !vcpu->domain->arch.vgic.enabled )
+        return false;
+
+    spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);
+
+    list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list)
+    {
+        spin_lock(&irq->irq_lock);
+        pending = irq_is_pending(irq) && irq->enabled;
+        spin_unlock(&irq->irq_lock);
+
+        if ( pending )
+            break;
+    }
+
+    spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);
+
+    return pending;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 22/45] ARM: new VGIC: Add MMIO handling framework

[Andre Przywara]

Add an MMIO handling framework to the VGIC emulation:
Each register is described by its offset, size (or number of bits per
IRQ, if applicable) and the read/write handler functions. We provide
initialization macros to describe each GIC register later easily.

Separate dispatch functions for read and write accesses are connected
to Xen's MMIO handling framework and binary-search for the responsible
register handler based on the offset address within the region.

The register handler prototype are courtesy of Christoffer Dall.

This is based on Linux commit 4493b1c4866a, written by Marc Zyngier.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- adjust indentation
- remove unneeded macros
- use unsigned types
- use new ilog2() implementation

 xen/arch/arm/vgic/vgic-mmio.c | 180 ++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h |  89 +++++++++++++++++++++
 2 files changed, 269 insertions(+)
 create mode 100644 xen/arch/arm/vgic/vgic-mmio.c
 create mode 100644 xen/arch/arm/vgic/vgic-mmio.h

diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
new file mode 100644
index 0000000000..866023a84d
--- /dev/null
+++ b/xen/arch/arm/vgic/vgic-mmio.c
@@ -0,0 +1,180 @@
+/*
+ * VGIC MMIO handling functions
+ * Imported from Linux ("new" KVM VGIC) and heavily adapted to Xen.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/bitops.h>
+#include <xen/lib.h>
+#include <xen/sched.h>
+#include <asm/new_vgic.h>
+#include <asm/byteorder.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+
+unsigned long vgic_mmio_read_raz(struct vcpu *vcpu,
+                                 paddr_t addr, unsigned int len)
+{
+    return 0;
+}
+
+unsigned long vgic_mmio_read_rao(struct vcpu *vcpu,
+                                 paddr_t addr, unsigned int len)
+{
+    return -1UL;
+}
+
+void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
+                        unsigned int len, unsigned long val)
+{
+    /* Ignore */
+}
+
+static int match_region(const void *key, const void *elt)
+{
+    const unsigned int offset = (unsigned long)key;
+    const struct vgic_register_region *region = elt;
+
+    if ( offset < region->reg_offset )
+        return -1;
+
+    if ( offset >= region->reg_offset + region->len )
+        return 1;
+
+    return 0;
+}
+
+static const struct vgic_register_region *
+vgic_find_mmio_region(const struct vgic_register_region *regions,
+                      int nr_regions, unsigned int offset)
+{
+    return bsearch((void *)(uintptr_t)offset, regions, nr_regions,
+                   sizeof(regions[0]), match_region);
+}
+
+static bool check_region(const struct domain *d,
+                         const struct vgic_register_region *region,
+                         paddr_t addr, int len)
+{
+    unsigned int flags, nr_irqs = d->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
+
+    switch ( len )
+    {
+    case sizeof(uint8_t):
+        flags = VGIC_ACCESS_8bit;
+        break;
+    case sizeof(uint32_t):
+        flags = VGIC_ACCESS_32bit;
+        break;
+    case sizeof(uint64_t):
+        flags = VGIC_ACCESS_64bit;
+        break;
+    default:
+        return false;
+    }
+
+    if ( (region->access_flags & flags) && IS_ALIGNED(addr, len) )
+    {
+        if ( !region->bits_per_irq )
+            return true;
+
+        /* Do we access a non-allocated IRQ? */
+        return VGIC_ADDR_TO_INTID(addr, region->bits_per_irq) < nr_irqs;
+    }
+
+    return false;
+}
+
+static const struct vgic_register_region *
+vgic_get_mmio_region(struct vcpu *vcpu, struct vgic_io_device *iodev,
+                     paddr_t addr, unsigned int len)
+{
+    const struct vgic_register_region *region;
+
+    region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
+                                   addr - gfn_to_gaddr(iodev->base_fn));
+    if ( !region || !check_region(vcpu->domain, region, addr, len) )
+        return NULL;
+
+    return region;
+}
+
+static int dispatch_mmio_read(struct vcpu *vcpu, mmio_info_t *info,
+                              register_t *r, void *priv)
+{
+    struct vgic_io_device *iodev = priv;
+    const struct vgic_register_region *region;
+    unsigned long data = 0;
+    paddr_t addr = info->gpa;
+    int len = 1U << info->dabt.size;
+
+    region = vgic_get_mmio_region(vcpu, iodev, addr, len);
+    if ( !region )
+    {
+        memset(r, 0, len);
+        return 0;
+    }
+
+    switch (iodev->iodev_type)
+    {
+    case IODEV_DIST:
+        data = region->read(vcpu, addr, len);
+        break;
+    case IODEV_REDIST:
+        data = region->read(iodev->redist_vcpu, addr, len);
+        break;
+    }
+
+    memcpy(r, &data, len);
+
+    return 1;
+}
+
+static int dispatch_mmio_write(struct vcpu *vcpu, mmio_info_t *info,
+                               register_t r, void *priv)
+{
+    struct vgic_io_device *iodev = priv;
+    const struct vgic_register_region *region;
+    unsigned long data = r;
+    paddr_t addr = info->gpa;
+    int len = 1U << info->dabt.size;
+
+    region = vgic_get_mmio_region(vcpu, iodev, addr, len);
+    if ( !region )
+        return 0;
+
+    switch (iodev->iodev_type)
+    {
+    case IODEV_DIST:
+        region->write(vcpu, addr, len, data);
+        break;
+    case IODEV_REDIST:
+        region->write(iodev->redist_vcpu, addr, len, data);
+        break;
+    }
+
+    return 1;
+}
+
+struct mmio_handler_ops vgic_io_ops = {
+    .read = dispatch_mmio_read,
+    .write = dispatch_mmio_write,
+};
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
new file mode 100644
index 0000000000..bf062a27ca
--- /dev/null
+++ b/xen/arch/arm/vgic/vgic-mmio.h
@@ -0,0 +1,89 @@
+/*
+ * Copyright (C) 2015, 2016 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __XEN_ARM_VGIC_VGIC_MMIO_H__
+#define __XEN_ARM_VGIC_VGIC_MMIO_H__
+
+struct vgic_register_region {
+    unsigned int reg_offset;
+    unsigned int len;
+    unsigned int bits_per_irq;
+    unsigned int access_flags;
+    unsigned long (*read)(struct vcpu *vcpu, paddr_t addr,
+                          unsigned int len);
+    void (*write)(struct vcpu *vcpu, paddr_t addr,
+                  unsigned int len, unsigned long val);
+};
+
+extern struct mmio_handler_ops vgic_io_ops;
+
+#define VGIC_ACCESS_8bit    1
+#define VGIC_ACCESS_32bit   2
+#define VGIC_ACCESS_64bit   4
+
+/*
+ * Generate a mask that covers the number of bytes required to address
+ * up to 1024 interrupts, each represented by <bits> bits. This assumes
+ * that <bits> is a power of two.
+ */
+#define VGIC_ADDR_IRQ_MASK(bits) (((bits) * 1024 / 8) - 1)
+
+/*
+ * (addr & mask) gives us the _byte_ offset for the INT ID.
+ * We multiply this by 8 the get the _bit_ offset, then divide this by
+ * the number of bits to learn the actual INT ID.
+ * But instead of a division (which requires a "long long div" implementation),
+ * we shift by the binary logarithm of <bits>.
+ * This assumes that <bits> is a power of two.
+ */
+#define VGIC_ADDR_TO_INTID(addr, bits)  (((addr) & VGIC_ADDR_IRQ_MASK(bits)) * \
+                                         8 >> ilog2(bits))
+
+/*
+ * Some VGIC registers store per-IRQ information, with a different number
+ * of bits per IRQ. For those registers this macro is used.
+ * The _WITH_LENGTH version instantiates registers with a fixed length
+ * and is mutually exclusive with the _PER_IRQ version.
+ */
+#define REGISTER_DESC_WITH_BITS_PER_IRQ(off, rd, wr, bpi, acc)  \
+    {                                                           \
+        .reg_offset = off,                                      \
+        .bits_per_irq = bpi,                                    \
+        .len = bpi * 1024 / 8,                                  \
+        .access_flags = acc,                                    \
+        .read = rd,                                             \
+        .write = wr,                                            \
+    }
+
+#define REGISTER_DESC_WITH_LENGTH(off, rd, wr, length, acc)     \
+    {                                                           \
+        .reg_offset = off,                                      \
+        .bits_per_irq = 0,                                      \
+        .len = length,                                          \
+        .access_flags = acc,                                    \
+        .read = rd,                                             \
+        .write = wr,                                            \
+    }
+
+unsigned long vgic_mmio_read_raz(struct vcpu *vcpu,
+                                 paddr_t addr, unsigned int len);
+
+unsigned long vgic_mmio_read_rao(struct vcpu *vcpu,
+                                 paddr_t addr, unsigned int len);
+
+void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
+                        unsigned int len, unsigned long val);
+
+#endif
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 23/45] ARM: new VGIC: Add GICv2 MMIO handling framework

[Andre Przywara]

Create vgic-mmio-v2.c to describe GICv2 emulation specific handlers
using the initializer macros provided by the VGIC MMIO framework.
Provide a function to register the GICv2 distributor registers to
the Xen MMIO framework.
The actual handler functions are still stubs in this patch.

This is based on Linux commit fb848db39661, written by Andre Przywara.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Reviewed-by:

 xen/arch/arm/vgic/vgic-mmio-v2.c | 83 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.c    | 25 ++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h    |  2 +
 xen/arch/arm/vgic/vgic.h         |  2 +
 4 files changed, 112 insertions(+)
 create mode 100644 xen/arch/arm/vgic/vgic-mmio-v2.c

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
new file mode 100644
index 0000000000..6f10cf16ca
--- /dev/null
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -0,0 +1,83 @@
+/*
+ * VGICv2 MMIO handling functions
+ * Imported from Linux ("new" KVM VGIC) and heavily adapted to Xen.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/bitops.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+#include <asm/new_vgic.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+
+static const struct vgic_register_region vgic_v2_dist_registers[] = {
+    REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 12,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_IGROUPR,
+        vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISENABLER,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICENABLER,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISPENDR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICPENDR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISACTIVER,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICACTIVER,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_IPRIORITYR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+        VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ITARGETSR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+        VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+    REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICFGR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 2,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_LENGTH(GICD_SGIR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
+        VGIC_ACCESS_32bit),
+    REGISTER_DESC_WITH_LENGTH(GICD_CPENDSGIR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 16,
+        VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+    REGISTER_DESC_WITH_LENGTH(GICD_SPENDSGIR,
+        vgic_mmio_read_raz, vgic_mmio_write_wi, 16,
+        VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+};
+
+unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev)
+{
+    dev->regions = vgic_v2_dist_registers;
+    dev->nr_regions = ARRAY_SIZE(vgic_v2_dist_registers);
+
+    return SZ_4K;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
index 866023a84d..a03e8d88b9 100644
--- a/xen/arch/arm/vgic/vgic-mmio.c
+++ b/xen/arch/arm/vgic/vgic-mmio.c
@@ -170,6 +170,31 @@ struct mmio_handler_ops vgic_io_ops = {
     .write = dispatch_mmio_write,
 };
 
+int vgic_register_dist_iodev(struct domain *d, gfn_t dist_base_fn,
+                             enum vgic_type type)
+{
+    struct vgic_io_device *io_device = &d->arch.vgic.dist_iodev;
+    unsigned int len;
+
+    switch ( type )
+    {
+    case VGIC_V2:
+        len = vgic_v2_init_dist_iodev(io_device);
+        break;
+    default:
+        BUG();
+    }
+
+    io_device->base_fn = dist_base_fn;
+    io_device->iodev_type = IODEV_DIST;
+    io_device->redist_vcpu = NULL;
+
+    register_mmio_handler(d, &vgic_io_ops, gfn_to_gaddr(dist_base_fn), len,
+                          io_device);
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
index bf062a27ca..c280668694 100644
--- a/xen/arch/arm/vgic/vgic-mmio.h
+++ b/xen/arch/arm/vgic/vgic-mmio.h
@@ -86,4 +86,6 @@ unsigned long vgic_mmio_read_rao(struct vcpu *vcpu,
 void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
                         unsigned int len, unsigned long val);
 
+unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
+
 #endif
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index 41cc0c5b54..7f221fd195 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -49,6 +49,8 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
 void vgic_v2_fold_lr_state(struct vcpu *vcpu);
 void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr);
 void vgic_v2_set_underflow(struct vcpu *vcpu);
+int vgic_register_dist_iodev(struct domain *d, gfn_t dist_base_fn,
+                             enum vgic_type);
 
 #endif
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 24/45] ARM: new VGIC: Add CTLR, TYPER and IIDR handlers

[Andre Przywara]

Those three registers are v2 emulation specific, so their implementation
lives entirely in vgic-mmio-v2.c. Also they are handled in one function,
as their implementation is pretty simple.
We choose to piggy-back on the existing KVM identification registers,
but use a different variant (major revision).
When the guest enables the distributor, we kick all VCPUs to get
potentially pending interrupts serviced.

This is based on Linux commit 2b0cda878965, written by Marc Zyngier.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- use new IIDR values (KVM product ID, Xen revision)
- add comment on handling GICD enablement
- use new vcpu_kick() function

 xen/arch/arm/vgic/vgic-mmio-v2.c | 63 +++++++++++++++++++++++++++++++++++++++-
 xen/arch/arm/vgic/vgic.c         | 15 ++++++++++
 xen/arch/arm/vgic/vgic.h         |  5 ++++
 3 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index 6f10cf16ca..43c1ab5906 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -20,9 +20,70 @@
 #include "vgic.h"
 #include "vgic-mmio.h"
 
+static unsigned long vgic_mmio_read_v2_misc(struct vcpu *vcpu,
+                                            paddr_t addr, unsigned int len)
+{
+    uint32_t value;
+
+    switch ( addr & 0x0c )      /* filter for the 4 registers handled here */
+    {
+    case GICD_CTLR:
+        value = vcpu->domain->arch.vgic.enabled ? GICD_CTL_ENABLE : 0;
+        break;
+    case GICD_TYPER:
+        value = vcpu->domain->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
+        value = (value >> 5) - 1;
+        value |= (vcpu->domain->max_vcpus - 1) << 5;
+        break;
+    case GICD_IIDR:
+        value = (PRODUCT_ID_KVM << 24) |
+                (VARIANT_ID_XEN << 16) |
+                (IMPLEMENTER_ARM << 0);
+        break;
+    default:
+        return 0;
+    }
+
+    return value;
+}
+
+static void vgic_mmio_write_v2_misc(struct vcpu *vcpu,
+                                    paddr_t addr, unsigned int len,
+                                    unsigned long val)
+{
+    struct vgic_dist *dist = &vcpu->domain->arch.vgic;
+    bool enabled;
+
+    switch ( addr & 0x0c )      /* filter for the 4 registers handled here */
+    {
+    case GICD_CTLR:
+        domain_lock(vcpu->domain);
+
+        /*
+         * Store the new enabled state in our distributor structure.
+         * Work out whether it was disabled before and now got enabled,
+         * so that we signal all VCPUs to check for interrupts to be injected.
+         */
+        enabled = dist->enabled;
+        dist->enabled = val & GICD_CTL_ENABLE;
+        enabled = !enabled && dist->enabled;
+
+        domain_unlock(vcpu->domain);
+
+        if ( enabled )
+            vgic_kick_vcpus(vcpu->domain);
+
+        break;
+    case GICD_TYPER:
+    case GICD_IIDR:
+        /* read-only, writes ignored */
+        return;
+    }
+}
+
 static const struct vgic_register_region vgic_v2_dist_registers[] = {
     REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 12,
+        vgic_mmio_read_v2_misc, vgic_mmio_write_v2_misc, 12,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_IGROUPR,
         vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index a1556e53e8..a3aaf20a73 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -674,6 +674,21 @@ int vgic_vcpu_pending_irq(struct vcpu *vcpu)
     return pending;
 }
 
+void vgic_kick_vcpus(struct domain *d)
+{
+    struct vcpu *vcpu;
+
+    /*
+     * We've injected an interrupt, time to find out who deserves
+     * a good kick...
+     */
+    for_each_vcpu( d, vcpu )
+    {
+        if ( vgic_vcpu_pending_irq(vcpu) )
+            vcpu_kick(vcpu);
+    }
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index 7f221fd195..e2d0f04440 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -17,6 +17,10 @@
 #ifndef __XEN_ARM_VGIC_VGIC_H__
 #define __XEN_ARM_VGIC_VGIC_H__
 
+#define PRODUCT_ID_KVM          0x4b        /* ASCII code X */
+#define VARIANT_ID_XEN          0x01
+#define IMPLEMENTER_ARM         0x43b
+
 #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
 
 static inline bool irq_is_pending(struct vgic_irq *irq)
@@ -37,6 +41,7 @@ struct vgic_irq *vgic_get_irq(struct domain *d, struct vcpu *vcpu,
 void vgic_put_irq(struct domain *d, struct vgic_irq *irq);
 void vgic_queue_irq_unlock(struct domain *d, struct vgic_irq *irq,
                            unsigned long flags);
+void vgic_kick_vcpus(struct domain *d);
 
 static inline void vgic_get_irq_kref(struct vgic_irq *irq)
 {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 25/45] ARM: new VGIC: Add ENABLE registers handlers

[Andre Przywara]

As the enable register handlers are shared between the v2 and v3
emulation, their implementation goes into vgic-mmio.c, to be easily
referenced from the v3 emulation as well later.
This introduces a vgic_sync_hardware_irq() function, which updates the
physical side of a hardware mapped virtual IRQ.
Because the existing locking order between vgic_irq->irq_lock and
irq_desc->lock dictates so, we drop the irq_lock and retake them in the
proper order.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- ASSERT on h/w IRQ and vIRQ staying in sync

 xen/arch/arm/vgic/vgic-mmio-v2.c |   4 +-
 xen/arch/arm/vgic/vgic-mmio.c    | 117 +++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h    |  11 ++++
 xen/arch/arm/vgic/vgic.c         |  40 +++++++++++++
 xen/arch/arm/vgic/vgic.h         |   3 +
 5 files changed, 173 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index 43c1ab5906..7efd1c4eb4 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -89,10 +89,10 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISENABLER,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICENABLER,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISPENDR,
         vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
index a03e8d88b9..99e1adb1ea 100644
--- a/xen/arch/arm/vgic/vgic-mmio.c
+++ b/xen/arch/arm/vgic/vgic-mmio.c
@@ -39,6 +39,123 @@ void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
     /* Ignore */
 }
 
+/*
+ * Read accesses to both GICD_ICENABLER and GICD_ISENABLER return the value
+ * of the enabled bit, so there is only one function for both here.
+ */
+unsigned long vgic_mmio_read_enable(struct vcpu *vcpu,
+                                    paddr_t addr, unsigned int len)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    uint32_t value = 0;
+    unsigned int i;
+
+    /* Loop over all IRQs affected by this read */
+    for ( i = 0; i < len * 8; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        if ( irq->enabled )
+            value |= (1U << i);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    return value;
+}
+
+void vgic_mmio_write_senable(struct vcpu *vcpu,
+                             paddr_t addr, unsigned int len,
+                             unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    unsigned int i;
+
+    for_each_set_bit( i, &val, len * 8 )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+        unsigned long flags;
+        irq_desc_t *desc;
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        if ( irq->enabled )            /* skip already enabled IRQs */
+        {
+            spin_unlock_irqrestore(&irq->irq_lock, flags);
+            vgic_put_irq(vcpu->domain, irq);
+            continue;
+        }
+
+        irq->enabled = true;
+        if ( irq->hw )
+        {
+            /*
+             * The irq cannot be a PPI, we only support delivery
+             * of SPIs to guests.
+             */
+            ASSERT(irq->hwintid >= VGIC_NR_PRIVATE_IRQS);
+
+            desc = irq_to_desc(irq->hwintid);
+        }
+        else
+            desc = NULL;
+
+        vgic_queue_irq_unlock(vcpu->domain, irq, flags);
+
+        if ( desc )
+            vgic_sync_hardware_irq(vcpu->domain, desc, irq);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
+void vgic_mmio_write_cenable(struct vcpu *vcpu,
+                 paddr_t addr, unsigned int len,
+                 unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    unsigned int i;
+
+    for_each_set_bit( i, &val, len * 8 )
+    {
+        struct vgic_irq *irq;
+        unsigned long flags;
+        irq_desc_t *desc;
+
+        irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        if ( !irq->enabled )            /* skip already disabled IRQs */
+        {
+            spin_unlock_irqrestore(&irq->irq_lock, flags);
+            vgic_put_irq(vcpu->domain, irq);
+            continue;
+        }
+
+        irq->enabled = false;
+
+        if ( irq->hw )
+        {
+            /*
+             * The irq cannot be a PPI, we only support delivery
+             * of SPIs to guests.
+             */
+            ASSERT(irq->hwintid >= VGIC_NR_PRIVATE_IRQS);
+
+            desc = irq_to_desc(irq->hwintid);
+        }
+        else
+            desc = NULL;
+
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+        if ( desc )
+            vgic_sync_hardware_irq(vcpu->domain, desc, irq);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
 static int match_region(const void *key, const void *elt)
 {
     const unsigned int offset = (unsigned long)key;
diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
index c280668694..a2cebd77f4 100644
--- a/xen/arch/arm/vgic/vgic-mmio.h
+++ b/xen/arch/arm/vgic/vgic-mmio.h
@@ -86,6 +86,17 @@ unsigned long vgic_mmio_read_rao(struct vcpu *vcpu,
 void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
                         unsigned int len, unsigned long val);
 
+unsigned long vgic_mmio_read_enable(struct vcpu *vcpu,
+                                    paddr_t addr, unsigned int len);
+
+void vgic_mmio_write_senable(struct vcpu *vcpu,
+                             paddr_t addr, unsigned int len,
+                             unsigned long val);
+
+void vgic_mmio_write_cenable(struct vcpu *vcpu,
+                             paddr_t addr, unsigned int len,
+                             unsigned long val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 #endif
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index a3aaf20a73..963d5e5f6e 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -689,6 +689,46 @@ void vgic_kick_vcpus(struct domain *d)
     }
 }
 
+static unsigned int translate_irq_type(bool is_level)
+{
+    return is_level ? IRQ_TYPE_LEVEL_HIGH : IRQ_TYPE_EDGE_RISING;
+}
+
+void vgic_sync_hardware_irq(struct domain *d,
+                            irq_desc_t *desc, struct vgic_irq *irq)
+{
+    unsigned long flags;
+
+    spin_lock_irqsave(&desc->lock, flags);
+    spin_lock(&irq->irq_lock);
+
+    /*
+     * We now forbid tinkering with the hardware IRQ association
+     * during a domain's lifetime.
+     */
+    ASSERT(irq->hw && desc->irq == irq->hwintid);
+
+    if ( irq->enabled )
+    {
+        /*
+         * We might end up from various callers, so check that the
+         * interrrupt is disabled before trying to change the config.
+         */
+        if ( irq_type_set_by_domain(d) &&
+             test_bit(_IRQ_DISABLED, &desc->status) )
+            gic_set_irq_type(desc, translate_irq_type(irq->config));
+
+        if ( irq->target_vcpu )
+            irq_set_affinity(desc, cpumask_of(irq->target_vcpu->processor));
+        desc->handler->enable(desc);
+    }
+    else
+        desc->handler->disable(desc);
+
+    spin_unlock(&irq->irq_lock);
+    spin_unlock_irqrestore(&desc->lock, flags);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index e2d0f04440..d135c26c80 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -51,6 +51,9 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
     atomic_inc(&irq->refcount);
 }
 
+void vgic_sync_hardware_irq(struct domain *d,
+                            irq_desc_t *desc, struct vgic_irq *irq);
+
 void vgic_v2_fold_lr_state(struct vcpu *vcpu);
 void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr);
 void vgic_v2_set_underflow(struct vcpu *vcpu);
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 26/45] ARM: new VGIC: Add PENDING registers handlers

[Andre Przywara]

The pending register handlers are shared between the v2 and v3
emulation, so their implementation goes into vgic-mmio.c, to be easily
referenced from the v3 emulation as well later.
For level triggered interrupts the real line level is unaffected by
this write, so we keep this state separate and combine it with the
device's level to get the actual pending state.
Hardware mapped IRQs need some special handling, as their hardware state
has to be coordinated with the virtual pending bit to avoid hanging
or masked interrupts.

This is based on Linux commit 96b298000db4, written by Andre Przywara.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- ASSERT on h/w IRQ and vIRQ staying in sync

 xen/arch/arm/vgic/vgic-mmio-v2.c |   4 +-
 xen/arch/arm/vgic/vgic-mmio.c    | 125 +++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h    |  11 ++++
 3 files changed, 138 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index 7efd1c4eb4..a48c554040 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -95,10 +95,10 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISPENDR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICPENDR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISACTIVER,
         vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
index 99e1adb1ea..15183c112c 100644
--- a/xen/arch/arm/vgic/vgic-mmio.c
+++ b/xen/arch/arm/vgic/vgic-mmio.c
@@ -156,6 +156,131 @@ void vgic_mmio_write_cenable(struct vcpu *vcpu,
     }
 }
 
+unsigned long vgic_mmio_read_pending(struct vcpu *vcpu,
+                                     paddr_t addr, unsigned int len)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    uint32_t value = 0;
+    unsigned int i;
+
+    /* Loop over all IRQs affected by this read */
+    for ( i = 0; i < len * 8; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        if ( irq_is_pending(irq) )
+            value |= (1U << i);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    return value;
+}
+
+void vgic_mmio_write_spending(struct vcpu *vcpu,
+                              paddr_t addr, unsigned int len,
+                              unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    unsigned int i;
+    unsigned long flags;
+    irq_desc_t *desc;
+
+    for_each_set_bit( i, &val, len * 8 )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+        irq->pending_latch = true;
+
+        /* To observe the locking order, just take the irq_desc pointer here. */
+        if ( irq->hw )
+            desc = irq_to_desc(irq->hwintid);
+        else
+            desc = NULL;
+
+        vgic_queue_irq_unlock(vcpu->domain, irq, flags);
+
+        /*
+         * When the VM sets the pending state for a HW interrupt on the virtual
+         * distributor we set the active state on the physical distributor,
+         * because the virtual interrupt can become active and then the guest
+         * can deactivate it.
+         */
+        if ( desc )
+        {
+            spin_lock_irqsave(&desc->lock, flags);
+            spin_lock(&irq->irq_lock);
+
+            /* This h/w IRQ should still be assigned to the virtual IRQ. */
+            ASSERT(irq->hw && desc->irq == irq->hwintid);
+
+            gic_set_active_state(desc, true);
+
+            spin_unlock(&irq->irq_lock);
+            spin_unlock_irqrestore(&desc->lock, flags);
+        }
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
+void vgic_mmio_write_cpending(struct vcpu *vcpu,
+                              paddr_t addr, unsigned int len,
+                              unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    unsigned int i;
+    unsigned long flags;
+    irq_desc_t *desc;
+
+    for_each_set_bit( i, &val, len * 8 )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+        irq->pending_latch = false;
+
+        /* To observe the locking order, just take the irq_desc pointer here. */
+        if ( irq->hw )
+            desc = irq_to_desc(irq->hwintid);
+        else
+            desc = NULL;
+
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+        /*
+         * We don't want the guest to effectively mask the physical
+         * interrupt by doing a write to SPENDR followed by a write to
+         * CPENDR for HW interrupts, so we clear the active state on
+         * the physical side if the virtual interrupt is not active.
+         * This may lead to taking an additional interrupt on the
+         * host, but that should not be a problem as the worst that
+         * can happen is an additional vgic injection.  We also clear
+         * the pending state to maintain proper semantics for edge HW
+         * interrupts.
+         */
+        if ( desc )
+        {
+            spin_lock_irqsave(&desc->lock, flags);
+            spin_lock(&irq->irq_lock);
+
+            /* This h/w IRQ should still be assigned to the virtual IRQ. */
+            ASSERT(irq->hw && desc->irq == irq->hwintid);
+
+            gic_set_pending_state(desc, false);
+            if (!irq->active)
+                gic_set_active_state(desc, false);
+
+            spin_unlock(&irq->irq_lock);
+            spin_unlock_irqrestore(&desc->lock, flags);
+        }
+
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
 static int match_region(const void *key, const void *elt)
 {
     const unsigned int offset = (unsigned long)key;
diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
index a2cebd77f4..5c927f28b0 100644
--- a/xen/arch/arm/vgic/vgic-mmio.h
+++ b/xen/arch/arm/vgic/vgic-mmio.h
@@ -97,6 +97,17 @@ void vgic_mmio_write_cenable(struct vcpu *vcpu,
                              paddr_t addr, unsigned int len,
                              unsigned long val);
 
+unsigned long vgic_mmio_read_pending(struct vcpu *vcpu,
+                                     paddr_t addr, unsigned int len);
+
+void vgic_mmio_write_spending(struct vcpu *vcpu,
+                              paddr_t addr, unsigned int len,
+                              unsigned long val);
+
+void vgic_mmio_write_cpending(struct vcpu *vcpu,
+                              paddr_t addr, unsigned int len,
+                              unsigned long val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 #endif
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 27/45] ARM: new VGIC: Add ACTIVE registers handlers

[Andre Przywara]

The active register handlers are shared between the v2 and v3 emulation,
so their implementation goes into vgic-mmio.c, to be easily referenced
from the v3 emulation as well later.
Since activation/deactivation of an interrupt may happen entirely in the
guest without it ever exiting, we need some extra logic to properly track
the active state.
For clearing the active state, we would basically have to halt the guest
to make sure this is properly propagated into the respective VCPUs.
This is not yet implemented in Xen.
Fortunately this feature is mostly used to reset a just in initialised
GIC, so chances are we are tasked to clear bits that are already zero.
Add a simple check to avoid pointless warnings in this case.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- print a line for every failing IRQ

 xen/arch/arm/vgic/vgic-mmio-v2.c |  4 +-
 xen/arch/arm/vgic/vgic-mmio.c    | 91 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h    | 11 +++++
 3 files changed, 104 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index a48c554040..724681e0f8 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -101,10 +101,10 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ISACTIVER,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICACTIVER,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+        vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_IPRIORITYR,
         vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
index 15183c112c..170017bde3 100644
--- a/xen/arch/arm/vgic/vgic-mmio.c
+++ b/xen/arch/arm/vgic/vgic-mmio.c
@@ -281,6 +281,97 @@ void vgic_mmio_write_cpending(struct vcpu *vcpu,
     }
 }
 
+/*
+ * The actual active bit for a virtual IRQ is held in the LR. Our shadow
+ * copy in struct vgic_irq is only synced when needed and may not be
+ * up-to-date all of the time.
+ * Returning the actual active state is quite costly (stopping all
+ * VCPUs processing any affected vIRQs), so we use a simple implementation
+ * to get the best possible answer.
+ */
+unsigned long vgic_mmio_read_active(struct vcpu *vcpu,
+                                    paddr_t addr, unsigned int len)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    uint32_t value = 0;
+    unsigned int i;
+
+    /* Loop over all IRQs affected by this read */
+    for ( i = 0; i < len * 8; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        if ( irq->active )
+            value |= (1U << i);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    return value;
+}
+
+/*
+ * We don't actually support clearing the active state of an IRQ (yet).
+ * However there is a chance that most guests use this for initialization.
+ * We check whether this MMIO access would actually affect any active IRQ,
+ * and only print our warning in this case. So clearing already non-active
+ * IRQs would not be moaned about in the logs.
+ */
+void vgic_mmio_write_cactive(struct vcpu *vcpu,
+                             paddr_t addr, unsigned int len,
+                             unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    unsigned int i;
+
+    for_each_set_bit( i, &val, len * 8 )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        /*
+         * If we know that the IRQ is active or we can't be sure about
+         * it (because it is currently in a CPU), log the not properly
+         * emulated MMIO access.
+         */
+        if ( irq->active || irq->vcpu )
+            printk(XENLOG_G_ERR
+                   "%pv: vGICD: IRQ%u: clearing active state not supported\n",
+                   vcpu, irq->intid);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
+/*
+ * We don't actually support setting the active state of an IRQ (yet).
+ * We check whether this MMIO access would actually affect any non-active IRQ,
+ * and only print our warning in this case.
+ */
+void vgic_mmio_write_sactive(struct vcpu *vcpu,
+                             paddr_t addr, unsigned int len,
+                             unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 1);
+    unsigned int i;
+
+    for_each_set_bit( i, &val, len * 8 )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        /*
+         * If we know that the IRQ is not active or we can't be sure about
+         * it (because it is currently in a CPU), log the not properly
+         * emulated MMIO access.
+         */
+        if ( !irq->active || irq->vcpu )
+            printk(XENLOG_G_ERR
+                   "%pv: vGICD: IRQ%u: setting active state not supported\n",
+                   vcpu, irq->intid);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
 static int match_region(const void *key, const void *elt)
 {
     const unsigned int offset = (unsigned long)key;
diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
index 5c927f28b0..832e2eb3d8 100644
--- a/xen/arch/arm/vgic/vgic-mmio.h
+++ b/xen/arch/arm/vgic/vgic-mmio.h
@@ -108,6 +108,17 @@ void vgic_mmio_write_cpending(struct vcpu *vcpu,
                               paddr_t addr, unsigned int len,
                               unsigned long val);
 
+unsigned long vgic_mmio_read_active(struct vcpu *vcpu,
+                                    paddr_t addr, unsigned int len);
+
+void vgic_mmio_write_cactive(struct vcpu *vcpu,
+                             paddr_t addr, unsigned int len,
+                             unsigned long val);
+
+void vgic_mmio_write_sactive(struct vcpu *vcpu,
+                             paddr_t addr, unsigned int len,
+                             unsigned long val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 #endif
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 28/45] ARM: new VGIC: Add PRIORITY registers handlers

[Andre Przywara]

The priority register handlers are shared between the v2 and v3 emulation,
so their implementation goes into vgic-mmio.c, to be easily referenced
from the v3 emulation as well later.

This is based on Linux commit 055658bf48fc, written by Andre Przywara.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- fix commit message

 xen/arch/arm/vgic/vgic-mmio-v2.c |  2 +-
 xen/arch/arm/vgic/vgic-mmio.c    | 47 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h    |  7 ++++++
 xen/arch/arm/vgic/vgic.h         |  2 ++
 4 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index 724681e0f8..d2d6a07e1b 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -107,7 +107,7 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_IPRIORITYR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+        vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
         VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ITARGETSR,
         vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
index 170017bde3..c6eef7b948 100644
--- a/xen/arch/arm/vgic/vgic-mmio.c
+++ b/xen/arch/arm/vgic/vgic-mmio.c
@@ -372,6 +372,53 @@ void vgic_mmio_write_sactive(struct vcpu *vcpu,
     }
 }
 
+unsigned long vgic_mmio_read_priority(struct vcpu *vcpu,
+                                      paddr_t addr, unsigned int len)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
+    unsigned int i;
+    uint32_t val = 0;
+
+    for ( i = 0; i < len; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        val |= (uint32_t)irq->priority << (i * 8);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    return val;
+}
+
+/*
+ * We currently don't handle changing the priority of an interrupt that
+ * is already pending on a VCPU. If there is a need for this, we would
+ * need to make this VCPU exit and re-evaluate the priorities, potentially
+ * leading to this interrupt getting presented now to the guest (if it has
+ * been masked by the priority mask before).
+ */
+void vgic_mmio_write_priority(struct vcpu *vcpu,
+                              paddr_t addr, unsigned int len,
+                              unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
+    unsigned int i;
+    unsigned long flags;
+
+    for ( i = 0; i < len; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+        /* Narrow the priority range to what we actually support */
+        irq->priority = (val >> (i * 8)) & GENMASK(7, 8 - VGIC_PRI_BITS);
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
 static int match_region(const void *key, const void *elt)
 {
     const unsigned int offset = (unsigned long)key;
diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
index 832e2eb3d8..b2d572d562 100644
--- a/xen/arch/arm/vgic/vgic-mmio.h
+++ b/xen/arch/arm/vgic/vgic-mmio.h
@@ -119,6 +119,13 @@ void vgic_mmio_write_sactive(struct vcpu *vcpu,
                              paddr_t addr, unsigned int len,
                              unsigned long val);
 
+unsigned long vgic_mmio_read_priority(struct vcpu *vcpu,
+                      paddr_t addr, unsigned int len);
+
+void vgic_mmio_write_priority(struct vcpu *vcpu,
+                  paddr_t addr, unsigned int len,
+                  unsigned long val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 #endif
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index d135c26c80..2feed9615f 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -21,6 +21,8 @@
 #define VARIANT_ID_XEN          0x01
 #define IMPLEMENTER_ARM         0x43b
 
+#define VGIC_PRI_BITS       5
+
 #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
 
 static inline bool irq_is_pending(struct vgic_irq *irq)
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 29/45] ARM: new VGIC: Add CONFIG registers handlers

[Andre Przywara]

The config register handlers are shared between the v2 and v3 emulation,
so their implementation goes into vgic-mmio.c, to be easily referenced
from the v3 emulation as well later.

This is based on Linux commit 79717e4ac09c, written by Andre Przywara.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Reviewed-by:

 xen/arch/arm/vgic/vgic-mmio-v2.c |  2 +-
 xen/arch/arm/vgic/vgic-mmio.c    | 54 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h    |  7 ++++++
 3 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index d2d6a07e1b..a28d0e459b 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -113,7 +113,7 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
         VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICFGR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 2,
+        vgic_mmio_read_config, vgic_mmio_write_config, 2,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_LENGTH(GICD_SGIR,
         vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
index c6eef7b948..1af76909f8 100644
--- a/xen/arch/arm/vgic/vgic-mmio.c
+++ b/xen/arch/arm/vgic/vgic-mmio.c
@@ -419,6 +419,60 @@ void vgic_mmio_write_priority(struct vcpu *vcpu,
     }
 }
 
+unsigned long vgic_mmio_read_config(struct vcpu *vcpu,
+                                    paddr_t addr, unsigned int len)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 2);
+    uint32_t value = 0;
+    int i;
+
+    for ( i = 0; i < len * 4; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        if ( irq->config == VGIC_CONFIG_EDGE )
+            value |= (2U << (i * 2));
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    return value;
+}
+
+void vgic_mmio_write_config(struct vcpu *vcpu,
+                            paddr_t addr, unsigned int len,
+                            unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 2);
+    int i;
+    unsigned long flags;
+
+    for ( i = 0; i < len * 4; i++ )
+    {
+        struct vgic_irq *irq;
+
+        /*
+         * The configuration cannot be changed for SGIs in general,
+         * for PPIs this is IMPLEMENTATION DEFINED. The arch timer
+         * code relies on PPIs being level triggered, so we also
+         * make them read-only here.
+         */
+        if ( intid + i < VGIC_NR_PRIVATE_IRQS )
+            continue;
+
+        irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        if ( test_bit(i * 2 + 1, &val) )
+            irq->config = VGIC_CONFIG_EDGE;
+        else
+            irq->config = VGIC_CONFIG_LEVEL;
+
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
 static int match_region(const void *key, const void *elt)
 {
     const unsigned int offset = (unsigned long)key;
diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
index b2d572d562..3566cf237c 100644
--- a/xen/arch/arm/vgic/vgic-mmio.h
+++ b/xen/arch/arm/vgic/vgic-mmio.h
@@ -126,6 +126,13 @@ void vgic_mmio_write_priority(struct vcpu *vcpu,
                   paddr_t addr, unsigned int len,
                   unsigned long val);
 
+unsigned long vgic_mmio_read_config(struct vcpu *vcpu,
+                    paddr_t addr, unsigned int len);
+
+void vgic_mmio_write_config(struct vcpu *vcpu,
+                paddr_t addr, unsigned int len,
+                unsigned long val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 #endif
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 30/45] ARM: new VGIC: Add TARGET registers handlers

[Andre Przywara]

The target register handlers are v2 emulation specific, so their
implementation lives entirely in vgic-mmio-v2.c.
We copy the old VGIC behaviour of assigning an IRQ to the first VCPU
set in the target mask instead of making it possibly pending on
multiple VCPUs.
We update the physical affinity of a hardware mapped vIRQ on the way.

This is based on Linux commit 2c234d6f1826, written by Andre Przywara.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- directly update affinity (avoid vgic_sync_hardware_irq() and lock)

 xen/arch/arm/vgic/vgic-mmio-v2.c | 59 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 58 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index a28d0e459b..b333de9ed7 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -81,6 +81,63 @@ static void vgic_mmio_write_v2_misc(struct vcpu *vcpu,
     }
 }
 
+static unsigned long vgic_mmio_read_target(struct vcpu *vcpu,
+                                           paddr_t addr, unsigned int len)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
+    uint32_t val = 0;
+    unsigned int i;
+
+    for ( i = 0; i < len; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        val |= (uint32_t)irq->targets << (i * 8);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    return val;
+}
+
+static void vgic_mmio_write_target(struct vcpu *vcpu,
+                                   paddr_t addr, unsigned int len,
+                                   unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
+    uint8_t cpu_mask = GENMASK(vcpu->domain->max_vcpus - 1, 0);
+    unsigned int i;
+    unsigned long flags;
+
+    /* GICD_ITARGETSR[0-7] are read-only */
+    if ( intid < VGIC_NR_PRIVATE_IRQS )
+        return;
+
+    for ( i = 0; i < len; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, NULL, intid + i);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        irq->targets = (val >> (i * 8)) & cpu_mask;
+        if ( irq->targets )
+        {
+            irq->target_vcpu = vcpu->domain->vcpu[ffs(irq->targets) - 1];
+            if ( irq->hw )
+            {
+                struct irq_desc *desc = irq_to_desc(irq->hwintid);
+
+                irq_set_affinity(desc, cpumask_of(irq->target_vcpu->processor));
+            }
+        }
+        else
+            irq->target_vcpu = NULL;
+
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
 static const struct vgic_register_region vgic_v2_dist_registers[] = {
     REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
         vgic_mmio_read_v2_misc, vgic_mmio_write_v2_misc, 12,
@@ -110,7 +167,7 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
         VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ITARGETSR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+        vgic_mmio_read_target, vgic_mmio_write_target, 8,
         VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
     REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICFGR,
         vgic_mmio_read_config, vgic_mmio_write_config, 2,
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 31/45] ARM: new VGIC: Add SGIR register handler

[Andre Przywara]

Triggering an IPI via this register is v2 specific, so the
implementation lives entirely in vgic-mmio-v2.c.

This is based on Linux commit 55cc01fb9004, written by Andre Przywara.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- remove stray rebase artefact

 xen/arch/arm/vgic/vgic-mmio-v2.c | 45 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index b333de9ed7..7e17cdc2ad 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -81,6 +81,49 @@ static void vgic_mmio_write_v2_misc(struct vcpu *vcpu,
     }
 }
 
+static void vgic_mmio_write_sgir(struct vcpu *source_vcpu,
+                                 paddr_t addr, unsigned int len,
+                                 unsigned long val)
+{
+    struct domain *d = source_vcpu->domain;
+    unsigned int nr_vcpus = d->max_vcpus;
+    unsigned int intid = val & GICD_SGI_INTID_MASK;
+    unsigned long targets = (val & GICD_SGI_TARGET_MASK) >>
+                            GICD_SGI_TARGET_SHIFT;
+    unsigned int vcpu_id;
+
+    switch ( val & GICD_SGI_TARGET_LIST_MASK )
+    {
+    case GICD_SGI_TARGET_LIST:                    /* as specified by targets */
+        targets &= GENMASK(nr_vcpus, 0);          /* limit to existing VCPUs */
+        break;
+    case GICD_SGI_TARGET_OTHERS:
+        targets = GENMASK(nr_vcpus, 0);           /* all, ...   */
+        targets &= ~(1U << source_vcpu->vcpu_id); /*   but self */
+        break;
+    case GICD_SGI_TARGET_SELF:                    /* this very vCPU only */
+        targets = (1U << source_vcpu->vcpu_id);
+        break;
+    case 0x3:                                     /* reserved */
+        return;
+    }
+
+    for_each_set_bit( vcpu_id, &targets, 8 )
+    {
+        struct vcpu *vcpu = d->vcpu[vcpu_id];
+        struct vgic_irq *irq = vgic_get_irq(d, vcpu, intid);
+        unsigned long flags;
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        irq->pending_latch = true;
+        irq->source |= 1U << source_vcpu->vcpu_id;
+
+        vgic_queue_irq_unlock(d, irq, flags);
+        vgic_put_irq(d, irq);
+    }
+}
+
 static unsigned long vgic_mmio_read_target(struct vcpu *vcpu,
                                            paddr_t addr, unsigned int len)
 {
@@ -173,7 +216,7 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_config, vgic_mmio_write_config, 2,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_LENGTH(GICD_SGIR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
+        vgic_mmio_read_raz, vgic_mmio_write_sgir, 4,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_LENGTH(GICD_CPENDSGIR,
         vgic_mmio_read_raz, vgic_mmio_write_wi, 16,
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 32/45] ARM: new VGIC: Add SGIPENDR register handlers

[Andre Przywara]

As this register is v2 specific, its implementation lives entirely
in vgic-mmio-v2.c.
This register allows setting the source mask of an IPI.

This is based on Linux commit ed40213ef9b0, written by Andre Przywara.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Reviewed-by:

 xen/arch/arm/vgic/vgic-mmio-v2.c | 81 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 79 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
index 7e17cdc2ad..81443bf1b9 100644
--- a/xen/arch/arm/vgic/vgic-mmio-v2.c
+++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
@@ -181,6 +181,83 @@ static void vgic_mmio_write_target(struct vcpu *vcpu,
     }
 }
 
+static unsigned long vgic_mmio_read_sgipend(struct vcpu *vcpu,
+                                            paddr_t addr, unsigned int len)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
+    uint32_t val = 0;
+    unsigned int i;
+
+    ASSERT(intid < VGIC_NR_SGIS);
+
+    for ( i = 0; i < len; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        val |= (uint32_t)irq->source << (i * 8);
+
+        vgic_put_irq(vcpu->domain, irq);
+    }
+
+    return val;
+}
+
+static void vgic_mmio_write_sgipendc(struct vcpu *vcpu,
+                                     paddr_t addr, unsigned int len,
+                                     unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
+    unsigned int i;
+    unsigned long flags;
+
+    ASSERT(intid < VGIC_NR_SGIS);
+
+    for ( i = 0; i < len; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        irq->source &= ~((val >> (i * 8)) & 0xff);
+        if ( !irq->source )
+            irq->pending_latch = false;
+
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
+static void vgic_mmio_write_sgipends(struct vcpu *vcpu,
+                                     paddr_t addr, unsigned int len,
+                                     unsigned long val)
+{
+    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
+    unsigned int i;
+    unsigned long flags;
+
+    ASSERT(intid < VGIC_NR_SGIS);
+
+    for ( i = 0; i < len; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        irq->source |= (val >> (i * 8)) & 0xff;
+
+        if ( irq->source )
+        {
+            irq->pending_latch = true;
+            vgic_queue_irq_unlock(vcpu->domain, irq, flags);
+        }
+        else
+        {
+            spin_unlock_irqrestore(&irq->irq_lock, flags);
+        }
+        vgic_put_irq(vcpu->domain, irq);
+    }
+}
+
 static const struct vgic_register_region vgic_v2_dist_registers[] = {
     REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
         vgic_mmio_read_v2_misc, vgic_mmio_write_v2_misc, 12,
@@ -219,10 +296,10 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
         vgic_mmio_read_raz, vgic_mmio_write_sgir, 4,
         VGIC_ACCESS_32bit),
     REGISTER_DESC_WITH_LENGTH(GICD_CPENDSGIR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 16,
+        vgic_mmio_read_sgipend, vgic_mmio_write_sgipendc, 16,
         VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
     REGISTER_DESC_WITH_LENGTH(GICD_SPENDSGIR,
-        vgic_mmio_read_raz, vgic_mmio_write_wi, 16,
+        vgic_mmio_read_sgipend, vgic_mmio_write_sgipends, 16,
         VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
 };
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 33/45] ARM: new VGIC: Handle hardware mapped IRQs

[Andre Przywara]

The VGIC supports virtual IRQs to be connected to a hardware IRQ, so
when a guest EOIs the virtual interrupt, it affects the state of that
corresponding interrupt on the hardware side at the same time.
Implement the interface that the Xen arch/core code expects to connect
the virtual and the physical world.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Reviewed-by:

 xen/arch/arm/vgic/vgic.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 963d5e5f6e..7ff680443c 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -689,6 +689,77 @@ void vgic_kick_vcpus(struct domain *d)
     }
 }
 
+struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
+                                      unsigned int virq)
+{
+    struct irq_desc *desc = NULL;
+    struct vgic_irq *irq = vgic_get_irq(d, v, virq);
+    unsigned long flags;
+
+    if ( !irq )
+        return NULL;
+
+    spin_lock_irqsave(&irq->irq_lock, flags);
+    if ( irq->hw )
+    {
+        ASSERT(irq->hwintid >= VGIC_NR_PRIVATE_IRQS);
+        desc = irq_to_desc(irq->hwintid);
+    }
+    spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+    vgic_put_irq(d, irq);
+
+    return desc;
+}
+
+/*
+ * was:
+ *      int kvm_vgic_map_phys_irq(struct vcpu *vcpu, u32 virt_irq, u32 phys_irq)
+ *      int kvm_vgic_unmap_phys_irq(struct vcpu *vcpu, unsigned int virt_irq)
+ */
+int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
+                        unsigned int virt_irq, struct irq_desc *desc,
+                        bool connect)
+{
+    struct vgic_irq *irq = vgic_get_irq(d, vcpu, virt_irq);
+    unsigned long flags;
+    int ret = 0;
+
+    if ( !irq )
+        return -EINVAL;
+
+    spin_lock_irqsave(&irq->irq_lock, flags);
+
+    if ( connect )                      /* assign a mapped IRQ */
+    {
+        /* The VIRQ should not be already enabled by the guest */
+        if ( !irq->hw && !irq->enabled )
+        {
+            irq->hw = true;
+            irq->hwintid = desc->irq;
+        }
+        else
+            ret = -EBUSY;
+    }
+    else                                /* remove a mapped IRQ */
+    {
+        if ( desc && irq->hwintid != desc->irq )
+        {
+            ret = -EINVAL;
+        }
+        else
+        {
+            irq->hw = false;
+            irq->hwintid = 0;
+        }
+    }
+
+    spin_unlock_irqrestore(&irq->irq_lock, flags);
+    vgic_put_irq(d, irq);
+
+    return ret;
+}
+
 static unsigned int translate_irq_type(bool is_level)
 {
     return is_level ? IRQ_TYPE_LEVEL_HIGH : IRQ_TYPE_EDGE_RISING;
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 34/45] ARM: new VGIC: Add event channel IRQ handling

[Andre Przywara]

The Xen core/arch code relies on two abstracted functions to inject an
event channel IRQ and to query its pending state.
Implement those to query the state of the new VGIC implementation.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Acked-by:

 xen/arch/arm/vgic/vgic.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 7ff680443c..277cf4a03e 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -689,6 +689,29 @@ void vgic_kick_vcpus(struct domain *d)
     }
 }
 
+void arch_evtchn_inject(struct vcpu *v)
+{
+    vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
+}
+
+bool vgic_evtchn_irq_pending(struct vcpu *v)
+{
+    struct vgic_irq *irq;
+    unsigned long flags;
+    bool pending;
+
+    /* Does not work for LPIs. */
+    ASSERT(!is_lpi(v->domain->arch.evtchn_irq));
+
+    irq = vgic_get_irq(v->domain, v, v->domain->arch.evtchn_irq);
+    spin_lock_irqsave(&irq->irq_lock, flags);
+    pending = irq_is_pending(irq);
+    spin_unlock_irqrestore(&irq->irq_lock, flags);
+    vgic_put_irq(v->domain, irq);
+
+    return pending;
+}
+
 struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
                                       unsigned int virq)
 {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 35/45] ARM: new VGIC: Handle virtual IRQ allocation/reservation

[Andre Przywara]

To find an unused virtual IRQ number Xen uses a scheme to track used
virtual IRQs.
Implement this interface in the new VGIC to make the Xen core/arch code
happy.
This is actually somewhat VGIC agnostic, so is mostly a copy of the code
from the old VGIC. But it has to live in the VGIC files, so we can't
easily reuse the existing implementation.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Acked-by:

 xen/arch/arm/vgic/vgic.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 277cf4a03e..d7ef6e32fc 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -712,6 +712,50 @@ bool vgic_evtchn_irq_pending(struct vcpu *v)
     return pending;
 }
 
+bool vgic_reserve_virq(struct domain *d, unsigned int virq)
+{
+    if ( virq >= vgic_num_irqs(d) )
+        return false;
+
+    return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
+}
+
+int vgic_allocate_virq(struct domain *d, bool spi)
+{
+    int first, end;
+    unsigned int virq;
+
+    if ( !spi )
+    {
+        /* We only allocate PPIs. SGIs are all reserved */
+        first = 16;
+        end = 32;
+    }
+    else
+    {
+        first = 32;
+        end = vgic_num_irqs(d);
+    }
+
+    /*
+     * There is no spinlock to protect allocated_irqs, therefore
+     * test_and_set_bit may fail. If so retry it.
+     */
+    do
+    {
+        virq = find_next_zero_bit(d->arch.vgic.allocated_irqs, end, first);
+        if ( virq >= end )
+            return -1;
+    } while ( test_and_set_bit(virq, d->arch.vgic.allocated_irqs) );
+
+    return virq;
+}
+
+void vgic_free_virq(struct domain *d, unsigned int virq)
+{
+    clear_bit(virq, d->arch.vgic.allocated_irqs);
+}
+
 struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
                                       unsigned int virq)
 {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 36/45] ARM: new VGIC: Dump virtual IRQ info

[Andre Przywara]

When we dump guest state on the Xen console, we also print the state of
IRQs that are on a VCPU.
Add the code to dump the state of an IRQ handled by the new VGIC.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Acked-by:

 xen/arch/arm/vgic/vgic.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index d7ef6e32fc..b56682f8e4 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -756,6 +756,31 @@ void vgic_free_virq(struct domain *d, unsigned int virq)
     clear_bit(virq, d->arch.vgic.allocated_irqs);
 }
 
+void gic_dump_vgic_info(struct vcpu *v)
+{
+    struct vgic_cpu *vgic_cpu = &v->arch.vgic;
+    struct vgic_irq *irq;
+    unsigned long flags;
+
+    spin_lock_irqsave(&v->arch.vgic.ap_list_lock, flags);
+
+    if ( !list_empty(&vgic_cpu->ap_list_head) )
+        printk("   active or pending interrupts queued:\n");
+
+    list_for_each_entry ( irq, &vgic_cpu->ap_list_head, ap_list )
+    {
+        spin_lock(&irq->irq_lock);
+        printk("     %s %s irq %u: %spending, %sactive, %senabled\n",
+               irq->hw ? "hardware" : "virtual",
+               irq->config == VGIC_CONFIG_LEVEL ? "level" : "edge",
+               irq->intid, irq_is_pending(irq) ? "" : "not ",
+               irq->active ? "" : "not ", irq->enabled ? "" : "not ");
+        spin_unlock(&irq->irq_lock);
+    }
+
+    spin_unlock_irqrestore(&v->arch.vgic.ap_list_lock, flags);
+}
+
 struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
                                       unsigned int virq)
 {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 37/45] ARM: new VGIC: Provide system register emulation stub

[Andre Przywara]

The Xen arch code traps system registers writes from the guest and will
relay anything GIC related to the VGIC.
Since this affects only GICv3 (which we don't yet emulate), provide a
stub implementation of vgic_emulate() for now.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
---
Changelog v1 ... v2:
- Add Acked-by:

 xen/arch/arm/vgic/vgic.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index b56682f8e4..73cd43f0fa 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -804,6 +804,13 @@ struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
     return desc;
 }
 
+bool vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
+{
+    ASSERT(current->domain->arch.vgic.version == GIC_V3);
+
+    return false;
+}
+
 /*
  * was:
  *      int kvm_vgic_map_phys_irq(struct vcpu *vcpu, u32 virt_irq, u32 phys_irq)
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 38/45] ARM: new VGIC: Implement arch_move_irqs()

[Andre Przywara]

When a VCPU moves to another CPU, we need to adjust the target affinity
of any hardware mapped vIRQs, to observe our "physical-follows-virtual"
policy.
Implement arch_move_irqs() to adjust the physical affinity of all hardware
mapped vIRQs targetting this VCPU.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- use irq_set_affinity() directly, avoid vgic_sync_hardware_irq() and lock

 xen/arch/arm/vgic/vgic.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 73cd43f0fa..1aae68ff83 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -781,6 +781,45 @@ void gic_dump_vgic_info(struct vcpu *v)
     spin_unlock_irqrestore(&v->arch.vgic.ap_list_lock, flags);
 }
 
+/**
+ * arch_move_irqs() - migrate the physical affinity of hardware mapped vIRQs
+ * @v:  the vCPU, already assigned to the new pCPU
+ *
+ * arch_move_irqs() updates the physical affinity of all virtual IRQs
+ * targetting this given vCPU. This only affects hardware mapped IRQs. The
+ * new pCPU to target is already set in v->processor.
+ * This is called by the core code after a vCPU has been migrated to a new
+ * physical CPU.
+ */
+void arch_move_irqs(struct vcpu *v)
+{
+    struct domain *d = v->domain;
+    unsigned int i;
+
+    /* We only target SPIs with this function */
+    for ( i = 0; i < d->arch.vgic.nr_spis; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(d, NULL, i + VGIC_NR_PRIVATE_IRQS);
+        unsigned long flags;
+
+        if ( !irq )
+            continue;
+
+        spin_lock_irqsave(&irq->irq_lock, flags);
+
+        /* only vIRQs that are not on a vCPU yet , but targetting this vCPU */
+        if ( irq->hw && !irq->vcpu && irq->target_vcpu == v)
+        {
+            irq_desc_t *desc = irq_to_desc(irq->hwintid);
+
+            irq_set_affinity(desc, cpumask_of(v->processor));
+        }
+
+        spin_unlock_irqrestore(&irq->irq_lock, flags);
+        vgic_put_irq(d, irq);
+    }
+}
+
 struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
                                       unsigned int virq)
 {
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 39/45] ARM: new VGIC: Add preliminary stub implementation

[Andre Przywara]

The ARM arch code requires an interrupt controller emulation to implement
vgic_clear_pending_irqs(), although it is suspected that it is actually
not necessary. Go with a stub for now to make the linker happy.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
 xen/arch/arm/vgic/vgic.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 1aae68ff83..002fec57e6 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -781,6 +781,14 @@ void gic_dump_vgic_info(struct vcpu *v)
     spin_unlock_irqrestore(&v->arch.vgic.ap_list_lock, flags);
 }
 
+void vgic_clear_pending_irqs(struct vcpu *v)
+{
+    /*
+     * TODO: It is unclear whether we really need this, so we might instead
+     * remove it on the caller site.
+     */
+}
+
 /**
  * arch_move_irqs() - migrate the physical affinity of hardware mapped vIRQs
  * @v:  the vCPU, already assigned to the new pCPU
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 40/45] ARM: new VGIC: vgic-init: register VGIC

[Andre Przywara]

This patch implements the function which is called by Xen when it wants
to register the virtual GIC.
This also implements vgic_max_vcpus() for the new VGIC, which reports
back the maximum number of VCPUs a certain GIC model supports.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
 xen/arch/arm/vgic/vgic-init.c | 60 +++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.c      | 22 ++++++++++++++++
 xen/arch/arm/vgic/vgic.h      |  3 +++
 3 files changed, 85 insertions(+)
 create mode 100644 xen/arch/arm/vgic/vgic-init.c

diff --git a/xen/arch/arm/vgic/vgic-init.c b/xen/arch/arm/vgic/vgic-init.c
new file mode 100644
index 0000000000..d091c92ed0
--- /dev/null
+++ b/xen/arch/arm/vgic/vgic-init.c
@@ -0,0 +1,60 @@
+/*
+ * Copyright (C) 2015, 2016 ARM Ltd.
+ * Imported from Linux ("new" KVM VGIC) and heavily adapted to Xen.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/sched.h>
+#include <asm/new_vgic.h>
+
+#include "vgic.h"
+
+/* CREATION */
+
+/**
+ * domain_vgic_register: create a virtual GIC
+ * @d: domain pointer
+ * @mmio_count: pointer to add number of required MMIO regions
+ *
+ * was: kvm_vgic_create
+ */
+int domain_vgic_register(struct domain *d, int *mmio_count)
+{
+    switch ( d->arch.vgic.version )
+    {
+    case GIC_V2:
+        *mmio_count = 1;
+        break;
+    default:
+        BUG();
+    }
+
+    if ( d->max_vcpus > domain_max_vcpus(d) )
+        return -E2BIG;
+
+    d->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
+    d->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
+    d->arch.vgic.vgic_redist_base = VGIC_ADDR_UNDEF;
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 002fec57e6..4b9664f313 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -946,6 +946,28 @@ void vgic_sync_hardware_irq(struct domain *d,
     spin_unlock_irqrestore(&desc->lock, flags);
 }
 
+unsigned int vgic_max_vcpus(const struct domain *d)
+{
+    unsigned int vgic_vcpu_limit;
+
+    switch ( d->arch.vgic.version )
+    {
+#ifdef CONFIG_HAS_GICV3
+    case GIC_V3:
+        vgic_vcpu_limit = VGIC_V3_MAX_CPUS;
+        break;
+#endif
+    case GIC_V2:
+        vgic_vcpu_limit = VGIC_V2_MAX_CPUS;
+        break;
+    default:
+        vgic_vcpu_limit = MAX_VIRT_CPUS;
+        break;
+    }
+
+    return min_t(unsigned int, MAX_VIRT_CPUS, vgic_vcpu_limit);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index 2feed9615f..deefbb3ef7 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -21,6 +21,9 @@
 #define VARIANT_ID_XEN          0x01
 #define IMPLEMENTER_ARM         0x43b
 
+#define VGIC_ADDR_UNDEF     INVALID_PADDR
+#define IS_VGIC_ADDR_UNDEF(_x)  ((_x) == VGIC_ADDR_UNDEF)
+
 #define VGIC_PRI_BITS       5
 
 #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 41/45] ARM: new VGIC: Add vgic_v2_enable

[Andre Przywara]

Enable the VGIC operation by properly initialising the registers
in the hypervisor GIC interface.

This is based on Linux commit f7b6985cc3d0, written by Eric Auger.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- move patch from later part in the series

 xen/arch/arm/vgic/vgic-v2.c | 6 ++++++
 xen/arch/arm/vgic/vgic.h    | 1 +
 2 files changed, 7 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
index bc5c7895f4..4662346327 100644
--- a/xen/arch/arm/vgic/vgic-v2.c
+++ b/xen/arch/arm/vgic/vgic-v2.c
@@ -219,6 +219,12 @@ void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr)
     gic_hw_ops->write_lr(lr, &lr_val);
 }
 
+void vgic_v2_enable(struct vcpu *vcpu)
+{
+    /* Get the show on the road... */
+    gic_hw_ops->update_hcr_status(GICH_HCR_EN, 1);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index deefbb3ef7..def1ac478a 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -62,6 +62,7 @@ void vgic_sync_hardware_irq(struct domain *d,
 void vgic_v2_fold_lr_state(struct vcpu *vcpu);
 void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr);
 void vgic_v2_set_underflow(struct vcpu *vcpu);
+void vgic_v2_enable(struct vcpu *vcpu);
 int vgic_register_dist_iodev(struct domain *d, gfn_t dist_base_fn,
                              enum vgic_type);
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 42/45] ARM: new VGIC: vgic-init: implement vgic_init

[Andre Przywara]

This patch allocates and initializes the data structures used to model
the vgic distributor and virtual cpu interfaces. At that stage the
number of IRQs and number of virtual CPUs is frozen.
Implement the various functions that the Xen arch code is expecting to
call during domain and VCPU setup to initialize the VGIC.
Their prototypes are already in existing header files.

This is based on Linux commit ad275b8bb1e6, written by Eric Auger.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- remove stray kvm_ prefix in comment
- use unsigned int
- ROUNDUP number of SPIs
- fix indentation

 xen/arch/arm/vgic/vgic-init.c | 199 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 199 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic-init.c b/xen/arch/arm/vgic/vgic-init.c
index d091c92ed0..d7695d69bf 100644
--- a/xen/arch/arm/vgic/vgic-init.c
+++ b/xen/arch/arm/vgic/vgic-init.c
@@ -15,11 +15,83 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <xen/lib.h>
 #include <xen/sched.h>
 #include <asm/new_vgic.h>
 
 #include "vgic.h"
 
+/*
+ * Initialization rules: there are multiple stages to the vgic
+ * initialization, both for the distributor and the CPU interfaces.  The basic
+ * idea is that even though the VGIC is not functional or not requested from
+ * user space, the critical path of the run loop can still call VGIC functions
+ * that just won't do anything, without them having to check additional
+ * initialization flags to ensure they don't look at uninitialized data
+ * structures.
+ *
+ * Distributor:
+ *
+ * - vgic_early_init(): initialization of static data that doesn't
+ *   depend on any sizing information or emulation type. No allocation
+ *   is allowed there.
+ *
+ * - vgic_init(): allocation and initialization of the generic data
+ *   structures that depend on sizing information (number of CPUs,
+ *   number of interrupts). Also initializes the vcpu specific data
+ *   structures. Can be executed lazily for GICv2.
+ *
+ * CPU Interface:
+ *
+ * - vgic_vcpu_early_init(): initialization of static data that
+ *   doesn't depend on any sizing information or emulation type. No
+ *   allocation is allowed there.
+ */
+
+/**
+ * vgic_vcpu_early_init() - Initialize static VGIC VCPU data structures
+ * @vcpu: The VCPU whose VGIC data structures whould be initialized
+ *
+ * Only do initialization, but do not actually enable the VGIC CPU interface
+ * yet.
+ */
+static void vgic_vcpu_early_init(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+    unsigned int i;
+
+    INIT_LIST_HEAD(&vgic_cpu->ap_list_head);
+    spin_lock_init(&vgic_cpu->ap_list_lock);
+
+    /*
+     * Enable and configure all SGIs to be edge-triggered and
+     * configure all PPIs as level-triggered.
+     */
+    for ( i = 0; i < VGIC_NR_PRIVATE_IRQS; i++ )
+    {
+        struct vgic_irq *irq = &vgic_cpu->private_irqs[i];
+
+        INIT_LIST_HEAD(&irq->ap_list);
+        spin_lock_init(&irq->irq_lock);
+        irq->intid = i;
+        irq->vcpu = NULL;
+        irq->target_vcpu = vcpu;
+        irq->targets = 1U << vcpu->vcpu_id;
+        atomic_set(&irq->refcount, 0);
+        if ( vgic_irq_is_sgi(i) )
+        {
+            /* SGIs */
+            irq->enabled = 1;
+            irq->config = VGIC_CONFIG_EDGE;
+        }
+        else
+        {
+            /* PPIs */
+            irq->config = VGIC_CONFIG_LEVEL;
+        }
+    }
+}
+
 /* CREATION */
 
 /**
@@ -50,6 +122,133 @@ int domain_vgic_register(struct domain *d, int *mmio_count)
     return 0;
 }
 
+/* INIT/DESTROY */
+
+/**
+ * domain_vgic_init: initialize the dist data structures
+ * @d: domain pointer
+ * @nr_spis: number of SPIs
+ */
+int domain_vgic_init(struct domain *d, unsigned int nr_spis)
+{
+    struct vgic_dist *dist = &d->arch.vgic;
+    unsigned int i;
+    int ret;
+
+    /* Limit the number of virtual SPIs supported to (1020 - 32) = 988  */
+    if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
+        return -EINVAL;
+    nr_spis = ROUNDUP(nr_spis, 32);
+
+    dist->nr_spis = nr_spis;
+    dist->spis = xzalloc_array(struct vgic_irq, nr_spis);
+    if ( !dist->spis )
+        return  -ENOMEM;
+
+    /*
+     * In the following code we do not take the irq struct lock since
+     * no other action on irq structs can happen while the VGIC is
+     * not initialized yet:
+     * If someone wants to inject an interrupt or does a MMIO access, we
+     * require prior initialization in case of a virtual GICv3 or trigger
+     * initialization when using a virtual GICv2.
+     */
+    for ( i = 0; i < nr_spis; i++ )
+    {
+        struct vgic_irq *irq = &dist->spis[i];
+
+        irq->intid = i + VGIC_NR_PRIVATE_IRQS;
+        INIT_LIST_HEAD(&irq->ap_list);
+        spin_lock_init(&irq->irq_lock);
+        irq->vcpu = NULL;
+        irq->target_vcpu = NULL;
+        atomic_set(&irq->refcount, 0);
+        if ( dist->version == GIC_V2 )
+            irq->targets = 0;
+        else
+            irq->mpidr = 0;
+    }
+
+    INIT_LIST_HEAD(&dist->lpi_list_head);
+    spin_lock_init(&dist->lpi_list_lock);
+
+    if ( dist->version == GIC_V2 )
+        ret = vgic_v2_map_resources(d);
+    else
+        ret = -ENXIO;
+
+    if ( ret )
+        return ret;
+
+    /* allocated_irqs() is used by Xen to find available vIRQs */
+    d->arch.vgic.allocated_irqs =
+        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
+    if ( !d->arch.vgic.allocated_irqs )
+        return -ENOMEM;
+
+    /* vIRQ0-15 (SGIs) are reserved */
+    for ( i = 0; i < NR_GIC_SGI; i++ )
+        set_bit(i, d->arch.vgic.allocated_irqs);
+
+    return 0;
+}
+
+/**
+ * vcpu_vgic_init() - Register VCPU-specific KVM iodevs
+ * was: kvm_vgic_vcpu_init()
+ * Xen: adding vgic_vx_enable() call
+ * @vcpu: pointer to the VCPU being created and initialized
+ */
+int vcpu_vgic_init(struct vcpu *vcpu)
+{
+    int ret = 0;
+
+    vgic_vcpu_early_init(vcpu);
+
+    if ( gic_hw_version() == GIC_V2 )
+        vgic_v2_enable(vcpu);
+    else
+        ret = -ENXIO;
+
+    return ret;
+}
+
+void domain_vgic_free(struct domain *d)
+{
+    struct vgic_dist *dist = &d->arch.vgic;
+        int i, ret;
+
+    for ( i = 0; i < dist->nr_spis; i++ )
+    {
+        struct vgic_irq *irq = vgic_get_irq(d, NULL, 32 + i);
+
+        if ( !irq->hw )
+            continue;
+
+        ret = release_guest_irq(d, irq->hwintid);
+        if ( ret )
+            dprintk(XENLOG_G_WARNING,
+                    "d%u: Failed to release virq %u ret = %d\n",
+                    d->domain_id, 32 + i, ret);
+    }
+
+    dist->ready = false;
+    dist->initialized = false;
+
+    xfree(dist->spis);
+    xfree(dist->allocated_irqs);
+    dist->nr_spis = 0;
+}
+
+int vcpu_vgic_free(struct vcpu *vcpu)
+{
+    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
+
+    INIT_LIST_HEAD(&vgic_cpu->ap_list_head);
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 43/45] ARM: new VGIC: vgic-init: implement map_resources

[Andre Przywara]

map_resources is the last initialization step needed before the first
VCPU is run. At that stage the code stores the MMIO base addresses used.
Also it registers the respective register frames with the MMIO framework.

This is based on Linux commit cbae53e663ea, written by Eric Auger.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- whitespace fixes

 xen/arch/arm/vgic/vgic-v2.c | 66 +++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.h    |  1 +
 2 files changed, 67 insertions(+)

diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
index 4662346327..c35f406a06 100644
--- a/xen/arch/arm/vgic/vgic-v2.c
+++ b/xen/arch/arm/vgic/vgic-v2.c
@@ -225,6 +225,72 @@ void vgic_v2_enable(struct vcpu *vcpu)
     gic_hw_ops->update_hcr_status(GICH_HCR_EN, 1);
 }
 
+int vgic_v2_map_resources(struct domain *d)
+{
+    struct vgic_dist *dist = &d->arch.vgic;
+    paddr_t cbase, csize;
+    paddr_t vbase;
+    int ret;
+
+    /*
+     * The hardware domain gets the hardware address.
+     * Guests get the virtual platform layout.
+     */
+    if ( is_hardware_domain(d) )
+    {
+        d->arch.vgic.vgic_dist_base = gic_v2_hw_data.dbase;
+        /*
+         * For the hardware domain, we always map the whole HW CPU
+         * interface region in order to match the device tree (the "reg"
+         * properties is copied as it is).
+         * Note that we assume the size of the CPU interface is always
+         * aligned to PAGE_SIZE.
+         */
+        cbase = gic_v2_hw_data.cbase; /* was: dist->vgic_cpu_base */
+        csize = gic_v2_hw_data.csize;
+        vbase = gic_v2_hw_data.vbase; /* was: kvm_vgic_global_state.vcpu_base */
+    }
+    else
+    {
+        d->arch.vgic.vgic_dist_base = GUEST_GICD_BASE;
+        /*
+         * The CPU interface exposed to the guest is always 8kB. We may
+         * need to add an offset to the virtual CPU interface base
+         * address when in the GIC is aliased to get a 8kB contiguous
+         * region.
+         */
+        BUILD_BUG_ON(GUEST_GICC_SIZE != SZ_8K);
+        cbase = GUEST_GICC_BASE;
+        csize = GUEST_GICC_SIZE;
+        vbase = gic_v2_hw_data.vbase + gic_v2_hw_data.aliased_offset;
+    }
+
+
+    ret = vgic_register_dist_iodev(d, gaddr_to_gfn(dist->vgic_dist_base),
+                                   VGIC_V2);
+    if ( ret )
+    {
+        gdprintk(XENLOG_ERR, "Unable to register VGIC MMIO regions\n");
+        return ret;
+    }
+
+    /*
+     * Map the gic virtual cpu interface in the gic cpu interface
+     * region of the guest.
+     */
+    ret = map_mmio_regions(d, gaddr_to_gfn(cbase), csize / PAGE_SIZE,
+                           maddr_to_mfn(vbase));
+    if ( ret )
+    {
+        gdprintk(XENLOG_ERR, "Unable to remap VGIC CPU to VCPU\n");
+        return ret;
+    }
+
+    dist->ready = true;
+
+        return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
index def1ac478a..7af982f100 100644
--- a/xen/arch/arm/vgic/vgic.h
+++ b/xen/arch/arm/vgic/vgic.h
@@ -63,6 +63,7 @@ void vgic_v2_fold_lr_state(struct vcpu *vcpu);
 void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr);
 void vgic_v2_set_underflow(struct vcpu *vcpu);
 void vgic_v2_enable(struct vcpu *vcpu);
+int vgic_v2_map_resources(struct domain *d);
 int vgic_register_dist_iodev(struct domain *d, gfn_t dist_base_fn,
                              enum vgic_type);
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 44/45] ARM: new VGIC: Allocate two pages for struct vcpu

[Andre Przywara]

At the moment we allocate exactly one page for struct vcpu on ARM, also
have a check in place to prevent it growing beyond 4KB.
As the struct includes the state of all 32 private (per-VCPU) interrupts,
we are at 3840 bytes on arm64 at the moment already. Growing the per-IRQ
VGIC structure even slightly makes the VCPU quickly exceed the 4K limit.
The new VGIC will need more space per virtual IRQ. I spent a few hours
trying to trim this down, but couldn't get it below 4KB, even with the
nasty hacks piling up to save some bytes here and there.
It turns out that beyond efficiency, maybe, there is no real technical
reason this struct has to fit in one page, so lifting the limit to two
pages seems like the most pragmatic solution.
Restrict this to compiling with the new VGIC and for ARM64 only.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- confine change to new VGIC and ARM64 only

 xen/arch/arm/domain.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 18b915d2e9..3fba05bda5 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -508,10 +508,25 @@ void dump_pageframe_info(struct domain *d)
 struct vcpu *alloc_vcpu_struct(void)
 {
     struct vcpu *v;
+
+    /*
+     * The new VGIC has a bigger per-IRQ structure, so we need more than one
+     * page on ARM64. Cowardly increase the limit in this case.
+     */
+#if defined(CONFIG_NEW_VGIC) && defined(CONFIG_ARM_64)
+    BUILD_BUG_ON(sizeof(*v) > 2 * PAGE_SIZE);
+    v = alloc_xenheap_pages(1, 0);
+    if ( v != NULL ) {
+        clear_page(v);
+        clear_page((void *)v + PAGE_SIZE);
+    }
+#else
     BUILD_BUG_ON(sizeof(*v) > PAGE_SIZE);
     v = alloc_xenheap_pages(0, 0);
     if ( v != NULL )
         clear_page(v);
+#endif
+
     return v;
 }
 
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Andre Przywara]

Now that we have both the old VGIC prepared to cope with a sibling and
the code for the new VGIC in place, lets add a Kconfig option to enable
the new code and wire it into the Xen build system.
This will add a compile time option to use either the "old" or the "new"
VGIC.
In the moment this is restricted to a vGIC-v2. To make the build system
happy, we provide a temporary dummy implementation of
vgic_v3_setup_hw() to allow building for now.

Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
---
Changelog v1 ... v2:
- add Kconfig help text
- use separate Makefile in vgic/ directory
- protect compilation without GICV3 support
- always include list_sort() in build

 xen/arch/arm/Kconfig       | 17 ++++++++++++++++-
 xen/arch/arm/Makefile      |  5 ++++-
 xen/arch/arm/vgic/Makefile |  5 +++++
 xen/arch/arm/vgic/vgic.c   | 10 ++++++++++
 xen/common/Makefile        |  1 +
 5 files changed, 36 insertions(+), 2 deletions(-)
 create mode 100644 xen/arch/arm/vgic/Makefile

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 2782ee6589..310f909768 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -48,7 +48,22 @@ config HAS_GICV3
 config HAS_ITS
         bool
         prompt "GICv3 ITS MSI controller support" if EXPERT = "y"
-        depends on HAS_GICV3
+        depends on HAS_GICV3 && !NEW_VGIC
+
+config NEW_VGIC
+        bool
+        prompt "Use new VGIC implementation"
+        ---help---
+
+        This is an alternative implementation of the ARM GIC interrupt
+        controller emulation, based on the Linux/KVM VGIC. It has a better
+        design and fixes many shortcomings of the existing GIC emulation in
+        Xen. It will eventually replace the existing/old VGIC.
+        However at the moment it lacks support for Dom0 using the ITS for
+        using MSIs.
+        Say Y if you want to help testing this new code or if you experience
+        problems with the standard emulation.
+        At the moment this implementation is not security supported.
 
 config SBSA_VUART_CONSOLE
 	bool "Emulated SBSA UART console support"
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 41d7366527..a9533b107e 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -16,7 +16,6 @@ obj-y += domain_build.o
 obj-y += domctl.o
 obj-$(EARLY_PRINTK) += early_printk.o
 obj-y += gic.o
-obj-y += gic-vgic.o
 obj-y += gic-v2.o
 obj-$(CONFIG_HAS_GICV3) += gic-v3.o
 obj-$(CONFIG_HAS_ITS) += gic-v3-its.o
@@ -47,10 +46,14 @@ obj-y += sysctl.o
 obj-y += time.o
 obj-y += traps.o
 obj-y += vcpreg.o
+subdir-$(CONFIG_NEW_VGIC) += vgic
+ifneq ($(CONFIG_NEW_VGIC),y)
+obj-y += gic-vgic.o
 obj-y += vgic.o
 obj-y += vgic-v2.o
 obj-$(CONFIG_HAS_GICV3) += vgic-v3.o
 obj-$(CONFIG_HAS_ITS) += vgic-v3-its.o
+endif
 obj-y += vm_event.o
 obj-y += vtimer.o
 obj-$(CONFIG_SBSA_VUART_CONSOLE) += vpl011.o
diff --git a/xen/arch/arm/vgic/Makefile b/xen/arch/arm/vgic/Makefile
new file mode 100644
index 0000000000..806826948e
--- /dev/null
+++ b/xen/arch/arm/vgic/Makefile
@@ -0,0 +1,5 @@
+obj-y += vgic.o
+obj-y += vgic-v2.o
+obj-y += vgic-mmio.o
+obj-y += vgic-mmio-v2.o
+obj-y += vgic-init.o
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 4b9664f313..342b95be31 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -968,6 +968,16 @@ unsigned int vgic_max_vcpus(const struct domain *d)
     return min_t(unsigned int, MAX_VIRT_CPUS, vgic_vcpu_limit);
 }
 
+#ifdef CONFIG_HAS_GICV3
+void vgic_v3_setup_hw(paddr_t dbase,
+                      unsigned int nr_rdist_regions,
+                      const struct rdist_region *regions,
+                      unsigned int intid_bits)
+{
+    /* Dummy implementation to allow building without actual vGICv3 support. */
+}
+#endif
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 3a349f478b..1668e14c4b 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -19,6 +19,7 @@ obj-y += keyhandler.o
 obj-$(CONFIG_KEXEC) += kexec.o
 obj-$(CONFIG_KEXEC) += kimage.o
 obj-y += lib.o
+obj-y += list_sort.o
 obj-$(CONFIG_LIVEPATCH) += livepatch.o livepatch_elf.o
 obj-y += lzo.o
 obj-$(CONFIG_HAS_MEM_ACCESS) += mem_access.o
-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 17/45] Add list_sort() routine from Linux

[Jan Beulich]

>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
> This pulls in Linux' list_sort.c, which is a merge sort implementation
> for linked lists. Apart from adding a full featured license header and
> adjusting the #include file, nothing has been changed in this code.

Please mention the version of Linux this was taken from, to ease
later determination of applicability to our tree of changes they do.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Jan Beulich]

>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -19,6 +19,7 @@ obj-y += keyhandler.o
>  obj-$(CONFIG_KEXEC) += kexec.o
>  obj-$(CONFIG_KEXEC) += kimage.o
>  obj-y += lib.o
> +obj-y += list_sort.o

Why here rather than in patch 17? And why also for x86? I think you
want a promptless Kconfig option that an arch can select if it needs
this code, unless or until common code makes use of it.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 01/45] ARM: VGIC: rename gic_event_needs_delivery()

[Julien Grall]

Hi Andre,

On 15/03/18 20:30, Andre Przywara wrote:
> gic_event_needs_delivery() is not named very intuitively, especially
> the gic_ prefix is somewhat misleading.
> Rename it to vgic_vcpu_pending_irq(), which makes it clear that this
> relates to the virtual GIC and is about interrupts.
> Also add a VCPU parameter, which makes the code more flexible in the
> future. The current VGIC expect this to be the current VCPU, so add
> an assert to spot any regressions.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 .. v2:
> - rename to vgic_vcpu_pending_irq()
> - add VCPU parameter
> 
>   xen/arch/arm/gic-vgic.c     | 16 ++++++++++++++--
>   xen/include/asm-arm/event.h |  2 +-
>   xen/include/asm-arm/gic.h   |  2 +-
>   3 files changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> index ecb07ceb40..61f093db50 100644
> --- a/xen/arch/arm/gic-vgic.c
> +++ b/xen/arch/arm/gic-vgic.c
> @@ -339,9 +339,18 @@ void gic_clear_pending_irqs(struct vcpu *v)
>           gic_remove_from_lr_pending(v, p);
>   }
>   
> -int gic_events_need_delivery(void)
> +/**
> + * vgic_vcpu_pending_irq() - determine if interrupts need to be injected
> + * @vcpu: The vCPU on which to check for interrupts.
> + *
> + * Checks whether there is an interrupt on the given VCPU which needs
> + * handling in the guest. This requires at least one IRQ to be pending
> + * and enabled.
> + *
> + * Returns: 1 if the guest should run to handle interrupts, 0 otherwise.
> + */
> +int vgic_vcpu_pending_irq(struct vcpu *v)
>   {
> -    struct vcpu *v = current;
>       struct pending_irq *p;
>       unsigned long flags;
>       const unsigned long apr = gic_hw_ops->read_apr(0);
> @@ -349,6 +358,9 @@ int gic_events_need_delivery(void)
>       int active_priority;
>       int rc = 0;
>   
> +    /* We rely on reading the VMCR, which is only accessible locally. */
> +    ASSERT(v == current);
> +
>       mask_priority = gic_hw_ops->read_vmcr_priority();
>       active_priority = find_next_bit(&apr, 32, 0);
>   
> diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
> index e8c2a6cb44..c7a415ef57 100644
> --- a/xen/include/asm-arm/event.h
> +++ b/xen/include/asm-arm/event.h
> @@ -24,7 +24,7 @@ static inline int local_events_need_delivery_nomask(void)
>        * interrupts disabled so this shouldn't be a problem in the general
>        * case.
>        */
> -    if ( gic_events_need_delivery() )
> +    if ( vgic_vcpu_pending_irq(current) )
>           return 1;
>   
>       if ( !vcpu_info(current, evtchn_upcall_pending) )
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index d568957dd1..49cb94f792 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -238,7 +238,7 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
>   
>   extern void vgic_sync_to_lrs(void);
>   extern void gic_clear_pending_irqs(struct vcpu *v);
> -extern int gic_events_need_delivery(void);
> +extern int vgic_vcpu_pending_irq(struct vcpu *v);
>   
>   extern void init_maintenance_interrupt(void);
>   extern void gic_raise_guest_irq(struct vcpu *v, unsigned int irq,
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 02/45] ARM: Implement vcpu_kick()

[Julien Grall]

Hi Andre,

On 15/03/18 20:30, Andre Przywara wrote:
> If we change something in a vCPU that affects its runnability or
> otherwise needs the vCPU's attention, we might need to tell the scheduler
> about it.
> We are using this in one place (vIRQ injection) at the moment, but will
> need this at more places soon.
> So let's factor out this functionality, using the already existing
> vcpu_kick() prototype (used in x86 only so far), to make this available
> to the rest of the Xen code.
> Also adjust the perfcounter name to reflect the new usage.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - use vcpu_kick() name from x86 and existing prototype
> - rename performance counter
> 
>   xen/arch/arm/domain.c            | 12 ++++++++++++
>   xen/arch/arm/vgic.c              | 11 +++--------
>   xen/include/asm-arm/perfc_defn.h |  3 ++-
>   3 files changed, 17 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index bc10f412ba..4462e62599 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -967,6 +967,18 @@ void vcpu_block_unless_event_pending(struct vcpu *v)
>           vcpu_unblock(current);
>   }
>   
> +void vcpu_kick(struct vcpu *vcpu)
> +{
> +    bool running = vcpu->is_running;
> +
> +    vcpu_unblock(vcpu);
> +    if ( running && vcpu != current )
> +    {
> +        perfc_incr(vcpu_kick);
> +        smp_send_event_check_mask(cpumask_of(vcpu->processor));
> +    }
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index eb09d9ca54..3fafdd0b66 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -26,6 +26,7 @@
>   #include <xen/sched.h>
>   #include <xen/perfc.h>
>   
> +#include <asm/event.h>
>   #include <asm/current.h>
>   
>   #include <asm/mmio.h>
> @@ -530,7 +531,6 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>       uint8_t priority;
>       struct pending_irq *iter, *n;
>       unsigned long flags;
> -    bool running;
>   
>       /*
>        * For edge triggered interrupts we always ignore a "falling edge".
> @@ -590,14 +590,9 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>       list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
>   out:
>       spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> +
>       /* we have a new higher priority irq, inject it into the guest */
> -    running = v->is_running;
> -    vcpu_unblock(v);
> -    if ( running && v != current )
> -    {
> -        perfc_incr(vgic_cross_cpu_intr_inject);
> -        smp_send_event_check_mask(cpumask_of(v->processor));
> -    }
> +    vcpu_kick(v);
>   
>       return;
>   }
> diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
> index 87866264ca..8922e9525a 100644
> --- a/xen/include/asm-arm/perfc_defn.h
> +++ b/xen/include/asm-arm/perfc_defn.h
> @@ -33,6 +33,8 @@ PERFCOUNTER(vpsci_cpu_suspend,         "vpsci: cpu_suspend")
>   PERFCOUNTER(vpsci_cpu_affinity_info,   "vpsci: cpu_affinity_info")
>   PERFCOUNTER(vpsci_features,            "vpsci: features")
>   
> +PERFCOUNTER(vcpu_kick,                 "vcpu: notify other vcpu")
> +
>   PERFCOUNTER(vgicd_reads,                "vgicd: read")
>   PERFCOUNTER(vgicd_writes,               "vgicd: write")
>   PERFCOUNTER(vgicr_reads,                "vgicr: read")
> @@ -44,7 +46,6 @@ PERFCOUNTER(vgic_sysreg_writes,         "vgic: sysreg write")
>   PERFCOUNTER(vgic_sgi_list  ,            "vgic: SGI send to list")
>   PERFCOUNTER(vgic_sgi_others,            "vgic: SGI send to others")
>   PERFCOUNTER(vgic_sgi_self,              "vgic: SGI send to self")
> -PERFCOUNTER(vgic_cross_cpu_intr_inject, "vgic: cross-CPU irq inject")
>   PERFCOUNTER(vgic_irq_migrates,          "vgic: irq migration")
>   
>   PERFCOUNTER(vuart_reads,  "vuart: read")
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Andre Przywara]

Hi,

On 16/03/18 10:48, Jan Beulich wrote:
>>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
>> --- a/xen/common/Makefile
>> +++ b/xen/common/Makefile
>> @@ -19,6 +19,7 @@ obj-y += keyhandler.o
>>  obj-$(CONFIG_KEXEC) += kexec.o
>>  obj-$(CONFIG_KEXEC) += kimage.o
>>  obj-y += lib.o
>> +obj-y += list_sort.o
> 
> Why here rather than in patch 17? And why also for x86? I think you
> want a promptless Kconfig option that an arch can select if it needs
> this code, unless or until common code makes use of it.

Yeah, I had obj-$(CONFIG_NEW_VGIC) before, but Julien disliked it.
Promptless Kconfig sounds good to me.

Thanks!
Andre

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Jan Beulich]

>>> On 16.03.18 at 12:10, <andre.przywara@linaro.org> wrote:
> On 16/03/18 10:48, Jan Beulich wrote:
>>>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
>>> --- a/xen/common/Makefile
>>> +++ b/xen/common/Makefile
>>> @@ -19,6 +19,7 @@ obj-y += keyhandler.o
>>>  obj-$(CONFIG_KEXEC) += kexec.o
>>>  obj-$(CONFIG_KEXEC) += kimage.o
>>>  obj-y += lib.o
>>> +obj-y += list_sort.o
>> 
>> Why here rather than in patch 17? And why also for x86? I think you
>> want a promptless Kconfig option that an arch can select if it needs
>> this code, unless or until common code makes use of it.
> 
> Yeah, I had obj-$(CONFIG_NEW_VGIC) before, but Julien disliked it.
> Promptless Kconfig sounds good to me.

And note I'm not asking for a VGIC option, but for a LIST_SORT one.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Andre Przywara]

Hi,

On 16/03/18 11:32, Jan Beulich wrote:
>>>> On 16.03.18 at 12:10, <andre.przywara@linaro.org> wrote:
>> On 16/03/18 10:48, Jan Beulich wrote:
>>>>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
>>>> --- a/xen/common/Makefile
>>>> +++ b/xen/common/Makefile
>>>> @@ -19,6 +19,7 @@ obj-y += keyhandler.o
>>>>  obj-$(CONFIG_KEXEC) += kexec.o
>>>>  obj-$(CONFIG_KEXEC) += kimage.o
>>>>  obj-y += lib.o
>>>> +obj-y += list_sort.o
>>>
>>> Why here rather than in patch 17? And why also for x86? I think you
>>> want a promptless Kconfig option that an arch can select if it needs
>>> this code, unless or until common code makes use of it.
>>
>> Yeah, I had obj-$(CONFIG_NEW_VGIC) before, but Julien disliked it.
>> Promptless Kconfig sounds good to me.
> 
> And note I'm not asking for a VGIC option, but for a LIST_SORT one.

Yeah, I got that ;-)
CONFIG_LIST_SORT or CONFIG_HAS_LIST_SORT?

Cheers,
Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Jan Beulich]

>>> On 16.03.18 at 16:13, <andre.przywara@linaro.org> wrote:
> Hi,
> 
> On 16/03/18 11:32, Jan Beulich wrote:
>>>>> On 16.03.18 at 12:10, <andre.przywara@linaro.org> wrote:
>>> On 16/03/18 10:48, Jan Beulich wrote:
>>>>>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
>>>>> --- a/xen/common/Makefile
>>>>> +++ b/xen/common/Makefile
>>>>> @@ -19,6 +19,7 @@ obj-y += keyhandler.o
>>>>>  obj-$(CONFIG_KEXEC) += kexec.o
>>>>>  obj-$(CONFIG_KEXEC) += kimage.o
>>>>>  obj-y += lib.o
>>>>> +obj-y += list_sort.o
>>>>
>>>> Why here rather than in patch 17? And why also for x86? I think you
>>>> want a promptless Kconfig option that an arch can select if it needs
>>>> this code, unless or until common code makes use of it.
>>>
>>> Yeah, I had obj-$(CONFIG_NEW_VGIC) before, but Julien disliked it.
>>> Promptless Kconfig sounds good to me.
>> 
>> And note I'm not asking for a VGIC option, but for a LIST_SORT one.
> 
> Yeah, I got that ;-)
> CONFIG_LIST_SORT or CONFIG_HAS_LIST_SORT?

The former (or CONFIG_NEEDS_LIST_SORT or CONFIG_WANT_LIST_SORT).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 09/45] ARM: GIC: Allow tweaking the active and pending state of an IRQ

[Andre Przywara]

Hi,

On 15/03/18 20:30, Andre Przywara wrote:
> When playing around with hardware mapped, level triggered virtual IRQs,
> there is the need to explicitly set the active or pending state of an
> interrupt at some point.
> To prepare the GIC for that, we introduce a set_active_state() and a
> set_pending_state() function to let the VGIC manipulate the state of
> an associated hardware IRQ.
> This takes care of properly setting the _IRQ_INPROGRESS bit.

after having discussed this with Julien off-line, we can simplify the
_IRQ_INPROGRESS setting. See below:

> For this it adds gicv2/3_peek_irq() helper functions, to read a bit in a
> bitmap spread over several MMIO registers.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> Changelog v1 ... v2:
> - properly set _IRQ_INPROGRESS bit
> - add gicv[23]_peek_irq() (pulled in from later patch)
> - move wrappers functions into gic.h
> 
>  xen/arch/arm/gic-v2.c     | 48 +++++++++++++++++++++++++++++++++++++++++++
>  xen/arch/arm/gic-v3.c     | 52 +++++++++++++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/gic.h | 24 ++++++++++++++++++++++
>  3 files changed, 124 insertions(+)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 7dfe6fc68d..c6fcbf59d0 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -243,6 +243,52 @@ static void gicv2_poke_irq(struct irq_desc *irqd, uint32_t offset)
>      writel_gicd(1U << (irqd->irq % 32), offset + (irqd->irq / 32) * 4);
>  }
>  
> +static bool gicv2_peek_irq(struct irq_desc *irqd, uint32_t offset)
> +{
> +    uint32_t reg;
> +
> +    reg = readl_gicd(offset + (irqd->irq / 32) * 4) & (1U << (irqd->irq % 32));
> +
> +    return reg;
> +}
> +
> +static void gicv2_set_active_state(struct irq_desc *irqd, bool active)
> +{
> +    ASSERT(spin_is_locked(&irqd->lock));
> +
> +    if ( active )
> +    {
> +        if ( test_bit(_IRQ_GUEST, &irqd->status) )
> +            set_bit(_IRQ_INPROGRESS, &irqd->status);
> +        gicv2_poke_irq(irqd, GICD_ISACTIVER);
> +    }
> +    else
> +    {
> +        gicv2_poke_irq(irqd, GICD_ICACTIVER);
> +        if ( !gicv2_peek_irq(irqd, GICD_ISPENDR) &&
> +             test_bit(_IRQ_GUEST, &irqd->status) )
> +            clear_bit(_IRQ_INPROGRESS, &irqd->status);

The check for the pending bit shouldn't be necessary:
- If the interrupt is (still) pending, clearing the active bit should
trigger it again. The _IRQ_INPROGRESS bit will be set in turn.
- If the interrupt is not pending, we need to clear the bit anyway.

So reading the pending state is not necessary, we can always
unconditionally clear the _IRQ_INPROGRESS bit, ideally before writing to
ICACTIVER.

> +    }
> +}
> +
> +static void gicv2_set_pending_state(struct irq_desc *irqd, bool pending)
> +{
> +    ASSERT(spin_is_locked(&irqd->lock));
> +
> +    if ( pending )
> +    {
> +        /* The INPROGRESS bit will be set when the interrupt fires. */
> +        gicv2_poke_irq(irqd, GICD_ISPENDR);
> +    }
> +    else
> +    {
> +        gicv2_poke_irq(irqd, GICD_ICPENDR);
> +        if ( !gicv2_peek_irq(irqd, GICD_ISACTIVER) &&
> +             test_bit(_IRQ_GUEST, &irqd->status) )
> +            clear_bit(_IRQ_INPROGRESS, &irqd->status);

We should not need to touch the _IRQ_INPROGRESS bit here. That bit
really shadows the *active* bit, so changing the pending state should
not matter here:
- If the h/w IRQ is active, the bit is set already and should remain so,
as Xen and the guest are still dealing with it. Clearing the h/w pending
state does not change that.
- If the h/w IRQ is not active, the _IRQ_INPROGRESS bit is not set, so
clearing it would be a NOP.
So we can remove  the _IRQ_INPROGRESS handling here completely.

I will amend the code accordingly, including the respective GICv3 parts.

Cheers,
Andre.

> +    }
> +}
> +
>  static void gicv2_set_irq_type(struct irq_desc *desc, unsigned int type)
>  {
>      uint32_t cfg, actual, edgebit;
> @@ -1277,6 +1323,8 @@ const static struct gic_hw_operations gicv2_ops = {
>      .eoi_irq             = gicv2_eoi_irq,
>      .deactivate_irq      = gicv2_dir_irq,
>      .read_irq            = gicv2_read_irq,
> +    .set_active_state    = gicv2_set_active_state,
> +    .set_pending_state   = gicv2_set_pending_state,
>      .set_irq_type        = gicv2_set_irq_type,
>      .set_irq_priority    = gicv2_set_irq_priority,
>      .send_SGI            = gicv2_send_SGI,
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 392cf91b58..316f2c4142 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -444,6 +444,19 @@ static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset, bool wait_for_rwp)
>          gicv3_wait_for_rwp(irqd->irq);
>  }
>  
> +static bool gicv3_peek_irq(struct irq_desc *irqd, u32 offset)
> +{
> +    void __iomem *base;
> +    unsigned int irq = irqd->irq;
> +
> +    if ( irq >= NR_GIC_LOCAL_IRQS)
> +        base = GICD + (irq / 32) * 4;
> +    else
> +        base = GICD_RDIST_SGI_BASE;
> +
> +    return !!(readl(base + offset) & (1U << (irq % 32)));
> +}
> +
>  static void gicv3_unmask_irq(struct irq_desc *irqd)
>  {
>      gicv3_poke_irq(irqd, GICD_ISENABLER, false);
> @@ -477,6 +490,43 @@ static unsigned int gicv3_read_irq(void)
>      return irq;
>  }
>  
> +static void gicv3_set_active_state(struct irq_desc *irqd, bool active)
> +{
> +    ASSERT(spin_is_locked(&irqd->lock));
> +
> +    if ( active )
> +    {
> +        if ( test_bit(_IRQ_GUEST, &irqd->status) )
> +            set_bit(_IRQ_INPROGRESS, &irqd->status);
> +        gicv3_poke_irq(irqd, GICD_ISACTIVER, false);
> +    }
> +    else
> +    {
> +        gicv3_poke_irq(irqd, GICD_ICACTIVER, false);
> +        if ( !gicv3_peek_irq(irqd, GICD_ISPENDR) &&
> +             test_bit(_IRQ_GUEST, &irqd->status) )
> +            clear_bit(_IRQ_INPROGRESS, &irqd->status);
> +    }
> +}
> +
> +static void gicv3_set_pending_state(struct irq_desc *irqd, bool pending)
> +{
> +    ASSERT(spin_is_locked(&irqd->lock));
> +
> +    if ( pending )
> +    {
> +        /* The INPROGRESS bit will be set when the interrupt fires. */
> +        gicv3_poke_irq(irqd, GICD_ISPENDR, false);
> +    }
> +    else
> +    {
> +        gicv3_poke_irq(irqd, GICD_ICPENDR, false);
> +        if ( !gicv3_peek_irq(irqd, GICD_ISACTIVER) &&
> +             test_bit(_IRQ_GUEST, &irqd->status) )
> +            clear_bit(_IRQ_INPROGRESS, &irqd->status);
> +    }
> +}
> +
>  static inline uint64_t gicv3_mpidr_to_affinity(int cpu)
>  {
>       uint64_t mpidr = cpu_logical_map(cpu);
> @@ -1766,6 +1816,8 @@ static const struct gic_hw_operations gicv3_ops = {
>      .eoi_irq             = gicv3_eoi_irq,
>      .deactivate_irq      = gicv3_dir_irq,
>      .read_irq            = gicv3_read_irq,
> +    .set_active_state    = gicv3_set_active_state,
> +    .set_pending_state   = gicv3_set_pending_state,
>      .set_irq_type        = gicv3_set_irq_type,
>      .set_irq_priority    = gicv3_set_irq_priority,
>      .send_SGI            = gicv3_send_sgi,
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 565b0875ca..21cf35f106 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -344,6 +344,10 @@ struct gic_hw_operations {
>      void (*deactivate_irq)(struct irq_desc *irqd);
>      /* Read IRQ id and Ack */
>      unsigned int (*read_irq)(void);
> +    /* Force the active state of an IRQ by accessing the distributor */
> +    void (*set_active_state)(struct irq_desc *irqd, bool state);
> +    /* Force the pending state of an IRQ by accessing the distributor */
> +    void (*set_pending_state)(struct irq_desc *irqd, bool state);
>      /* Set IRQ type */
>      void (*set_irq_type)(struct irq_desc *desc, unsigned int type);
>      /* Set IRQ priority */
> @@ -392,6 +396,26 @@ static inline unsigned int gic_get_nr_lrs(void)
>      return gic_hw_ops->info->nr_lrs;
>  }
>  
> +/*
> + * Set the active state of an IRQ. This should be used with care, as this
> + * directly forces the active bit, without considering the GIC state machine.
> + * For private IRQs this only works for those of the current CPU.
> + */
> +static inline void gic_set_active_state(struct irq_desc *irqd, bool state)
> +{
> +    gic_hw_ops->set_active_state(irqd, state);
> +}
> +
> +/*
> + * Set the pending state of an IRQ. This should be used with care, as this
> + * directly forces the pending bit, without considering the GIC state machine.
> + * For private IRQs this only works for those of the current CPU.
> + */
> +static inline void gic_set_pending_state(struct irq_desc *irqd, bool state)
> +{
> +    gic_hw_ops->set_pending_state(irqd, state);
> +}
> +
>  void register_gic_ops(const struct gic_hw_operations *ops);
>  int gic_make_hwdom_dt_node(const struct domain *d,
>                             const struct dt_device_node *gic,
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 01/45] ARM: VGIC: rename gic_event_needs_delivery()

[Stefano Stabellini]

On Thu, 15 Mar 2018, Andre Przywara wrote:
> gic_event_needs_delivery() is not named very intuitively, especially
> the gic_ prefix is somewhat misleading.
> Rename it to vgic_vcpu_pending_irq(), which makes it clear that this
> relates to the virtual GIC and is about interrupts.
> Also add a VCPU parameter, which makes the code more flexible in the
> future. The current VGIC expect this to be the current VCPU, so add
> an assert to spot any regressions.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changelog v1 .. v2:
> - rename to vgic_vcpu_pending_irq()
> - add VCPU parameter                                                            
> 
>  xen/arch/arm/gic-vgic.c     | 16 ++++++++++++++--
>  xen/include/asm-arm/event.h |  2 +-
>  xen/include/asm-arm/gic.h   |  2 +-
>  3 files changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> index ecb07ceb40..61f093db50 100644
> --- a/xen/arch/arm/gic-vgic.c
> +++ b/xen/arch/arm/gic-vgic.c
> @@ -339,9 +339,18 @@ void gic_clear_pending_irqs(struct vcpu *v)
>          gic_remove_from_lr_pending(v, p);
>  }
>  
> -int gic_events_need_delivery(void)
> +/**
> + * vgic_vcpu_pending_irq() - determine if interrupts need to be injected
> + * @vcpu: The vCPU on which to check for interrupts.
> + *
> + * Checks whether there is an interrupt on the given VCPU which needs
> + * handling in the guest. This requires at least one IRQ to be pending
> + * and enabled.
> + *
> + * Returns: 1 if the guest should run to handle interrupts, 0 otherwise.
> + */
> +int vgic_vcpu_pending_irq(struct vcpu *v)
>  {
> -    struct vcpu *v = current;
>      struct pending_irq *p;
>      unsigned long flags;
>      const unsigned long apr = gic_hw_ops->read_apr(0);
> @@ -349,6 +358,9 @@ int gic_events_need_delivery(void)
>      int active_priority;
>      int rc = 0;
>  
> +    /* We rely on reading the VMCR, which is only accessible locally. */
> +    ASSERT(v == current);
> +
>      mask_priority = gic_hw_ops->read_vmcr_priority();
>      active_priority = find_next_bit(&apr, 32, 0);
>  
> diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
> index e8c2a6cb44..c7a415ef57 100644
> --- a/xen/include/asm-arm/event.h
> +++ b/xen/include/asm-arm/event.h
> @@ -24,7 +24,7 @@ static inline int local_events_need_delivery_nomask(void)
>       * interrupts disabled so this shouldn't be a problem in the general
>       * case.
>       */
> -    if ( gic_events_need_delivery() )
> +    if ( vgic_vcpu_pending_irq(current) )
>          return 1;
>  
>      if ( !vcpu_info(current, evtchn_upcall_pending) )
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index d568957dd1..49cb94f792 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -238,7 +238,7 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
>  
>  extern void vgic_sync_to_lrs(void);
>  extern void gic_clear_pending_irqs(struct vcpu *v);
> -extern int gic_events_need_delivery(void);
> +extern int vgic_vcpu_pending_irq(struct vcpu *v);
>  
>  extern void init_maintenance_interrupt(void);
>  extern void gic_raise_guest_irq(struct vcpu *v, unsigned int irq,
> -- 
> 2.14.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 02/45] ARM: Implement vcpu_kick()

[Stefano Stabellini]

On Thu, 15 Mar 2018, Andre Przywara wrote:
> If we change something in a vCPU that affects its runnability or
> otherwise needs the vCPU's attention, we might need to tell the scheduler
> about it.
> We are using this in one place (vIRQ injection) at the moment, but will
> need this at more places soon.
> So let's factor out this functionality, using the already existing
> vcpu_kick() prototype (used in x86 only so far), to make this available
> to the rest of the Xen code.
> Also adjust the perfcounter name to reflect the new usage.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changelog v1 ... v2:
> - use vcpu_kick() name from x86 and existing prototype
> - rename performance counter
> 
>  xen/arch/arm/domain.c            | 12 ++++++++++++
>  xen/arch/arm/vgic.c              | 11 +++--------
>  xen/include/asm-arm/perfc_defn.h |  3 ++-
>  3 files changed, 17 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index bc10f412ba..4462e62599 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -967,6 +967,18 @@ void vcpu_block_unless_event_pending(struct vcpu *v)
>          vcpu_unblock(current);
>  }
>  
> +void vcpu_kick(struct vcpu *vcpu)
> +{
> +    bool running = vcpu->is_running;
> +
> +    vcpu_unblock(vcpu);
> +    if ( running && vcpu != current )
> +    {
> +        perfc_incr(vcpu_kick);
> +        smp_send_event_check_mask(cpumask_of(vcpu->processor));
> +    }
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index eb09d9ca54..3fafdd0b66 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -26,6 +26,7 @@
>  #include <xen/sched.h>
>  #include <xen/perfc.h>
>  
> +#include <asm/event.h>
>  #include <asm/current.h>
>  
>  #include <asm/mmio.h>
> @@ -530,7 +531,6 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>      uint8_t priority;
>      struct pending_irq *iter, *n;
>      unsigned long flags;
> -    bool running;
>  
>      /*
>       * For edge triggered interrupts we always ignore a "falling edge".
> @@ -590,14 +590,9 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>      list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
>  out:
>      spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> +
>      /* we have a new higher priority irq, inject it into the guest */
> -    running = v->is_running;
> -    vcpu_unblock(v);
> -    if ( running && v != current )
> -    {
> -        perfc_incr(vgic_cross_cpu_intr_inject);
> -        smp_send_event_check_mask(cpumask_of(v->processor));
> -    }
> +    vcpu_kick(v);
>  
>      return;
>  }
> diff --git a/xen/include/asm-arm/perfc_defn.h b/xen/include/asm-arm/perfc_defn.h
> index 87866264ca..8922e9525a 100644
> --- a/xen/include/asm-arm/perfc_defn.h
> +++ b/xen/include/asm-arm/perfc_defn.h
> @@ -33,6 +33,8 @@ PERFCOUNTER(vpsci_cpu_suspend,         "vpsci: cpu_suspend")
>  PERFCOUNTER(vpsci_cpu_affinity_info,   "vpsci: cpu_affinity_info")
>  PERFCOUNTER(vpsci_features,            "vpsci: features")
>  
> +PERFCOUNTER(vcpu_kick,                 "vcpu: notify other vcpu")
> +
>  PERFCOUNTER(vgicd_reads,                "vgicd: read")
>  PERFCOUNTER(vgicd_writes,               "vgicd: write")
>  PERFCOUNTER(vgicr_reads,                "vgicr: read")
> @@ -44,7 +46,6 @@ PERFCOUNTER(vgic_sysreg_writes,         "vgic: sysreg write")
>  PERFCOUNTER(vgic_sgi_list  ,            "vgic: SGI send to list")
>  PERFCOUNTER(vgic_sgi_others,            "vgic: SGI send to others")
>  PERFCOUNTER(vgic_sgi_self,              "vgic: SGI send to self")
> -PERFCOUNTER(vgic_cross_cpu_intr_inject, "vgic: cross-CPU irq inject")
>  PERFCOUNTER(vgic_irq_migrates,          "vgic: irq migration")
>  
>  PERFCOUNTER(vuart_reads,  "vuart: read")
> -- 
> 2.14.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 04/45] xen/arm: vgic: Override the group in lr everytime

[Stefano Stabellini]

On Thu, 15 Mar 2018, Andre Przywara wrote:
> From: Julien Grall <julien.grall@arm.com>
> 
> At the moment, write_lr is assuming the caller will set correctly the
> group. However the group should always be 0 when the guest is using
> vGICv2 and 1 for vGICv3. As the caller should not care about the group,
> override it directly.
> 
> With that change, write_lr is now behaving like update_lr for the group.
> 
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes:
> - Add Andre's reviewed-by
> 
>  xen/arch/arm/gic-v2.c     |  4 +---
>  xen/arch/arm/gic-v3.c     | 11 ++++++++---
>  xen/include/asm-arm/gic.h |  1 -
>  3 files changed, 9 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index f16e17c1a3..fc105c08b8 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -469,7 +469,6 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
>      lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
>      lr_reg->state     = (lrv >> GICH_V2_LR_STATE_SHIFT) & GICH_V2_LR_STATE_MASK;
>      lr_reg->hw_status = (lrv >> GICH_V2_LR_HW_SHIFT) & GICH_V2_LR_HW_MASK;
> -    lr_reg->grp       = (lrv >> GICH_V2_LR_GRP_SHIFT) & GICH_V2_LR_GRP_MASK;
>  }
>  
>  static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
> @@ -483,8 +482,7 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
>            ((uint32_t)(lr_reg->state & GICH_V2_LR_STATE_MASK)
>                                     << GICH_V2_LR_STATE_SHIFT) |
>            ((uint32_t)(lr_reg->hw_status & GICH_V2_LR_HW_MASK)
> -                                       << GICH_V2_LR_HW_SHIFT)  |
> -          ((uint32_t)(lr_reg->grp & GICH_V2_LR_GRP_MASK) << GICH_V2_LR_GRP_SHIFT) );
> +                                       << GICH_V2_LR_HW_SHIFT));
>  
>      writel_gich(lrv, GICH_LR + lr * 4);
>  }
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 09b49a07d5..0dfa1a1e08 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1012,7 +1012,6 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
>      lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
>      lr_reg->state     = (lrv >> ICH_LR_STATE_SHIFT) & ICH_LR_STATE_MASK;
>      lr_reg->hw_status = (lrv >> ICH_LR_HW_SHIFT) & ICH_LR_HW_MASK;
> -    lr_reg->grp       = (lrv >> ICH_LR_GRP_SHIFT) & ICH_LR_GRP_MASK;
>  }
>  
>  static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
> @@ -1023,8 +1022,14 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
>          ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
>          ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT)|
>          ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) |
> -        ((u64)(lr->hw_status & ICH_LR_HW_MASK) << ICH_LR_HW_SHIFT)  |
> -        ((u64)(lr->grp & ICH_LR_GRP_MASK) << ICH_LR_GRP_SHIFT) );
> +        ((u64)(lr->hw_status & ICH_LR_HW_MASK) << ICH_LR_HW_SHIFT) );
> +
> +    /*
> +     * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
> +     * would result in a FIQ, which will not be expected by the guest OS.
> +     */
> +    if ( current->domain->arch.vgic.version == GIC_V3 )
> +        lrv |= ICH_LR_GRP1;
>  
>      gicv3_ich_write_lr(lr_reg, lrv);
>  }
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 49cb94f792..1eb08b856e 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -211,7 +211,6 @@ struct gic_lr {
>     uint8_t priority;
>     uint8_t state;
>     uint8_t hw_status;
> -   uint8_t grp;
>  };
>  
>  enum gic_version {
> -- 
> 2.14.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 05/45] xen/arm: gic: Use bool instead of uint8_t for the hw_status in gic_lr

[Stefano Stabellini]

On Thu, 15 Mar 2018, Andre Przywara wrote:
> From: Julien Grall <julien.grall@arm.com>
> 
> hw_status can only be 1 or 0. So convert to a bool.
> 
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
> Changes:
> - Remove == *LR_HW as it is pointless
> - Add Andre's reviewed-by
> 
>  xen/arch/arm/gic-v2.c     | 9 +++++----
>  xen/arch/arm/gic-v3.c     | 8 +++++---
>  xen/include/asm-arm/gic.h | 2 +-
>  3 files changed, 11 insertions(+), 8 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index fc105c08b8..9d589115bd 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -468,7 +468,7 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
>      lr_reg->virq = (lrv >> GICH_V2_LR_VIRTUAL_SHIFT) & GICH_V2_LR_VIRTUAL_MASK;
>      lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
>      lr_reg->state     = (lrv >> GICH_V2_LR_STATE_SHIFT) & GICH_V2_LR_STATE_MASK;
> -    lr_reg->hw_status = (lrv >> GICH_V2_LR_HW_SHIFT) & GICH_V2_LR_HW_MASK;
> +    lr_reg->hw_status = lrv & GICH_V2_LR_HW;
>  }
>  
>  static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
> @@ -480,9 +480,10 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
>            ((uint32_t)(lr_reg->priority & GICH_V2_LR_PRIORITY_MASK)
>                                        << GICH_V2_LR_PRIORITY_SHIFT) |
>            ((uint32_t)(lr_reg->state & GICH_V2_LR_STATE_MASK)
> -                                   << GICH_V2_LR_STATE_SHIFT) |
> -          ((uint32_t)(lr_reg->hw_status & GICH_V2_LR_HW_MASK)
> -                                       << GICH_V2_LR_HW_SHIFT));
> +                                   << GICH_V2_LR_STATE_SHIFT) );
> +
> +    if ( lr_reg->hw_status )
> +        lrv |= GICH_V2_LR_HW;
>  
>      writel_gich(lrv, GICH_LR + lr * 4);
>  }
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 0dfa1a1e08..f761ae60d6 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1011,7 +1011,7 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
>  
>      lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
>      lr_reg->state     = (lrv >> ICH_LR_STATE_SHIFT) & ICH_LR_STATE_MASK;
> -    lr_reg->hw_status = (lrv >> ICH_LR_HW_SHIFT) & ICH_LR_HW_MASK;
> +    lr_reg->hw_status = lrv & ICH_LR_HW;
>  }
>  
>  static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
> @@ -1021,8 +1021,10 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
>      lrv = ( ((u64)(lr->pirq & ICH_LR_PHYSICAL_MASK) << ICH_LR_PHYSICAL_SHIFT)|
>          ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
>          ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT)|
> -        ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) |
> -        ((u64)(lr->hw_status & ICH_LR_HW_MASK) << ICH_LR_HW_SHIFT) );
> +        ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) );
> +
> +    if ( lr->hw_status )
> +        lrv |= ICH_LR_HW;
>  
>      /*
>       * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 1eb08b856e..daec51499c 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -210,7 +210,7 @@ struct gic_lr {
>     uint32_t virq;
>     uint8_t priority;
>     uint8_t state;
> -   uint8_t hw_status;
> +   bool hw_status;
>  };
>  
>  enum gic_version {
> -- 
> 2.14.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 06/45] xen/arm: gic: Split the field state in gic_lr in 2 fields active and pending

[Stefano Stabellini]

On Thu, 15 Mar 2018, Andre Przywara wrote:
> From: Julien Grall <julien.grall@arm.com>
> 
> Mostly making the code nicer to read.
> 
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> Changes:
> - Use 1ULL
> - Remove pointless == *_STATE_*
> 
>  xen/arch/arm/gic-v2.c             | 15 +++++++++++----
>  xen/arch/arm/gic-v3.c             | 12 +++++++++---
>  xen/arch/arm/gic-vgic.c           |  6 +++---
>  xen/include/asm-arm/gic.h         |  3 ++-
>  xen/include/asm-arm/gic_v3_defs.h |  2 ++
>  5 files changed, 27 insertions(+), 11 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 9d589115bd..6dae5c1e55 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -51,6 +51,8 @@
>  #define GICH_V2_LR_PHYSICAL_SHIFT  10
>  #define GICH_V2_LR_STATE_MASK      0x3
>  #define GICH_V2_LR_STATE_SHIFT     28
> +#define GICH_V2_LR_PENDING         (1U << 28)
> +#define GICH_V2_LR_ACTIVE          (1U << 29)
>  #define GICH_V2_LR_PRIORITY_SHIFT  23
>  #define GICH_V2_LR_PRIORITY_MASK   0x1f
>  #define GICH_V2_LR_HW_SHIFT        31
> @@ -467,7 +469,8 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
>      lr_reg->pirq = (lrv >> GICH_V2_LR_PHYSICAL_SHIFT) & GICH_V2_LR_PHYSICAL_MASK;
>      lr_reg->virq = (lrv >> GICH_V2_LR_VIRTUAL_SHIFT) & GICH_V2_LR_VIRTUAL_MASK;
>      lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
> -    lr_reg->state     = (lrv >> GICH_V2_LR_STATE_SHIFT) & GICH_V2_LR_STATE_MASK;
> +    lr_reg->pending = lrv & GICH_V2_LR_PENDING;
> +    lr_reg->active = lrv & GICH_V2_LR_ACTIVE;
>      lr_reg->hw_status = lrv & GICH_V2_LR_HW;
>  }
>  
> @@ -478,9 +481,13 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
>      lrv = ( ((lr_reg->pirq & GICH_V2_LR_PHYSICAL_MASK) << GICH_V2_LR_PHYSICAL_SHIFT) |
>            ((lr_reg->virq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT)   |
>            ((uint32_t)(lr_reg->priority & GICH_V2_LR_PRIORITY_MASK)
> -                                      << GICH_V2_LR_PRIORITY_SHIFT) |
> -          ((uint32_t)(lr_reg->state & GICH_V2_LR_STATE_MASK)
> -                                   << GICH_V2_LR_STATE_SHIFT) );
> +                                      << GICH_V2_LR_PRIORITY_SHIFT) );
> +
> +    if ( lr_reg->active )
> +        lrv |= GICH_V2_LR_ACTIVE;
> +
> +    if ( lr_reg->pending )
> +        lrv |= GICH_V2_LR_PENDING;
>  
>      if ( lr_reg->hw_status )
>          lrv |= GICH_V2_LR_HW;
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index f761ae60d6..6547b5eb0d 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1010,7 +1010,8 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
>      lr_reg->virq = (lrv >> ICH_LR_VIRTUAL_SHIFT) & ICH_LR_VIRTUAL_MASK;
>  
>      lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
> -    lr_reg->state     = (lrv >> ICH_LR_STATE_SHIFT) & ICH_LR_STATE_MASK;
> +    lr_reg->pending   = lrv & ICH_LR_STATE_PENDING;
> +    lr_reg->active    = lrv & ICH_LR_STATE_ACTIVE;
>      lr_reg->hw_status = lrv & ICH_LR_HW;
>  }
>  
> @@ -1020,8 +1021,13 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
>  
>      lrv = ( ((u64)(lr->pirq & ICH_LR_PHYSICAL_MASK) << ICH_LR_PHYSICAL_SHIFT)|
>          ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
> -        ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT)|
> -        ((u64)(lr->state & ICH_LR_STATE_MASK) << ICH_LR_STATE_SHIFT) );
> +        ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT) );
> +
> +    if ( lr->active )
> +        lrv |= ICH_LR_STATE_ACTIVE;
> +
> +    if ( lr->pending )
> +        lrv |= ICH_LR_STATE_PENDING;
>  
>      if ( lr->hw_status )
>          lrv |= ICH_LR_HW;
> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> index e3cb47e80e..d831b35525 100644
> --- a/xen/arch/arm/gic-vgic.c
> +++ b/xen/arch/arm/gic-vgic.c
> @@ -189,7 +189,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>          return;
>      }
>  
> -    if ( lr_val.state & GICH_LR_ACTIVE )
> +    if ( lr_val.active )
>      {
>          set_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
>          if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
> @@ -197,7 +197,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>          {
>              if ( p->desc == NULL )
>              {
> -                lr_val.state |= GICH_LR_PENDING;
> +                lr_val.pending = true;
>                  gic_hw_ops->write_lr(i, &lr_val);
>              }
>              else
> @@ -205,7 +205,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>                           irq, v->domain->domain_id, v->vcpu_id, i);
>          }
>      }
> -    else if ( lr_val.state & GICH_LR_PENDING )
> +    else if ( lr_val.pending )
>      {
>          int q __attribute__ ((unused)) = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
>  #ifdef GIC_DEBUG
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index daec51499c..c32861d4fa 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -209,7 +209,8 @@ struct gic_lr {
>     /* Virtual IRQ */
>     uint32_t virq;
>     uint8_t priority;
> -   uint8_t state;
> +   bool active;
> +   bool pending;
>     bool hw_status;
>  };

I like the readability but dislike the increase memory usage. I would
have kept a single uint8_t and I would have used status flags as an
approach, maybe I would have improved on those flags.

That said, it doesn't bother me enough to nack the patch :-)


> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index ccb72cf0f1..d9827bd84c 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -171,6 +171,8 @@
>  #define ICH_LR_PHYSICAL_SHIFT        32
>  #define ICH_LR_STATE_MASK            0x3
>  #define ICH_LR_STATE_SHIFT           62
> +#define ICH_LR_STATE_PENDING         (1ULL << 62)
> +#define ICH_LR_STATE_ACTIVE          (1ULL << 63)
>  #define ICH_LR_PRIORITY_MASK         0xff
>  #define ICH_LR_PRIORITY_SHIFT        48
>  #define ICH_LR_HW_MASK               0x1
> -- 
> 2.14.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 07/45] xen/arm: GIC: Only set pirq in the LR when hw_status is set

[Stefano Stabellini]

On Thu, 15 Mar 2018, Andre Przywara wrote:
> From: Julien Grall <julien.grall@arm.com>
> 
> The field pirq should only be valid when the virtual interrupt
> is associated to a physical interrupt.
> 
> This change will help to extend gic_lr for supporting specific virtual
> interrupt field (e.g eoi, source) that clashes with the PIRQ field.
> 
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
> Changes:
> - Add Andre's reviewed-by
> 
>  xen/arch/arm/gic-v2.c     | 13 ++++++++++---
>  xen/arch/arm/gic-v3.c     | 10 +++++++---
>  xen/include/asm-arm/gic.h |  2 +-
>  3 files changed, 18 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 6dae5c1e55..2f012692e0 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -466,20 +466,24 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
>      uint32_t lrv;
>  
>      lrv          = readl_gich(GICH_LR + lr * 4);
> -    lr_reg->pirq = (lrv >> GICH_V2_LR_PHYSICAL_SHIFT) & GICH_V2_LR_PHYSICAL_MASK;
>      lr_reg->virq = (lrv >> GICH_V2_LR_VIRTUAL_SHIFT) & GICH_V2_LR_VIRTUAL_MASK;
>      lr_reg->priority = (lrv >> GICH_V2_LR_PRIORITY_SHIFT) & GICH_V2_LR_PRIORITY_MASK;
>      lr_reg->pending = lrv & GICH_V2_LR_PENDING;
>      lr_reg->active = lrv & GICH_V2_LR_ACTIVE;
>      lr_reg->hw_status = lrv & GICH_V2_LR_HW;
> +
> +    if ( lr_reg->hw_status )
> +    {
> +        lr_reg->pirq = lrv >> GICH_V2_LR_PHYSICAL_SHIFT;
> +        lr_reg->pirq &= GICH_V2_LR_PHYSICAL_MASK;
> +    }
>  }
>  
>  static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
>  {
>      uint32_t lrv = 0;
>  
> -    lrv = ( ((lr_reg->pirq & GICH_V2_LR_PHYSICAL_MASK) << GICH_V2_LR_PHYSICAL_SHIFT) |
> -          ((lr_reg->virq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT)   |
> +    lrv = (((lr_reg->virq & GICH_V2_LR_VIRTUAL_MASK) << GICH_V2_LR_VIRTUAL_SHIFT)   |
>            ((uint32_t)(lr_reg->priority & GICH_V2_LR_PRIORITY_MASK)
>                                        << GICH_V2_LR_PRIORITY_SHIFT) );
>  
> @@ -490,7 +494,10 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
>          lrv |= GICH_V2_LR_PENDING;
>  
>      if ( lr_reg->hw_status )
> +    {
>          lrv |= GICH_V2_LR_HW;
> +        lrv |= lr_reg->pirq << GICH_V2_LR_PHYSICAL_SHIFT;
> +    }
>  
>      writel_gich(lrv, GICH_LR + lr * 4);
>  }
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 6547b5eb0d..e901210b78 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1006,21 +1006,22 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
>  
>      lrv = gicv3_ich_read_lr(lr);
>  
> -    lr_reg->pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
>      lr_reg->virq = (lrv >> ICH_LR_VIRTUAL_SHIFT) & ICH_LR_VIRTUAL_MASK;
>  
>      lr_reg->priority  = (lrv >> ICH_LR_PRIORITY_SHIFT) & ICH_LR_PRIORITY_MASK;
>      lr_reg->pending   = lrv & ICH_LR_STATE_PENDING;
>      lr_reg->active    = lrv & ICH_LR_STATE_ACTIVE;
>      lr_reg->hw_status = lrv & ICH_LR_HW;
> +
> +    if ( lr_reg->hw_status )
> +        lr_reg->pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
>  }
>  
>  static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
>  {
>      uint64_t lrv = 0;
>  
> -    lrv = ( ((u64)(lr->pirq & ICH_LR_PHYSICAL_MASK) << ICH_LR_PHYSICAL_SHIFT)|
> -        ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
> +    lrv = ( ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
>          ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT) );
>  
>      if ( lr->active )
> @@ -1030,7 +1031,10 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
>          lrv |= ICH_LR_STATE_PENDING;
>  
>      if ( lr->hw_status )
> +    {
>          lrv |= ICH_LR_HW;
> +        lrv |= (uint64_t)lr->pirq << ICH_LR_PHYSICAL_SHIFT;
> +    }
>  
>      /*
>       * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index c32861d4fa..545901b120 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -204,7 +204,7 @@ union gic_state_data {
>   * The LR register format is different for GIC HW version
>   */
>  struct gic_lr {
> -   /* Physical IRQ */
> +   /* Physical IRQ -> Only set when hw_status is set. */
>     uint32_t pirq;
>     /* Virtual IRQ */
>     uint32_t virq;
> -- 
> 2.14.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 08/45] ARM: GIC: extend LR read/write functions to cover EOI and source

[Stefano Stabellini]

On Thu, 15 Mar 2018, Andre Przywara wrote:
> From: Julien Grall <julien.grall@arm.com>
> 
> So far our LR read/write functions do not handle the EOI bit and the
> source CPUID bits in an LR, because the current VGIC implementation does
> not use them.
> Extend the gic_lr data structure to hold these bits of information by
> using a union to differentiate field used depending on whether the vIRQ
> has a corresponding pIRQ.
> 
> This allows the new VGIC to use this information.
> 
> This is based on the original patch sent by Andre Przywara [1].
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2018-03/msg00435.html
> 
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
> Changes:
> - Reset authorship
> - Rework source
> - Add source support on GICv3
> 
>  xen/arch/arm/gic-v2.c             | 26 +++++++++++++++++++++++---
>  xen/arch/arm/gic-v3.c             | 34 +++++++++++++++++++++++++++++++---
>  xen/include/asm-arm/gic.h         | 16 ++++++++++++++--
>  xen/include/asm-arm/gic_v3_defs.h |  2 ++
>  4 files changed, 70 insertions(+), 8 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 2f012692e0..7dfe6fc68d 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -474,8 +474,17 @@ static void gicv2_read_lr(int lr, struct gic_lr *lr_reg)
>  
>      if ( lr_reg->hw_status )
>      {
> -        lr_reg->pirq = lrv >> GICH_V2_LR_PHYSICAL_SHIFT;
> -        lr_reg->pirq &= GICH_V2_LR_PHYSICAL_MASK;
> +        lr_reg->hw.pirq = lrv >> GICH_V2_LR_PHYSICAL_SHIFT;
> +        lr_reg->hw.pirq &= GICH_V2_LR_PHYSICAL_MASK;
> +    }
> +    else
> +    {
> +        lr_reg->virt.eoi = (lrv & GICH_V2_LR_MAINTENANCE_IRQ);
> +        if ( lr_reg->virq < NR_GIC_SGI )
> +        {
> +            lr_reg->virt.source = (lrv >> GICH_V2_LR_CPUID_SHIFT)
> +                & GICH_V2_LR_CPUID_MASK;
> +        }
>      }
>  }
>  
> @@ -496,7 +505,18 @@ static void gicv2_write_lr(int lr, const struct gic_lr *lr_reg)
>      if ( lr_reg->hw_status )
>      {
>          lrv |= GICH_V2_LR_HW;
> -        lrv |= lr_reg->pirq << GICH_V2_LR_PHYSICAL_SHIFT;
> +        lrv |= lr_reg->hw.pirq << GICH_V2_LR_PHYSICAL_SHIFT;
> +    }
> +    else
> +    {
> +        if ( lr_reg->virt.eoi )
> +            lrv |= GICH_V2_LR_MAINTENANCE_IRQ;
> +        /*
> +         * This is only valid for SGI, but it does not matter to always
> +         * read it as it should be 0 by default.
> +         */
> +        ASSERT(!lr_reg->virt.source || lr_reg->virq < NR_GIC_SGI);
> +        lrv |= (uint32_t)lr_reg->virt.source << GICH_V2_LR_CPUID_SHIFT;
>      }
>  
>      writel_gich(lrv, GICH_LR + lr * 4);
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index e901210b78..392cf91b58 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1014,12 +1014,25 @@ static void gicv3_read_lr(int lr, struct gic_lr *lr_reg)
>      lr_reg->hw_status = lrv & ICH_LR_HW;
>  
>      if ( lr_reg->hw_status )
> -        lr_reg->pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
> +        lr_reg->hw.pirq = (lrv >> ICH_LR_PHYSICAL_SHIFT) & ICH_LR_PHYSICAL_MASK;
> +    else
> +    {
> +        lr_reg->virt.eoi = (lrv & ICH_LR_MAINTENANCE_IRQ);
> +        /* Source only exists for SGI and in GICv2 compatible mode */
> +        if ( lr_reg->virq < NR_GIC_SGI &&
> +             current->domain->arch.vgic.version == GIC_V2 )
> +        {
> +            lr_reg->virt.source = (lrv >> ICH_LR_CPUID_SHIFT)
> +                & ICH_LR_CPUID_MASK;
> +        }
> +    }
>  }
>  
>  static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
>  {
>      uint64_t lrv = 0;
> +    const enum gic_version vgic_version = current->domain->arch.vgic.version;
> +
>  
>      lrv = ( ((u64)(lr->virq & ICH_LR_VIRTUAL_MASK)  << ICH_LR_VIRTUAL_SHIFT) |
>          ((u64)(lr->priority & ICH_LR_PRIORITY_MASK) << ICH_LR_PRIORITY_SHIFT) );
> @@ -1033,14 +1046,29 @@ static void gicv3_write_lr(int lr_reg, const struct gic_lr *lr)
>      if ( lr->hw_status )
>      {
>          lrv |= ICH_LR_HW;
> -        lrv |= (uint64_t)lr->pirq << ICH_LR_PHYSICAL_SHIFT;
> +        lrv |= (uint64_t)lr->hw.pirq << ICH_LR_PHYSICAL_SHIFT;
> +    }
> +    else
> +    {
> +        if ( lr->virt.eoi )
> +            lrv |= ICH_LR_MAINTENANCE_IRQ;
> +        /* Source is only set in GICv2 compatible mode */
> +        if ( vgic_version == GIC_V2 )
> +        {
> +            /*
> +             * This is only valid for SGI, but it does not matter to always
> +             * read it as it should be 0 by default.
> +             */
> +            ASSERT(!lr->virt.source || lr->virq < NR_GIC_SGI);
> +            lrv |= (uint64_t)lr->virt.source << ICH_LR_CPUID_SHIFT;
> +        }
>      }
>  
>      /*
>       * When the guest is using vGICv3, all the IRQs are Group 1. Group 0
>       * would result in a FIQ, which will not be expected by the guest OS.
>       */
> -    if ( current->domain->arch.vgic.version == GIC_V3 )
> +    if ( vgic_version == GIC_V3 )
>          lrv |= ICH_LR_GRP1;
>  
>      gicv3_ich_write_lr(lr_reg, lrv);
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 545901b120..565b0875ca 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -204,14 +204,26 @@ union gic_state_data {
>   * The LR register format is different for GIC HW version
>   */
>  struct gic_lr {
> -   /* Physical IRQ -> Only set when hw_status is set. */
> -   uint32_t pirq;
>     /* Virtual IRQ */
>     uint32_t virq;
>     uint8_t priority;
>     bool active;
>     bool pending;
>     bool hw_status;
> +   union
> +   {
> +       /* Only filled when there are a corresponding pIRQ (hw_state = true) */
> +       struct
> +       {
> +           uint32_t pirq;
> +       } hw;
> +       /* Only filled when there are no corresponding pIRQ (hw_state = false) */
> +       struct
> +       {
> +           bool eoi;
> +           uint8_t source;      /* GICv2 only */
> +       } virt;
> +   };
>  };
>  
>  enum gic_version {
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index d9827bd84c..10a2aeea93 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -167,6 +167,8 @@
>  
>  #define ICH_LR_VIRTUAL_MASK          0xffff
>  #define ICH_LR_VIRTUAL_SHIFT         0
> +#define ICH_LR_CPUID_MASK            0x7
> +#define ICH_LR_CPUID_SHIFT           10
>  #define ICH_LR_PHYSICAL_MASK         0x3ff
>  #define ICH_LR_PHYSICAL_SHIFT        32
>  #define ICH_LR_STATE_MASK            0x3
> -- 
> 2.14.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 06/45] xen/arm: gic: Split the field state in gic_lr in 2 fields active and pending

[Julien Grall]

On 16/03/2018 21:34, Stefano Stabellini wrote:
> On Thu, 15 Mar 2018, Andre Przywara wrote:
>> From: Julien Grall <julien.grall@arm.com>
>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>> index daec51499c..c32861d4fa 100644
>> --- a/xen/include/asm-arm/gic.h
>> +++ b/xen/include/asm-arm/gic.h
>> @@ -209,7 +209,8 @@ struct gic_lr {
>>      /* Virtual IRQ */
>>      uint32_t virq;
>>      uint8_t priority;
>> -   uint8_t state;
>> +   bool active;
>> +   bool pending;
>>      bool hw_status;
>>   };
> 
> I like the readability but dislike the increase memory usage. I would
> have kept a single uint8_t and I would have used status flags as an
> approach, maybe I would have improved on those flags.

Why is that important? gic_lr will only be allocated on the stack...

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 06/45] xen/arm: gic: Split the field state in gic_lr in 2 fields active and pending

[Stefano Stabellini]

On Fri, 16 Mar 2018, Julien Grall wrote:
> On 16/03/2018 21:34, Stefano Stabellini wrote:
> > On Thu, 15 Mar 2018, Andre Przywara wrote:
> > > From: Julien Grall <julien.grall@arm.com>
> > > diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> > > index daec51499c..c32861d4fa 100644
> > > --- a/xen/include/asm-arm/gic.h
> > > +++ b/xen/include/asm-arm/gic.h
> > > @@ -209,7 +209,8 @@ struct gic_lr {
> > >      /* Virtual IRQ */
> > >      uint32_t virq;
> > >      uint8_t priority;
> > > -   uint8_t state;
> > > +   bool active;
> > > +   bool pending;
> > >      bool hw_status;
> > >   };
> > 
> > I like the readability but dislike the increase memory usage. I would
> > have kept a single uint8_t and I would have used status flags as an
> > approach, maybe I would have improved on those flags.
> 
> Why is that important? gic_lr will only be allocated on the stack...

You are right, so it is even less important than I thought.

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 21/45] ARM: new VGIC: Implement vgic_vcpu_pending_irq

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Tell Xen whether a particular VCPU has an IRQ that needs handling
> in the guest. This is used to decide whether a VCPU is runnable or
> if a hypercall should be preempted to let the guest handle the IRQ.
> 
> This is based on Linux commit 90eee56c5f90, written by Eric Auger.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> Changelog v1 ... v2:
> - adjust to new vgic_vcpu_pending_irq() prototype, drop wrapper
> 
>   xen/arch/arm/vgic/vgic.c | 37 +++++++++++++++++++++++++++++++++++++
>   1 file changed, 37 insertions(+)
> 
> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index 035c8c8f74..a1556e53e8 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -637,6 +637,43 @@ void vgic_sync_to_lrs(void)
>       gic_hw_ops->update_hcr_status(GICH_HCR_EN, 1);
>   }
>   
> +/**
> + * vgic_vcpu_pending_irq() - determine if interrupts need to be injected
> + * @vcpu: The vCPU on which to check for interrupts.
> + *
> + * Checks whether there is an interrupt on the given VCPU which needs
> + * handling in the guest. This requires at least one IRQ to be pending
> + * and enabled.
> + *
> + * Returns: 1 if the guest should run to handle interrupts, 0 otherwise.

This does not match with the current implementation as you return bool.

> + */
> +int vgic_vcpu_pending_irq(struct vcpu *vcpu)
> +{
> +    struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic;
> +    struct vgic_irq *irq;
> +    bool pending = false;
> +    unsigned long flags;
> +
> +    if ( !vcpu->domain->arch.vgic.enabled )
> +        return false;

Please don't mix bool and int. Ideally the interface should return a 
bool and not an int.

> +
> +    spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);
> +
> +    list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list)
> +    {
> +        spin_lock(&irq->irq_lock);
> +        pending = irq_is_pending(irq) && irq->enabled;
> +        spin_unlock(&irq->irq_lock);
> +
> +        if ( pending )
> +            break;
> +    }
> +
> +    spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);
> +
> +    return pending;
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 22/45] ARM: new VGIC: Add MMIO handling framework

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Add an MMIO handling framework to the VGIC emulation:
> Each register is described by its offset, size (or number of bits per
> IRQ, if applicable) and the read/write handler functions. We provide
> initialization macros to describe each GIC register later easily.
> 
> Separate dispatch functions for read and write accesses are connected
> to Xen's MMIO handling framework and binary-search for the responsible
> register handler based on the offset address within the region.
> 
> The register handler prototype are courtesy of Christoffer Dall.
> 
> This is based on Linux commit 4493b1c4866a, written by Marc Zyngier.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - adjust indentation
> - remove unneeded macros
> - use unsigned types
> - use new ilog2() implementation
> 
>   xen/arch/arm/vgic/vgic-mmio.c | 180 ++++++++++++++++++++++++++++++++++++++++++
>   xen/arch/arm/vgic/vgic-mmio.h |  89 +++++++++++++++++++++
>   2 files changed, 269 insertions(+)
>   create mode 100644 xen/arch/arm/vgic/vgic-mmio.c
>   create mode 100644 xen/arch/arm/vgic/vgic-mmio.h
> 
> diff --git a/xen/arch/arm/vgic/vgic-mmio.c b/xen/arch/arm/vgic/vgic-mmio.c
> new file mode 100644
> index 0000000000..866023a84d
> --- /dev/null
> +++ b/xen/arch/arm/vgic/vgic-mmio.c
> @@ -0,0 +1,180 @@
> +/*
> + * VGIC MMIO handling functions
> + * Imported from Linux ("new" KVM VGIC) and heavily adapted to Xen.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <xen/bitops.h>
> +#include <xen/lib.h>
> +#include <xen/sched.h>
> +#include <asm/new_vgic.h>
> +#include <asm/byteorder.h>
> +
> +#include "vgic.h"
> +#include "vgic-mmio.h"
> +
> +unsigned long vgic_mmio_read_raz(struct vcpu *vcpu,
> +                                 paddr_t addr, unsigned int len)
> +{
> +    return 0;
> +}
> +
> +unsigned long vgic_mmio_read_rao(struct vcpu *vcpu,
> +                                 paddr_t addr, unsigned int len)
> +{
> +    return -1UL;
> +}
> +
> +void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
> +                        unsigned int len, unsigned long val)
> +{
> +    /* Ignore */
> +}
> +
> +static int match_region(const void *key, const void *elt)
> +{
> +    const unsigned int offset = (unsigned long)key;
> +    const struct vgic_register_region *region = elt;
> +
> +    if ( offset < region->reg_offset )
> +        return -1;
> +
> +    if ( offset >= region->reg_offset + region->len )
> +        return 1;
> +
> +    return 0;
> +}
> +
> +static const struct vgic_register_region *
> +vgic_find_mmio_region(const struct vgic_register_region *regions,
> +                      int nr_regions, unsigned int offset)
> +{
> +    return bsearch((void *)(uintptr_t)offset, regions, nr_regions,
> +                   sizeof(regions[0]), match_region);
> +}
> +
> +static bool check_region(const struct domain *d,
> +                         const struct vgic_register_region *region,
> +                         paddr_t addr, int len)
> +{
> +    unsigned int flags, nr_irqs = d->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
> +
> +    switch ( len )
> +    {
> +    case sizeof(uint8_t):
> +        flags = VGIC_ACCESS_8bit;
> +        break;
> +    case sizeof(uint32_t):
> +        flags = VGIC_ACCESS_32bit;
> +        break;
> +    case sizeof(uint64_t):
> +        flags = VGIC_ACCESS_64bit;
> +        break;
> +    default:
> +        return false;
> +    }
> +
> +    if ( (region->access_flags & flags) && IS_ALIGNED(addr, len) )
> +    {
> +        if ( !region->bits_per_irq )
> +            return true;
> +
> +        /* Do we access a non-allocated IRQ? */
> +        return VGIC_ADDR_TO_INTID(addr, region->bits_per_irq) < nr_irqs;
> +    }
> +
> +    return false;
> +}
> +
> +static const struct vgic_register_region *
> +vgic_get_mmio_region(struct vcpu *vcpu, struct vgic_io_device *iodev,
> +                     paddr_t addr, unsigned int len)
> +{
> +    const struct vgic_register_region *region;
> +
> +    region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> +                                   addr - gfn_to_gaddr(iodev->base_fn));
> +    if ( !region || !check_region(vcpu->domain, region, addr, len) )
> +        return NULL;
> +
> +    return region;
> +}
> +
> +static int dispatch_mmio_read(struct vcpu *vcpu, mmio_info_t *info,
> +                              register_t *r, void *priv)
> +{
> +    struct vgic_io_device *iodev = priv;
> +    const struct vgic_register_region *region;
> +    unsigned long data = 0;
> +    paddr_t addr = info->gpa;
> +    int len = 1U << info->dabt.size;
> +
> +    region = vgic_get_mmio_region(vcpu, iodev, addr, len);
> +    if ( !region )
> +    {
> +        memset(r, 0, len);
> +        return 0;
> +    }
> +
> +    switch (iodev->iodev_type)
> +    {
> +    case IODEV_DIST:
> +        data = region->read(vcpu, addr, len);
> +        break;
> +    case IODEV_REDIST:
> +        data = region->read(iodev->redist_vcpu, addr, len);
> +        break;
> +    }
> +
> +    memcpy(r, &data, len);
> +
> +    return 1;
> +}
> +
> +static int dispatch_mmio_write(struct vcpu *vcpu, mmio_info_t *info,
> +                               register_t r, void *priv)
> +{
> +    struct vgic_io_device *iodev = priv;
> +    const struct vgic_register_region *region;
> +    unsigned long data = r;
> +    paddr_t addr = info->gpa;
> +    int len = 1U << info->dabt.size;
> +
> +    region = vgic_get_mmio_region(vcpu, iodev, addr, len);
> +    if ( !region )
> +        return 0;
> +
> +    switch (iodev->iodev_type)
> +    {
> +    case IODEV_DIST:
> +        region->write(vcpu, addr, len, data);
> +        break;
> +    case IODEV_REDIST:
> +        region->write(iodev->redist_vcpu, addr, len, data);
> +        break;
> +    }
> +
> +    return 1;
> +}
> +
> +struct mmio_handler_ops vgic_io_ops = {
> +    .read = dispatch_mmio_read,
> +    .write = dispatch_mmio_write,
> +};
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
> new file mode 100644
> index 0000000000..bf062a27ca
> --- /dev/null
> +++ b/xen/arch/arm/vgic/vgic-mmio.h
> @@ -0,0 +1,89 @@
> +/*
> + * Copyright (C) 2015, 2016 ARM Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +#ifndef __XEN_ARM_VGIC_VGIC_MMIO_H__
> +#define __XEN_ARM_VGIC_VGIC_MMIO_H__
> +
> +struct vgic_register_region {
> +    unsigned int reg_offset;
> +    unsigned int len;
> +    unsigned int bits_per_irq;
> +    unsigned int access_flags;
> +    unsigned long (*read)(struct vcpu *vcpu, paddr_t addr,
> +                          unsigned int len);
> +    void (*write)(struct vcpu *vcpu, paddr_t addr,
> +                  unsigned int len, unsigned long val);
> +};
> +
> +extern struct mmio_handler_ops vgic_io_ops;
> +
> +#define VGIC_ACCESS_8bit    1
> +#define VGIC_ACCESS_32bit   2
> +#define VGIC_ACCESS_64bit   4
> +
> +/*
> + * Generate a mask that covers the number of bytes required to address
> + * up to 1024 interrupts, each represented by <bits> bits. This assumes
> + * that <bits> is a power of two.
> + */
> +#define VGIC_ADDR_IRQ_MASK(bits) (((bits) * 1024 / 8) - 1)
> +
> +/*
> + * (addr & mask) gives us the _byte_ offset for the INT ID.
> + * We multiply this by 8 the get the _bit_ offset, then divide this by
> + * the number of bits to learn the actual INT ID.
> + * But instead of a division (which requires a "long long div" implementation),
> + * we shift by the binary logarithm of <bits>.
> + * This assumes that <bits> is a power of two.
> + */
> +#define VGIC_ADDR_TO_INTID(addr, bits)  (((addr) & VGIC_ADDR_IRQ_MASK(bits)) * \
> +                                         8 >> ilog2(bits))
> +
> +/*
> + * Some VGIC registers store per-IRQ information, with a different number
> + * of bits per IRQ. For those registers this macro is used.
> + * The _WITH_LENGTH version instantiates registers with a fixed length
> + * and is mutually exclusive with the _PER_IRQ version.
> + */
> +#define REGISTER_DESC_WITH_BITS_PER_IRQ(off, rd, wr, bpi, acc)  \
> +    {                                                           \
> +        .reg_offset = off,                                      \
> +        .bits_per_irq = bpi,                                    \
> +        .len = bpi * 1024 / 8,                                  \
> +        .access_flags = acc,                                    \
> +        .read = rd,                                             \
> +        .write = wr,                                            \
> +    }
> +
> +#define REGISTER_DESC_WITH_LENGTH(off, rd, wr, length, acc)     \
> +    {                                                           \
> +        .reg_offset = off,                                      \
> +        .bits_per_irq = 0,                                      \
> +        .len = length,                                          \
> +        .access_flags = acc,                                    \
> +        .read = rd,                                             \
> +        .write = wr,                                            \
> +    }
> +
> +unsigned long vgic_mmio_read_raz(struct vcpu *vcpu,
> +                                 paddr_t addr, unsigned int len);
> +
> +unsigned long vgic_mmio_read_rao(struct vcpu *vcpu,
> +                                 paddr_t addr, unsigned int len);
> +
> +void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
> +                        unsigned int len, unsigned long val);
> +
> +#endif
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 24/45] ARM: new VGIC: Add CTLR, TYPER and IIDR handlers

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
> index 7f221fd195..e2d0f04440 100644
> --- a/xen/arch/arm/vgic/vgic.h
> +++ b/xen/arch/arm/vgic/vgic.h
> @@ -17,6 +17,10 @@
>   #ifndef __XEN_ARM_VGIC_VGIC_H__
>   #define __XEN_ARM_VGIC_VGIC_H__
>   
> +#define PRODUCT_ID_KVM          0x4b        /* ASCII code X */

I think you mean 'ASCII code K'.

Also, do you want to copy the commit message about the identifier in the 
code?

> +#define VARIANT_ID_XEN          0x01
> +#define IMPLEMENTER_ARM         0x43b
> +
>   #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
>   
>   static inline bool irq_is_pending(struct vgic_irq *irq)
> @@ -37,6 +41,7 @@ struct vgic_irq *vgic_get_irq(struct domain *d, struct vcpu *vcpu,
>   void vgic_put_irq(struct domain *d, struct vgic_irq *irq);
>   void vgic_queue_irq_unlock(struct domain *d, struct vgic_irq *irq,
>                              unsigned long flags);
> +void vgic_kick_vcpus(struct domain *d);
>   
>   static inline void vgic_get_irq_kref(struct vgic_irq *irq)
>   {
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 25/45] ARM: new VGIC: Add ENABLE registers handlers

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> +void vgic_mmio_write_cenable(struct vcpu *vcpu,
> +                 paddr_t addr, unsigned int len,
> +                 unsigned long val)

NIT: Indentation.

[...]

> diff --git a/xen/arch/arm/vgic/vgic-mmio.h b/xen/arch/arm/vgic/vgic-mmio.h
> index c280668694..a2cebd77f4 100644
> --- a/xen/arch/arm/vgic/vgic-mmio.h
> +++ b/xen/arch/arm/vgic/vgic-mmio.h
> @@ -86,6 +86,17 @@ unsigned long vgic_mmio_read_rao(struct vcpu *vcpu,
>   void vgic_mmio_write_wi(struct vcpu *vcpu, paddr_t addr,
>                           unsigned int len, unsigned long val);
>   
> +unsigned long vgic_mmio_read_enable(struct vcpu *vcpu,
> +                                    paddr_t addr, unsigned int len);
> +
> +void vgic_mmio_write_senable(struct vcpu *vcpu,
> +                             paddr_t addr, unsigned int len,
> +                             unsigned long val);
> +
> +void vgic_mmio_write_cenable(struct vcpu *vcpu,
> +                             paddr_t addr, unsigned int len,
> +                             unsigned long val);
> +
>   unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>   
>   #endif
> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index a3aaf20a73..963d5e5f6e 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -689,6 +689,46 @@ void vgic_kick_vcpus(struct domain *d)
>       }
>   }
>   
> +static unsigned int translate_irq_type(bool is_level)
> +{
> +    return is_level ? IRQ_TYPE_LEVEL_HIGH : IRQ_TYPE_EDGE_RISING;
> +}
> +
> +void vgic_sync_hardware_irq(struct domain *d,
> +                            irq_desc_t *desc, struct vgic_irq *irq)
> +{
> +    unsigned long flags;
> +
> +    spin_lock_irqsave(&desc->lock, flags);
> +    spin_lock(&irq->irq_lock);
> +
> +    /*
> +     * We now forbid tinkering with the hardware IRQ association

NIT: The 'now' looks a bit weird here.

With that fixed:

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 26/45] ARM: new VGIC: Add PENDING registers handlers

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The pending register handlers are shared between the v2 and v3
> emulation, so their implementation goes into vgic-mmio.c, to be easily
> referenced from the v3 emulation as well later.
> For level triggered interrupts the real line level is unaffected by
> this write, so we keep this state separate and combine it with the
> device's level to get the actual pending state.
> Hardware mapped IRQs need some special handling, as their hardware state
> has to be coordinated with the virtual pending bit to avoid hanging
> or masked interrupts.
> 
> This is based on Linux commit 96b298000db4, written by Andre Przywara.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 27/45] ARM: new VGIC: Add ACTIVE registers handlers

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The active register handlers are shared between the v2 and v3 emulation,
> so their implementation goes into vgic-mmio.c, to be easily referenced
> from the v3 emulation as well later.
> Since activation/deactivation of an interrupt may happen entirely in the
> guest without it ever exiting, we need some extra logic to properly track
> the active state.
> For clearing the active state, we would basically have to halt the guest
> to make sure this is properly propagated into the respective VCPUs.
> This is not yet implemented in Xen.
> Fortunately this feature is mostly used to reset a just in initialised
> GIC, so chances are we are tasked to clear bits that are already zero.
> Add a simple check to avoid pointless warnings in this case.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 06/45] xen/arm: gic: Split the field state in gic_lr in 2 fields active and pending

[Andre Przywara]

Hi,

On 16/03/18 22:52, Stefano Stabellini wrote:
> On Fri, 16 Mar 2018, Julien Grall wrote:
>> On 16/03/2018 21:34, Stefano Stabellini wrote:
>>> On Thu, 15 Mar 2018, Andre Przywara wrote:
>>>> From: Julien Grall <julien.grall@arm.com>
>>>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>>>> index daec51499c..c32861d4fa 100644
>>>> --- a/xen/include/asm-arm/gic.h
>>>> +++ b/xen/include/asm-arm/gic.h
>>>> @@ -209,7 +209,8 @@ struct gic_lr {
>>>>      /* Virtual IRQ */
>>>>      uint32_t virq;
>>>>      uint8_t priority;
>>>> -   uint8_t state;
>>>> +   bool active;
>>>> +   bool pending;
>>>>      bool hw_status;
>>>>   };
>>>
>>> I like the readability but dislike the increase memory usage. I would
>>> have kept a single uint8_t and I would have used status flags as an
>>> approach, maybe I would have improved on those flags.
>>
>> Why is that important? gic_lr will only be allocated on the stack...
> 
> You are right, so it is even less important than I thought.

... especially given that this patch increases it from 11 to 12 bytes,
just to fill up the padding. And actually patch 04 decreased the size of
the structure by that one byte.
So it was 12 bytes before this series, is 12 bytes after this patch and
will be 12 bytes after the whole series.
So actually no change at all.

> Acked-by: Stefano Stabellini <sstabellini@kernel.org>

Thanks!

Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 09/45] ARM: GIC: Allow tweaking the active and pending state of an IRQ

[Julien Grall]

On 03/16/2018 04:05 PM, Andre Przywara wrote:
> Hi,

Hi Andre,

> On 15/03/18 20:30, Andre Przywara wrote:
>> +    }
>> +}
>> +
>> +static void gicv2_set_pending_state(struct irq_desc *irqd, bool pending)
>> +{
>> +    ASSERT(spin_is_locked(&irqd->lock));
>> +
>> +    if ( pending )
>> +    {
>> +        /* The INPROGRESS bit will be set when the interrupt fires. */
>> +        gicv2_poke_irq(irqd, GICD_ISPENDR);
>> +    }
>> +    else
>> +    {
>> +        gicv2_poke_irq(irqd, GICD_ICPENDR);
>> +        if ( !gicv2_peek_irq(irqd, GICD_ISACTIVER) &&
>> +             test_bit(_IRQ_GUEST, &irqd->status) )
>> +            clear_bit(_IRQ_INPROGRESS, &irqd->status);
> 
> We should not need to touch the _IRQ_INPROGRESS bit here. That bit
> really shadows the *active* bit, so changing the pending state should
> not matter here:
> - If the h/w IRQ is active, the bit is set already and should remain so,
> as Xen and the guest are still dealing with it. Clearing the h/w pending
> state does not change that.
> - If the h/w IRQ is not active, the _IRQ_INPROGRESS bit is not set, so
> clearing it would be a NOP.
> So we can remove  the _IRQ_INPROGRESS handling here completely.
> 
> I will amend the code accordingly, including the respective GICv3 parts.

Thank you for summarizing our discussion. Is it still making sense to 
document how those helpers should be called?

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 28/45] ARM: new VGIC: Add PRIORITY registers handlers

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The priority register handlers are shared between the v2 and v3 emulation,
> so their implementation goes into vgic-mmio.c, to be easily referenced
> from the v3 emulation as well later.
> 
> This is based on Linux commit 055658bf48fc, written by Andre Przywara.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 30/45] ARM: new VGIC: Add TARGET registers handlers

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The target register handlers are v2 emulation specific, so their
> implementation lives entirely in vgic-mmio-v2.c.
> We copy the old VGIC behaviour of assigning an IRQ to the first VCPU
> set in the target mask instead of making it possibly pending on
> multiple VCPUs.
> We update the physical affinity of a hardware mapped vIRQ on the way.
> 
> This is based on Linux commit 2c234d6f1826, written by Andre Przywara.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - directly update affinity (avoid vgic_sync_hardware_irq() and lock)
> 
>   xen/arch/arm/vgic/vgic-mmio-v2.c | 59 +++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 58 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
> index a28d0e459b..b333de9ed7 100644
> --- a/xen/arch/arm/vgic/vgic-mmio-v2.c
> +++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
> @@ -81,6 +81,63 @@ static void vgic_mmio_write_v2_misc(struct vcpu *vcpu,
>       }
>   }
>   
> +static unsigned long vgic_mmio_read_target(struct vcpu *vcpu,
> +                                           paddr_t addr, unsigned int len)
> +{
> +    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
> +    uint32_t val = 0;
> +    unsigned int i;
> +
> +    for ( i = 0; i < len; i++ )
> +    {
> +        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, vcpu, intid + i);
> +
> +        val |= (uint32_t)irq->targets << (i * 8);
> +
> +        vgic_put_irq(vcpu->domain, irq);
> +    }
> +
> +    return val;
> +}
> +
> +static void vgic_mmio_write_target(struct vcpu *vcpu,
> +                                   paddr_t addr, unsigned int len,
> +                                   unsigned long val)
> +{
> +    uint32_t intid = VGIC_ADDR_TO_INTID(addr, 8);
> +    uint8_t cpu_mask = GENMASK(vcpu->domain->max_vcpus - 1, 0);
> +    unsigned int i;
> +    unsigned long flags;
> +
> +    /* GICD_ITARGETSR[0-7] are read-only */
> +    if ( intid < VGIC_NR_PRIVATE_IRQS )
> +        return;
> +
> +    for ( i = 0; i < len; i++ )
> +    {
> +        struct vgic_irq *irq = vgic_get_irq(vcpu->domain, NULL, intid + i);
> +
> +        spin_lock_irqsave(&irq->irq_lock, flags);
> +
> +        irq->targets = (val >> (i * 8)) & cpu_mask;
> +        if ( irq->targets )
> +        {
> +            irq->target_vcpu = vcpu->domain->vcpu[ffs(irq->targets) - 1];
> +            if ( irq->hw )
> +            {
> +                struct irq_desc *desc = irq_to_desc(irq->hwintid);
> +
> +                irq_set_affinity(desc, cpumask_of(irq->target_vcpu->processor));
> +            }
> +        }
> +        else
> +            irq->target_vcpu = NULL;
> +
> +        spin_unlock_irqrestore(&irq->irq_lock, flags);
> +        vgic_put_irq(vcpu->domain, irq);
> +    }
> +}
> +
>   static const struct vgic_register_region vgic_v2_dist_registers[] = {
>       REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
>           vgic_mmio_read_v2_misc, vgic_mmio_write_v2_misc, 12,
> @@ -110,7 +167,7 @@ static const struct vgic_register_region vgic_v2_dist_registers[] = {
>           vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
>           VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
>       REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ITARGETSR,
> -        vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
> +        vgic_mmio_read_target, vgic_mmio_write_target, 8,
>           VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
>       REGISTER_DESC_WITH_BITS_PER_IRQ(GICD_ICFGR,
>           vgic_mmio_read_config, vgic_mmio_write_config, 2,
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 31/45] ARM: new VGIC: Add SGIR register handler

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Triggering an IPI via this register is v2 specific, so the
> implementation lives entirely in vgic-mmio-v2.c.
> 
> This is based on Linux commit 55cc01fb9004, written by Andre Przywara.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> Changelog v1 ... v2:
> - remove stray rebase artefact
> 
>   xen/arch/arm/vgic/vgic-mmio-v2.c | 45 +++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 44 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c b/xen/arch/arm/vgic/vgic-mmio-v2.c
> index b333de9ed7..7e17cdc2ad 100644
> --- a/xen/arch/arm/vgic/vgic-mmio-v2.c
> +++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
> @@ -81,6 +81,49 @@ static void vgic_mmio_write_v2_misc(struct vcpu *vcpu,
>       }
>   }
>   
> +static void vgic_mmio_write_sgir(struct vcpu *source_vcpu,
> +                                 paddr_t addr, unsigned int len,
> +                                 unsigned long val)
> +{
> +    struct domain *d = source_vcpu->domain;
> +    unsigned int nr_vcpus = d->max_vcpus;
> +    unsigned int intid = val & GICD_SGI_INTID_MASK;
> +    unsigned long targets = (val & GICD_SGI_TARGET_MASK) >>
> +                            GICD_SGI_TARGET_SHIFT;
> +    unsigned int vcpu_id;
> +
> +    switch ( val & GICD_SGI_TARGET_LIST_MASK )
> +    {
> +    case GICD_SGI_TARGET_LIST:                    /* as specified by targets */
> +        targets &= GENMASK(nr_vcpus, 0);          /* limit to existing VCPUs */

Shouldn't it be 'nr_vcpus - 1'?

> +        break;
> +    case GICD_SGI_TARGET_OTHERS:
> +        targets = GENMASK(nr_vcpus, 0);           /* all, ...   */
> +        targets &= ~(1U << source_vcpu->vcpu_id); /*   but self */
> +        break;
> +    case GICD_SGI_TARGET_SELF:                    /* this very vCPU only */
> +        targets = (1U << source_vcpu->vcpu_id);
> +        break;
> +    case 0x3:                                     /* reserved */
> +        return;
> +    }

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 38/45] ARM: new VGIC: Implement arch_move_irqs()

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> When a VCPU moves to another CPU, we need to adjust the target affinity
> of any hardware mapped vIRQs, to observe our "physical-follows-virtual"
> policy.
> Implement arch_move_irqs() to adjust the physical affinity of all hardware
> mapped vIRQs targetting this VCPU.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - use irq_set_affinity() directly, avoid vgic_sync_hardware_irq() and lock
> 
>   xen/arch/arm/vgic/vgic.c | 39 +++++++++++++++++++++++++++++++++++++++
>   1 file changed, 39 insertions(+)
> 
> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index 73cd43f0fa..1aae68ff83 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -781,6 +781,45 @@ void gic_dump_vgic_info(struct vcpu *v)
>       spin_unlock_irqrestore(&v->arch.vgic.ap_list_lock, flags);
>   }
>   
> +/**
> + * arch_move_irqs() - migrate the physical affinity of hardware mapped vIRQs
> + * @v:  the vCPU, already assigned to the new pCPU
> + *
> + * arch_move_irqs() updates the physical affinity of all virtual IRQs
> + * targetting this given vCPU. This only affects hardware mapped IRQs. The
> + * new pCPU to target is already set in v->processor.
> + * This is called by the core code after a vCPU has been migrated to a new
> + * physical CPU.
> + */
> +void arch_move_irqs(struct vcpu *v)
> +{
> +    struct domain *d = v->domain;
> +    unsigned int i;
> +
> +    /* We only target SPIs with this function */
> +    for ( i = 0; i < d->arch.vgic.nr_spis; i++ )
> +    {
> +        struct vgic_irq *irq = vgic_get_irq(d, NULL, i + VGIC_NR_PRIVATE_IRQS);
> +        unsigned long flags;
> +
> +        if ( !irq )
> +            continue;
> +
> +        spin_lock_irqsave(&irq->irq_lock, flags);
> +
> +        /* only vIRQs that are not on a vCPU yet , but targetting this vCPU */
> +        if ( irq->hw && !irq->vcpu && irq->target_vcpu == v)
> +        {
> +            irq_desc_t *desc = irq_to_desc(irq->hwintid);
> +
> +            irq_set_affinity(desc, cpumask_of(v->processor));
> +        }
> +
> +        spin_unlock_irqrestore(&irq->irq_lock, flags);
> +        vgic_put_irq(d, irq);
> +    }
> +}
> +
>   struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
>                                         unsigned int virq)
>   {
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 39/45] ARM: new VGIC: Add preliminary stub implementation

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The ARM arch code requires an interrupt controller emulation to implement
> vgic_clear_pending_irqs(), although it is suspected that it is actually
> not necessary. Go with a stub for now to make the linker happy.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
>   xen/arch/arm/vgic/vgic.c | 8 ++++++++
>   1 file changed, 8 insertions(+)
> 
> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index 1aae68ff83..002fec57e6 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -781,6 +781,14 @@ void gic_dump_vgic_info(struct vcpu *v)
>       spin_unlock_irqrestore(&v->arch.vgic.ap_list_lock, flags);
>   }
>   
> +void vgic_clear_pending_irqs(struct vcpu *v)
> +{
> +    /*
> +     * TODO: It is unclear whether we really need this, so we might instead
> +     * remove it on the caller site.
> +     */
> +}
> +
>   /**
>    * arch_move_irqs() - migrate the physical affinity of hardware mapped vIRQs
>    * @v:  the vCPU, already assigned to the new pCPU
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 41/45] ARM: new VGIC: Add vgic_v2_enable

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Enable the VGIC operation by properly initialising the registers
> in the hypervisor GIC interface.
> 
> This is based on Linux commit f7b6985cc3d0, written by Eric Auger.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - move patch from later part in the series
> 
>   xen/arch/arm/vgic/vgic-v2.c | 6 ++++++
>   xen/arch/arm/vgic/vgic.h    | 1 +
>   2 files changed, 7 insertions(+)
> 
> diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
> index bc5c7895f4..4662346327 100644
> --- a/xen/arch/arm/vgic/vgic-v2.c
> +++ b/xen/arch/arm/vgic/vgic-v2.c
> @@ -219,6 +219,12 @@ void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr)
>       gic_hw_ops->write_lr(lr, &lr_val);
>   }
>   
> +void vgic_v2_enable(struct vcpu *vcpu)
> +{
> +    /* Get the show on the road... */
> +    gic_hw_ops->update_hcr_status(GICH_HCR_EN, 1);
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
> index deefbb3ef7..def1ac478a 100644
> --- a/xen/arch/arm/vgic/vgic.h
> +++ b/xen/arch/arm/vgic/vgic.h
> @@ -62,6 +62,7 @@ void vgic_sync_hardware_irq(struct domain *d,
>   void vgic_v2_fold_lr_state(struct vcpu *vcpu);
>   void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr);
>   void vgic_v2_set_underflow(struct vcpu *vcpu);
> +void vgic_v2_enable(struct vcpu *vcpu);
>   int vgic_register_dist_iodev(struct domain *d, gfn_t dist_base_fn,
>                                enum vgic_type);
>   
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 44/45] ARM: new VGIC: Allocate two pages for struct vcpu

[Julien Grall]

Hi Andre,

Title: Please update it.

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> At the moment we allocate exactly one page for struct vcpu on ARM, also
> have a check in place to prevent it growing beyond 4KB.
> As the struct includes the state of all 32 private (per-VCPU) interrupts,
> we are at 3840 bytes on arm64 at the moment already. Growing the per-IRQ
> VGIC structure even slightly makes the VCPU quickly exceed the 4K limit.
> The new VGIC will need more space per virtual IRQ. I spent a few hours
> trying to trim this down, but couldn't get it below 4KB, even with the
> nasty hacks piling up to save some bytes here and there.
> It turns out that beyond efficiency, maybe, there is no real technical
> reason this struct has to fit in one page, so lifting the limit to two
> pages seems like the most pragmatic solution.
> Restrict this to compiling with the new VGIC and for ARM64 only.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> Changelog v1 ... v2:
> - confine change to new VGIC and ARM64 only
> 
>   xen/arch/arm/domain.c | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 18b915d2e9..3fba05bda5 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -508,10 +508,25 @@ void dump_pageframe_info(struct domain *d)
>   struct vcpu *alloc_vcpu_struct(void)
>   {
>       struct vcpu *v;
> +
> +    /*
> +     * The new VGIC has a bigger per-IRQ structure, so we need more than one
> +     * page on ARM64. Cowardly increase the limit in this case.
> +     */
> +#if defined(CONFIG_NEW_VGIC) && defined(CONFIG_ARM_64)
> +    BUILD_BUG_ON(sizeof(*v) > 2 * PAGE_SIZE);
> +    v = alloc_xenheap_pages(1, 0);
> +    if ( v != NULL ) {
> +        clear_page(v);
> +        clear_page((void *)v + PAGE_SIZE);
> +    }
> +#else
>       BUILD_BUG_ON(sizeof(*v) > PAGE_SIZE);
>       v = alloc_xenheap_pages(0, 0);
>       if ( v != NULL )
>           clear_page(v);
> +#endif.
All this logic can just be simplified using get_order_* helpers. This 
will avoid #ifdef like that in alloc_vcpu_struct which is really really 
ugly.


Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 44/45] ARM: new VGIC: Allocate two pages for struct vcpu

[Julien Grall]

Hi Andre,

On 03/19/2018 10:00 AM, Julien Grall wrote:
> Hi Andre,
> 
> Title: Please update it.
> 
> On 03/15/2018 08:30 PM, Andre Przywara wrote:
>> At the moment we allocate exactly one page for struct vcpu on ARM, also
>> have a check in place to prevent it growing beyond 4KB.
>> As the struct includes the state of all 32 private (per-VCPU) interrupts,
>> we are at 3840 bytes on arm64 at the moment already. Growing the per-IRQ
>> VGIC structure even slightly makes the VCPU quickly exceed the 4K limit.
>> The new VGIC will need more space per virtual IRQ. I spent a few hours
>> trying to trim this down, but couldn't get it below 4KB, even with the
>> nasty hacks piling up to save some bytes here and there.
>> It turns out that beyond efficiency, maybe, there is no real technical
>> reason this struct has to fit in one page, so lifting the limit to two
>> pages seems like the most pragmatic solution.
>> Restrict this to compiling with the new VGIC and for ARM64 only.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
>> ---
>> Changelog v1 ... v2:
>> - confine change to new VGIC and ARM64 only
>>
>>   xen/arch/arm/domain.c | 15 +++++++++++++++
>>   1 file changed, 15 insertions(+)
>>
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index 18b915d2e9..3fba05bda5 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -508,10 +508,25 @@ void dump_pageframe_info(struct domain *d)
>>   struct vcpu *alloc_vcpu_struct(void)
>>   {
>>       struct vcpu *v;
>> +
>> +    /*
>> +     * The new VGIC has a bigger per-IRQ structure, so we need more 
>> than one
>> +     * page on ARM64. Cowardly increase the limit in this case.
>> +     */
>> +#if defined(CONFIG_NEW_VGIC) && defined(CONFIG_ARM_64)
>> +    BUILD_BUG_ON(sizeof(*v) > 2 * PAGE_SIZE);
>> +    v = alloc_xenheap_pages(1, 0);
>> +    if ( v != NULL ) {
>> +        clear_page(v);
>> +        clear_page((void *)v + PAGE_SIZE);
>> +    }
>> +#else
>>       BUILD_BUG_ON(sizeof(*v) > PAGE_SIZE);
>>       v = alloc_xenheap_pages(0, 0);
>>       if ( v != NULL )
>>           clear_page(v);
>> +#endif.
> All this logic can just be simplified using get_order_* helpers. This 
> will avoid #ifdef like that in alloc_vcpu_struct which is really really 
> ugly.

To clarify, what I suggest is:

unsigned int order = get_order(...);
BUILD_BUG_ON(sizeof(*v) > 2 * PAGE_SIZE);

v = alloc_xenheap_pages(order, 0);
if ( v != NULL )
{
    for ( i = 0; i < (1U << order) )
       clear_page(v + (i * PAGE_SIZE));
}

Cheers,

> 
> 
> Cheers,
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 10/45] ARM: GIC: Allow reading pending state of a hardware IRQ

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> To synchronize level triggered interrupts which are mapped into a guest,
> we need to update the virtual line level at certain points in time.
> For a hardware mapped interrupt the GIC is the only place where we can
> easily access this information.
> Implement a gic_hw_operations member to return the pending state of a
> particular interrupt. Due to hardware limitations this only works for
> private interrupts of the current CPU, so there is no CPU field in the
> prototype.
> This adds gicv2/3_peek_irq() helper functions, to read a bit in a bitmap
> spread over several MMIO registers.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - remove introduction of gicv[23]_peek_irq() (now one patch earlier)
> - move wrapper functions into gic.h
> 
>   xen/arch/arm/gic-v2.c     |  6 ++++++
>   xen/arch/arm/gic-v3.c     |  6 ++++++
>   xen/include/asm-arm/gic.h | 11 +++++++++++
>   3 files changed, 23 insertions(+)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index c6fcbf59d0..52880d658c 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -591,6 +591,11 @@ static unsigned int gicv2_read_apr(int apr_reg)
>      return readl_gich(GICH_APR);
>   }
>   
> +static bool gicv2_read_pending_state(struct irq_desc *irqd)
> +{
> +    return gicv2_peek_irq(irqd, GICD_ISPENDR);
> +}
> +
>   static void gicv2_irq_enable(struct irq_desc *desc)
>   {
>       unsigned long flags;
> @@ -1336,6 +1341,7 @@ const static struct gic_hw_operations gicv2_ops = {
>       .write_lr            = gicv2_write_lr,
>       .read_vmcr_priority  = gicv2_read_vmcr_priority,
>       .read_apr            = gicv2_read_apr,
> +    .read_pending_state  = gicv2_read_pending_state,
>       .make_hwdom_dt_node  = gicv2_make_hwdom_dt_node,
>       .make_hwdom_madt     = gicv2_make_hwdom_madt,
>       .get_hwdom_extra_madt_size = gicv2_get_hwdom_extra_madt_size,
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 316f2c4142..008be87572 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1161,6 +1161,11 @@ static unsigned int gicv3_read_apr(int apr_reg)
>       }
>   }
>   
> +static bool gicv3_read_pending_state(struct irq_desc *irqd)
> +{
> +    return gicv3_peek_irq(irqd, GICD_ISPENDR);
> +}
> +
>   static void gicv3_irq_enable(struct irq_desc *desc)
>   {
>       unsigned long flags;
> @@ -1829,6 +1834,7 @@ static const struct gic_hw_operations gicv3_ops = {
>       .write_lr            = gicv3_write_lr,
>       .read_vmcr_priority  = gicv3_read_vmcr_priority,
>       .read_apr            = gicv3_read_apr,
> +    .read_pending_state  = gicv3_read_pending_state,
>       .secondary_init      = gicv3_secondary_cpu_init,
>       .make_hwdom_dt_node  = gicv3_make_hwdom_dt_node,
>       .make_hwdom_madt     = gicv3_make_hwdom_madt,
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 21cf35f106..bd7802667f 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -372,6 +372,8 @@ struct gic_hw_operations {
>       unsigned int (*read_vmcr_priority)(void);
>       /* Read APRn register */
>       unsigned int (*read_apr)(int apr_reg);
> +    /* Query the pending state of an interrupt at the distributor level. */
> +    bool (*read_pending_state)(struct irq_desc *irqd);
>       /* Secondary CPU init */
>       int (*secondary_init)(void);
>       /* Create GIC node for the hardware domain */
> @@ -416,6 +418,15 @@ static inline void gic_set_pending_state(struct irq_desc *irqd, bool state)
>       gic_hw_ops->set_pending_state(irqd, state);
>   }
>   
> +/*
> + * Read the pending state of an interrupt from the distributor.
> + * For private IRQs this only works for those of the current CPU.
> + */
> +static inline bool gic_read_pending_state(struct irq_desc *irqd)
> +{
> +    return gic_hw_ops->read_pending_state(irqd);
> +}
> +
>   void register_gic_ops(const struct gic_hw_operations *ops);
>   int gic_make_hwdom_dt_node(const struct domain *d,
>                              const struct dt_device_node *gic,
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 11/45] ARM: timer: Handle level triggered IRQs correctly

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The ARM Generic Timer uses a level-sensitive interrupt semantic. We
> easily catch when the line goes high, as this triggers the hardware IRQ.
> However we have to sync the state of the interrupt condition at certain
> points to catch when the line goes low and we can remove the vtimer vIRQ
> from the vGIC (and the LR).
> The VGIC in Xen so far only implemented edge triggered vIRQs, really, so
> we need to add new functionality to re-sample the interrupt state.
> Do this only when the new VGIC is in use.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Please address my comments from :
<fd911249-cc73-977b-9176-b5a20ae569ef@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - restrict to new VGIC
> - add TODO: comment
> 
>   xen/arch/arm/time.c     | 36 ++++++++++++++++++++++++++++++++++++
>   xen/arch/arm/traps.c    | 11 +++++++++++
>   xen/include/xen/timer.h |  2 ++
>   3 files changed, 49 insertions(+)
> 
> diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c
> index c11fcfeadd..c0ae781ecd 100644
> --- a/xen/arch/arm/time.c
> +++ b/xen/arch/arm/time.c
> @@ -263,6 +263,42 @@ static void vtimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
>       vgic_inject_irq(current->domain, current, current->arch.virt_timer.irq, true);
>   }
>   
> +/**
> + * vtimer_sync() - update the state of the virtual timer after a guest run
> + * @vcpu: The VCPU to sync the arch timer state
> + *
> + * After returning from a guest, update the state of the virtual interrupt
> + * line, to model the level triggered interrupt correctly.
> + * If the guest has handled a timer interrupt, the virtual interrupt line
> + * needs to be lowered explicitly. vgic_inject_irq() takes care of that.
> + */
> +void vtimer_sync(struct vcpu *vcpu)
> +{
> +    struct vtimer *vtimer = &vcpu->arch.virt_timer;
> +    uint32_t vtimer_ctl = READ_SYSREG32(CNTV_CTL_EL0);
> +    bool level;
> +
> +    /*
> +     * Technically the mask should include the CNTx_CTL_MASK bit here,
> +     * to catch if the timer interrupt is masked. However Xen always masks
> +     * the timer upon entering the hypervisor, leaving it up to the guest
> +     * to un-mask it. So we would always read a "low" level, despite the
> +     * condition being actually "high".
> +     * Ignoring the mask bit solves this (for now).
> +     * Another possible check would be to compare the value of CNTVCT_EL0
> +     * against vtimer->cval and derive the interrupt state from that.
> +     */
> +    vtimer_ctl &= (CNTx_CTL_ENABLE | CNTx_CTL_PENDING);
> +    level = (vtimer_ctl == (CNTx_CTL_ENABLE | CNTx_CTL_PENDING));
> +
> +     /*
> +      * TODO: The proper fix for this is to make vtimer vIRQ hardware mapped,
> +      * but this requires reworking the arch timer to implement this.
> +      */
> +
> +    vgic_inject_irq(vcpu->domain, vcpu, vtimer->irq, level);
> +}
> +
>   /*
>    * Arch timer interrupt really ought to be level triggered, since the
>    * design of the timer/comparator mechanism is based around that
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 7411bff7a7..46464d7bb9 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -2024,6 +2024,17 @@ static void enter_hypervisor_head(struct cpu_user_regs *regs)
>           if ( current->arch.hcr_el2 & HCR_VA )
>               current->arch.hcr_el2 = READ_SYSREG(HCR_EL2);
>   
> +#ifdef CONFIG_NEW_VGIC
> +        /*
> +         * We need to update the state of our emulated devices using level
> +         * triggered interrupts before syncing back the VGIC state.
> +         *
> +         * TODO: Investigate whether this is necessary to do on every
> +         * trap and how it can be optimised.
> +         */
> +        vtimer_sync(current);
> +#endif
> +
>           vgic_sync_from_lrs(current);
>       }
>   }
> diff --git a/xen/include/xen/timer.h b/xen/include/xen/timer.h
> index 4513260b0d..eddbbf3903 100644
> --- a/xen/include/xen/timer.h
> +++ b/xen/include/xen/timer.h
> @@ -94,6 +94,8 @@ DECLARE_PER_CPU(s_time_t, timer_deadline);
>   /* Arch-defined function to reprogram timer hardware for new deadline. */
>   int reprogram_timer(s_time_t timeout);
>   
> +void vtimer_sync(struct vcpu *vcpu);
> +
>   /* Calculate the aligned first tick time for a given periodic timer. */
>   s_time_t align_timer(s_time_t firsttick, uint64_t period);
>   
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 12/45] ARM: evtchn: Handle level triggered IRQs correctly

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The event channel IRQ has level triggered semantics, however the current
> VGIC treats everything as edge triggered.
> To correctly process those IRQs, we have to lower the (virtual) IRQ line
> at some point in time, depending on whether ther interrupt condition
> still prevails.
> Check the per-VCPU evtchn_upcall_pending variable to make the interrupt
> line match its status, and call this function upon every hypervisor
> entry.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
>   xen/arch/arm/domain.c       | 7 +++++++
>   xen/arch/arm/traps.c        | 1 +
>   xen/include/asm-arm/event.h | 1 +
>   3 files changed, 9 insertions(+)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 4462e62599..18b915d2e9 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -954,6 +954,13 @@ void vcpu_mark_events_pending(struct vcpu *v)
>       vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
>   }
>   
> +void vcpu_update_evtchn_irq(struct vcpu *v)
> +{
> +    bool pending = vcpu_info(v, evtchn_upcall_pending);
> +
> +    vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, pending);
> +}
> +
>   /* The ARM spec declares that even if local irqs are masked in
>    * the CPSR register, an irq should wake up a cpu from WFI anyway.
>    * For this reason we need to check for irqs that need delivery,
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 46464d7bb9..c13223a69f 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -2033,6 +2033,7 @@ static void enter_hypervisor_head(struct cpu_user_regs *regs)
>            * trap and how it can be optimised.
>            */
>           vtimer_sync(current);
> +        vcpu_update_evtchn_irq(current);
>   #endif
>   
>           vgic_sync_from_lrs(current);
> diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
> index c7a415ef57..2f51864043 100644
> --- a/xen/include/asm-arm/event.h
> +++ b/xen/include/asm-arm/event.h
> @@ -6,6 +6,7 @@
>   
>   void vcpu_kick(struct vcpu *v);
>   void vcpu_mark_events_pending(struct vcpu *v);
> +void vcpu_update_evtchn_irq(struct vcpu *v);
>   void vcpu_block_unless_event_pending(struct vcpu *v);
>   
>   static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 13/45] ARM: vPL011: Use the VGIC's level triggered IRQs handling if available

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The emulated ARM SBSA UART is using level triggered IRQ semantics,
> however the current VGIC can only handle edge triggered IRQs, really.
> Disable the existing workaround for this problem in case we have the
> new VGIC in place, which can properly handle level triggered IRQs.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - remove unneeded "!!"
> 
>   xen/arch/arm/vpl011.c | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> index 5dcf4bec18..a281eabd7e 100644
> --- a/xen/arch/arm/vpl011.c
> +++ b/xen/arch/arm/vpl011.c
> @@ -54,6 +54,7 @@ static void vpl011_update_interrupt_status(struct domain *d)
>        */
>       ASSERT(spin_is_locked(&vpl011->lock));
>   
> +#ifndef CONFIG_NEW_VGIC
>       /*
>        * TODO: PL011 interrupts are level triggered which means
>        * that interrupt needs to be set/clear instead of being
> @@ -71,6 +72,9 @@ static void vpl011_update_interrupt_status(struct domain *d)
>           vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, true);
>   
>       vpl011->shadow_uartmis = uartmis;
> +#else
> +    vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, uartmis);
> +#endif
>   }
>   
>   static uint8_t vpl011_read_data(struct domain *d)
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 14/45] ARM: new VGIC: Add data structure definitions

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Add a new header file for the new and improved GIC implementation.
> The big change is that we now have a struct vgic_irq per IRQ instead
> of spreading all the information over various bitmaps in the ranks.
> 
> We include this new header conditionally from within the old header
> file for the time being to avoid touching all the users.
> 
> This is based on Linux commit b18b57787f5e, written by Christoffer Dall.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - define CONFIG_EDGE to false instead of using enum
> - remove not needed forward declaration
> - remove unneeded macros
> - add emacs footer
> 
>   xen/include/asm-arm/new_vgic.h | 198 +++++++++++++++++++++++++++++++++++++++++
>   xen/include/asm-arm/vgic.h     |   6 ++
>   2 files changed, 204 insertions(+)
>   create mode 100644 xen/include/asm-arm/new_vgic.h
> 
> diff --git a/xen/include/asm-arm/new_vgic.h b/xen/include/asm-arm/new_vgic.h
> new file mode 100644
> index 0000000000..97d622bff6
> --- /dev/null
> +++ b/xen/include/asm-arm/new_vgic.h
> @@ -0,0 +1,198 @@
> +/*
> + * Copyright (C) 2015, 2016 ARM Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +#ifndef __ASM_ARM_NEW_VGIC_H
> +#define __ASM_ARM_NEW_VGIC_H
> +
> +#include <asm/atomic.h>
> +#include <asm/mmio.h>
> +#include <xen/list.h>
> +#include <xen/mm.h>
> +#include <xen/spinlock.h>
> +
> +#define VGIC_V3_MAX_CPUS        255
> +#define VGIC_V2_MAX_CPUS        8
> +#define VGIC_NR_SGIS            16
> +#define VGIC_NR_PPIS            16
> +#define VGIC_NR_PRIVATE_IRQS    (VGIC_NR_SGIS + VGIC_NR_PPIS)
> +#define VGIC_MAX_PRIVATE        (VGIC_NR_PRIVATE_IRQS - 1)
> +#define VGIC_MAX_SPI            1019
> +#define VGIC_MAX_RESERVED       1023
> +#define VGIC_MIN_LPI            8192
> +
> +#define irq_is_ppi(irq) ((irq) >= VGIC_NR_SGIS && (irq) < VGIC_NR_PRIVATE_IRQS)
> +#define irq_is_spi(irq) ((irq) >= VGIC_NR_PRIVATE_IRQS && \
> +                         (irq) <= VGIC_MAX_SPI)
> +
> +enum vgic_type {
> +    VGIC_V2,        /* Good ol' GICv2 */
> +    VGIC_V3,        /* New fancy GICv3 */
> +};
> +
> +#define VGIC_V2_MAX_LRS         (1 << 6)
> +#define VGIC_V3_MAX_LRS         16
> +#define VGIC_V3_LR_INDEX(lr)    (VGIC_V3_MAX_LRS - 1 - lr)
> +
> +#define VGIC_CONFIG_EDGE        false
> +#define VGIC_CONFIG_LEVEL       true
> +
> +struct vgic_irq {
> +    struct list_head ap_list;
> +
> +    struct vcpu *vcpu;          /*
> +                                 * SGIs and PPIs: The VCPU
> +                                 * SPIs and LPIs: The VCPU whose ap_list
> +                                 * this is queued on.
> +                                 */
> +
> +    struct vcpu *target_vcpu;   /*
> +                                 * The VCPU that this interrupt should
> +                                 * be sent to, as a result of the
> +                                 * targets reg (v2) or the affinity reg (v3).
> +                                 */
> +
> +    spinlock_t irq_lock;        /* Protects the content of the struct */
> +    uint32_t intid;             /* Guest visible INTID */
> +    atomic_t refcount;          /* Used for LPIs */
> +    uint32_t hwintid;           /* HW INTID number */
> +    union
> +    {
> +        struct {
> +            uint8_t targets;    /* GICv2 target VCPUs mask */
> +            uint8_t source;     /* GICv2 SGIs only */
> +        };
> +        uint32_t mpidr;         /* GICv3 target VCPU */
> +    };
> +    uint8_t priority;
> +    bool line_level:1;          /* Level only */
> +    bool pending_latch:1;       /*
> +                                 * The pending latch state used to
> +                                 * calculate the pending state for both
> +                                 * level and edge triggered IRQs.
> +                                 */
> +    bool active:1;              /* not used for LPIs */
> +    bool enabled:1;
> +    bool hw:1;                  /* Tied to HW IRQ */
> +    bool config:1;              /* Level or edge */
> +    struct list_head lpi_list;  /* Used to link all LPIs together */
> +};
> +
> +enum iodev_type {
> +    IODEV_DIST,
> +    IODEV_REDIST,
> +};
> +
> +struct vgic_io_device {
> +    gfn_t base_fn;
> +    struct vcpu *redist_vcpu;
> +    const struct vgic_register_region *regions;
> +    enum iodev_type iodev_type;
> +    unsigned int nr_regions;
> +};
> +
> +struct vgic_dist {
> +    bool                ready;
> +    bool                initialized;
> +
> +    /* vGIC model the kernel emulates for the guest (GICv2 or GICv3) */
> +    uint32_t            version;
> +
> +    /* Do injected MSIs require an additional device ID? */
> +    bool                msis_require_devid;
> +
> +    unsigned int        nr_spis;
> +
> +    /* base addresses in guest physical address space: */
> +    paddr_t             vgic_dist_base;     /* distributor */
> +    union
> +    {
> +        /* either a GICv2 CPU interface */
> +        paddr_t         vgic_cpu_base;
> +        /* or a number of GICv3 redistributor regions */
> +        struct
> +        {
> +            paddr_t     vgic_redist_base;
> +            paddr_t     vgic_redist_free_offset;
> +        };
> +    };
> +
> +    /* distributor enabled */
> +    bool                enabled;
> +
> +    struct vgic_irq     *spis;
> +    unsigned long       *allocated_irqs; /* bitmap of IRQs allocated */
> +
> +    struct vgic_io_device   dist_iodev;
> +
> +    bool                has_its;
> +
> +    /*
> +     * Contains the attributes and gpa of the LPI configuration table.
> +     * Since we report GICR_TYPER.CommonLPIAff as 0b00, we can share
> +     * one address across all redistributors.
> +     * GICv3 spec: 6.1.2 "LPI Configuration tables"
> +     */
> +    uint64_t            propbaser;
> +
> +    /* Protects the lpi_list and the count value below. */
> +    spinlock_t          lpi_list_lock;
> +    struct list_head    lpi_list_head;
> +    unsigned int        lpi_list_count;
> +};
> +
> +struct vgic_cpu {
> +    struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS];
> +
> +    struct list_head ap_list_head;
> +    spinlock_t ap_list_lock;    /* Protects the ap_list */
> +
> +    unsigned int used_lrs;
> +
> +    /*
> +     * List of IRQs that this VCPU should consider because they are either
> +     * Active or Pending (hence the name; AP list), or because they recently
> +     * were one of the two and need to be migrated off this list to another
> +     * VCPU.
> +     */
> +
> +    /*
> +     * Members below are used with GICv3 emulation only and represent
> +     * parts of the redistributor.
> +     */
> +    struct vgic_io_device   rd_iodev;
> +    struct vgic_io_device   sgi_iodev;
> +
> +    /* Contains the attributes and gpa of the LPI pending tables. */
> +    uint64_t pendbaser;
> +
> +    bool lpis_enabled;
> +
> +    /* Cache guest priority bits */
> +    uint32_t num_pri_bits;
> +
> +    /* Cache guest interrupt ID bits */
> +    uint32_t num_id_bits;
> +};
> +
> +#endif /* __ASM_ARM_NEW_VGIC_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 0787ba9549..2a58ea30fe 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -18,6 +18,10 @@
>   #ifndef __ASM_ARM_VGIC_H__
>   #define __ASM_ARM_VGIC_H__
>   
> +#ifdef CONFIG_NEW_VGIC
> +#include <asm/new_vgic.h>
> +#else
> +
>   #include <xen/bitops.h>
>   #include <xen/radix-tree.h>
>   #include <xen/rbtree.h>
> @@ -299,6 +303,8 @@ extern bool vgic_to_sgi(struct vcpu *v, register_t sgir,
>                           const struct sgi_target *target);
>   extern bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq);
>   
> +#endif /* !CONFIG_NEW_VGIC */
> +
>   /*** Common VGIC functions used by Xen arch code ****/
>   
>   /*
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 15/45] ARM: new VGIC: Add acccessor to new struct vgic_irq instance

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> The new VGIC implementation centers around a struct vgic_irq instance
> per virtual IRQ.
> Provide a function to retrieve the right instance for a given IRQ
> number and (in case of private interrupts) the right VCPU.
> This also includes the corresponding put function, which does nothing
> for private interrupts and SPIs, but handles the ref-counting for LPIs.
> 
> This is based on Linux commit 64a959d66e47, written by Christoffer Dall.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> Changelog v1 ... v2:
> - reorder header file inclusion

Please address my comments made in

<54b15802-2f72-7e7c-ada2-a3b77f81b2f4@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 16/45] ARM: new VGIC: Implement virtual IRQ injection

[Julien Grall]

Hi,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Provide a vgic_queue_irq_unlock() function which decides whether a
> given IRQ needs to be queued to a VCPU's ap_list.
> This should be called whenever an IRQ becomes pending or enabled,
> either as a result of a hardware IRQ injection, from devices emulated by
> Xen (like the architected timer) or from MMIO accesses to the distributor
> emulation.
> Also provides the necessary functions to allow to inject an IRQ to a guest.
> Since this is the first code that starts using our locking mechanism,
> we add some (hopefully) clear documentation of our locking strategy and
> requirements along with this patch.
> 
> This is based on Linux commit 81eeb95ddbab, written by Christoffer Dall.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 18/45] ARM: new VGIC: Add IRQ sorting

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Adds the sorting function to cover the case where you have more IRQs
> to consider than you have LRs. We consider their priorities.
> This uses the new sort_list() implementation imported from Linux.
> 
> This is based on Linux commit 8e4447457965, written by Christoffer Dall.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 19/45] ARM: new VGIC: Add IRQ sync/flush framework

[Julien Grall]

Hi,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> Implement the framework for syncing IRQs between our emulation and the
> list registers, which represent the guest's view of IRQs.
> This is done in vgic_sync_from_lrs() and vgic_sync_to_lrs(), which
> get called on guest entry and exit, respectively.
> The code talking to the actual GICv2/v3 hardware is added in the
> following patches.
> 
> This is based on Linux commit 0919e84c0fc1, written by Marc Zyngier.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
> ---
> Changelog v1 ... v2:
> - make functions void

Hmmm, the function were already void in the previous version. However, 
you switch from static inline to static. Did I miss anything?

[...]

> +static void vgic_set_underflow(struct vcpu *vcpu)
> +{
> +    ASSERT(vcpu == current);
> +
> +    gic_hw_ops->update_hcr_status(GICH_HCR_UIE, 1);

The second is a boolean, so please use true.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 20/45] ARM: new VGIC: Add GICv2 world switch backend

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> +
> +	/*
> +	 * Level-triggered mapped IRQs are special because we only
> +	 * observe rising edges as input to the VGIC.
> +	 *
> +	 * If the guest never acked the interrupt we have to sample
> +	 * the physical line and set the line level, because the
> +	 * device state could have changed or we simply need to
> +	 * process the still pending interrupt later.
> +	 *
> +	 * If this causes us to lower the level, we have to also clear
> +	 * the physical active state, since we will otherwise never be
> +	 * told when the interrupt becomes asserted again.
> +	 */

Again, the indentation looks wrong.

> +        if ( vgic_irq_is_mapped_level(irq) && lr_val.pending )
> +        {
> +            struct irq_desc *irqd;
> +
> +            ASSERT(irq->hwintid >= VGIC_NR_PRIVATE_IRQS);
> +
> +            irqd = irq_to_desc(irq->hwintid);
> +            irq->line_level = gic_read_pending_state(irqd);
> +
> +            if ( !irq->line_level )
> +                gic_set_active_state(irqd, false);
> +        }
> +
> +        spin_unlock_irqrestore(&irq->irq_lock, flags);
> +        vgic_put_irq(vcpu->domain, irq);
> +    }
> +
> +    gic_hw_ops->update_hcr_status(GICH_HCR_EN, 0);

Please use false instead 0.

[...]

> +void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr)
> +{
> +    struct gic_lr lr_val = {0};
> +
> +    lr_val.virq = irq->intid;
> +
> +    if ( irq_is_pending(irq) )
> +    {
> +        lr_val.pending = true;
> +
> +        if ( irq->config == VGIC_CONFIG_EDGE )
> +            irq->pending_latch = false;
> +
> +        if ( vgic_irq_is_sgi(irq->intid) )
> +        {
> +            u32 src = ffs(irq->source);
> +
> +            BUG_ON(!src);
> +            lr_val.virt.source = (src - 1);
> +            irq->source &= ~(1 << (src - 1));
> +            if ( irq->source )
> +                irq->pending_latch = true;
> +        }
> +    }
> +
> +    lr_val.active = irq->active;
> +
> +    if ( irq->hw )
> +    {
> +        lr_val.hw_status = 1;

Please use true.

> +        lr_val.hw.pirq = irq->hwintid;
> +        /*
> +         * Never set pending+active on a HW interrupt, as the
> +         * pending state is kept at the physical distributor
> +         * level.
> +         */
> +        if ( irq->active && irq_is_pending(irq) )
> +            lr_val.pending = false;
> +    }
> +    else
> +    {
> +        if ( irq->config == VGIC_CONFIG_LEVEL )
> +            lr_val.virt.eoi = 1;

Please use true.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 31/45] ARM: new VGIC: Add SGIR register handler

[Andre Przywara]

Hi,

On 19/03/18 09:47, Julien Grall wrote:
> Hi Andre,
> 
> On 03/15/2018 08:30 PM, Andre Przywara wrote:
>> Triggering an IPI via this register is v2 specific, so the
>> implementation lives entirely in vgic-mmio-v2.c.
>>
>> This is based on Linux commit 55cc01fb9004, written by Andre Przywara.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
>> ---
>> Changelog v1 ... v2:
>> - remove stray rebase artefact
>>
>>   xen/arch/arm/vgic/vgic-mmio-v2.c | 45
>> +++++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 44 insertions(+), 1 deletion(-)
>>
>> diff --git a/xen/arch/arm/vgic/vgic-mmio-v2.c
>> b/xen/arch/arm/vgic/vgic-mmio-v2.c
>> index b333de9ed7..7e17cdc2ad 100644
>> --- a/xen/arch/arm/vgic/vgic-mmio-v2.c
>> +++ b/xen/arch/arm/vgic/vgic-mmio-v2.c
>> @@ -81,6 +81,49 @@ static void vgic_mmio_write_v2_misc(struct vcpu *vcpu,
>>       }
>>   }
>>   +static void vgic_mmio_write_sgir(struct vcpu *source_vcpu,
>> +                                 paddr_t addr, unsigned int len,
>> +                                 unsigned long val)
>> +{
>> +    struct domain *d = source_vcpu->domain;
>> +    unsigned int nr_vcpus = d->max_vcpus;
>> +    unsigned int intid = val & GICD_SGI_INTID_MASK;
>> +    unsigned long targets = (val & GICD_SGI_TARGET_MASK) >>
>> +                            GICD_SGI_TARGET_SHIFT;
>> +    unsigned int vcpu_id;
>> +
>> +    switch ( val & GICD_SGI_TARGET_LIST_MASK )
>> +    {
>> +    case GICD_SGI_TARGET_LIST:                    /* as specified by
>> targets */
>> +        targets &= GENMASK(nr_vcpus, 0);          /* limit to
>> existing VCPUs */
> 
> Shouldn't it be 'nr_vcpus - 1'?

D'oh! Indeed looks like it.
Thanks for catching this!

Cheers,
Andre.

>> +        break;
>> +    case GICD_SGI_TARGET_OTHERS:
>> +        targets = GENMASK(nr_vcpus, 0);           /* all, ...   */
>> +        targets &= ~(1U << source_vcpu->vcpu_id); /*   but self */
>> +        break;
>> +    case GICD_SGI_TARGET_SELF:                    /* this very vCPU
>> only */
>> +        targets = (1U << source_vcpu->vcpu_id);
>> +        break;
>> +    case 0x3:                                     /* reserved */
>> +        return;
>> +    }
> 
> Cheers,
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 19/45] ARM: new VGIC: Add IRQ sync/flush framework

[Andre Przywara]

Hi,

On 19/03/18 14:17, Julien Grall wrote:
> Hi,
> 
> On 03/15/2018 08:30 PM, Andre Przywara wrote:
>> Implement the framework for syncing IRQs between our emulation and the
>> list registers, which represent the guest's view of IRQs.
>> This is done in vgic_sync_from_lrs() and vgic_sync_to_lrs(), which
>> get called on guest entry and exit, respectively.
>> The code talking to the actual GICv2/v3 hardware is added in the
>> following patches.
>>
>> This is based on Linux commit 0919e84c0fc1, written by Marc Zyngier.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>
>> ---
>> Changelog v1 ... v2:
>> - make functions void
> 
> Hmmm, the function were already void in the previous version. However,
> you switch from static inline to static. Did I miss anything?

Ah right, I just saw the diff hitting on the prototype line ;-)
For the records: static inline in a .c file is a red herring:
https://www.kernel.org/doc/local/inline.html
summary: Use static inline in header files, just static in .c files.
inline is a hint, and the compiler knows best what to do. Really.

Will adjust the change log.

Cheers,
Andre.

> 
> [...]
> 
>> +static void vgic_set_underflow(struct vcpu *vcpu)
>> +{
>> +    ASSERT(vcpu == current);
>> +
>> +    gic_hw_ops->update_hcr_status(GICH_HCR_UIE, 1);
> 
> The second is a boolean, so please use true.
> 
> Cheers,
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 09/45] ARM: GIC: Allow tweaking the active and pending state of an IRQ

[Andre Przywara]

Hi,

On 19/03/18 09:30, Julien Grall wrote:
> 
> 
> On 03/16/2018 04:05 PM, Andre Przywara wrote:
>> Hi,
> 
> Hi Andre,
> 
>> On 15/03/18 20:30, Andre Przywara wrote:
>>> +    }
>>> +}
>>> +
>>> +static void gicv2_set_pending_state(struct irq_desc *irqd, bool
>>> pending)
>>> +{
>>> +    ASSERT(spin_is_locked(&irqd->lock));
>>> +
>>> +    if ( pending )
>>> +    {
>>> +        /* The INPROGRESS bit will be set when the interrupt fires. */
>>> +        gicv2_poke_irq(irqd, GICD_ISPENDR);
>>> +    }
>>> +    else
>>> +    {
>>> +        gicv2_poke_irq(irqd, GICD_ICPENDR);
>>> +        if ( !gicv2_peek_irq(irqd, GICD_ISACTIVER) &&
>>> +             test_bit(_IRQ_GUEST, &irqd->status) )
>>> +            clear_bit(_IRQ_INPROGRESS, &irqd->status);
>>
>> We should not need to touch the _IRQ_INPROGRESS bit here. That bit
>> really shadows the *active* bit, so changing the pending state should
>> not matter here:
>> - If the h/w IRQ is active, the bit is set already and should remain so,
>> as Xen and the guest are still dealing with it. Clearing the h/w pending
>> state does not change that.
>> - If the h/w IRQ is not active, the _IRQ_INPROGRESS bit is not set, so
>> clearing it would be a NOP.
>> So we can remove  the _IRQ_INPROGRESS handling here completely.
>>
>> I will amend the code accordingly, including the respective GICv3 parts.
> 
> Thank you for summarizing our discussion. Is it still making sense to
> document how those helpers should be called?

Which helpers? The set_{pending,active}_state() functions? I already put
some kind of warning before the (wrapper) prototypes:
/*
 * Set the active state of an IRQ. This should be used with care, as
 * this directly forces the active bit, without considering the GIC
 * state machine.
 * For private IRQs this only works for those of the current CPU.
 */

Cheers,
Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 09/45] ARM: GIC: Allow tweaking the active and pending state of an IRQ

[Julien Grall]

Hi Andre,

On 03/19/2018 05:54 PM, Andre Przywara wrote:
> Which helpers? The set_{pending,active}_state() functions? I already put
> some kind of warning before the (wrapper) prototypes:
> /*
>   * Set the active state of an IRQ. This should be used with care, as
>   * this directly forces the active bit, without considering the GIC
>   * state machine.
>   * For private IRQs this only works for those of the current CPU.
>   */

Oh yes, I missed that. Sorry for the noise.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 40/45] ARM: new VGIC: vgic-init: register VGIC

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index 002fec57e6..4b9664f313 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -946,6 +946,28 @@ void vgic_sync_hardware_irq(struct domain *d,
>       spin_unlock_irqrestore(&desc->lock, flags);
>   }
>   
> +unsigned int vgic_max_vcpus(const struct domain *d)
> +{
> +    unsigned int vgic_vcpu_limit;
> +
> +    switch ( d->arch.vgic.version )
> +    {
> +#ifdef CONFIG_HAS_GICV3
> +    case GIC_V3:
> +        vgic_vcpu_limit = VGIC_V3_MAX_CPUS;
> +        break;
> +#endif

It is a bit strange that you handle GICV3 here but don't in 
domain_vgic_register.

> +    case GIC_V2:
> +        vgic_vcpu_limit = VGIC_V2_MAX_CPUS;
> +        break;
> +    default:
> +        vgic_vcpu_limit = MAX_VIRT_CPUS;

I feel this is a bit odd. We only support GICv2 and GICv3 and the enum 
has two values. Likely GCC will complain if CONFIG_HAS_GICV3 is set 
because default label is not used.

Lastly, I can't see how you handle the corner case mentioned in the 
current vGIC:

     /*
      * Since evtchn_init would call domain_max_vcpus for poll_mask
      * allocation when the vgic_ops haven't been initialised yet,
      * we return MAX_VIRT_CPUS if d->arch.vgic.handler is null.
      */

The comment in the code would also be very useful as the reason to call 
vgic_max_vcpus before the full initialization is not that straightforward.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 42/45] ARM: new VGIC: vgic-init: implement vgic_init

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> +/* INIT/DESTROY */
> +
> +/**
> + * domain_vgic_init: initialize the dist data structures
> + * @d: domain pointer
> + * @nr_spis: number of SPIs
> + */
> +int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> +{
> +    struct vgic_dist *dist = &d->arch.vgic;
> +    unsigned int i;
> +    int ret;
> +
> +    /* Limit the number of virtual SPIs supported to (1020 - 32) = 988  */
> +    if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
> +        return -EINVAL;
> +    nr_spis = ROUNDUP(nr_spis, 32);

I think the roundup is misplaced. Here you might allow the guest to 
inject IRQs 1020-1023. From the Spec, using those values in the LRs for 
the vINTID field will result to unpredictable behavior.

So you want to move the ROUNDUP before the check. This will limit the 
SPI INT ID to 992, but this is not a big deal for now.


Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 43/45] ARM: new VGIC: vgic-init: implement map_resources

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> map_resources is the last initialization step needed before the first
> VCPU is run. At that stage the code stores the MMIO base addresses used.
> Also it registers the respective register frames with the MMIO framework.
> 
> This is based on Linux commit cbae53e663ea, written by Eric Auger.
> 
> Signed-off-by: Andre Przywara <andre.przywara@linaro.org>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changelog v1 ... v2:
> - whitespace fixes
> 
>   xen/arch/arm/vgic/vgic-v2.c | 66 +++++++++++++++++++++++++++++++++++++++++++++
>   xen/arch/arm/vgic/vgic.h    |  1 +
>   2 files changed, 67 insertions(+)
> 
> diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
> index 4662346327..c35f406a06 100644
> --- a/xen/arch/arm/vgic/vgic-v2.c
> +++ b/xen/arch/arm/vgic/vgic-v2.c
> @@ -225,6 +225,72 @@ void vgic_v2_enable(struct vcpu *vcpu)
>       gic_hw_ops->update_hcr_status(GICH_HCR_EN, 1);
>   }
>   
> +int vgic_v2_map_resources(struct domain *d)
> +{
> +    struct vgic_dist *dist = &d->arch.vgic;
> +    paddr_t cbase, csize;
> +    paddr_t vbase;
> +    int ret;
> +
> +    /*
> +     * The hardware domain gets the hardware address.
> +     * Guests get the virtual platform layout.
> +     */
> +    if ( is_hardware_domain(d) )
> +    {
> +        d->arch.vgic.vgic_dist_base = gic_v2_hw_data.dbase;
> +        /*
> +         * For the hardware domain, we always map the whole HW CPU
> +         * interface region in order to match the device tree (the "reg"
> +         * properties is copied as it is).
> +         * Note that we assume the size of the CPU interface is always
> +         * aligned to PAGE_SIZE.
> +         */
> +        cbase = gic_v2_hw_data.cbase; /* was: dist->vgic_cpu_base */
> +        csize = gic_v2_hw_data.csize;
> +        vbase = gic_v2_hw_data.vbase; /* was: kvm_vgic_global_state.vcpu_base */
> +    }
> +    else
> +    {
> +        d->arch.vgic.vgic_dist_base = GUEST_GICD_BASE;
> +        /*
> +         * The CPU interface exposed to the guest is always 8kB. We may
> +         * need to add an offset to the virtual CPU interface base
> +         * address when in the GIC is aliased to get a 8kB contiguous
> +         * region.
> +         */
> +        BUILD_BUG_ON(GUEST_GICC_SIZE != SZ_8K);
> +        cbase = GUEST_GICC_BASE;
> +        csize = GUEST_GICC_SIZE;
> +        vbase = gic_v2_hw_data.vbase + gic_v2_hw_data.aliased_offset;
> +    }
> +
> +
> +    ret = vgic_register_dist_iodev(d, gaddr_to_gfn(dist->vgic_dist_base),
> +                                   VGIC_V2);
> +    if ( ret )
> +    {
> +        gdprintk(XENLOG_ERR, "Unable to register VGIC MMIO regions\n");
> +        return ret;
> +    }
> +
> +    /*
> +     * Map the gic virtual cpu interface in the gic cpu interface
> +     * region of the guest.
> +     */
> +    ret = map_mmio_regions(d, gaddr_to_gfn(cbase), csize / PAGE_SIZE,
> +                           maddr_to_mfn(vbase));
> +    if ( ret )
> +    {
> +        gdprintk(XENLOG_ERR, "Unable to remap VGIC CPU to VCPU\n");
> +        return ret;
> +    }
> +
> +    dist->ready = true;
> +
> +        return 0;
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/arch/arm/vgic/vgic.h b/xen/arch/arm/vgic/vgic.h
> index def1ac478a..7af982f100 100644
> --- a/xen/arch/arm/vgic/vgic.h
> +++ b/xen/arch/arm/vgic/vgic.h
> @@ -63,6 +63,7 @@ void vgic_v2_fold_lr_state(struct vcpu *vcpu);
>   void vgic_v2_populate_lr(struct vcpu *vcpu, struct vgic_irq *irq, int lr);
>   void vgic_v2_set_underflow(struct vcpu *vcpu);
>   void vgic_v2_enable(struct vcpu *vcpu);
> +int vgic_v2_map_resources(struct domain *d);
>   int vgic_register_dist_iodev(struct domain *d, gfn_t dist_base_fn,
>                                enum vgic_type);
>   
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Julien Grall]

Hi Andre,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> diff --git a/xen/arch/arm/vgic/Makefile b/xen/arch/arm/vgic/Makefile
> new file mode 100644
> index 0000000000..806826948e
> --- /dev/null
> +++ b/xen/arch/arm/vgic/Makefile
> @@ -0,0 +1,5 @@
> +obj-y += vgic.o
> +obj-y += vgic-v2.o
> +obj-y += vgic-mmio.o
> +obj-y += vgic-mmio-v2.o
> +obj-y += vgic-init.o
> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index 4b9664f313..342b95be31 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -968,6 +968,16 @@ unsigned int vgic_max_vcpus(const struct domain *d)
>       return min_t(unsigned int, MAX_VIRT_CPUS, vgic_vcpu_limit);
>   }
>   
> +#ifdef CONFIG_HAS_GICV3
> +void vgic_v3_setup_hw(paddr_t dbase,
> +                      unsigned int nr_rdist_regions,
> +                      const struct rdist_region *regions,
> +                      unsigned int intid_bits)
> +{
> +    /* Dummy implementation to allow building without actual vGICv3 support. */
> +}
> +#endif

Why not just avoid selecting HAS_GICV3?

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 00/45] New VGIC(-v2) implementation

[Julien Grall]

Hi,

On 03/15/2018 08:30 PM, Andre Przywara wrote:
> tl;dr: Coarse changelog below, individual patches have changelogs as
> well. git branch:
> http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/vgic-new/v2
> git://linux-arm.org/xen-ap.git branch vgic-new/v2
> 
> Another update, addressing the review comments. Nothing too outstanding this
> time, the most interesting changes include:
> - removing the split-out preparatory patches, which are already merged
> - changing the setting and clearing of _IRQ_INPROGRESS
> - including Julien's LR access rework series
> - restricting new level IRQ handling to the new VGIC
> - fix multiple SGI handling (mimicing the recent Linux/KVM patch)
> - ASSERTing that h/w IRQs stay connected to their virtual IRQs
> - directly update h/w affinity, without taking the desc lock
> - restrict 8K struct vcpu to new VGIC and ARM64
> - use separate Makefile for vgic/ directory
> - many minor changes to address whitespace issues and usage of unsigned,
>    also extending comments
> 
> A summarising changelog can be found below, each individual patch has
> its own changelog as well.
> 
> There are some things that have (still) not been covered yet:
> - struct VCPU still allocates two pages on ARM64 when using the new VGIC now.
> We could try to look if we can allocate some parts of struct vcpu instead of
> embedding sub-structures into it.
> - vGICv3 support is not implemented, but should be fairly straight-forward to
> add, as the design incorporated this already. Will look at this next.
> - There is a possible DOS vector on the VCPU ap_list, which holds pending
> vIRQs. A guest can make this list rather long, which forces the hypervisor
> to hold the list lock when iterating the list. This should be bounded by
> the number of emulated vIRQs though, and there are ideas how to mitigate
> this issue. Those fixes would be posted on top as fixes later.
> - There is no ITS support, though the VGIC code itself is more ready for that
> than the old VGIC ever was. However due to differences between the Xen
> and KVM architecture the ITS bits are not easy to port over to Xen.
> - Do we need to call vgic_evtchn_irq_pending() in
> local_event_needs_delivery_nomask()? The event channel IRQ should be covered
> by the VGIC already.

Thank you for summarising the open questions! I will log them on jira 
once the series is merged.

Meanwhile, I have committed patches #1 - #8.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 02/45] ARM: Implement vcpu_kick()

[Jan Beulich]

>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
> If we change something in a vCPU that affects its runnability or
> otherwise needs the vCPU's attention, we might need to tell the scheduler
> about it.
> We are using this in one place (vIRQ injection) at the moment, but will
> need this at more places soon.
> So let's factor out this functionality, using the already existing
> vcpu_kick() prototype (used in x86 only so far), to make this available
> to the rest of the Xen code.

Having just seen this among the commits having gone in recently -
was it considered to make this a common function? The
implementations currently differ, but I'm not sure I see why that
needs to be. With x86's vcpu_kick_softirq() handler doing nothing
I could see the ARM implementation be suitable for x86, just like
I could see the x86 implementation be suitable for ARM.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 00/45] New VGIC(-v2) implementation

[Andre Przywara]

Hi,

On 20/03/18 08:30, Julien Grall wrote:
> Hi,
> 
> On 03/15/2018 08:30 PM, Andre Przywara wrote:
>> tl;dr: Coarse changelog below, individual patches have changelogs as
>> well. git branch:
>> http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/vgic-new/v2
>>
>> git://linux-arm.org/xen-ap.git branch vgic-new/v2
>>
>> Another update, addressing the review comments. Nothing too
>> outstanding this
>> time, the most interesting changes include:
>> - removing the split-out preparatory patches, which are already merged
>> - changing the setting and clearing of _IRQ_INPROGRESS
>> - including Julien's LR access rework series
>> - restricting new level IRQ handling to the new VGIC
>> - fix multiple SGI handling (mimicing the recent Linux/KVM patch)
>> - ASSERTing that h/w IRQs stay connected to their virtual IRQs
>> - directly update h/w affinity, without taking the desc lock
>> - restrict 8K struct vcpu to new VGIC and ARM64
>> - use separate Makefile for vgic/ directory
>> - many minor changes to address whitespace issues and usage of unsigned,
>>    also extending comments
>>
>> A summarising changelog can be found below, each individual patch has
>> its own changelog as well.
>>
>> There are some things that have (still) not been covered yet:
>> - struct VCPU still allocates two pages on ARM64 when using the new
>> VGIC now.
>> We could try to look if we can allocate some parts of struct vcpu
>> instead of
>> embedding sub-structures into it.
>> - vGICv3 support is not implemented, but should be fairly
>> straight-forward to
>> add, as the design incorporated this already. Will look at this next.
>> - There is a possible DOS vector on the VCPU ap_list, which holds pending
>> vIRQs. A guest can make this list rather long, which forces the
>> hypervisor
>> to hold the list lock when iterating the list. This should be bounded by
>> the number of emulated vIRQs though, and there are ideas how to mitigate
>> this issue. Those fixes would be posted on top as fixes later.
>> - There is no ITS support, though the VGIC code itself is more ready
>> for that
>> than the old VGIC ever was. However due to differences between the Xen
>> and KVM architecture the ITS bits are not easy to port over to Xen.
>> - Do we need to call vgic_evtchn_irq_pending() in
>> local_event_needs_delivery_nomask()? The event channel IRQ should be
>> covered
>> by the VGIC already.
> 
> Thank you for summarising the open questions! I will log them on jira
> once the series is merged.

Good idea, thanks!

> Meanwhile, I have committed patches #1 - #8.

Many thanks for that! Only 37 to go ;-)
Will try to send a v3 later today.

Cheers,
Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 45/45] ARM: VGIC: wire new VGIC(-v2) files into Xen build system

[Andre Przywara]

Hi,

On 20/03/18 03:13, Julien Grall wrote:
> Hi Andre,
> 
> On 03/15/2018 08:30 PM, Andre Przywara wrote:
>> diff --git a/xen/arch/arm/vgic/Makefile b/xen/arch/arm/vgic/Makefile
>> new file mode 100644
>> index 0000000000..806826948e
>> --- /dev/null
>> +++ b/xen/arch/arm/vgic/Makefile
>> @@ -0,0 +1,5 @@
>> +obj-y += vgic.o
>> +obj-y += vgic-v2.o
>> +obj-y += vgic-mmio.o
>> +obj-y += vgic-mmio-v2.o
>> +obj-y += vgic-init.o
>> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
>> index 4b9664f313..342b95be31 100644
>> --- a/xen/arch/arm/vgic/vgic.c
>> +++ b/xen/arch/arm/vgic/vgic.c
>> @@ -968,6 +968,16 @@ unsigned int vgic_max_vcpus(const struct domain *d)
>>       return min_t(unsigned int, MAX_VIRT_CPUS, vgic_vcpu_limit);
>>   }
>>   +#ifdef CONFIG_HAS_GICV3
>> +void vgic_v3_setup_hw(paddr_t dbase,
>> +                      unsigned int nr_rdist_regions,
>> +                      const struct rdist_region *regions,
>> +                      unsigned int intid_bits)
>> +{
>> +    /* Dummy implementation to allow building without actual vGICv3
>> support. */
>> +}
>> +#endif
> 
> Why not just avoid selecting HAS_GICV3?

Because "config ARM_64" selects HAS_GICV3, and I didn't dare to touch
this. Shouldn't be around for long anyways.

Cheers,
Andre.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 40/45] ARM: new VGIC: vgic-init: register VGIC

[Andre Przywara]

Hi,

On 20/03/18 01:17, Julien Grall wrote:
> Hi Andre,
> 
> On 03/15/2018 08:30 PM, Andre Przywara wrote:
>> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
>> index 002fec57e6..4b9664f313 100644
>> --- a/xen/arch/arm/vgic/vgic.c
>> +++ b/xen/arch/arm/vgic/vgic.c
>> @@ -946,6 +946,28 @@ void vgic_sync_hardware_irq(struct domain *d,
>>       spin_unlock_irqrestore(&desc->lock, flags);
>>   }
>>   +unsigned int vgic_max_vcpus(const struct domain *d)
>> +{
>> +    unsigned int vgic_vcpu_limit;
>> +
>> +    switch ( d->arch.vgic.version )
>> +    {
>> +#ifdef CONFIG_HAS_GICV3
>> +    case GIC_V3:
>> +        vgic_vcpu_limit = VGIC_V3_MAX_CPUS;
>> +        break;
>> +#endif
> 
> It is a bit strange that you handle GICV3 here but don't in
> domain_vgic_register.

Fair enough.

>> +    case GIC_V2:
>> +        vgic_vcpu_limit = VGIC_V2_MAX_CPUS;
>> +        break;
>> +    default:
>> +        vgic_vcpu_limit = MAX_VIRT_CPUS;
> 
> I feel this is a bit odd. We only support GICv2 and GICv3 and the enum
> has two values. Likely GCC will complain if CONFIG_HAS_GICV3 is set
> because default label is not used.

AFAICT (and have tested) at least my GCC never complains about
"default:", even if every enum member has already been used. I think
it's good style to catch those cases should the enum get extended for
some reason.
Plus we have this already in arch_domain_create() (in switch
get_hw_version()).

> Lastly, I can't see how you handle the corner case mentioned in the
> current vGIC:
> 
>     /*
>      * Since evtchn_init would call domain_max_vcpus for poll_mask
>      * allocation when the vgic_ops haven't been initialised yet,
>      * we return MAX_VIRT_CPUS if d->arch.vgic.handler is null.
>      */

Do we need this still with Andrew's latest patch?

> The comment in the code would also be very useful as the reason to call
> vgic_max_vcpus before the full initialization is not that straightforward.

Otherwise this smells like we need to have enum gic_version extended, to
reserve the 0 case? enum gic_version {GIC_INVALID, GIC_V2, GIC_V3};?

That should cover the not-yet-initialised case, shouldn't it?

Cheers,
Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 02/45] ARM: Implement vcpu_kick()

[Julien Grall]

Hi Jan,

On 03/20/2018 10:35 AM, Jan Beulich wrote:
>>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
>> If we change something in a vCPU that affects its runnability or
>> otherwise needs the vCPU's attention, we might need to tell the scheduler
>> about it.
>> We are using this in one place (vIRQ injection) at the moment, but will
>> need this at more places soon.
>> So let's factor out this functionality, using the already existing
>> vcpu_kick() prototype (used in x86 only so far), to make this available
>> to the rest of the Xen code.
> 
> Having just seen this among the commits having gone in recently -
> was it considered to make this a common function? The
> implementations currently differ, but I'm not sure I see why that
> needs to be. With x86's vcpu_kick_softirq() handler doing nothing
> I could see the ARM implementation be suitable for x86, just like
> I could see the x86 implementation be suitable for ARM.
I considered it when reviewing the patch but discard it I wasn't 
entirely sure if it was possible to make it common and  I wanted this 
series to move forward (it is 50 patches series)!

I would be happy to consider any patch to make them common. My 
preference would tend to go towards the Arm solution as it has a 
slightly smaller overhead to kick a vCPU. Indeed the x86 version 
requires to raise a softirq and then send an IPI to the other CPU.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 40/45] ARM: new VGIC: vgic-init: register VGIC

[Julien Grall]

On 03/20/2018 05:11 PM, Andre Przywara wrote:
> Hi,
> 
> On 20/03/18 01:17, Julien Grall wrote:
>> Hi Andre,
>>
>> On 03/15/2018 08:30 PM, Andre Przywara wrote:
>>> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
>>> index 002fec57e6..4b9664f313 100644
>>> --- a/xen/arch/arm/vgic/vgic.c
>>> +++ b/xen/arch/arm/vgic/vgic.c
>>> @@ -946,6 +946,28 @@ void vgic_sync_hardware_irq(struct domain *d,
>>>        spin_unlock_irqrestore(&desc->lock, flags);
>>>    }
>>>    +unsigned int vgic_max_vcpus(const struct domain *d)
>>> +{
>>> +    unsigned int vgic_vcpu_limit;
>>> +
>>> +    switch ( d->arch.vgic.version )
>>> +    {
>>> +#ifdef CONFIG_HAS_GICV3
>>> +    case GIC_V3:
>>> +        vgic_vcpu_limit = VGIC_V3_MAX_CPUS;
>>> +        break;
>>> +#endif
>>
>> It is a bit strange that you handle GICV3 here but don't in
>> domain_vgic_register.
> 
> Fair enough.
> 
>>> +    case GIC_V2:
>>> +        vgic_vcpu_limit = VGIC_V2_MAX_CPUS;
>>> +        break;
>>> +    default:
>>> +        vgic_vcpu_limit = MAX_VIRT_CPUS;
>>
>> I feel this is a bit odd. We only support GICv2 and GICv3 and the enum
>> has two values. Likely GCC will complain if CONFIG_HAS_GICV3 is set
>> because default label is not used.
> 
> AFAICT (and have tested) at least my GCC never complains about
> "default:", even if every enum member has already been used. I think
> it's good style to catch those cases should the enum get extended for
> some reason.

Well, the compiler will usually tell you if you miss one case. In that 
circumstance you put a value that may not make sense for that new item 
in the enum and you will have some trouble to catch it.

So the default should really be a BUG() or ASSERT_UNREACHABLE().

> Plus we have this already in arch_domain_create() (in switch
> get_hw_version()).

See my point above.

> 
>> Lastly, I can't see how you handle the corner case mentioned in the
>> current vGIC:
>>
>>      /*
>>       * Since evtchn_init would call domain_max_vcpus for poll_mask
>>       * allocation when the vgic_ops haven't been initialised yet,
>>       * we return MAX_VIRT_CPUS if d->arch.vgic.handler is null.
>>       */
> 
> Do we need this still with Andrew's latest patch?

Well depends which series is going first. If it is yours, then you still 
need it and Andrew has to fix it.

> 
>> The comment in the code would also be very useful as the reason to call
>> vgic_max_vcpus before the full initialization is not that straightforward.
> 
> Otherwise this smells like we need to have enum gic_version extended, to
> reserve the 0 case? enum gic_version {GIC_INVALID, GIC_V2, GIC_V3};?
> 
> That should cover the not-yet-initialised case, shouldn't it?

I think so. But see above.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 02/45] ARM: Implement vcpu_kick()

[Jan Beulich]

>>> On 21.03.18 at 05:10, <julien.grall@arm.com> wrote:
> On 03/20/2018 10:35 AM, Jan Beulich wrote:
>>>>> On 15.03.18 at 21:30, <andre.przywara@linaro.org> wrote:
>>> If we change something in a vCPU that affects its runnability or
>>> otherwise needs the vCPU's attention, we might need to tell the scheduler
>>> about it.
>>> We are using this in one place (vIRQ injection) at the moment, but will
>>> need this at more places soon.
>>> So let's factor out this functionality, using the already existing
>>> vcpu_kick() prototype (used in x86 only so far), to make this available
>>> to the rest of the Xen code.
>> 
>> Having just seen this among the commits having gone in recently -
>> was it considered to make this a common function? The
>> implementations currently differ, but I'm not sure I see why that
>> needs to be. With x86's vcpu_kick_softirq() handler doing nothing
>> I could see the ARM implementation be suitable for x86, just like
>> I could see the x86 implementation be suitable for ARM.
> I considered it when reviewing the patch but discard it I wasn't 
> entirely sure if it was possible to make it common and  I wanted this 
> series to move forward (it is 50 patches series)!
> 
> I would be happy to consider any patch to make them common. My 
> preference would tend to go towards the Arm solution as it has a 
> slightly smaller overhead to kick a vCPU. Indeed the x86 version 
> requires to raise a softirq and then send an IPI to the other CPU.

Yes, that's my preference too. I'll send something after 4.11 has
settled.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel