* [PATCH v8 00/27] arm64: Dom0 ITS emulation
@ 2017-04-12 0:44 Andre Przywara
2017-04-12 0:44 ` [PATCH v8 01/27] ARM: GICv3: propagate number of host LPIs to GICv3 guest Andre Przywara
` (27 more replies)
0 siblings, 28 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
this is a reworked version of the second part of the ITS emulation series,
where the first part has been merged already.
It extends the concept of letting Xen deal with irq_to_pending()
potentially returning NULL, by making sure the retrieval and usage
of the pending_irq pointer is always happening under the VCPU VGIC lock
(patch 02 and 03). This allows us to free the memory for the pending_irqs
when a device gets unmapped.
Patch 20 contains a relatively easy solution to some LPI unmap/map corner
case (DISCARD;MAPTI sequence using the same virtual LPI number while the
VLPI is pending in some LR).
On top of these changes I addressed the remaining comments from v5 and v6.
For a detailed changelog see below.
Cheers,
Andre
----------------------------------
This series adds support for emulation of an ARM GICv3 ITS interrupt
controller. For hardware which relies on the ITS to provide interrupts for
its peripherals this code is needed to get a machine booted into Dom0 at
all. ITS emulation for DomUs is only really useful with PCI passthrough,
which is not yet available for ARM. It is expected that this feature
will be co-developed with the ITS DomU code. However this code drop here
considered DomU emulation already, to keep later architectural changes
to a minimum.
This is technical preview version to allow early testing of the feature.
Things not (properly) addressed in this release:
- The MOVALL command is not emulated. In our case there is really nothing
to do here. We might need to revisit this in the future for DomU support.
- The INVALL command might need some rework to be more efficient. Currently
we iterate over all mapped LPIs, which might take a bit longer.
- Indirect tables are not supported. This affects both the host and the
virtual side.
- The command queue locking is currently suboptimal and should be made more
fine-grained in the future, if possible.
- We don't move the LPIs on the host to the pCPU where the target VCPU
is currently running on. Doing so would involve sending ITS commands to the
host. We should investigate whether this is feasible during scheduling.
- MOVI doesn't move pending IRQs.
- We need to properly investigate the possible interaction when devices get
removed. This requires to properly clean up and remove any associated
resources like pending or in-flight LPIs, for instance.
Some generic design principles:
* The current GIC code statically allocates structures for each supported
IRQ (both for the host and the guest), which due to the potentially
millions of LPI interrupts is not feasible to copy for the ITS.
So we refrain from introducing the ITS as a first class Xen interrupt
controller, also we don't hold struct irq_desc's or struct pending_irq's
for each possible LPI.
Fortunately LPIs are only interesting to guests, so we get away with
storing only the virtual IRQ number and the guest VCPU for each allocated
host LPI, which can be stashed into one uint64_t. This data is stored in
a two-level table, which is both memory efficient and quick to access.
We hook into the existing IRQ handling and VGIC code to avoid accessing
the normal structures, providing alternative methods for getting the
needed information (priority, is enabled?) for LPIs.
Whenever a guest maps a device, we allocate the maximum required number
of struct pending_irq's, so that any triggering LPI can find its data
structure. Upon the guest actually mapping the LPI, this pointer to the
corresponding pending_irq gets entered into a radix tree, so that it can
be quickly looked up.
* On the guest side we (later will) have to deal with malicious guests
trying to hog Xen with mapping requests for a lot of LPIs, for instance.
As the ITS actually uses system memory for storing status information,
we use this memory (which the guest has to provide) to naturally limit
a guest. Whenever we need information from any of the ITS tables, we
temporarily map them (which is cheap on arm64) and copy the required data.
* An obvious approach to handling some guest ITS commands would be to
propagate them to the host, for instance to map devices and LPIs and
to enable or disable LPIs.
However this (later with DomU support) will create an attack vector, as
a malicious guest could try to fill the host command queue with
propagated commands.
So we try to avoid this situation: Dom0 sending a device mapping (MAPD)
command is the only time we allow queuing commands to the host ITS command
queue, as this seems to be the only reliable way of getting the
required information at the moment. However at the same time we map all
events to LPIs already, also enable them. This avoids sending commands
later at runtime, as we can deal with mappings and LPI enabling/disabling
internally.
To accomodate the tech preview nature of this feature at the moment, there
is a Kconfig option to enable it. Also it is supported on arm64 only, which
will most likely not change in the future.
This leads to some hideous constructs like an #ifdef'ed header file with
empty function stubs to accomodate arm32 and non-ITS builds, which share
some generic code paths with the ITS emulation.
The number of supported LPIs can be limited on the command line, in case
the number reported by the hardware is too high. As Xen cannot foresee how
many interrupts the guests will need, we cater for as many as possible.
The command line parameter is called max-lpi-bits and expresses the number
of bits required to hold an interrupt ID. It defaults to 20, if that is
lower than the number supported by the hardware.
This code boots Dom0 on an ARM Fast Model with ITS support. I tried to
address the issues seen by people running the previous versions on real
hardware, though couldn't verify this here for myself.
So any testing, bug reports (and possibly even fixes) are very welcome.
The code can also be found on the its/v6 branch here:
git://linux-arm.org/xen-ap.git
http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/its/v6
Cheers,
Andre
Changelog v7 ... v8:
- drop list parameter and rename to gicv3_its_make_hwdwom_dt_nodes()
- remove rebase artifacts
- add irq_enter/irq_exit() calls
- propagates number of host LPIs and number of event IDs to Dom0
- add proper coverage of all addresses in ITS MMIO handler
- avoid vcmd_lock for CBASER writes
- fix missing irqsave/irqrestore on VGIC VCPU lock
- move struct pending_irq use under the VGIC VCPU lock
- protect gic_raise_guest_irq() against NULL pending_irq
- improve device and collection table entry size documentation
- count number of ITSes to increase mmio_count
- rework MAPD, DISCARD, MAPTI and MOVI to take proper locks
- properly rollback failing MAPD and MAPTI calls
- rework functions to update property table
- return error on vgic_access_guest_memory crossing page boundary
- make sure CREADR access is atomic
- add PRISTINE_LPI handling
Changelog v6 ... v7:
- got rid of GIC_IRQ_GUEST_LPI_PENDING
- typo and comment fix in 09/36
- rebased against latest staging
- added Reviewed-by: and Acked-by: tags
Changelog v5 ... v6:
- reordered patches to allow splitting the series
- introduced functions later to avoid warnings on intermediate builds
- refactored common code changes into separate patches
- dropped GENMASK_ULL and BIT_ULL (both patches and their usage later)
- rework locking in MMIO register reads and writes
- protect new code from being executed without an ITS being configured
- fix vgic_access_guest_memory (now a separate patch)
- some more comments and TODOs
Changelog v4 ... v5:
- adding many comments
- spinlock asserts
- rename r_host_lpis to max_host_lpi_ids
- remove max_its_device_bits command line
- add warning on high number of LPIs
- avoid potential leak on host MAPD
- properly handle nr_events rounding
- remove unmap_all_devices(), replace with ASSERT
- add barriers for (lockless) host LPI lookups
- add proper locking in ITS and redist MMIO register handling
- rollback failing device mapping
- fix various printks
- add vgic_access_guest_memory() and use it
- (getting rid of page mapping functions and helpers)
- drop table mapping / unmapping on redist/ITS enable/disable
- minor reworks in functions as per review comments
- fix ITS enablement check
- move lpi_to_pending() and lpi_get_priority() to vgic_ops
- move do_LPI() to gic_hw_ops
- whitespace and hard tabs fixes
- introduce ITS domain init function (and use it for the rbtree)
- enable IRQs around do_LPI
- implement TODOs for later optimizations
- add "v" prefix to variables holding virtual properties
- provide locked and normal versions of read/write_itte
- only CLEAR LPI if not already guest visible (plus comment)
- update LPI property on MAPTI
- store vcpu_id in pending_irq for LPIs (helps INVALL)
- improve INVALL implementation to only cover LPIs on this VCPU
- improve virtual BASE register initialization
- limit number of virtual LPIs to 24 bits (Linux bug at 32??)
- only inject LPIs if redistributor is actually enabled
Changelog v3 .. v4:
- make HAS_ITS depend on EXPERT
- introduce new patch 02 to initialize host ITS early
- fix cmd_lock init position
- introduce warning on high number of LPI allocations
- various int -> unsigned fixes
- adding and improving comments
- rate limit ITS command queue full msg
- drop unneeded checks
- validate against allowed number of device IDs
- avoid memory leaks when removing devices
- improve algorithm for finding free host LPI
- convert unmap_all_devices from goto to while loop
- add message on remapping ITS device
- name virtual device / event IDs properly
- use atomic read when reading ITT entry
Changelog v2 .. v3:
- preallocate struct pending_irq's
- map ITS and redistributor tables only on demand
- store property, enable and pending bit in struct pending_irq
- improve error checking and handling
- add comments
Changelog v1 .. v2:
- clean up header file inclusion
- rework host ITS table allocation: observe attributes, many fixes
- remove patch 1 to export __flush_dcache_area, use existing function instead
- use number of LPIs internally instead of number of bits
- keep host_its_list as private as possible
- keep struct its_devices private
- rework gicv3_its_map_guest_devices
- fix rbtree issues
- more error handling and propagation
- cope with GICv4 implementations (but no virtual LPI features!)
- abstract host and guest ITSes by using doorbell addresses
- join per-redistributor variables into one per-CPU structure
- fix data types (unsigned int)
- many minor bug fixes
(Rough) changelog RFC-v2 .. v1:
- split host ITS driver into gic-v3-lpi.c and gic-v3-its.c part
- rename virtual ITS driver file to vgic-v3-its.c
- use macros and named constants for all magic numbers
- use atomic accessors for accessing the host LPI data
- remove leftovers from connecting virtual and host ITSes
- bail out if host ITS is disabled in the DT
- rework map/unmap_guest_pages():
- split off p2m part as get/put_guest_pages (to be done on allocation)
- get rid of vmap, using map_domain_page() instead
- delay allocation of virtual tables until actual LPI/ITS enablement
- properly size both virtual and physical tables upon allocation
- fix put_domain() locking issues in physdev_op and LPI handling code
- add and extend comments in various areas
- fix lotsa coding style and white space issues, including comment style
- add locking to data structures not yet covered
- fix various locking issues
- use an rbtree to deal with ITS devices (instead of a list)
- properly handle memory attributes for ITS tables
- handle cacheable/non-cacheable ITS table mappings
- sanitize guest provided ITS/LPI table attributes
- fix breakage on non-GICv2 compatible host GICv3 controllers
- add command line parameters on top of Kconfig options
- properly wait for an ITS to become quiescient before enabling it
- handle host ITS command queue errors
- actually wait for host ITS command completion (READR==WRITER)
- fix ARM32 compilation
- various patch splits and reorderings
Andre Przywara (26):
ARM: GICv3: propagate number of host LPIs to GICv3 guest
ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock
ARM: GIC: Add checks for NULL pointer pending_irq's
ARM: GICv3: introduce separate pending_irq structs for LPIs
ARM: GICv3: forward pending LPIs to guests
ARM: GICv3: enable ITS and LPIs on the host
ARM: vGICv3: handle virtual LPI pending and property tables
ARM: vGICv3: re-use vgic_reg64_check_access
ARM: GIC: export vgic_init_pending_irq()
ARM: VGIC: add vcpu_id to struct pending_irq
ARM: vGIC: advertise LPI support
ARM: vITS: add command handling stub and MMIO emulation
ARM: vITS: introduce translation table walks
ARM: vITS: handle CLEAR command
ARM: vITS: handle INT command
ARM: vITS: handle MAPC command
ARM: vITS: handle MAPD command
ARM: vITS: handle MAPTI command
ARM: GICv3: handle unmapped LPIs
ARM: vITS: handle MOVI command
ARM: vITS: handle DISCARD command
ARM: vITS: handle INV command
ARM: vITS: handle INVALL command
ARM: vITS: increase mmio_count for each ITS
ARM: vITS: create and initialize virtual ITSes for Dom0
ARM: vITS: create ITS subnodes for Dom0 DT
Vijaya Kumar K (1):
ARM: introduce vgic_access_guest_memory()
xen/arch/arm/gic-v2.c | 7 +
xen/arch/arm/gic-v3-its.c | 196 ++++++
xen/arch/arm/gic-v3-lpi.c | 104 +++
xen/arch/arm/gic-v3.c | 29 +-
xen/arch/arm/gic.c | 65 +-
xen/arch/arm/vgic-v2.c | 15 +
xen/arch/arm/vgic-v3-its.c | 1373 +++++++++++++++++++++++++++++++++++++-
xen/arch/arm/vgic-v3.c | 289 +++++++-
xen/arch/arm/vgic.c | 75 ++-
xen/include/asm-arm/domain.h | 12 +-
xen/include/asm-arm/gic.h | 2 +
xen/include/asm-arm/gic_v3_its.h | 47 ++
xen/include/asm-arm/vgic-emul.h | 9 +
xen/include/asm-arm/vgic.h | 13 +-
14 files changed, 2192 insertions(+), 44 deletions(-)
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* [PATCH v8 01/27] ARM: GICv3: propagate number of host LPIs to GICv3 guest
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 10:06 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 02/27] ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock Andre Przywara
` (26 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The host supports a certain number of LPI identifiers, as stored in
the GICD_TYPER register.
Store this number from the hardware register in vgic_v3_hw to allow
injecting the very same number into a guest (Dom0).
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic-v3.c | 6 +++++-
xen/arch/arm/vgic-v3.c | 7 ++++++-
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/vgic.h | 3 ++-
4 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index a559e5e..29c8964 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1579,6 +1579,7 @@ static int __init gicv3_init(void)
{
int res, i;
uint32_t reg;
+ unsigned int intid_bits;
if ( !cpu_has_gicv3 )
{
@@ -1622,8 +1623,11 @@ static int __init gicv3_init(void)
i, r->base, r->base + r->size);
}
+ reg = readl_relaxed(GICD + GICD_TYPER);
+ intid_bits = GICD_TYPE_ID_BITS(reg);
+
vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions,
- gicv3.rdist_stride);
+ gicv3.rdist_stride, intid_bits);
gicv3_init_v2();
spin_lock_init(&gicv3.lock);
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index d10757a..ebcca22 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -57,18 +57,21 @@ static struct {
unsigned int nr_rdist_regions;
const struct rdist_region *regions;
uint32_t rdist_stride; /* Re-distributor stride */
+ unsigned int intid_bits; /* Number of interrupt ID bits */
} vgic_v3_hw;
void vgic_v3_setup_hw(paddr_t dbase,
unsigned int nr_rdist_regions,
const struct rdist_region *regions,
- uint32_t rdist_stride)
+ uint32_t rdist_stride,
+ unsigned int intid_bits)
{
vgic_v3_hw.enabled = 1;
vgic_v3_hw.dbase = dbase;
vgic_v3_hw.nr_rdist_regions = nr_rdist_regions;
vgic_v3_hw.regions = regions;
vgic_v3_hw.rdist_stride = rdist_stride;
+ vgic_v3_hw.intid_bits = intid_bits;
}
static struct vcpu *vgic_v3_irouter_to_vcpu(struct domain *d, uint64_t irouter)
@@ -1482,6 +1485,8 @@ static int vgic_v3_domain_init(struct domain *d)
first_cpu += size / d->arch.vgic.rdist_stride;
}
+
+ d->arch.vgic.intid_bits = vgic_v3_hw.intid_bits;
}
else
{
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 6de8082..7c3829d 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -111,6 +111,7 @@ struct arch_domain
uint32_t rdist_stride; /* Re-Distributor stride */
struct rb_root its_devices; /* Devices mapped to an ITS */
spinlock_t its_devices_lock; /* Protects the its_devices tree */
+ unsigned int intid_bits;
#endif
} vgic;
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 544867a..df75064 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -346,7 +346,8 @@ struct rdist_region;
void vgic_v3_setup_hw(paddr_t dbase,
unsigned int nr_rdist_regions,
const struct rdist_region *regions,
- uint32_t rdist_stride);
+ uint32_t rdist_stride,
+ unsigned int intid_bits);
#endif
#endif /* __ASM_ARM_VGIC_H__ */
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 02/27] ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
2017-04-12 0:44 ` [PATCH v8 01/27] ARM: GICv3: propagate number of host LPIs to GICv3 guest Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 10:13 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's Andre Przywara
` (25 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
So far irq_to_pending() is just a convenience function to lookup
in statically allocated arrays. This will change with LPIs, which are
more dynamic.
So move the irq_to_pending() call under the VGIC VCPU lock, so we
only use this pointer while holding the lock.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic.c | 5 ++++-
xen/arch/arm/vgic.c | 8 +++++---
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index da19130..dcb1783 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -402,10 +402,13 @@ static inline void gic_add_to_lr_pending(struct vcpu *v, struct pending_irq *n)
void gic_remove_from_queues(struct vcpu *v, unsigned int virtual_irq)
{
- struct pending_irq *p = irq_to_pending(v, virtual_irq);
+ struct pending_irq *p;
unsigned long flags;
spin_lock_irqsave(&v->arch.vgic.lock, flags);
+
+ p = irq_to_pending(v, virtual_irq);
+
if ( !list_empty(&p->lr_queue) )
list_del_init(&p->lr_queue);
spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 83569b0..c953f13 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -466,14 +466,16 @@ void vgic_clear_pending_irqs(struct vcpu *v)
void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
{
uint8_t priority;
- struct pending_irq *iter, *n = irq_to_pending(v, virq);
+ struct pending_irq *iter, *n;
unsigned long flags;
bool running;
- priority = vgic_get_virq_priority(v, virq);
-
spin_lock_irqsave(&v->arch.vgic.lock, flags);
+ n = irq_to_pending(v, virq);
+
+ priority = vgic_get_virq_priority(v, virq);
+
/* vcpu offline */
if ( test_bit(_VPF_down, &v->pause_flags) )
{
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
2017-04-12 0:44 ` [PATCH v8 01/27] ARM: GICv3: propagate number of host LPIs to GICv3 guest Andre Przywara
2017-04-12 0:44 ` [PATCH v8 02/27] ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 10:25 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 04/27] ARM: GICv3: introduce separate pending_irq structs for LPIs Andre Przywara
` (24 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
For LPIs the struct pending_irq's are dynamically allocated and the
pointers will be stored in a radix tree. Since an LPI can be "unmapped"
at any time, teach the VGIC how to handle with irq_to_pending() returning
a NULL pointer.
We just do nothing in this case or clean up the LR if the virtual LPI
number was still in an LR.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic.c | 37 ++++++++++++++++++++++++++++++++-----
xen/arch/arm/vgic.c | 6 ++++++
2 files changed, 38 insertions(+), 5 deletions(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index dcb1783..62ae3b8 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -408,9 +408,15 @@ void gic_remove_from_queues(struct vcpu *v, unsigned int virtual_irq)
spin_lock_irqsave(&v->arch.vgic.lock, flags);
p = irq_to_pending(v, virtual_irq);
-
- if ( !list_empty(&p->lr_queue) )
- list_del_init(&p->lr_queue);
+ /*
+ * If an LPIs has been removed meanwhile, it has been cleaned up
+ * already, so nothing to remove here.
+ */
+ if ( p )
+ {
+ if ( !list_empty(&p->lr_queue) )
+ list_del_init(&p->lr_queue);
+ }
spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
}
@@ -418,6 +424,10 @@ void gic_raise_inflight_irq(struct vcpu *v, unsigned int virtual_irq)
{
struct pending_irq *n = irq_to_pending(v, virtual_irq);
+ /* If an LPI has been removed meanwhile, there is nothing left to raise. */
+ if ( !n )
+ return;
+
ASSERT(spin_is_locked(&v->arch.vgic.lock));
if ( list_empty(&n->lr_queue) )
@@ -437,6 +447,11 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
{
int i;
unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
+ struct pending_irq *p = irq_to_pending(v, virtual_irq);
+
+ if ( !p )
+ /* An unmapped LPI does not need to be raised. */
+ return;
ASSERT(spin_is_locked(&v->arch.vgic.lock));
@@ -445,12 +460,12 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
i = find_first_zero_bit(&this_cpu(lr_mask), nr_lrs);
if (i < nr_lrs) {
set_bit(i, &this_cpu(lr_mask));
- gic_set_lr(i, irq_to_pending(v, virtual_irq), GICH_LR_PENDING);
+ gic_set_lr(i, p, GICH_LR_PENDING);
return;
}
}
- gic_add_to_lr_pending(v, irq_to_pending(v, virtual_irq));
+ gic_add_to_lr_pending(v, p);
}
static void gic_update_one_lr(struct vcpu *v, int i)
@@ -464,7 +479,19 @@ static void gic_update_one_lr(struct vcpu *v, int i)
gic_hw_ops->read_lr(i, &lr_val);
irq = lr_val.virq;
+
p = irq_to_pending(v, irq);
+ /* An LPI might have been unmapped, in which case we just clean up here. */
+ if ( !p )
+ {
+ ASSERT(is_lpi(irq));
+
+ gic_hw_ops->clear_lr(i);
+ clear_bit(i, &this_cpu(lr_mask));
+
+ return;
+ }
+
if ( lr_val.state & GICH_LR_ACTIVE )
{
set_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index c953f13..b9fc837 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -473,6 +473,12 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
spin_lock_irqsave(&v->arch.vgic.lock, flags);
n = irq_to_pending(v, virq);
+ /* If an LPI has been removed, there is nothing to inject here. */
+ if ( !n )
+ {
+ spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+ return;
+ }
priority = vgic_get_virq_priority(v, virq);
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 04/27] ARM: GICv3: introduce separate pending_irq structs for LPIs
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (2 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 10:35 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests Andre Przywara
` (23 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
For the same reason that allocating a struct irq_desc for each
possible LPI is not an option, having a struct pending_irq for each LPI
is also not feasible. We only care about mapped LPIs, so we can get away
with having struct pending_irq's only for them.
Maintain a radix tree per domain where we drop the pointer to the
respective pending_irq. The index used is the virtual LPI number.
The memory for the actual structures has been allocated already per
device at device mapping time.
Teach the existing VGIC functions to find the right pointer when being
given a virtual LPI number.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v2.c | 8 ++++++++
xen/arch/arm/vgic-v3.c | 23 +++++++++++++++++++++++
xen/arch/arm/vgic.c | 2 ++
xen/include/asm-arm/domain.h | 2 ++
xen/include/asm-arm/vgic.h | 2 ++
5 files changed, 37 insertions(+)
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index dc9f95b..0587569 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -702,10 +702,18 @@ static void vgic_v2_domain_free(struct domain *d)
/* Nothing to be cleanup for this driver */
}
+static struct pending_irq *vgic_v2_lpi_to_pending(struct domain *d,
+ unsigned int vlpi)
+{
+ /* Dummy function, no LPIs on a VGICv2. */
+ BUG();
+}
+
static const struct vgic_ops vgic_v2_ops = {
.vcpu_init = vgic_v2_vcpu_init,
.domain_init = vgic_v2_domain_init,
.domain_free = vgic_v2_domain_free,
+ .lpi_to_pending = vgic_v2_lpi_to_pending,
.max_vcpus = 8,
};
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index ebcca22..f462610 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1454,6 +1454,9 @@ static int vgic_v3_domain_init(struct domain *d)
d->arch.vgic.nr_regions = rdist_count;
d->arch.vgic.rdist_regions = rdist_regions;
+ rwlock_init(&d->arch.vgic.pend_lpi_tree_lock);
+ radix_tree_init(&d->arch.vgic.pend_lpi_tree);
+
/*
* Domain 0 gets the hardware address.
* Guests get the virtual platform layout.
@@ -1533,14 +1536,34 @@ static int vgic_v3_domain_init(struct domain *d)
static void vgic_v3_domain_free(struct domain *d)
{
vgic_v3_its_free_domain(d);
+ radix_tree_destroy(&d->arch.vgic.pend_lpi_tree, NULL);
xfree(d->arch.vgic.rdist_regions);
}
+/*
+ * Looks up a virtual LPI number in our tree of mapped LPIs. This will return
+ * the corresponding struct pending_irq, which we also use to store the
+ * enabled and pending bit plus the priority.
+ * Returns NULL if an LPI cannot be found (or no LPIs are supported).
+ */
+static struct pending_irq *vgic_v3_lpi_to_pending(struct domain *d,
+ unsigned int lpi)
+{
+ struct pending_irq *pirq;
+
+ read_lock(&d->arch.vgic.pend_lpi_tree_lock);
+ pirq = radix_tree_lookup(&d->arch.vgic.pend_lpi_tree, lpi);
+ read_unlock(&d->arch.vgic.pend_lpi_tree_lock);
+
+ return pirq;
+}
+
static const struct vgic_ops v3_ops = {
.vcpu_init = vgic_v3_vcpu_init,
.domain_init = vgic_v3_domain_init,
.domain_free = vgic_v3_domain_free,
.emulate_reg = vgic_v3_emulate_reg,
+ .lpi_to_pending = vgic_v3_lpi_to_pending,
/*
* We use both AFF1 and AFF0 in (v)MPIDR. Thus, the max number of CPU
* that can be supported is up to 4096(==256*16) in theory.
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index b9fc837..c2bfdb1 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -439,6 +439,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
* are used for SPIs; the rests are used for per cpu irqs */
if ( irq < 32 )
n = &v->arch.vgic.pending_irqs[irq];
+ else if ( is_lpi(irq) )
+ n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
else
n = &v->domain->arch.vgic.pending_irqs[irq - 32];
return n;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 7c3829d..3d8e84c 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -111,6 +111,8 @@ struct arch_domain
uint32_t rdist_stride; /* Re-Distributor stride */
struct rb_root its_devices; /* Devices mapped to an ITS */
spinlock_t its_devices_lock; /* Protects the its_devices tree */
+ struct radix_tree_root pend_lpi_tree; /* Stores struct pending_irq's */
+ rwlock_t pend_lpi_tree_lock; /* Protects the pend_lpi_tree */
unsigned int intid_bits;
#endif
} vgic;
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index df75064..c9075a9 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -134,6 +134,8 @@ struct vgic_ops {
void (*domain_free)(struct domain *d);
/* vGIC sysreg/cpregs emulate */
bool (*emulate_reg)(struct cpu_user_regs *regs, union hsr hsr);
+ /* lookup the struct pending_irq for a given LPI interrupt */
+ struct pending_irq *(*lpi_to_pending)(struct domain *d, unsigned int vlpi);
/* Maximum number of vCPU supported */
const unsigned int max_vcpus;
};
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (3 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 04/27] ARM: GICv3: introduce separate pending_irq structs for LPIs Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 10:44 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 06/27] ARM: GICv3: enable ITS and LPIs on the host Andre Przywara
` (22 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Upon receiving an LPI on the host, we need to find the right VCPU and
virtual IRQ number to get this IRQ injected.
Iterate our two-level LPI table to find this information quickly when
the host takes an LPI. Call the existing injection function to let the
GIC emulation deal with this interrupt.
Also we enhance struct pending_irq to cache the pending bit and the
priority information for LPIs. Reading the information from there is
faster than accessing the property table from guest memory. Also it
use some padding area, so does not require more memory.
This introduces a do_LPI() as a hardware gic_ops and a function to
retrieve the (cached) priority value of an LPI and a vgic_ops.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic-v2.c | 7 ++++
xen/arch/arm/gic-v3-lpi.c | 71 ++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 1 +
xen/arch/arm/gic.c | 8 ++++-
xen/arch/arm/vgic-v2.c | 7 ++++
xen/arch/arm/vgic-v3.c | 12 +++++++
xen/arch/arm/vgic.c | 7 +++-
xen/include/asm-arm/domain.h | 3 +-
xen/include/asm-arm/gic.h | 2 ++
xen/include/asm-arm/gic_v3_its.h | 8 +++++
xen/include/asm-arm/vgic.h | 2 ++
11 files changed, 125 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 270a136..ffbe47c 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -1217,6 +1217,12 @@ static int __init gicv2_init(void)
return 0;
}
+static void gicv2_do_LPI(unsigned int lpi)
+{
+ /* No LPIs in a GICv2 */
+ BUG();
+}
+
const static struct gic_hw_operations gicv2_ops = {
.info = &gicv2_info,
.init = gicv2_init,
@@ -1244,6 +1250,7 @@ const static struct gic_hw_operations gicv2_ops = {
.make_hwdom_madt = gicv2_make_hwdom_madt,
.map_hwdom_extra_mappings = gicv2_map_hwdown_extra_mappings,
.iomem_deny_access = gicv2_iomem_deny_access,
+ .do_LPI = gicv2_do_LPI,
};
/* Set up the GIC */
diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
index 292f2d0..44f6315 100644
--- a/xen/arch/arm/gic-v3-lpi.c
+++ b/xen/arch/arm/gic-v3-lpi.c
@@ -136,6 +136,77 @@ uint64_t gicv3_get_redist_address(unsigned int cpu, bool use_pta)
return per_cpu(lpi_redist, cpu).redist_id << 16;
}
+/*
+ * Handle incoming LPIs, which are a bit special, because they are potentially
+ * numerous and also only get injected into guests. Treat them specially here,
+ * by just looking up their target vCPU and virtual LPI number and hand it
+ * over to the injection function.
+ * Please note that LPIs are edge-triggered only, also have no active state,
+ * so spurious interrupts on the host side are no issue (we can just ignore
+ * them).
+ * Also a guest cannot expect that firing interrupts that haven't been
+ * fully configured yet will reach the CPU, so we don't need to care about
+ * this special case.
+ */
+void gicv3_do_LPI(unsigned int lpi)
+{
+ struct domain *d;
+ union host_lpi *hlpip, hlpi;
+ struct vcpu *vcpu;
+
+ irq_enter();
+
+ /* EOI the LPI already. */
+ WRITE_SYSREG32(lpi, ICC_EOIR1_EL1);
+
+ /* Find out if a guest mapped something to this physical LPI. */
+ hlpip = gic_get_host_lpi(lpi);
+ if ( !hlpip )
+ goto out;
+
+ hlpi.data = read_u64_atomic(&hlpip->data);
+
+ /*
+ * Unmapped events are marked with an invalid LPI ID. We can safely
+ * ignore them, as they have no further state and no-one can expect
+ * to see them if they have not been mapped.
+ */
+ if ( hlpi.virt_lpi == INVALID_LPI )
+ goto out;
+
+ d = rcu_lock_domain_by_id(hlpi.dom_id);
+ if ( !d )
+ goto out;
+
+ /* Make sure we don't step beyond the vcpu array. */
+ if ( hlpi.vcpu_id >= d->max_vcpus )
+ {
+ rcu_unlock_domain(d);
+ goto out;
+ }
+
+ vcpu = d->vcpu[hlpi.vcpu_id];
+
+ /* Check if the VCPU is ready to receive LPIs. */
+ if ( vcpu->arch.vgic.flags & VGIC_V3_LPIS_ENABLED )
+ /*
+ * TODO: Investigate what to do here for potential interrupt storms.
+ * As we keep all host LPIs enabled, for disabling LPIs we would need
+ * to queue a ITS host command, which we avoid so far during a guest's
+ * runtime. Also re-enabling would trigger a host command upon the
+ * guest sending a command, which could be an attack vector for
+ * hogging the host command queue.
+ * See the thread around here for some background:
+ * https://lists.xen.org/archives/html/xen-devel/2016-12/msg00003.html
+ */
+ vgic_vcpu_inject_irq(vcpu, hlpi.virt_lpi);
+
+ rcu_unlock_domain(d);
+
+out:
+ irq_exit();
+}
+
static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
{
uint64_t val;
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 29c8964..8140c5f 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1674,6 +1674,7 @@ static const struct gic_hw_operations gicv3_ops = {
.make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
.make_hwdom_madt = gicv3_make_hwdom_madt,
.iomem_deny_access = gicv3_iomem_deny_access,
+ .do_LPI = gicv3_do_LPI,
};
static int __init gicv3_dt_preinit(struct dt_device_node *node, const void *data)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 62ae3b8..d752352 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -735,7 +735,13 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
do_IRQ(regs, irq, is_fiq);
local_irq_disable();
}
- else if (unlikely(irq < 16))
+ else if ( is_lpi(irq) )
+ {
+ local_irq_enable();
+ gic_hw_ops->do_LPI(irq);
+ local_irq_disable();
+ }
+ else if ( unlikely(irq < 16) )
{
do_sgi(regs, irq);
}
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index 0587569..df91940 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -709,11 +709,18 @@ static struct pending_irq *vgic_v2_lpi_to_pending(struct domain *d,
BUG();
}
+static int vgic_v2_lpi_get_priority(struct domain *d, unsigned int vlpi)
+{
+ /* Dummy function, no LPIs on a VGICv2. */
+ BUG();
+}
+
static const struct vgic_ops vgic_v2_ops = {
.vcpu_init = vgic_v2_vcpu_init,
.domain_init = vgic_v2_domain_init,
.domain_free = vgic_v2_domain_free,
.lpi_to_pending = vgic_v2_lpi_to_pending,
+ .lpi_get_priority = vgic_v2_lpi_get_priority,
.max_vcpus = 8,
};
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index f462610..c059dbd 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1558,12 +1558,24 @@ static struct pending_irq *vgic_v3_lpi_to_pending(struct domain *d,
return pirq;
}
+/* Retrieve the priority of an LPI from its struct pending_irq. */
+static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
+{
+ struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
+
+ if ( !p )
+ return GIC_PRI_IRQ;
+
+ return p->lpi_priority;
+}
+
static const struct vgic_ops v3_ops = {
.vcpu_init = vgic_v3_vcpu_init,
.domain_init = vgic_v3_domain_init,
.domain_free = vgic_v3_domain_free,
.emulate_reg = vgic_v3_emulate_reg,
.lpi_to_pending = vgic_v3_lpi_to_pending,
+ .lpi_get_priority = vgic_v3_lpi_get_priority,
/*
* We use both AFF1 and AFF0 in (v)MPIDR. Thus, the max number of CPU
* that can be supported is up to 4096(==256*16) in theory.
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index c2bfdb1..b6fe34f 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -226,10 +226,15 @@ struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq)
static int vgic_get_virq_priority(struct vcpu *v, unsigned int virq)
{
- struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
+ struct vgic_irq_rank *rank;
unsigned long flags;
int priority;
+ /* LPIs don't have a rank, also store their priority separately. */
+ if ( is_lpi(virq) )
+ return v->domain->arch.vgic.handler->lpi_get_priority(v->domain, virq);
+
+ rank = vgic_rank_irq(v, virq);
vgic_lock_rank(v, rank, flags);
priority = rank->priority[virq & INTERRUPT_RANK_MASK];
vgic_unlock_rank(v, rank, flags);
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 3d8e84c..ebaea35 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -260,7 +260,8 @@ struct arch_vcpu
/* GICv3: redistributor base and flags for this vCPU */
paddr_t rdist_base;
-#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
+#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
+#define VGIC_V3_LPIS_ENABLED (1 << 1)
uint8_t flags;
} vgic;
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 836a103..42963c0 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -366,6 +366,8 @@ struct gic_hw_operations {
int (*map_hwdom_extra_mappings)(struct domain *d);
/* Deny access to GIC regions */
int (*iomem_deny_access)(const struct domain *d);
+ /* Handle LPIs, which require special handling */
+ void (*do_LPI)(unsigned int lpi);
};
void register_gic_ops(const struct gic_hw_operations *ops);
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index 29559a3..7470779 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -134,6 +134,8 @@ void gicv3_its_dt_init(const struct dt_device_node *node);
bool gicv3_its_host_has_its(void);
+void gicv3_do_LPI(unsigned int lpi);
+
int gicv3_lpi_init_rdist(void __iomem * rdist_base);
/* Initialize the host structures for LPIs and the host ITSes. */
@@ -175,6 +177,12 @@ static inline bool gicv3_its_host_has_its(void)
return false;
}
+static inline void gicv3_do_LPI(unsigned int lpi)
+{
+ /* We don't enable LPIs without an ITS. */
+ BUG();
+}
+
static inline int gicv3_lpi_init_rdist(void __iomem * rdist_base)
{
return -ENODEV;
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index c9075a9..7efa164 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -72,6 +72,7 @@ struct pending_irq
#define GIC_INVALID_LR (uint8_t)~0
uint8_t lr;
uint8_t priority;
+ uint8_t lpi_priority; /* Caches the priority if this is an LPI. */
/* inflight is used to append instances of pending_irq to
* vgic.inflight_irqs */
struct list_head inflight;
@@ -136,6 +137,7 @@ struct vgic_ops {
bool (*emulate_reg)(struct cpu_user_regs *regs, union hsr hsr);
/* lookup the struct pending_irq for a given LPI interrupt */
struct pending_irq *(*lpi_to_pending)(struct domain *d, unsigned int vlpi);
+ int (*lpi_get_priority)(struct domain *d, uint32_t vlpi);
/* Maximum number of vCPU supported */
const unsigned int max_vcpus;
};
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 06/27] ARM: GICv3: enable ITS and LPIs on the host
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (4 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables Andre Przywara
` (21 subsequent siblings)
27 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Now that the host part of the ITS code is in place, we can enable the
ITS and also LPIs on each redistributor to get the show rolling.
At this point there would be no LPIs mapped, as guests don't know about
the ITS yet.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
xen/arch/arm/gic-v3-its.c | 4 ++++
xen/arch/arm/gic-v3.c | 18 ++++++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 07280b3..aebc257 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -505,6 +505,10 @@ static int gicv3_its_init_single_its(struct host_its *hw_its)
return -ENOMEM;
writeq_relaxed(0, hw_its->its_base + GITS_CWRITER);
+ /* Now enable interrupt translation and command processing on that ITS. */
+ reg = readl_relaxed(hw_its->its_base + GITS_CTLR);
+ writel_relaxed(reg | GITS_CTLR_ENABLE, hw_its->its_base + GITS_CTLR);
+
return 0;
}
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 8140c5f..d539d6c 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -620,6 +620,21 @@ static int gicv3_enable_redist(void)
return 0;
}
+/* Enable LPIs on this redistributor (only useful when the host has an ITS). */
+static bool gicv3_enable_lpis(void)
+{
+ uint32_t val;
+
+ val = readl_relaxed(GICD_RDIST_BASE + GICR_TYPER);
+ if ( !(val & GICR_TYPER_PLPIS) )
+ return false;
+
+ val = readl_relaxed(GICD_RDIST_BASE + GICR_CTLR);
+ writel_relaxed(val | GICR_CTLR_ENABLE_LPIS, GICD_RDIST_BASE + GICR_CTLR);
+
+ return true;
+}
+
static int __init gicv3_populate_rdist(void)
{
int i;
@@ -731,11 +746,14 @@ static int gicv3_cpu_init(void)
if ( gicv3_enable_redist() )
return -ENODEV;
+ /* If the host has any ITSes, enable LPIs now. */
if ( gicv3_its_host_has_its() )
{
ret = gicv3_its_setup_collection(smp_processor_id());
if ( ret )
return ret;
+ if ( !gicv3_enable_lpis() )
+ return -EBUSY;
}
/* Set priority on PPI and SGI interrupts */
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (5 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 06/27] ARM: GICv3: enable ITS and LPIs on the host Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 10:55 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 08/27] ARM: introduce vgic_access_guest_memory() Andre Przywara
` (20 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Allow a guest to provide the address and size for the memory regions
it has reserved for the GICv3 pending and property tables.
We sanitise the various fields of the respective redistributor
registers.
The MMIO read and write accesses are protected by locks, to avoid any
changing of the property or pending table address while a redistributor
is live and also to protect the non-atomic vgic_reg64_extract() function
on the MMIO read side.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Julien Grall <julien.grall@arm.com>
---
xen/arch/arm/vgic-v3.c | 164 +++++++++++++++++++++++++++++++++++++++----
xen/include/asm-arm/domain.h | 5 ++
2 files changed, 157 insertions(+), 12 deletions(-)
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index c059dbd..e15c875 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -233,12 +233,29 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
goto read_reserved;
case VREG64(GICR_PROPBASER):
- /* LPI's not implemented */
- goto read_as_zero_64;
+ if ( !v->domain->arch.vgic.has_its )
+ goto read_as_zero_64;
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+
+ vgic_lock(v);
+ *r = vgic_reg64_extract(v->domain->arch.vgic.rdist_propbase, info);
+ vgic_unlock(v);
+ return 1;
case VREG64(GICR_PENDBASER):
- /* LPI's not implemented */
- goto read_as_zero_64;
+ {
+ unsigned long flags;
+
+ if ( !v->domain->arch.vgic.has_its )
+ goto read_as_zero_64;
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+
+ spin_lock_irqsave(&v->arch.vgic.lock, flags);
+ *r = vgic_reg64_extract(v->arch.vgic.rdist_pendbase, info);
+ *r &= ~GICR_PENDBASER_PTZ; /* WO, reads as 0 */
+ spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+ return 1;
+ }
case 0x0080:
goto read_reserved;
@@ -335,11 +352,95 @@ read_unknown:
return 1;
}
+static uint64_t vgic_sanitise_field(uint64_t reg, uint64_t field_mask,
+ int field_shift,
+ uint64_t (*sanitise_fn)(uint64_t))
+{
+ uint64_t field = (reg & field_mask) >> field_shift;
+
+ field = sanitise_fn(field) << field_shift;
+
+ return (reg & ~field_mask) | field;
+}
+
+/* We want to avoid outer shareable. */
+static uint64_t vgic_sanitise_shareability(uint64_t field)
+{
+ switch ( field )
+ {
+ case GIC_BASER_OuterShareable:
+ return GIC_BASER_InnerShareable;
+ default:
+ return field;
+ }
+}
+
+/* Avoid any inner non-cacheable mapping. */
+static uint64_t vgic_sanitise_inner_cacheability(uint64_t field)
+{
+ switch ( field )
+ {
+ case GIC_BASER_CACHE_nCnB:
+ case GIC_BASER_CACHE_nC:
+ return GIC_BASER_CACHE_RaWb;
+ default:
+ return field;
+ }
+}
+
+/* Non-cacheable or same-as-inner are OK. */
+static uint64_t vgic_sanitise_outer_cacheability(uint64_t field)
+{
+ switch ( field )
+ {
+ case GIC_BASER_CACHE_SameAsInner:
+ case GIC_BASER_CACHE_nC:
+ return field;
+ default:
+ return GIC_BASER_CACHE_nC;
+ }
+}
+
+static uint64_t sanitize_propbaser(uint64_t reg)
+{
+ reg = vgic_sanitise_field(reg, GICR_PROPBASER_SHAREABILITY_MASK,
+ GICR_PROPBASER_SHAREABILITY_SHIFT,
+ vgic_sanitise_shareability);
+ reg = vgic_sanitise_field(reg, GICR_PROPBASER_INNER_CACHEABILITY_MASK,
+ GICR_PROPBASER_INNER_CACHEABILITY_SHIFT,
+ vgic_sanitise_inner_cacheability);
+ reg = vgic_sanitise_field(reg, GICR_PROPBASER_OUTER_CACHEABILITY_MASK,
+ GICR_PROPBASER_OUTER_CACHEABILITY_SHIFT,
+ vgic_sanitise_outer_cacheability);
+
+ reg &= ~GICR_PROPBASER_RES0_MASK;
+
+ return reg;
+}
+
+static uint64_t sanitize_pendbaser(uint64_t reg)
+{
+ reg = vgic_sanitise_field(reg, GICR_PENDBASER_SHAREABILITY_MASK,
+ GICR_PENDBASER_SHAREABILITY_SHIFT,
+ vgic_sanitise_shareability);
+ reg = vgic_sanitise_field(reg, GICR_PENDBASER_INNER_CACHEABILITY_MASK,
+ GICR_PENDBASER_INNER_CACHEABILITY_SHIFT,
+ vgic_sanitise_inner_cacheability);
+ reg = vgic_sanitise_field(reg, GICR_PENDBASER_OUTER_CACHEABILITY_MASK,
+ GICR_PENDBASER_OUTER_CACHEABILITY_SHIFT,
+ vgic_sanitise_outer_cacheability);
+
+ reg &= ~GICR_PENDBASER_RES0_MASK;
+
+ return reg;
+}
+
static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
uint32_t gicr_reg,
register_t r)
{
struct hsr_dabt dabt = info->dabt;
+ uint64_t reg;
switch ( gicr_reg )
{
@@ -370,36 +471,75 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
goto write_impl_defined;
case VREG64(GICR_SETLPIR):
- /* LPI is not implemented */
+ /* LPIs without an ITS are not implemented */
goto write_ignore_64;
case VREG64(GICR_CLRLPIR):
- /* LPI is not implemented */
+ /* LPIs without an ITS are not implemented */
goto write_ignore_64;
case 0x0050:
goto write_reserved;
case VREG64(GICR_PROPBASER):
- /* LPI is not implemented */
- goto write_ignore_64;
+ if ( !v->domain->arch.vgic.has_its )
+ goto write_ignore_64;
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+
+ vgic_lock(v);
+
+ /*
+ * Writing PROPBASER with any redistributor having LPIs enabled
+ * is UNPREDICTABLE.
+ */
+ if ( !(v->domain->arch.vgic.rdists_enabled) )
+ {
+ reg = v->domain->arch.vgic.rdist_propbase;
+ vgic_reg64_update(®, r, info);
+ reg = sanitize_propbaser(reg);
+ v->domain->arch.vgic.rdist_propbase = reg;
+ }
+
+ vgic_unlock(v);
+
+ return 1;
case VREG64(GICR_PENDBASER):
- /* LPI is not implemented */
- goto write_ignore_64;
+ {
+ unsigned long flags;
+
+ if ( !v->domain->arch.vgic.has_its )
+ goto write_ignore_64;
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+
+ spin_lock_irqsave(&v->arch.vgic.lock, flags);
+
+ /* Writing PENDBASER with LPIs enabled is UNPREDICTABLE. */
+ if ( !(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED) )
+ {
+ reg = v->arch.vgic.rdist_pendbase;
+ vgic_reg64_update(®, r, info);
+ reg = sanitize_pendbaser(reg);
+ v->arch.vgic.rdist_pendbase = reg;
+ }
+
+ spin_unlock_irqrestore(&v->arch.vgic.lock, false);
+
+ return 1;
+ }
case 0x0080:
goto write_reserved;
case VREG64(GICR_INVLPIR):
- /* LPI is not implemented */
+ /* LPIs without an ITS are not implemented */
goto write_ignore_64;
case 0x00A8:
goto write_reserved;
case VREG64(GICR_INVALLR):
- /* LPI is not implemented */
+ /* LPIs without an ITS are not implemented */
goto write_ignore_64;
case 0x00B8:
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index ebaea35..b2d98bb 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -109,11 +109,15 @@ struct arch_domain
} *rdist_regions;
int nr_regions; /* Number of rdist regions */
uint32_t rdist_stride; /* Re-Distributor stride */
+ unsigned long int nr_lpis;
+ uint64_t rdist_propbase;
struct rb_root its_devices; /* Devices mapped to an ITS */
spinlock_t its_devices_lock; /* Protects the its_devices tree */
struct radix_tree_root pend_lpi_tree; /* Stores struct pending_irq's */
rwlock_t pend_lpi_tree_lock; /* Protects the pend_lpi_tree */
unsigned int intid_bits;
+ bool rdists_enabled; /* Is any redistributor enabled? */
+ bool has_its;
#endif
} vgic;
@@ -260,6 +264,7 @@ struct arch_vcpu
/* GICv3: redistributor base and flags for this vCPU */
paddr_t rdist_base;
+ uint64_t rdist_pendbase;
#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
#define VGIC_V3_LPIS_ENABLED (1 << 1)
uint8_t flags;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 08/27] ARM: introduce vgic_access_guest_memory()
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (6 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 12:29 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 09/27] ARM: vGICv3: re-use vgic_reg64_check_access Andre Przywara
` (19 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
This function allows to copy a chunk of data from and to guest physical
memory. It looks up the associated page from the guest's p2m tree
and maps this page temporarily for the time of the access.
This function was originally written by Vijaya as part of an earlier series:
https://patchwork.kernel.org/patch/8177251
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/vgic.h | 3 +++
2 files changed, 53 insertions(+)
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index b6fe34f..e385b43 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -20,6 +20,7 @@
#include <xen/bitops.h>
#include <xen/lib.h>
#include <xen/init.h>
+#include <xen/domain_page.h>
#include <xen/softirq.h>
#include <xen/irq.h>
#include <xen/sched.h>
@@ -602,6 +603,55 @@ void vgic_free_virq(struct domain *d, unsigned int virq)
}
/*
+ * Temporarily map one physical guest page and copy data to or from it.
+ * The data to be copied cannot cross a page boundary.
+ */
+int vgic_access_guest_memory(struct domain *d, paddr_t gpa, void *buf,
+ uint32_t size, bool_t is_write)
+{
+ struct page_info *page;
+ uint64_t offset;
+ p2m_type_t p2mt;
+ int ret = 0;
+ void *p;
+
+ page = get_page_from_gfn(d, paddr_to_pfn(gpa), &p2mt, P2M_ALLOC);
+ if ( !page )
+ {
+ printk(XENLOG_G_ERR "d%d: vITS: Failed to get table entry\n",
+ d->domain_id);
+ return -EINVAL;
+ }
+
+ if ( !p2m_is_ram(p2mt) )
+ {
+ put_page(page);
+ printk(XENLOG_G_ERR "d%d: vITS: memory used by the ITS should be RAM.",
+ d->domain_id);
+ return -EINVAL;
+ }
+
+ p = __map_domain_page(page);
+ /* Offset within the mapped page */
+ offset = gpa & ~PAGE_MASK;
+ /* Do not cross a page boundary. */
+ if ( size <= (PAGE_SIZE - offset) )
+ {
+ if ( is_write )
+ memcpy(p + offset, buf, size);
+ else
+ memcpy(buf, p + offset, size);
+ }
+ else
+ ret = -ENOSPC;
+
+ unmap_domain_page(p);
+ put_page(page);
+
+ return ret;
+}
+
+/*
* Local variables:
* mode: C
* c-file-style: "BSD"
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 7efa164..6b17802 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -313,6 +313,9 @@ extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
int vgic_v2_init(struct domain *d, int *mmio_count);
int vgic_v3_init(struct domain *d, int *mmio_count);
+int vgic_access_guest_memory(struct domain *d, paddr_t gpa, void *buf,
+ uint32_t size, bool_t is_write);
+
extern int domain_vgic_register(struct domain *d, int *mmio_count);
extern int vcpu_vgic_free(struct vcpu *v);
extern bool vgic_to_sgi(struct vcpu *v, register_t sgir,
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 09/27] ARM: vGICv3: re-use vgic_reg64_check_access
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (7 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 08/27] ARM: introduce vgic_access_guest_memory() Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 10/27] ARM: GIC: export vgic_init_pending_irq() Andre Przywara
` (18 subsequent siblings)
27 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
vgic_reg64_check_access() checks for a valid access width of a 64-bit
MMIO register, which is useful beyond the current GICv3 emulation only.
Move this function to the vgic-emul.h to be easily reusable.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Julien Grall <julien.grall@arm.com>
---
xen/arch/arm/vgic-v3.c | 9 ---------
xen/include/asm-arm/vgic-emul.h | 9 +++++++++
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index e15c875..849a6f3 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -161,15 +161,6 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
}
}
-static inline bool vgic_reg64_check_access(struct hsr_dabt dabt)
-{
- /*
- * 64 bits registers can be accessible using 32-bit and 64-bit unless
- * stated otherwise (See 8.1.3 ARM IHI 0069A).
- */
- return ( dabt.size == DABT_DOUBLE_WORD || dabt.size == DABT_WORD );
-}
-
static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
uint32_t gicr_reg,
register_t *r)
diff --git a/xen/include/asm-arm/vgic-emul.h b/xen/include/asm-arm/vgic-emul.h
index 184a1f0..e52fbaa 100644
--- a/xen/include/asm-arm/vgic-emul.h
+++ b/xen/include/asm-arm/vgic-emul.h
@@ -12,6 +12,15 @@
#define VRANGE32(start, end) start ... end + 3
#define VRANGE64(start, end) start ... end + 7
+/*
+ * 64 bits registers can be accessible using 32-bit and 64-bit unless
+ * stated otherwise (See 8.1.3 ARM IHI 0069A).
+ */
+static inline bool vgic_reg64_check_access(struct hsr_dabt dabt)
+{
+ return ( dabt.size == DABT_DOUBLE_WORD || dabt.size == DABT_WORD );
+}
+
#endif /* __ASM_ARM_VGIC_EMUL_H__ */
/*
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 10/27] ARM: GIC: export vgic_init_pending_irq()
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (8 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 09/27] ARM: vGICv3: re-use vgic_reg64_check_access Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 11/27] ARM: VGIC: add vcpu_id to struct pending_irq Andre Przywara
` (17 subsequent siblings)
27 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
For LPIs we later want to dynamically allocate struct pending_irqs.
Let's export the vgic_init_pending_irq() to be able to reuse it.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Julien Grall <julien.grall@arm.com>
---
xen/arch/arm/vgic.c | 2 +-
xen/include/asm-arm/vgic.h | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index e385b43..c8fe89c 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -61,7 +61,7 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
return vgic_get_rank(v, rank);
}
-static void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
+void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
{
INIT_LIST_HEAD(&p->inflight);
INIT_LIST_HEAD(&p->lr_queue);
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 6b17802..e2111a5 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -302,6 +302,7 @@ extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq);
extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
extern void vgic_clear_pending_irqs(struct vcpu *v);
+extern void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq);
extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
extern struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq);
extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, int b, int n, int s);
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 11/27] ARM: VGIC: add vcpu_id to struct pending_irq
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (9 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 10/27] ARM: GIC: export vgic_init_pending_irq() Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 12:32 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 12/27] ARM: vGIC: advertise LPI support Andre Przywara
` (16 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The target CPU for an LPI is encoded in the interrupt translation table
entry, so can't be easily derived from just an LPI number (short of
walking *all* tables and find the matching LPI).
To avoid this in case we need to know the VCPU (for the INVALL command,
for instance), put the VCPU ID in the struct pending_irq, so that it is
easily accessible.
We use the remaining 8 bits of padding space for that to avoid enlarging
the size of struct pending_irq. The number of VCPUs is limited to 127
at the moment anyway.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/include/asm-arm/vgic.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index e2111a5..02732db 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -73,6 +73,7 @@ struct pending_irq
uint8_t lr;
uint8_t priority;
uint8_t lpi_priority; /* Caches the priority if this is an LPI. */
+ uint8_t lpi_vcpu_id; /* The VCPU for an LPI. */
/* inflight is used to append instances of pending_irq to
* vgic.inflight_irqs */
struct list_head inflight;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 12/27] ARM: vGIC: advertise LPI support
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (10 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 11/27] ARM: VGIC: add vcpu_id to struct pending_irq Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 12:38 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 13/27] ARM: vITS: add command handling stub and MMIO emulation Andre Przywara
` (15 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
To let a guest know about the availability of virtual LPIs, set the
respective bits in the virtual GIC registers and let a guest control
the LPI enable bit.
Only report the LPI capability if the host has initialized at least
one ITS.
This removes a "TBD" comment, as we now populate the processor number
in the GICR_TYPE register.
Advertise 24 bits worth of LPIs to the guest.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 62 insertions(+), 5 deletions(-)
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 849a6f3..33cd5f7 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -170,8 +170,19 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case VREG32(GICR_CTLR):
- /* We have not implemented LPI's, read zero */
- goto read_as_zero_32;
+ {
+ unsigned long flags;
+
+ if ( !v->domain->arch.vgic.has_its )
+ goto read_as_zero_32;
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+
+ spin_lock_irqsave(&v->arch.vgic.lock, flags);
+ *r = vgic_reg32_extract(!!(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED),
+ info);
+ spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+ return 1;
+ }
case VREG32(GICR_IIDR):
if ( dabt.size != DABT_WORD ) goto bad_width;
@@ -183,16 +194,20 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
uint64_t typer, aff;
if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
- /* TBD: Update processor id in [23:8] when ITS support is added */
aff = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 56 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
typer = aff;
+ /* We use the VCPU ID as the redistributor ID in bits[23:8] */
+ typer |= (v->vcpu_id & 0xffff) << 8;
if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
typer |= GICR_TYPER_LAST;
+ if ( v->domain->arch.vgic.has_its )
+ typer |= GICR_TYPER_PLPIS;
+
*r = vgic_reg64_extract(typer, info);
return 1;
@@ -426,6 +441,25 @@ static uint64_t sanitize_pendbaser(uint64_t reg)
return reg;
}
+static void vgic_vcpu_enable_lpis(struct vcpu *v)
+{
+ uint64_t reg = v->domain->arch.vgic.rdist_propbase;
+ unsigned int nr_lpis = BIT((reg & 0x1f) + 1);
+
+ if ( nr_lpis < LPI_OFFSET )
+ nr_lpis = 0;
+ else
+ nr_lpis -= LPI_OFFSET;
+
+ if ( !v->domain->arch.vgic.rdists_enabled )
+ {
+ v->domain->arch.vgic.nr_lpis = nr_lpis;
+ v->domain->arch.vgic.rdists_enabled = true;
+ }
+
+ v->arch.vgic.flags |= VGIC_V3_LPIS_ENABLED;
+}
+
static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
uint32_t gicr_reg,
register_t r)
@@ -436,8 +470,26 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case VREG32(GICR_CTLR):
- /* LPI's not implemented */
- goto write_ignore_32;
+ {
+ unsigned long flags;
+
+ if ( !v->domain->arch.vgic.has_its )
+ goto write_ignore_32;
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+
+ vgic_lock(v); /* protects rdists_enabled */
+ spin_lock_irqsave(&v->arch.vgic.lock, flags);
+
+ /* LPIs can only be enabled once, but never disabled again. */
+ if ( (r & GICR_CTLR_ENABLE_LPIS) &&
+ !(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED) )
+ vgic_vcpu_enable_lpis(v);
+
+ spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+ vgic_unlock(v);
+
+ return 1;
+ }
case VREG32(GICR_IIDR):
/* RO */
@@ -1058,6 +1110,11 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
typer = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
+ if ( v->domain->arch.vgic.has_its )
+ {
+ typer |= GICD_TYPE_LPIS;
+ irq_bits = v->domain->arch.vgic.intid_bits;
+ }
typer |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
*r = vgic_reg32_extract(typer, info);
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 13/27] ARM: vITS: add command handling stub and MMIO emulation
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (11 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 12/27] ARM: vGIC: advertise LPI support Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 12:59 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 14/27] ARM: vITS: introduce translation table walks Andre Przywara
` (14 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Emulate the memory mapped ITS registers and provide a stub to introduce
the ITS command handling framework (but without actually emulating any
commands at this time).
This fixes a misnomer in our virtual ITS structure, where the spec is
confusingly using ID_bits in GITS_TYPER to denote the number of event IDs
(in contrast to GICD_TYPER, where it means number of LPIs).
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 523 ++++++++++++++++++++++++++++++++++++++-
xen/include/asm-arm/gic_v3_its.h | 3 +
2 files changed, 525 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 065ffe2..a60f9b2 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -19,6 +19,16 @@
* along with this program; If not, see <http://www.gnu.org/licenses/>.
*/
+/*
+ * Locking order:
+ *
+ * its->vcmd_lock (protects the command queue)
+ * its->its_lock (protects the translation tables)
+ * d->its_devices_lock (protects the device RB tree)
+ * v->vgic.lock (protects the struct pending_irq)
+ * d->pend_lpi_tree_lock (protects the radix tree)
+ */
+
#include <xen/bitops.h>
#include <xen/config.h>
#include <xen/domain_page.h>
@@ -43,7 +53,7 @@
struct virt_its {
struct domain *d;
unsigned int devid_bits;
- unsigned int intid_bits;
+ unsigned int evid_bits;
spinlock_t vcmd_lock; /* Protects the virtual command buffer, which */
uint64_t cwriter; /* consists of CWRITER and CREADR and those */
uint64_t creadr; /* shadow variables cwriter and creadr. */
@@ -53,6 +63,7 @@ struct virt_its {
uint64_t baser_dev, baser_coll; /* BASER0 and BASER1 for the guest */
unsigned int max_collections;
unsigned int max_devices;
+ /* changing "enabled" requires to hold *both* the vcmd_lock and its_lock */
bool enabled;
};
@@ -67,6 +78,9 @@ struct vits_itte
uint16_t pad;
};
+#define GITS_BASER_RO_MASK (GITS_BASER_TYPE_MASK | \
+ (31UL << GITS_BASER_ENTRY_SIZE_SHIFT))
+
int vgic_v3_its_init_domain(struct domain *d)
{
spin_lock_init(&d->arch.vgic.its_devices_lock);
@@ -80,6 +94,513 @@ void vgic_v3_its_free_domain(struct domain *d)
ASSERT(RB_EMPTY_ROOT(&d->arch.vgic.its_devices));
}
+/**************************************
+ * Functions that handle ITS commands *
+ **************************************/
+
+static uint64_t its_cmd_mask_field(uint64_t *its_cmd, unsigned int word,
+ unsigned int shift, unsigned int size)
+{
+ return (le64_to_cpu(its_cmd[word]) >> shift) & (BIT(size) - 1);
+}
+
+#define its_cmd_get_command(cmd) its_cmd_mask_field(cmd, 0, 0, 8)
+#define its_cmd_get_deviceid(cmd) its_cmd_mask_field(cmd, 0, 32, 32)
+#define its_cmd_get_size(cmd) its_cmd_mask_field(cmd, 1, 0, 5)
+#define its_cmd_get_id(cmd) its_cmd_mask_field(cmd, 1, 0, 32)
+#define its_cmd_get_physical_id(cmd) its_cmd_mask_field(cmd, 1, 32, 32)
+#define its_cmd_get_collection(cmd) its_cmd_mask_field(cmd, 2, 0, 16)
+#define its_cmd_get_target_addr(cmd) its_cmd_mask_field(cmd, 2, 16, 32)
+#define its_cmd_get_validbit(cmd) its_cmd_mask_field(cmd, 2, 63, 1)
+#define its_cmd_get_ittaddr(cmd) (its_cmd_mask_field(cmd, 2, 8, 44) << 8)
+
+#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
+#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
+
+/*
+ * Must be called with the vcmd_lock held.
+ * TODO: Investigate whether we can be smarter here and don't need to hold
+ * the lock all of the time.
+ */
+static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
+{
+ paddr_t addr = its->cbaser & GENMASK(51, 12);
+ uint64_t command[4];
+
+ ASSERT(spin_is_locked(&its->vcmd_lock));
+
+ if ( its->cwriter >= ITS_CMD_BUFFER_SIZE(its->cbaser) )
+ return -1;
+
+ while ( its->creadr != its->cwriter )
+ {
+ int ret;
+
+ ret = vgic_access_guest_memory(d, addr + its->creadr,
+ command, sizeof(command), false);
+ if ( ret )
+ return ret;
+
+ switch ( its_cmd_get_command(command) )
+ {
+ case GITS_CMD_SYNC:
+ /* We handle ITS commands synchronously, so we ignore SYNC. */
+ break;
+ default:
+ gdprintk(XENLOG_WARNING, "ITS: unhandled ITS command %lu\n",
+ its_cmd_get_command(command));
+ break;
+ }
+
+ write_u64_atomic(&its->creadr, (its->creadr + ITS_CMD_SIZE) %
+ ITS_CMD_BUFFER_SIZE(its->cbaser));
+
+ if ( ret )
+ gdprintk(XENLOG_WARNING,
+ "ITS: ITS command error %d while handling command %lu\n",
+ ret, its_cmd_get_command(command));
+ }
+
+ return 0;
+}
+
+/*****************************
+ * ITS registers read access *
+ *****************************/
+
+/* Identifying as an ARM IP, using "X" as the product ID. */
+#define GITS_IIDR_VALUE 0x5800034c
+
+static int vgic_v3_its_mmio_read(struct vcpu *v, mmio_info_t *info,
+ register_t *r, void *priv)
+{
+ struct virt_its *its = priv;
+ uint64_t reg;
+
+ switch ( info->gpa & 0xffff )
+ {
+ case VREG32(GITS_CTLR):
+ {
+ /*
+ * We try to avoid waiting for the command queue lock and report
+ * non-quiescent if that lock is already taken.
+ */
+ bool have_cmd_lock;
+
+ if ( info->dabt.size != DABT_WORD ) goto bad_width;
+
+ have_cmd_lock = spin_trylock(&its->vcmd_lock);
+ spin_lock(&its->its_lock);
+ if ( its->enabled )
+ reg = GITS_CTLR_ENABLE;
+ else
+ reg = 0;
+
+ if ( have_cmd_lock && its->cwriter == its->creadr )
+ reg |= GITS_CTLR_QUIESCENT;
+
+ spin_unlock(&its->its_lock);
+ if ( have_cmd_lock )
+ spin_unlock(&its->vcmd_lock);
+
+ *r = vgic_reg32_extract(reg, info);
+ break;
+ }
+ case VREG32(GITS_IIDR):
+ if ( info->dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_extract(GITS_IIDR_VALUE, info);
+ break;
+ case VREG64(GITS_TYPER):
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+
+ reg = GITS_TYPER_PHYSICAL;
+ reg |= (sizeof(struct vits_itte) - 1) << GITS_TYPER_ITT_SIZE_SHIFT;
+ reg |= (its->evid_bits - 1) << GITS_TYPER_IDBITS_SHIFT;
+ reg |= (its->devid_bits - 1) << GITS_TYPER_DEVIDS_SHIFT;
+ *r = vgic_reg64_extract(reg, info);
+ break;
+ case VRANGE32(0x0018, 0x001C):
+ goto read_reserved;
+ case VRANGE32(0x0020, 0x003C):
+ goto read_impl_defined;
+ case VRANGE32(0x0040, 0x007C):
+ goto read_reserved;
+ case VREG64(GITS_CBASER):
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+ spin_lock(&its->its_lock);
+ *r = vgic_reg64_extract(its->cbaser, info);
+ spin_unlock(&its->its_lock);
+ break;
+ case VREG64(GITS_CWRITER):
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+
+ reg = its->cwriter;
+ *r = vgic_reg64_extract(reg, info);
+ break;
+ case VREG64(GITS_CREADR):
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+
+ reg = its->creadr;
+ *r = vgic_reg64_extract(reg, info);
+ break;
+ case VRANGE64(0x0098, 0x00F8):
+ goto read_reserved;
+ case VREG64(GITS_BASER0): /* device table */
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+ spin_lock(&its->its_lock);
+ *r = vgic_reg64_extract(its->baser_dev, info);
+ spin_unlock(&its->its_lock);
+ break;
+ case VREG64(GITS_BASER1): /* collection table */
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+ spin_lock(&its->its_lock);
+ *r = vgic_reg64_extract(its->baser_coll, info);
+ spin_unlock(&its->its_lock);
+ break;
+ case VRANGE64(GITS_BASER2, GITS_BASER7):
+ goto read_as_zero_64;
+ case VRANGE32(0x0140, 0xBFFC):
+ goto read_reserved;
+ case VRANGE32(0xC000, 0xFFCC):
+ goto read_impl_defined;
+ case VRANGE32(0xFFD0, 0xFFE4):
+ goto read_impl_defined;
+ case VREG32(GITS_PIDR2):
+ if ( info->dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_extract(GIC_PIDR2_ARCH_GICv3, info);
+ break;
+ case VRANGE32(0xFFEC, 0xFFFC):
+ goto read_impl_defined;
+ default:
+ printk(XENLOG_G_ERR
+ "%pv: vGITS: unhandled read r%d offset %#04lx\n",
+ v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+ return 0;
+ }
+
+ return 1;
+
+read_as_zero_64:
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+ *r = 0;
+
+ return 1;
+
+read_impl_defined:
+ printk(XENLOG_G_DEBUG
+ "%pv: vGITS: RAZ on implementation defined register offset %#04lx\n",
+ v, info->gpa & 0xffff);
+ *r = 0;
+ return 1;
+
+read_reserved:
+ printk(XENLOG_G_DEBUG
+ "%pv: vGITS: RAZ on reserved register offset %#04lx\n",
+ v, info->gpa & 0xffff);
+ *r = 0;
+ return 1;
+
+bad_width:
+ printk(XENLOG_G_ERR "vGIIS: bad read width %d r%d offset %#04lx\n",
+ info->dabt.size, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+ domain_crash_synchronous();
+
+ return 0;
+}
+
+/******************************
+ * ITS registers write access *
+ ******************************/
+
+static unsigned int its_baser_table_size(uint64_t baser)
+{
+ unsigned int ret, page_size[4] = {SZ_4K, SZ_16K, SZ_64K, SZ_64K};
+
+ ret = page_size[(baser >> GITS_BASER_PAGE_SIZE_SHIFT) & 3];
+
+ return ret * ((baser & GITS_BASER_SIZE_MASK) + 1);
+}
+
+static unsigned int its_baser_nr_entries(uint64_t baser)
+{
+ unsigned int entry_size = GITS_BASER_ENTRY_SIZE(baser);
+
+ return its_baser_table_size(baser) / entry_size;
+}
+
+/* Must be called with the ITS lock held. */
+static bool vgic_v3_verify_its_status(struct virt_its *its, bool status)
+{
+ ASSERT(spin_is_locked(&its->its_lock));
+
+ if ( !status )
+ return false;
+
+ if ( !(its->cbaser & GITS_VALID_BIT) ||
+ !(its->baser_dev & GITS_VALID_BIT) ||
+ !(its->baser_coll & GITS_VALID_BIT) )
+ {
+ printk(XENLOG_G_WARNING "d%d tried to enable ITS without having the tables configured.\n",
+ its->d->domain_id);
+ return false;
+ }
+
+ return true;
+}
+
+static void sanitize_its_base_reg(uint64_t *reg)
+{
+ uint64_t r = *reg;
+
+ /* Avoid outer shareable. */
+ switch ( (r >> GITS_BASER_SHAREABILITY_SHIFT) & 0x03 )
+ {
+ case GIC_BASER_OuterShareable:
+ r &= ~GITS_BASER_SHAREABILITY_MASK;
+ r |= GIC_BASER_InnerShareable << GITS_BASER_SHAREABILITY_SHIFT;
+ break;
+ default:
+ break;
+ }
+
+ /* Avoid any inner non-cacheable mapping. */
+ switch ( (r >> GITS_BASER_INNER_CACHEABILITY_SHIFT) & 0x07 )
+ {
+ case GIC_BASER_CACHE_nCnB:
+ case GIC_BASER_CACHE_nC:
+ r &= ~GITS_BASER_INNER_CACHEABILITY_MASK;
+ r |= GIC_BASER_CACHE_RaWb << GITS_BASER_INNER_CACHEABILITY_SHIFT;
+ break;
+ default:
+ break;
+ }
+
+ /* Only allow non-cacheable or same-as-inner. */
+ switch ( (r >> GITS_BASER_OUTER_CACHEABILITY_SHIFT) & 0x07 )
+ {
+ case GIC_BASER_CACHE_SameAsInner:
+ case GIC_BASER_CACHE_nC:
+ break;
+ default:
+ r &= ~GITS_BASER_OUTER_CACHEABILITY_MASK;
+ r |= GIC_BASER_CACHE_nC << GITS_BASER_OUTER_CACHEABILITY_SHIFT;
+ break;
+ }
+
+ *reg = r;
+}
+
+static int vgic_v3_its_mmio_write(struct vcpu *v, mmio_info_t *info,
+ register_t r, void *priv)
+{
+ struct domain *d = v->domain;
+ struct virt_its *its = priv;
+ uint64_t reg;
+ uint32_t reg32;
+
+ switch ( info->gpa & 0xffff )
+ {
+ case VREG32(GITS_CTLR):
+ {
+ uint32_t ctlr;
+
+ if ( info->dabt.size != DABT_WORD ) goto bad_width;
+
+ /*
+ * We need to take the vcmd_lock to prevent a guest from disabling
+ * the ITS while commands are still processed.
+ */
+ spin_lock(&its->vcmd_lock);
+ spin_lock(&its->its_lock);
+ ctlr = its->enabled ? GITS_CTLR_ENABLE : 0;
+ reg32 = ctlr;
+ vgic_reg32_update(®32, r, info);
+
+ if ( ctlr ^ reg32 )
+ its->enabled = vgic_v3_verify_its_status(its,
+ reg32 & GITS_CTLR_ENABLE);
+ spin_unlock(&its->its_lock);
+ spin_unlock(&its->vcmd_lock);
+ return 1;
+ }
+
+ case VREG32(GITS_IIDR):
+ goto write_ignore_32;
+ case VREG32(GITS_TYPER):
+ goto write_ignore_32;
+ case VRANGE32(0x0018, 0x001C):
+ goto write_reserved;
+ case VRANGE32(0x0020, 0x003C):
+ goto write_impl_defined;
+ case VRANGE32(0x0040, 0x007C):
+ goto write_reserved;
+ case VREG64(GITS_CBASER):
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+
+ spin_lock(&its->its_lock);
+ /* Changing base registers with the ITS enabled is UNPREDICTABLE. */
+ if ( its->enabled )
+ {
+ spin_unlock(&its->its_lock);
+ gdprintk(XENLOG_WARNING,
+ "ITS: tried to change CBASER with the ITS enabled.\n");
+ return 1;
+ }
+
+ reg = its->cbaser;
+ vgic_reg64_update(®, r, info);
+ sanitize_its_base_reg(®);
+
+ its->cbaser = reg;
+ its->creadr = 0;
+ spin_unlock(&its->its_lock);
+
+ return 1;
+
+ case VREG64(GITS_CWRITER):
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+
+ spin_lock(&its->vcmd_lock);
+ reg = ITS_CMD_OFFSET(its->cwriter);
+ vgic_reg64_update(®, r, info);
+ its->cwriter = ITS_CMD_OFFSET(reg);
+
+ if ( its->enabled )
+ if ( vgic_its_handle_cmds(d, its) )
+ gdprintk(XENLOG_WARNING, "error handling ITS commands\n");
+
+ spin_unlock(&its->vcmd_lock);
+
+ return 1;
+
+ case VREG64(GITS_CREADR):
+ goto write_ignore_64;
+
+ case VRANGE32(0x0098, 0x00FC):
+ goto write_reserved;
+ case VREG64(GITS_BASER0): /* device table */
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+
+ spin_lock(&its->its_lock);
+
+ /*
+ * Changing base registers with the ITS enabled is UNPREDICTABLE,
+ * we choose to ignore it, but warn.
+ */
+ if ( its->enabled )
+ {
+ spin_unlock(&its->its_lock);
+ gdprintk(XENLOG_WARNING, "ITS: tried to change BASER with the ITS enabled.\n");
+
+ return 1;
+ }
+
+ reg = its->baser_dev;
+ vgic_reg64_update(®, r, info);
+
+ /* We don't support indirect tables for now. */
+ reg &= ~(GITS_BASER_RO_MASK | GITS_BASER_INDIRECT);
+ reg |= (sizeof(uint64_t) - 1) << GITS_BASER_ENTRY_SIZE_SHIFT;
+ reg |= GITS_BASER_TYPE_DEVICE << GITS_BASER_TYPE_SHIFT;
+ sanitize_its_base_reg(®);
+
+ if ( reg & GITS_VALID_BIT )
+ {
+ its->max_devices = its_baser_nr_entries(reg);
+ if ( its->max_devices > BIT(its->devid_bits) )
+ its->max_devices = BIT(its->devid_bits);
+ }
+ else
+ its->max_devices = 0;
+
+ its->baser_dev = reg;
+ spin_unlock(&its->its_lock);
+ return 1;
+ case VREG64(GITS_BASER1): /* collection table */
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+
+ spin_lock(&its->its_lock);
+ /*
+ * Changing base registers with the ITS enabled is UNPREDICTABLE,
+ * we choose to ignore it, but warn.
+ */
+ if ( its->enabled )
+ {
+ spin_unlock(&its->its_lock);
+ gdprintk(XENLOG_INFO, "ITS: tried to change BASER with the ITS enabled.\n");
+ return 1;
+ }
+
+ reg = its->baser_coll;
+ vgic_reg64_update(®, r, info);
+ /* No indirect tables for the collection table. */
+ reg &= ~(GITS_BASER_RO_MASK | GITS_BASER_INDIRECT);
+ reg |= (sizeof(uint16_t) - 1) << GITS_BASER_ENTRY_SIZE_SHIFT;
+ reg |= GITS_BASER_TYPE_COLLECTION << GITS_BASER_TYPE_SHIFT;
+ sanitize_its_base_reg(®);
+
+ if ( reg & GITS_VALID_BIT )
+ its->max_collections = its_baser_nr_entries(reg);
+ else
+ its->max_collections = 0;
+ its->baser_coll = reg;
+ spin_unlock(&its->its_lock);
+ return 1;
+ case VRANGE64(GITS_BASER2, GITS_BASER7):
+ goto write_ignore_64;
+ case VRANGE32(0x0140, 0xBFFC):
+ goto write_reserved;
+ case VRANGE32(0xC000, 0xFFCC):
+ goto write_impl_defined;
+ case VRANGE32(0xFFD0, 0xFFE4): /* IMPDEF identification registers */
+ goto write_impl_defined;
+ case VREG32(GITS_PIDR2):
+ goto write_ignore_32;
+ case VRANGE32(0xFFEC, 0xFFFC): /* IMPDEF identification registers */
+ goto write_impl_defined;
+ default:
+ printk(XENLOG_G_ERR
+ "%pv: vGITS: unhandled write r%d offset %#04lx\n",
+ v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+ return 0;
+ }
+
+ return 1;
+
+write_ignore_64:
+ if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
+ return 1;
+
+write_ignore_32:
+ if ( info->dabt.size != DABT_WORD ) goto bad_width;
+ return 1;
+
+write_impl_defined:
+ printk(XENLOG_G_DEBUG
+ "%pv: vGITS: WI on implementation defined register offset %#04lx\n",
+ v, info->gpa & 0xffff);
+ return 1;
+
+write_reserved:
+ printk(XENLOG_G_DEBUG
+ "%pv: vGITS: WI on implementation defined register offset %#04lx\n",
+ v, info->gpa & 0xffff);
+ return 1;
+
+bad_width:
+ printk(XENLOG_G_ERR "vGITS: bad write width %d r%d offset %#08lx\n",
+ info->dabt.size, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+
+ domain_crash_synchronous();
+
+ return 0;
+}
+
+static const struct mmio_handler_ops vgic_its_mmio_handler = {
+ .read = vgic_v3_its_mmio_read,
+ .write = vgic_v3_its_mmio_write,
+};
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index 7470779..40f4ef5 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -35,6 +35,7 @@
#define GITS_BASER5 0x128
#define GITS_BASER6 0x130
#define GITS_BASER7 0x138
+#define GITS_PIDR2 GICR_PIDR2
/* Register bits */
#define GITS_VALID_BIT BIT(63)
@@ -57,6 +58,7 @@
#define GITS_TYPER_ITT_SIZE_MASK (0xfUL << GITS_TYPER_ITT_SIZE_SHIFT)
#define GITS_TYPER_ITT_SIZE(r) ((((r) & GITS_TYPER_ITT_SIZE_MASK) >> \
GITS_TYPER_ITT_SIZE_SHIFT) + 1)
+#define GITS_TYPER_PHYSICAL (1U << 0)
#define GITS_BASER_INDIRECT BIT(62)
#define GITS_BASER_INNER_CACHEABILITY_SHIFT 59
@@ -76,6 +78,7 @@
(((reg >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
#define GITS_BASER_SHAREABILITY_SHIFT 10
#define GITS_BASER_PAGE_SIZE_SHIFT 8
+#define GITS_BASER_SIZE_MASK 0xff
#define GITS_BASER_SHAREABILITY_MASK (0x3ULL << GITS_BASER_SHAREABILITY_SHIFT)
#define GITS_BASER_OUTER_CACHEABILITY_MASK (0x7ULL << GITS_BASER_OUTER_CACHEABILITY_SHIFT)
#define GITS_BASER_INNER_CACHEABILITY_MASK (0x7ULL << GITS_BASER_INNER_CACHEABILITY_SHIFT)
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 14/27] ARM: vITS: introduce translation table walks
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (12 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 13/27] ARM: vITS: add command handling stub and MMIO emulation Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 13:22 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 15/27] ARM: vITS: handle CLEAR command Andre Przywara
` (13 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The ITS stores the target (v)CPU and the (virtual) LPI number in tables.
Introduce functions to walk those tables and translate an device ID -
event ID pair into a pair of virtual LPI and vCPU.
We map those tables on demand - which is cheap on arm64 - and copy the
respective entries before using them, to avoid the guest tampering with
them meanwhile.
To allow compiling without warnings, we declare two functions as
non-static for the moment, which two later patches will fix.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 183 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 183 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index a60f9b2..632ab84 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -78,6 +78,7 @@ struct vits_itte
uint16_t pad;
};
+#define UNMAPPED_COLLECTION ((uint16_t)~0)
#define GITS_BASER_RO_MASK (GITS_BASER_TYPE_MASK | \
(31UL << GITS_BASER_ENTRY_SIZE_SHIFT))
@@ -94,6 +95,188 @@ void vgic_v3_its_free_domain(struct domain *d)
ASSERT(RB_EMPTY_ROOT(&d->arch.vgic.its_devices));
}
+/*
+ * The physical address is encoded slightly differently depending on
+ * the used page size: the highest four bits are stored in the lowest
+ * four bits of the field for 64K pages.
+ */
+static paddr_t get_baser_phys_addr(uint64_t reg)
+{
+ if ( reg & BIT(9) )
+ return (reg & GENMASK(47, 16)) |
+ ((reg & GENMASK(15, 12)) << 36);
+ else
+ return reg & GENMASK(47, 12);
+}
+
+/*
+ * Our collection table encoding:
+ * Just contains the 16-bit VCPU ID of the respective vCPU.
+ */
+#define COLL_TABLE_ENTRY_SIZE 2UL
+
+/* Must be called with the ITS lock held. */
+static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
+ uint16_t collid)
+{
+ paddr_t addr = get_baser_phys_addr(its->baser_coll);
+ uint16_t vcpu_id;
+ int ret;
+
+ ASSERT(spin_is_locked(&its->its_lock));
+
+ if ( collid >= its->max_collections )
+ return NULL;
+
+ ret = vgic_access_guest_memory(its->d, addr + collid * sizeof(uint16_t),
+ &vcpu_id, sizeof(vcpu_id), false);
+ if ( ret )
+ return NULL;
+
+ if ( vcpu_id == UNMAPPED_COLLECTION || vcpu_id >= its->d->max_vcpus )
+ return NULL;
+
+ return its->d->vcpu[vcpu_id];
+}
+
+/*
+ * Our device table encodings:
+ * Contains the guest physical address of the Interrupt Translation Table in
+ * bits [51:8], and the size of it is encoded as the number of bits minus one
+ * in the lowest 5 bits of the word.
+ */
+#define DEV_TABLE_ENTRY_SIZE 8UL
+#define DEV_TABLE_ITT_ADDR(x) ((x) & GENMASK(51, 8))
+#define DEV_TABLE_ITT_SIZE(x) (BIT(((x) & GENMASK(4, 0)) + 1))
+#define DEV_TABLE_ENTRY(addr, bits) \
+ (((addr) & GENMASK(51, 8)) | (((bits) - 1) & GENMASK(4, 0)))
+
+/*
+ * Lookup the address of the Interrupt Translation Table associated with
+ * that device ID.
+ * TODO: add support for walking indirect tables.
+ */
+static int its_get_itt(struct virt_its *its, uint32_t devid,
+ uint64_t *itt)
+{
+ paddr_t addr = get_baser_phys_addr(its->baser_dev);
+
+ if ( devid >= its->max_devices )
+ return -EINVAL;
+
+ return vgic_access_guest_memory(its->d, addr + devid * sizeof(uint64_t),
+ itt, sizeof(*itt), false);
+}
+
+/*
+ * Lookup the address of the Interrupt Translation Table associated with
+ * a device ID and return the address of the ITTE belonging to the event ID
+ * (which is an index into that table).
+ */
+static paddr_t its_get_itte_address(struct virt_its *its,
+ uint32_t devid, uint32_t evid)
+{
+ uint64_t itt;
+ int ret;
+
+ ret = its_get_itt(its, devid, &itt);
+ if ( ret )
+ return INVALID_PADDR;
+
+ if ( evid >= DEV_TABLE_ITT_SIZE(itt) ||
+ DEV_TABLE_ITT_ADDR(itt) == INVALID_PADDR )
+ return INVALID_PADDR;
+
+ return DEV_TABLE_ITT_ADDR(itt) + evid * sizeof(struct vits_itte);
+}
+
+/*
+ * Queries the collection and device tables to get the vCPU and virtual
+ * LPI number for a given guest event. This first accesses the guest memory
+ * to resolve the address of the ITTE, then reads the ITTE entry at this
+ * address and puts the result in vcpu_ptr and vlpi_ptr.
+ * Must be called with the ITS lock held.
+ */
+static bool read_itte_locked(struct virt_its *its, uint32_t devid,
+ uint32_t evid, struct vcpu **vcpu_ptr,
+ uint32_t *vlpi_ptr)
+{
+ paddr_t addr;
+ struct vits_itte itte;
+ struct vcpu *vcpu;
+
+ ASSERT(spin_is_locked(&its->its_lock));
+
+ addr = its_get_itte_address(its, devid, evid);
+ if ( addr == INVALID_PADDR )
+ return false;
+
+ if ( vgic_access_guest_memory(its->d, addr, &itte, sizeof(itte), false) )
+ return false;
+
+ vcpu = get_vcpu_from_collection(its, itte.collection);
+ if ( !vcpu )
+ return false;
+
+ *vcpu_ptr = vcpu;
+ *vlpi_ptr = itte.vlpi;
+ return true;
+}
+
+/*
+ * This function takes care of the locking by taking the its_lock itself, so
+ * a caller shall not hold this. Before returning, the lock is dropped again.
+ */
+bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
+ struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
+{
+ bool ret;
+
+ spin_lock(&its->its_lock);
+ ret = read_itte_locked(its, devid, evid, vcpu_ptr, vlpi_ptr);
+ spin_unlock(&its->its_lock);
+
+ return ret;
+}
+
+/*
+ * Queries the collection and device tables to translate the device ID and
+ * event ID and find the appropriate ITTE. The given collection ID and the
+ * virtual LPI number are then stored into that entry.
+ * If vcpu_ptr is provided, returns the VCPU belonging to that collection.
+ * Must be called with the ITS lock held.
+ */
+bool write_itte_locked(struct virt_its *its, uint32_t devid,
+ uint32_t evid, uint32_t collid, uint32_t vlpi,
+ struct vcpu **vcpu_ptr)
+{
+ paddr_t addr;
+ struct vits_itte itte;
+
+ ASSERT(spin_is_locked(&its->its_lock));
+
+ if ( collid >= its->max_collections )
+ return false;
+
+ if ( vlpi >= its->d->arch.vgic.nr_lpis )
+ return false;
+
+ addr = its_get_itte_address(its, devid, evid);
+ if ( addr == INVALID_PADDR )
+ return false;
+
+ itte.collection = collid;
+ itte.vlpi = vlpi;
+
+ if ( vgic_access_guest_memory(its->d, addr, &itte, sizeof(itte), true) )
+ return false;
+
+ if ( vcpu_ptr )
+ *vcpu_ptr = get_vcpu_from_collection(its, collid);
+
+ return true;
+}
+
/**************************************
* Functions that handle ITS commands *
**************************************/
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 15/27] ARM: vITS: handle CLEAR command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (13 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 14/27] ARM: vITS: introduce translation table walks Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 14:10 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 16/27] ARM: vITS: handle INT command Andre Przywara
` (12 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
This introduces the ITS command handler for the CLEAR command, which
clears the pending state of an LPI.
This removes a not-yet injected, but already queued IRQ from a VCPU.
As read_itte() is now eventually used, we add the static keyword.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 52 ++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 50 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 632ab84..e24ab60 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -227,8 +227,8 @@ static bool read_itte_locked(struct virt_its *its, uint32_t devid,
* This function takes care of the locking by taking the its_lock itself, so
* a caller shall not hold this. Before returning, the lock is dropped again.
*/
-bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
- struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
+static bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
+ struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
{
bool ret;
@@ -297,6 +297,51 @@ static uint64_t its_cmd_mask_field(uint64_t *its_cmd, unsigned int word,
#define its_cmd_get_validbit(cmd) its_cmd_mask_field(cmd, 2, 63, 1)
#define its_cmd_get_ittaddr(cmd) (its_cmd_mask_field(cmd, 2, 8, 44) << 8)
+/*
+ * CLEAR removes the pending state from an LPI. */
+static int its_handle_clear(struct virt_its *its, uint64_t *cmdptr)
+{
+ uint32_t devid = its_cmd_get_deviceid(cmdptr);
+ uint32_t eventid = its_cmd_get_id(cmdptr);
+ struct pending_irq *p;
+ struct vcpu *vcpu;
+ uint32_t vlpi;
+ unsigned long flags;
+
+ /* Translate the DevID/EvID pair into a vCPU/vLPI pair. */
+ if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
+ return -1;
+
+ spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
+
+ p = its->d->arch.vgic.handler->lpi_to_pending(its->d, vlpi);
+ if ( !p )
+ {
+ spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
+ return -1;
+ }
+
+ /*
+ * If the LPI is already visible on the guest, it is too late to
+ * clear the pending state. However this is a benign race that can
+ * happen on real hardware, too: If the LPI has already been forwarded
+ * to a CPU interface, a CLEAR request reaching the redistributor has
+ * no effect on that LPI anymore. Since LPIs are edge triggered and
+ * have no active state, we don't need to care about this here.
+ */
+ if ( !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
+ {
+ /* Remove a pending, but not yet injected guest IRQ. */
+ clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
+ list_del_init(&p->inflight);
+ list_del_init(&p->lr_queue);
+ }
+
+ spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
+
+ return 0;
+}
+
#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
@@ -326,6 +371,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
switch ( its_cmd_get_command(command) )
{
+ case GITS_CMD_CLEAR:
+ ret = its_handle_clear(its, command);
+ break;
case GITS_CMD_SYNC:
/* We handle ITS commands synchronously, so we ignore SYNC. */
break;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 16/27] ARM: vITS: handle INT command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (14 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 15/27] ARM: vITS: handle CLEAR command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 14:50 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 17/27] ARM: vITS: handle MAPC command Andre Przywara
` (11 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The INT command sets a given LPI identified by a DeviceID/EventID pair
as pending and thus triggers it to be injected.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index e24ab60..eeb1306 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -342,6 +342,24 @@ static int its_handle_clear(struct virt_its *its, uint64_t *cmdptr)
return 0;
}
+static int its_handle_int(struct virt_its *its, uint64_t *cmdptr)
+{
+ uint32_t devid = its_cmd_get_deviceid(cmdptr);
+ uint32_t eventid = its_cmd_get_id(cmdptr);
+ struct vcpu *vcpu;
+ uint32_t vlpi;
+
+ if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
+ return -1;
+
+ if ( vlpi == INVALID_LPI )
+ return -1;
+
+ vgic_vcpu_inject_irq(vcpu, vlpi);
+
+ return 0;
+}
+
#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
@@ -374,6 +392,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_CLEAR:
ret = its_handle_clear(its, command);
break;
+ case GITS_CMD_INT:
+ ret = its_handle_int(its, command);
+ break;
case GITS_CMD_SYNC:
/* We handle ITS commands synchronously, so we ignore SYNC. */
break;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 17/27] ARM: vITS: handle MAPC command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (15 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 16/27] ARM: vITS: handle INT command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 14:51 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 18/27] ARM: vITS: handle MAPD command Andre Przywara
` (10 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The MAPC command associates a given collection ID with a given
redistributor, thus mapping collections to VCPUs.
We just store the vcpu_id in the collection table for that.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index eeb1306..6e505cb 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -116,6 +116,21 @@ static paddr_t get_baser_phys_addr(uint64_t reg)
#define COLL_TABLE_ENTRY_SIZE 2UL
/* Must be called with the ITS lock held. */
+static int its_set_collection(struct virt_its *its, uint16_t collid,
+ uint16_t vcpu_id)
+{
+ paddr_t addr = get_baser_phys_addr(its->baser_coll);
+
+ ASSERT(spin_is_locked(&its->its_lock));
+
+ if ( collid >= its->max_collections )
+ return -ENOENT;
+
+ return vgic_access_guest_memory(its->d, addr + collid * sizeof(uint16_t),
+ &vcpu_id, sizeof(vcpu_id), true);
+}
+
+/* Must be called with the ITS lock held. */
static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
uint16_t collid)
{
@@ -360,6 +375,29 @@ static int its_handle_int(struct virt_its *its, uint64_t *cmdptr)
return 0;
}
+static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
+{
+ uint32_t collid = its_cmd_get_collection(cmdptr);
+ uint64_t rdbase = its_cmd_mask_field(cmdptr, 2, 16, 44);
+
+ if ( collid >= its->max_collections )
+ return -1;
+
+ if ( rdbase >= its->d->max_vcpus )
+ return -1;
+
+ spin_lock(&its->its_lock);
+
+ if ( its_cmd_get_validbit(cmdptr) )
+ its_set_collection(its, collid, rdbase);
+ else
+ its_set_collection(its, collid, UNMAPPED_COLLECTION);
+
+ spin_unlock(&its->its_lock);
+
+ return 0;
+}
+
#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
@@ -395,6 +433,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_INT:
ret = its_handle_int(its, command);
break;
+ case GITS_CMD_MAPC:
+ ret = its_handle_mapc(its, command);
+ break;
case GITS_CMD_SYNC:
/* We handle ITS commands synchronously, so we ignore SYNC. */
break;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (16 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 17/27] ARM: vITS: handle MAPC command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 15:21 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 19/27] ARM: vITS: handle MAPTI command Andre Przywara
` (9 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The MAPD command maps a device by associating a memory region for
storing ITEs with a certain device ID. Since it features a valid bit,
MAPD also covers the "unmap" functionality, which we also cover here.
We store the given guest physical address in the device table, and, if
this command comes from Dom0, tell the host ITS driver about this new
mapping, so it can issue the corresponding host MAPD command and create
the required tables. We take care of rolling back actions should one
step fail.
Upon unmapping a device we make sure we clean up all associated
resources and release the memory again.
We use our existing guest memory access function to find the right ITT
entry and store the mapping there (in guest memory).
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic-v3-its.c | 60 +++++++++++++++++
xen/arch/arm/gic-v3-lpi.c | 18 +++++
xen/arch/arm/vgic-v3-its.c | 137 +++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic_v3_its.h | 6 ++
4 files changed, 221 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index aebc257..900c9d1 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -800,6 +800,66 @@ out:
return ret;
}
+/* Must be called with the its_device_lock held. */
+static struct its_device *get_its_device(struct domain *d, paddr_t vdoorbell,
+ uint32_t vdevid)
+{
+ struct rb_node *node = d->arch.vgic.its_devices.rb_node;
+ struct its_device *dev;
+
+ ASSERT(spin_is_locked(&d->arch.vgic.its_devices_lock));
+
+ while (node)
+ {
+ int cmp;
+
+ dev = rb_entry(node, struct its_device, rbnode);
+ cmp = compare_its_guest_devices(dev, vdoorbell, vdevid);
+
+ if ( !cmp )
+ return dev;
+
+ if ( cmp > 0 )
+ node = node->rb_left;
+ else
+ node = node->rb_right;
+ }
+
+ return NULL;
+}
+
+static uint32_t get_host_lpi(struct its_device *dev, uint32_t eventid)
+{
+ uint32_t host_lpi = INVALID_LPI;
+
+ if ( dev && (eventid < dev->eventids) )
+ host_lpi = dev->host_lpi_blocks[eventid / LPI_BLOCK] +
+ (eventid % LPI_BLOCK);
+
+ return host_lpi;
+}
+
+int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
+ uint32_t vdevid, uint32_t veventid)
+{
+ struct its_device *dev;
+ uint32_t host_lpi = INVALID_LPI;
+
+ spin_lock(&d->arch.vgic.its_devices_lock);
+ dev = get_its_device(d, vdoorbell_address, vdevid);
+ if ( dev && veventid <= dev->eventids )
+ host_lpi = get_host_lpi(dev, veventid);
+ spin_unlock(&d->arch.vgic.its_devices_lock);
+
+ if ( host_lpi == INVALID_LPI )
+ return -EINVAL;
+
+ gicv3_lpi_update_host_entry(host_lpi, d->domain_id,
+ INVALID_VCPU_ID, INVALID_LPI);
+
+ return 0;
+}
+
/* Scan the DT for any ITS nodes and create a list of host ITSes out of it. */
void gicv3_its_dt_init(const struct dt_device_node *node)
{
diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
index 44f6315..d427539 100644
--- a/xen/arch/arm/gic-v3-lpi.c
+++ b/xen/arch/arm/gic-v3-lpi.c
@@ -207,6 +207,24 @@ out:
irq_exit();
}
+void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
+ unsigned int vcpu_id, uint32_t virt_lpi)
+{
+ union host_lpi *hlpip, hlpi;
+
+ ASSERT(host_lpi >= LPI_OFFSET);
+
+ host_lpi -= LPI_OFFSET;
+
+ hlpip = &lpi_data.host_lpis[host_lpi / HOST_LPIS_PER_PAGE][host_lpi % HOST_LPIS_PER_PAGE];
+
+ hlpi.virt_lpi = virt_lpi;
+ hlpi.dom_id = domain_id;
+ hlpi.vcpu_id = vcpu_id;
+
+ write_u64_atomic(&hlpip->data, hlpi.data);
+}
+
static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
{
uint64_t val;
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 6e505cb..104017e 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -52,6 +52,7 @@
*/
struct virt_its {
struct domain *d;
+ paddr_t doorbell_address;
unsigned int devid_bits;
unsigned int evid_bits;
spinlock_t vcmd_lock; /* Protects the virtual command buffer, which */
@@ -166,6 +167,20 @@ static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
#define DEV_TABLE_ENTRY(addr, bits) \
(((addr) & GENMASK(51, 8)) | (((bits) - 1) & GENMASK(4, 0)))
+/* Set the address of an ITT for a given device ID. */
+static int its_set_itt_address(struct virt_its *its, uint32_t devid,
+ paddr_t itt_address, uint32_t nr_bits)
+{
+ paddr_t addr = get_baser_phys_addr(its->baser_dev);
+ uint64_t itt_entry = DEV_TABLE_ENTRY(itt_address, nr_bits);
+
+ if ( devid >= its->max_devices )
+ return -ENOENT;
+
+ return vgic_access_guest_memory(its->d, addr + devid * sizeof(uint64_t),
+ &itt_entry, sizeof(itt_entry), true);
+}
+
/*
* Lookup the address of the Interrupt Translation Table associated with
* that device ID.
@@ -398,6 +413,125 @@ static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
return 0;
}
+/* Must be called with the ITS lock held. */
+static int its_discard_event(struct virt_its *its,
+ uint32_t vdevid, uint32_t vevid)
+{
+ struct pending_irq *p;
+ unsigned long flags;
+ struct vcpu *vcpu;
+ uint32_t vlpi;
+
+ ASSERT(spin_is_locked(&its->its_lock));
+
+ if ( !read_itte_locked(its, vdevid, vevid, &vcpu, &vlpi) )
+ return -ENOENT;
+
+ if ( vlpi == INVALID_LPI )
+ return -ENOENT;
+
+ /* Lock this VCPU's VGIC to make sure nobody is using the pending_irq. */
+ spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
+
+ /* Remove the pending_irq from the tree. */
+ write_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
+ p = radix_tree_delete(&its->d->arch.vgic.pend_lpi_tree, vlpi);
+ write_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
+
+ if ( !p )
+ {
+ spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
+
+ return -ENOENT;
+ }
+
+ /* Cleanup the pending_irq, to be reusable later. */
+ list_del_init(&p->inflight);
+ list_del_init(&p->lr_queue);
+ p->status = 0;
+ p->lr = GIC_INVALID_LR;
+ p->irq = INVALID_LPI;
+
+ spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
+
+ /* Remove the corresponding host LPI entry */
+ return gicv3_remove_guest_event(its->d, its->doorbell_address,
+ vdevid, vevid);
+}
+
+static int its_unmap_device(struct virt_its *its, uint32_t devid)
+{
+ uint64_t itt, evid;
+ int ret;
+
+ spin_lock(&its->its_lock);
+
+ ret = its_get_itt(its, devid, &itt);
+ if ( ret )
+ {
+ spin_unlock(&its->its_lock);
+ return ret;
+ }
+
+ for ( evid = 0; evid < DEV_TABLE_ITT_SIZE(itt); evid++ )
+ /* Don't care about errors here, clean up as much as possible. */
+ its_discard_event(its, devid, evid);
+
+ spin_unlock(&its->its_lock);
+
+ return 0;
+}
+
+static int its_handle_mapd(struct virt_its *its, uint64_t *cmdptr)
+{
+ /* size and devid get validated by the functions called below. */
+ uint32_t devid = its_cmd_get_deviceid(cmdptr);
+ unsigned int size = its_cmd_get_size(cmdptr) + 1;
+ bool valid = its_cmd_get_validbit(cmdptr);
+ paddr_t itt_addr = its_cmd_get_ittaddr(cmdptr);
+ int ret;
+
+ /* Sanitize the number of events. */
+ if ( valid && (size > its->evid_bits) )
+ return -1;
+
+ /*
+ * There is no easy and clean way for Xen to know the ITS device ID of a
+ * particular (PCI) device, so we have to rely on the guest telling
+ * us about it. For *now* we are just using the device ID *Dom0* uses,
+ * because the driver there has the actual knowledge.
+ * Eventually this will be replaced with a dedicated hypercall to
+ * announce pass-through of devices.
+ */
+ if ( is_hardware_domain(its->d) )
+ {
+ if ( !valid )
+ /* Discard all events and remove pending LPIs. */
+ its_unmap_device(its, devid);
+
+ /*
+ * Dom0's ITSes are mapped 1:1, so both addresses are the same.
+ * Also the device IDs are equal.
+ */
+ ret = gicv3_its_map_guest_device(its->d, its->doorbell_address, devid,
+ its->doorbell_address, devid,
+ BIT(size), valid);
+ if ( ret && valid )
+ return ret;
+ }
+
+ spin_lock(&its->its_lock);
+
+ if ( valid )
+ ret = its_set_itt_address(its, devid, itt_addr, size);
+ else
+ ret = its_set_itt_address(its, devid, INVALID_PADDR, 1);
+
+ spin_unlock(&its->its_lock);
+
+ return ret;
+}
+
#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
@@ -436,6 +570,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_MAPC:
ret = its_handle_mapc(its, command);
break;
+ case GITS_CMD_MAPD:
+ ret = its_handle_mapd(its, command);
+ break;
case GITS_CMD_SYNC:
/* We handle ITS commands synchronously, so we ignore SYNC. */
break;
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index 40f4ef5..60ffdb6 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -169,6 +169,12 @@ int gicv3_its_map_guest_device(struct domain *d,
int gicv3_allocate_host_lpi_block(struct domain *d, uint32_t *first_lpi);
void gicv3_free_host_lpi_block(uint32_t first_lpi);
+int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
+ uint32_t vdevid, uint32_t veventid);
+
+void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
+ unsigned int vcpu_id, uint32_t virt_lpi);
+
#else
static inline void gicv3_its_dt_init(const struct dt_device_node *node)
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 19/27] ARM: vITS: handle MAPTI command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (17 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 18/27] ARM: vITS: handle MAPD command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 16:18 ` Julien Grall
` (2 more replies)
2017-04-12 0:44 ` [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs Andre Przywara
` (8 subsequent siblings)
27 siblings, 3 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The MAPTI commands associates a DeviceID/EventID pair with a LPI/CPU
pair and actually instantiates LPI interrupts.
We connect the already allocated host LPI to this virtual LPI, so that
any triggering LPI on the host can be quickly forwarded to a guest.
Beside entering the VCPU and the virtual LPI number in the respective
host LPI entry, we also initialize and add the already allocated
struct pending_irq to our radix tree, so that we can now easily find it
by its virtual LPI number.
We also read the property table to update the enabled bit and the
priority for our new LPI, as we might have missed this during an earlier
INVALL call (which only checks mapped LPIs).
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic-v3-its.c | 35 ++++++++++++
xen/arch/arm/vgic-v3-its.c | 114 +++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic_v3_its.h | 3 ++
3 files changed, 152 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 900c9d1..fa1f2d5 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -860,6 +860,41 @@ int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
return 0;
}
+/*
+ * Connects the event ID for an already assigned device to the given VCPU/vLPI
+ * pair. The corresponding physical LPI is already mapped on the host side
+ * (when assigning the physical device to the guest), so we just connect the
+ * target VCPU/vLPI pair to that interrupt to inject it properly if it fires.
+ * Returns a pointer to the already allocated struct pending_irq that is
+ * meant to be used by that event.
+ */
+struct pending_irq *gicv3_assign_guest_event(struct domain *d,
+ paddr_t vdoorbell_address,
+ uint32_t vdevid, uint32_t veventid,
+ struct vcpu *v, uint32_t virt_lpi)
+{
+ struct its_device *dev;
+ struct pending_irq *pirq = NULL;
+ uint32_t host_lpi = 0;
+
+ spin_lock(&d->arch.vgic.its_devices_lock);
+ dev = get_its_device(d, vdoorbell_address, vdevid);
+ if ( dev )
+ {
+ host_lpi = get_host_lpi(dev, veventid);
+ pirq = &dev->pend_irqs[veventid];
+ }
+ spin_unlock(&d->arch.vgic.its_devices_lock);
+
+ if ( !host_lpi || !pirq )
+ return NULL;
+
+ gicv3_lpi_update_host_entry(host_lpi, d->domain_id,
+ v ? v->vcpu_id : INVALID_VCPU_ID, virt_lpi);
+
+ return pirq;
+}
+
/* Scan the DT for any ITS nodes and create a list of host ITSes out of it. */
void gicv3_its_dt_init(const struct dt_device_node *node)
{
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 104017e..b7e61b2 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -390,6 +390,34 @@ static int its_handle_int(struct virt_its *its, uint64_t *cmdptr)
return 0;
}
+/*
+ * For a given virtual LPI read the enabled bit and priority from the virtual
+ * property table and update the virtual IRQ's state in the given pending_irq.
+ * Must be called with the respective VGIC VCPU lock held.
+ */
+static int update_lpi_property(struct domain *d, uint32_t vlpi,
+ struct pending_irq *p)
+{
+ paddr_t addr;
+ uint8_t property;
+ int ret;
+
+ addr = d->arch.vgic.rdist_propbase & GENMASK(51, 12);
+
+ ret = vgic_access_guest_memory(d, addr + vlpi - LPI_OFFSET,
+ &property, sizeof(property), false);
+ if ( ret )
+ return ret;
+
+ p->lpi_priority = property & LPI_PROP_PRIO_MASK;
+ if ( property & LPI_PROP_ENABLED )
+ set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+ else
+ clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+
+ return 0;
+}
+
static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
{
uint32_t collid = its_cmd_get_collection(cmdptr);
@@ -532,6 +560,88 @@ static int its_handle_mapd(struct virt_its *its, uint64_t *cmdptr)
return ret;
}
+static int its_handle_mapti(struct virt_its *its, uint64_t *cmdptr)
+{
+ uint32_t devid = its_cmd_get_deviceid(cmdptr);
+ uint32_t eventid = its_cmd_get_id(cmdptr);
+ uint32_t intid = its_cmd_get_physical_id(cmdptr), _intid;
+ uint16_t collid = its_cmd_get_collection(cmdptr);
+ struct pending_irq *pirq;
+ struct vcpu *vcpu = NULL;
+ int ret = -1;
+
+ if ( its_cmd_get_command(cmdptr) == GITS_CMD_MAPI )
+ intid = eventid;
+
+ spin_lock(&its->its_lock);
+ /*
+ * Check whether there is a valid existing mapping. If yes, behavior is
+ * unpredictable, we choose to ignore this command here.
+ * This makes sure we start with a pristine pending_irq below.
+ */
+ if ( read_itte_locked(its, devid, eventid, &vcpu, &_intid) &&
+ _intid != INVALID_LPI )
+ {
+ spin_unlock(&its->its_lock);
+ return -1;
+ }
+
+ /* Enter the mapping in our virtual ITS tables. */
+ if ( !write_itte_locked(its, devid, eventid, collid, intid, &vcpu) )
+ {
+ spin_unlock(&its->its_lock);
+ return -1;
+ }
+
+ spin_unlock(&its->its_lock);
+
+ /*
+ * Connect this virtual LPI to the corresponding host LPI, which is
+ * determined by the same device ID and event ID on the host side.
+ * This returns us the corresponding, still unused pending_irq.
+ */
+ pirq = gicv3_assign_guest_event(its->d, its->doorbell_address,
+ devid, eventid, vcpu, intid);
+ if ( !pirq )
+ goto out_remove_mapping;
+
+ vgic_init_pending_irq(pirq, intid);
+
+ /*
+ * Now read the guest's property table to initialize our cached state.
+ * It can't fire at this time, because it is not known to the host yet.
+ * We don't need the VGIC VCPU lock here, because the pending_irq isn't
+ * in the radix tree yet.
+ */
+ ret = update_lpi_property(its->d, intid, pirq);
+ if ( ret )
+ goto out_remove_host_entry;
+
+ pirq->lpi_vcpu_id = vcpu->vcpu_id;
+
+ /*
+ * Now insert the pending_irq into the domain's LPI tree, so that
+ * it becomes live.
+ */
+ write_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
+ ret = radix_tree_insert(&its->d->arch.vgic.pend_lpi_tree, intid, pirq);
+ write_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
+
+ if ( !ret )
+ return 0;
+
+out_remove_host_entry:
+ gicv3_remove_guest_event(its->d, its->doorbell_address, devid, eventid);
+
+out_remove_mapping:
+ spin_lock(&its->its_lock);
+ write_itte_locked(its, devid, eventid,
+ UNMAPPED_COLLECTION, INVALID_LPI, NULL);
+ spin_unlock(&its->its_lock);
+
+ return ret;
+}
+
#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
@@ -573,6 +683,10 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_MAPD:
ret = its_handle_mapd(its, command);
break;
+ case GITS_CMD_MAPI:
+ case GITS_CMD_MAPTI:
+ ret = its_handle_mapti(its, command);
+ break;
case GITS_CMD_SYNC:
/* We handle ITS commands synchronously, so we ignore SYNC. */
break;
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index 60ffdb6..7b16aeb 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -172,6 +172,9 @@ void gicv3_free_host_lpi_block(uint32_t first_lpi);
int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
uint32_t vdevid, uint32_t veventid);
+struct pending_irq *gicv3_assign_guest_event(struct domain *d, paddr_t doorbell,
+ uint32_t devid, uint32_t eventid,
+ struct vcpu *v, uint32_t virt_lpi);
void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
unsigned int vcpu_id, uint32_t virt_lpi);
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (18 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 19/27] ARM: vITS: handle MAPTI command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 9:46 ` Andre Przywara
2017-04-12 16:49 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 21/27] ARM: vITS: handle MOVI command Andre Przywara
` (7 subsequent siblings)
27 siblings, 2 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
When LPIs get unmapped by a guest, they might still be in some LR of
some VCPU. Nevertheless we remove the corresponding pending_irq
(possibly freeing it), and detect this case (irq_to_pending() returns
NULL) when the LR gets cleaned up later.
However a *new* LPI may get mapped with the same number while the old
LPI is *still* in some LR. To avoid getting the wrong state, we mark
every newly mapped LPI as PRISTINE, which means: has never been in an
LR before. If we detect the LPI in an LR anyway, it must have been an
older one, which we can simply retire.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic.c | 17 ++++++++++++++++-
xen/arch/arm/vgic-v3-its.c | 5 +++++
xen/include/asm-arm/vgic.h | 1 +
3 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index d752352..e8c3202 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -373,6 +373,8 @@ static inline void gic_set_lr(int lr, struct pending_irq *p,
{
ASSERT(!local_irq_is_enabled());
+ clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status);
+
gic_hw_ops->update_lr(lr, p, state);
set_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
@@ -510,7 +512,17 @@ static void gic_update_one_lr(struct vcpu *v, int i)
}
else if ( lr_val.state & GICH_LR_PENDING )
{
- int q __attribute__ ((unused)) = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
+ int q __attribute__ ((unused));
+
+ if ( test_and_clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status) )
+ {
+ gic_hw_ops->clear_lr(i);
+ clear_bit(i, &this_cpu(lr_mask));
+
+ return;
+ }
+
+ q = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
#ifdef GIC_DEBUG
if ( q )
gdprintk(XENLOG_DEBUG, "trying to inject irq=%d into d%dv%d, when it is already pending in LR%d\n",
@@ -522,6 +534,9 @@ static void gic_update_one_lr(struct vcpu *v, int i)
gic_hw_ops->clear_lr(i);
clear_bit(i, &this_cpu(lr_mask));
+ if ( test_and_clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status) )
+ return;
+
if ( p->desc != NULL )
clear_bit(_IRQ_INPROGRESS, &p->desc->status);
clear_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index b7e61b2..0765810 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -618,6 +618,11 @@ static int its_handle_mapti(struct virt_its *its, uint64_t *cmdptr)
goto out_remove_host_entry;
pirq->lpi_vcpu_id = vcpu->vcpu_id;
+ /*
+ * Mark this LPI as new, so any older (now unmapped) LPI in any LR
+ * can be easily recognised as such.
+ */
+ set_bit(GIC_IRQ_GUEST_PRISTINE_LPI, pirq->status);
/*
* Now insert the pending_irq into the domain's LPI tree, so that
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 02732db..b1a7525 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -66,6 +66,7 @@ struct pending_irq
#define GIC_IRQ_GUEST_VISIBLE 2
#define GIC_IRQ_GUEST_ENABLED 3
#define GIC_IRQ_GUEST_MIGRATING 4
+#define GIC_IRQ_GUEST_PRISTINE_LPI 5
unsigned long status;
struct irq_desc *desc; /* only set it the irq corresponds to a physical irq */
unsigned int irq;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 21/27] ARM: vITS: handle MOVI command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (19 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 16:59 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 22/27] ARM: vITS: handle DISCARD command Andre Przywara
` (6 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The MOVI command moves the interrupt affinity from one redistributor
(read: VCPU) to another.
For now migration of "live" LPIs is not yet implemented, but we store
the changed affinity in the host LPI structure and in our virtual ITTE.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic-v3-its.c | 24 +++++++++++++++++
xen/arch/arm/gic-v3-lpi.c | 15 +++++++++++
xen/arch/arm/vgic-v3-its.c | 57 ++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic_v3_its.h | 4 +++
4 files changed, 100 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index fa1f2d5..1a08d43 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -895,6 +895,30 @@ struct pending_irq *gicv3_assign_guest_event(struct domain *d,
return pirq;
}
+/* Changes the target VCPU for a given host LPI assigned to a domain. */
+int gicv3_lpi_change_vcpu(struct domain *d, paddr_t vdoorbell,
+ uint32_t vdevid, uint32_t veventid,
+ unsigned int vcpu_id)
+{
+ uint32_t host_lpi;
+ struct its_device *dev;
+
+ spin_lock(&d->arch.vgic.its_devices_lock);
+ dev = get_its_device(d, vdoorbell, vdevid);
+ if ( dev )
+ host_lpi = get_host_lpi(dev, veventid);
+ else
+ host_lpi = 0;
+ spin_unlock(&d->arch.vgic.its_devices_lock);
+
+ if ( !host_lpi )
+ return -ENOENT;
+
+ gicv3_lpi_update_host_vcpuid(host_lpi, vcpu_id);
+
+ return 0;
+}
+
/* Scan the DT for any ITS nodes and create a list of host ITSes out of it. */
void gicv3_its_dt_init(const struct dt_device_node *node)
{
diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
index d427539..6af5ad9 100644
--- a/xen/arch/arm/gic-v3-lpi.c
+++ b/xen/arch/arm/gic-v3-lpi.c
@@ -225,6 +225,21 @@ void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
write_u64_atomic(&hlpip->data, hlpi.data);
}
+int gicv3_lpi_update_host_vcpuid(uint32_t host_lpi, unsigned int vcpu_id)
+{
+ union host_lpi *hlpip;
+
+ ASSERT(host_lpi >= LPI_OFFSET);
+
+ host_lpi -= LPI_OFFSET;
+
+ hlpip = &lpi_data.host_lpis[host_lpi / HOST_LPIS_PER_PAGE][host_lpi % HOST_LPIS_PER_PAGE];
+
+ write_u16_atomic(&hlpip->vcpu_id, vcpu_id);
+
+ return 0;
+}
+
static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
{
uint64_t val;
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 0765810..be9de08 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -647,6 +647,57 @@ out_remove_mapping:
return ret;
}
+static int its_handle_movi(struct virt_its *its, uint64_t *cmdptr)
+{
+ uint32_t devid = its_cmd_get_deviceid(cmdptr);
+ uint32_t eventid = its_cmd_get_id(cmdptr);
+ int collid = its_cmd_get_collection(cmdptr);
+ unsigned long flags;
+ struct pending_irq *p;
+ struct vcpu *ovcpu, *nvcpu;
+ uint32_t vlpi;
+ int ret = -1;
+
+ spin_lock(&its->its_lock);
+ /* Check for a mapped LPI and get the LPI number. */
+ if ( !read_itte_locked(its, devid, eventid, &ovcpu, &vlpi) )
+ goto out_unlock;
+
+ if ( vlpi == INVALID_LPI )
+ {
+ spin_unlock(&its->its_lock);
+ return -1;
+ }
+
+ /* Check the new collection ID and get the new VCPU pointer */
+ nvcpu = get_vcpu_from_collection(its, collid);
+ if ( !nvcpu )
+ goto out_unlock;
+
+ spin_lock_irqsave(&ovcpu->arch.vgic.lock, flags);
+
+ /* Update our cached vcpu_id in the pending_irq. */
+ p = its->d->arch.vgic.handler->lpi_to_pending(its->d, vlpi);
+ p->lpi_vcpu_id = nvcpu->vcpu_id;
+
+ /* Now store the new collection in the translation table. */
+ if ( !write_itte_locked(its, devid, eventid, collid, vlpi, &nvcpu) )
+ goto out_unlock;
+
+ spin_unlock_irqrestore(&ovcpu->arch.vgic.lock, flags);
+ spin_unlock(&its->its_lock);
+
+ /* TODO: lookup currently-in-guest virtual IRQs and migrate them? */
+
+ return gicv3_lpi_change_vcpu(its->d, its->doorbell_address,
+ devid, eventid, nvcpu->vcpu_id);
+
+out_unlock:
+ spin_unlock(&its->its_lock);
+
+ return ret;
+}
+
#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
@@ -692,6 +743,12 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_MAPTI:
ret = its_handle_mapti(its, command);
break;
+ case GITS_CMD_MOVALL:
+ gdprintk(XENLOG_G_INFO, "ITS: ignoring MOVALL command\n");
+ break;
+ case GITS_CMD_MOVI:
+ ret = its_handle_movi(its, command);
+ break;
case GITS_CMD_SYNC:
/* We handle ITS commands synchronously, so we ignore SYNC. */
break;
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index 7b16aeb..ee69e9b 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -175,8 +175,12 @@ int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
struct pending_irq *gicv3_assign_guest_event(struct domain *d, paddr_t doorbell,
uint32_t devid, uint32_t eventid,
struct vcpu *v, uint32_t virt_lpi);
+int gicv3_lpi_change_vcpu(struct domain *d, paddr_t doorbell,
+ uint32_t devid, uint32_t eventid,
+ unsigned int vcpu_id);
void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
unsigned int vcpu_id, uint32_t virt_lpi);
+int gicv3_lpi_update_host_vcpuid(uint32_t host_lpi, unsigned int vcpu_id);
#else
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 22/27] ARM: vITS: handle DISCARD command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (20 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 21/27] ARM: vITS: handle MOVI command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 17:06 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 23/27] ARM: vITS: handle INV command Andre Przywara
` (5 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The DISCARD command drops the connection between a DeviceID/EventID
and an LPI/collection pair.
We mark the respective structure entries as not allocated and make
sure that any queued IRQs are removed.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 30 +++++++++++++++++++++++++++---
1 file changed, 27 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index be9de08..09cb3af 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -276,9 +276,9 @@ static bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
* If vcpu_ptr is provided, returns the VCPU belonging to that collection.
* Must be called with the ITS lock held.
*/
-bool write_itte_locked(struct virt_its *its, uint32_t devid,
- uint32_t evid, uint32_t collid, uint32_t vlpi,
- struct vcpu **vcpu_ptr)
+static bool write_itte_locked(struct virt_its *its, uint32_t devid,
+ uint32_t evid, uint32_t collid, uint32_t vlpi,
+ struct vcpu **vcpu_ptr)
{
paddr_t addr;
struct vits_itte itte;
@@ -698,6 +698,27 @@ out_unlock:
return ret;
}
+static int its_handle_discard(struct virt_its *its, uint64_t *cmdptr)
+{
+ uint32_t devid = its_cmd_get_deviceid(cmdptr);
+ uint32_t eventid = its_cmd_get_id(cmdptr);
+ int ret;
+
+ spin_lock(&its->its_lock);
+
+ /* Remove from the radix tree and remove the host entry. */
+ ret = its_discard_event(its, devid, eventid);
+
+ /* Remove from the guest's ITTE. */
+ if ( ret || write_itte_locked(its, devid, eventid,
+ UNMAPPED_COLLECTION, INVALID_LPI, NULL) )
+ ret = -1;
+
+ spin_unlock(&its->its_lock);
+
+ return ret;
+}
+
#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
@@ -730,6 +751,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_CLEAR:
ret = its_handle_clear(its, command);
break;
+ case GITS_CMD_DISCARD:
+ ret = its_handle_discard(its, command);
+ break;
case GITS_CMD_INT:
ret = its_handle_int(its, command);
break;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 23/27] ARM: vITS: handle INV command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (21 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 22/27] ARM: vITS: handle DISCARD command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 17:20 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 24/27] ARM: vITS: handle INVALL command Andre Przywara
` (4 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The INV command instructs the ITS to update the configuration data for
a given LPI by re-reading its entry from the property table.
We don't need to care so much about the priority value, but enabling
or disabling an LPI has some effect: We remove or push virtual LPIs
to their VCPUs, also check the virtual pending bit if an LPI gets enabled.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 09cb3af..f2789c5 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -418,6 +418,68 @@ static int update_lpi_property(struct domain *d, uint32_t vlpi,
return 0;
}
+/*
+ * Checks whether an LPI that got enabled or disabled needs to change
+ * something in the VGIC (added or removed from the LR or queues).
+ * Must be called with the VCPU VGIC lock held.
+ */
+static void update_lpi_vgic_status(struct vcpu *v, struct pending_irq *p,
+ uint32_t vlpi)
+{
+ ASSERT(spin_is_locked(&v->arch.vgic.lock));
+
+ if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+ {
+ if ( !list_empty(&p->inflight) &&
+ !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
+ gic_raise_guest_irq(v, vlpi, p->lpi_priority);
+ }
+ else
+ {
+ clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+ list_del_init(&p->lr_queue);
+ }
+}
+
+static int its_handle_inv(struct virt_its *its, uint64_t *cmdptr)
+{
+ struct domain *d = its->d;
+ uint32_t devid = its_cmd_get_deviceid(cmdptr);
+ uint32_t eventid = its_cmd_get_id(cmdptr);
+ struct pending_irq *p;
+ unsigned long flags;
+ struct vcpu *vcpu;
+ uint32_t vlpi;
+ int ret = -1;
+
+ /* Translate the event into a vCPU/vLPI pair. */
+ if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
+ return -1;
+
+ if ( vlpi == INVALID_LPI )
+ return -1;
+
+ spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
+
+ p = d->arch.vgic.handler->lpi_to_pending(d, vlpi);
+ if ( !p )
+ goto out_unlock;
+
+ /* Read the property table and update our cached status. */
+ if ( update_lpi_property(d, vlpi, p) )
+ goto out_unlock;
+
+ /* Check whether the LPI needs to go on a VCPU. */
+ update_lpi_vgic_status(vcpu, p, vlpi);
+
+ ret = 0;
+
+out_unlock:
+ spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
+
+ return ret;
+}
+
static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
{
uint32_t collid = its_cmd_get_collection(cmdptr);
@@ -757,6 +819,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_INT:
ret = its_handle_int(its, command);
break;
+ case GITS_CMD_INV:
+ ret = its_handle_inv(its, command);
+ break;
case GITS_CMD_MAPC:
ret = its_handle_mapc(its, command);
break;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 24/27] ARM: vITS: handle INVALL command
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (22 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 23/27] ARM: vITS: handle INV command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 17:26 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 25/27] ARM: vITS: increase mmio_count for each ITS Andre Przywara
` (3 subsequent siblings)
27 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
The INVALL command instructs an ITS to invalidate the configuration
data for all LPIs associated with a given redistributor (read: VCPU).
This is nasty to emulate exactly with our architecture, so we just
iterate over all mapped LPIs and filter for those from that particular
VCPU.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 66 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index f2789c5..9b5032b 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -480,6 +480,69 @@ out_unlock:
return ret;
}
+/*
+ * INVALL updates the per-LPI configuration status for every LPI mapped to
+ * a particular redistributor.
+ * We iterate over all mapped LPIs in our radix tree and update those.
+ */
+static int its_handle_invall(struct virt_its *its, uint64_t *cmdptr)
+{
+ uint32_t collid = its_cmd_get_collection(cmdptr);
+ struct vcpu *vcpu;
+ struct pending_irq *pirqs[16];
+ uint64_t vlpi = 0; /* 64-bit to catch overflows */
+ unsigned int nr_lpis, i;
+ unsigned long flags;
+ int ret = 0;
+
+ /*
+ * As this implementation walks over all mapped LPIs, it might take
+ * too long for a real guest, so we might want to revisit this
+ * implementation for DomUs.
+ * However this command is very rare, also we don't expect many
+ * LPIs to be actually mapped, so it's fine for Dom0 to use.
+ */
+ ASSERT(is_hardware_domain(its->d));
+
+ spin_lock(&its->its_lock);
+ vcpu = get_vcpu_from_collection(its, collid);
+ spin_unlock(&its->its_lock);
+
+ spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
+ read_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
+
+ do
+ {
+ nr_lpis = radix_tree_gang_lookup(&its->d->arch.vgic.pend_lpi_tree,
+ (void **)pirqs, vlpi,
+ ARRAY_SIZE(pirqs));
+
+ for ( i = 0; i < nr_lpis; i++ )
+ {
+ /* We only care about LPIs on our VCPU. */
+ if ( pirqs[i]->lpi_vcpu_id != vcpu->vcpu_id )
+ continue;
+
+ vlpi = pirqs[i]->irq;
+ /* If that fails for a single LPI, carry on to handle the rest. */
+ ret = update_lpi_property(its->d, vlpi, pirqs[i]);
+ if ( !ret )
+ update_lpi_vgic_status(vcpu, pirqs[i], vlpi);
+ }
+ /*
+ * Loop over the next gang of pending_irqs until we reached the end of
+ * a (fully populated) tree or the lookup function returns less LPIs than
+ * it has been asked for.
+ */
+ } while ( (++vlpi < its->d->arch.vgic.nr_lpis) &&
+ (nr_lpis == ARRAY_SIZE(pirqs)) );
+
+ read_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
+ spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
+
+ return ret;
+}
+
static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
{
uint32_t collid = its_cmd_get_collection(cmdptr);
@@ -822,6 +885,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
case GITS_CMD_INV:
ret = its_handle_inv(its, command);
break;
+ case GITS_CMD_INVALL:
+ ret = its_handle_invall(its, command);
+ break;
case GITS_CMD_MAPC:
ret = its_handle_mapc(its, command);
break;
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 25/27] ARM: vITS: increase mmio_count for each ITS
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (23 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 24/27] ARM: vITS: handle INVALL command Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 26/27] ARM: vITS: create and initialize virtual ITSes for Dom0 Andre Przywara
` (2 subsequent siblings)
27 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Increase the count of MMIO regions needed by one for each ITS Dom0 has
to emulate. We emulate the ITSes 1:1 from the hardware, so the number
is the number of host ITSes.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 15 +++++++++++++++
xen/arch/arm/vgic-v3.c | 3 +++
xen/include/asm-arm/gic_v3_its.h | 7 +++++++
3 files changed, 25 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 9b5032b..1622e53 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -83,6 +83,21 @@ struct vits_itte
#define GITS_BASER_RO_MASK (GITS_BASER_TYPE_MASK | \
(31UL << GITS_BASER_ENTRY_SIZE_SHIFT))
+unsigned int vgic_v3_its_count(const struct domain *d)
+{
+ struct host_its *hw_its;
+ unsigned int ret = 0;
+
+ /* Only Dom0 can use emulated ITSes so far. */
+ if ( !is_hardware_domain(d) )
+ return 0;
+
+ list_for_each_entry(hw_its, &host_its_list, entry)
+ ret++;
+
+ return ret;
+}
+
int vgic_v3_its_init_domain(struct domain *d)
{
spin_lock_init(&d->arch.vgic.its_devices_lock);
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 33cd5f7..e8ed658 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1784,6 +1784,9 @@ int vgic_v3_init(struct domain *d, int *mmio_count)
/* GICD region + number of Redistributors */
*mmio_count = vgic_v3_rdist_count(d) + 1;
+ /* one region per ITS */
+ *mmio_count += vgic_v3_its_count(d);
+
register_vgic_ops(d, &v3_ops);
return 0;
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index ee69e9b..ae79d5c 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -137,6 +137,8 @@ void gicv3_its_dt_init(const struct dt_device_node *node);
bool gicv3_its_host_has_its(void);
+unsigned int vgic_v3_its_count(const struct domain *d);
+
void gicv3_do_LPI(unsigned int lpi);
int gicv3_lpi_init_rdist(void __iomem * rdist_base);
@@ -193,6 +195,11 @@ static inline bool gicv3_its_host_has_its(void)
return false;
}
+static inline unsigned int vgic_v3_its_count(const struct domain *d)
+{
+ return 0;
+}
+
static inline void gicv3_do_LPI(unsigned int lpi)
{
/* We don't enable LPIs without an ITS. */
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 26/27] ARM: vITS: create and initialize virtual ITSes for Dom0
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (24 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 25/27] ARM: vITS: increase mmio_count for each ITS Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 27/27] ARM: vITS: create ITS subnodes for Dom0 DT Andre Przywara
2017-04-12 14:13 ` [PATCH v8 00/27] arm64: Dom0 ITS emulation Julien Grall
27 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
For each hardware ITS create and initialize a virtual ITS for Dom0.
We use the same memory mapped address to keep the doorbell working.
This introduces a function to initialize a virtual ITS.
We maintain a list of virtual ITSes, at the moment for the only
purpose of later being able to free them again.
We configure the virtual ITSes to match the hardware ones, that is we
keep the number of device ID bits and event ID bits the same as the host
ITS.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/vgic-v3-its.c | 74 ++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/vgic-v3.c | 4 +++
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/gic_v3_its.h | 4 +++
4 files changed, 83 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 1622e53..956b59c 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -52,6 +52,7 @@
*/
struct virt_its {
struct domain *d;
+ struct list_head vits_list;
paddr_t doorbell_address;
unsigned int devid_bits;
unsigned int evid_bits;
@@ -100,14 +101,49 @@ unsigned int vgic_v3_its_count(const struct domain *d)
int vgic_v3_its_init_domain(struct domain *d)
{
+ int ret;
+
+ INIT_LIST_HEAD(&d->arch.vgic.vits_list);
spin_lock_init(&d->arch.vgic.its_devices_lock);
d->arch.vgic.its_devices = RB_ROOT;
+ if ( is_hardware_domain(d) )
+ {
+ struct host_its *hw_its;
+
+ list_for_each_entry(hw_its, &host_its_list, entry)
+ {
+ /*
+ * For each host ITS create a virtual ITS using the same
+ * base and thus doorbell address.
+ * Use the same number of device ID and event ID bits as the host.
+ */
+ ret = vgic_v3_its_init_virtual(d, hw_its->addr,
+ hw_its->devid_bits,
+ hw_its->evid_bits);
+ if ( ret )
+ {
+ vgic_v3_its_free_domain(d);
+ return ret;
+ }
+ else
+ d->arch.vgic.has_its = true;
+ }
+ }
+
return 0;
}
void vgic_v3_its_free_domain(struct domain *d)
{
+ struct virt_its *pos, *temp;
+
+ list_for_each_entry_safe( pos, temp, &d->arch.vgic.vits_list, vits_list )
+ {
+ list_del(&pos->vits_list);
+ xfree(pos);
+ }
+
ASSERT(RB_EMPTY_ROOT(&d->arch.vgic.its_devices));
}
@@ -1377,6 +1413,44 @@ static const struct mmio_handler_ops vgic_its_mmio_handler = {
.write = vgic_v3_its_mmio_write,
};
+int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr,
+ unsigned int devid_bits, unsigned int evid_bits)
+{
+ struct virt_its *its;
+ uint64_t base_attr;
+
+ its = xzalloc(struct virt_its);
+ if ( !its )
+ return -ENOMEM;
+
+ base_attr = GIC_BASER_InnerShareable << GITS_BASER_SHAREABILITY_SHIFT;
+ base_attr |= GIC_BASER_CACHE_SameAsInner << GITS_BASER_OUTER_CACHEABILITY_SHIFT;
+ base_attr |= GIC_BASER_CACHE_RaWaWb << GITS_BASER_INNER_CACHEABILITY_SHIFT;
+
+ its->cbaser = base_attr;
+ base_attr |= 0ULL << GITS_BASER_PAGE_SIZE_SHIFT; /* 4K pages */
+ its->baser_dev = GITS_BASER_TYPE_DEVICE << GITS_BASER_TYPE_SHIFT;
+ its->baser_dev |= (DEV_TABLE_ENTRY_SIZE - 1) << GITS_BASER_ENTRY_SIZE_SHIFT;
+ its->baser_dev |= base_attr;
+ its->baser_coll = GITS_BASER_TYPE_COLLECTION << GITS_BASER_TYPE_SHIFT;
+ its->baser_coll |= (COLL_TABLE_ENTRY_SIZE - 1) <<
+ GITS_BASER_ENTRY_SIZE_SHIFT;
+ its->baser_coll |= base_attr;
+ its->d = d;
+ its->doorbell_address = guest_addr + ITS_DOORBELL_OFFSET;
+ its->devid_bits = devid_bits;
+ its->evid_bits = evid_bits;
+ spin_lock_init(&its->vcmd_lock);
+ spin_lock_init(&its->its_lock);
+
+ register_mmio_handler(d, &vgic_its_mmio_handler, guest_addr, SZ_64K, its);
+
+ /* Register the virtual ITSes to be able to clean them up later. */
+ list_add_tail(&its->vits_list, &d->arch.vgic.vits_list);
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index e8ed658..e576e31 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1695,6 +1695,10 @@ static int vgic_v3_domain_init(struct domain *d)
d->arch.vgic.rdist_regions[0].first_cpu = 0;
}
+ /*
+ * For a hardware domain, this will iterate over the host ITSes
+ * and maps one virtual ITS per host ITS at the same address.
+ */
ret = vgic_v3_its_init_domain(d);
if ( ret )
return ret;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index b2d98bb..92f4ce5 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -115,6 +115,7 @@ struct arch_domain
spinlock_t its_devices_lock; /* Protects the its_devices tree */
struct radix_tree_root pend_lpi_tree; /* Stores struct pending_irq's */
rwlock_t pend_lpi_tree_lock; /* Protects the pend_lpi_tree */
+ struct list_head vits_list; /* List of virtual ITSes */
unsigned int intid_bits;
bool rdists_enabled; /* Is any redistributor enabled? */
bool has_its;
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index ae79d5c..dbae6ee 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -158,6 +158,10 @@ int gicv3_its_setup_collection(unsigned int cpu);
int vgic_v3_its_init_domain(struct domain *d);
void vgic_v3_its_free_domain(struct domain *d);
+/* Create and register a virtual ITS at the given guest address. */
+int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr,
+ unsigned int devid_bits, unsigned int evid_bits);
+
/*
* Map a device on the host by allocating an ITT on the host (ITS).
* "nr_event" specifies how many events (interrupts) this device will need.
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v8 27/27] ARM: vITS: create ITS subnodes for Dom0 DT
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (25 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 26/27] ARM: vITS: create and initialize virtual ITSes for Dom0 Andre Przywara
@ 2017-04-12 0:44 ` Andre Przywara
2017-04-12 14:13 ` [PATCH v8 00/27] arm64: Dom0 ITS emulation Julien Grall
27 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 0:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Dom0 expects all ITSes in the system to be propagated to be able to
use MSIs.
Create Dom0 DT nodes for each hardware ITS, keeping the register frame
address the same, as the doorbell address that the Dom0 drivers program
into the BARs has to match the hardware.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
xen/arch/arm/gic-v3-its.c | 73 ++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 4 ++-
xen/include/asm-arm/gic_v3_its.h | 12 +++++++
3 files changed, 88 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 1a08d43..1fb06ca 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -20,6 +20,7 @@
#include <xen/lib.h>
#include <xen/delay.h>
+#include <xen/libfdt/libfdt.h>
#include <xen/mm.h>
#include <xen/rbtree.h>
#include <xen/sched.h>
@@ -919,6 +920,78 @@ int gicv3_lpi_change_vcpu(struct domain *d, paddr_t vdoorbell,
return 0;
}
+/*
+ * Create the respective guest DT nodes from a list of host ITSes.
+ * This copies the reg property, so the guest sees the ITS at the same address
+ * as the host.
+ */
+int gicv3_its_make_hwdom_dt_nodes(const struct domain *d,
+ const struct dt_device_node *gic,
+ void *fdt)
+{
+ uint32_t len;
+ int res;
+ const void *prop = NULL;
+ const struct dt_device_node *its = NULL;
+ const struct host_its *its_data;
+
+ if ( list_empty(&host_its_list) )
+ return 0;
+
+ /* The sub-nodes require the ranges property */
+ prop = dt_get_property(gic, "ranges", &len);
+ if ( !prop )
+ {
+ printk(XENLOG_ERR "Can't find ranges property for the gic node\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ res = fdt_property(fdt, "ranges", prop, len);
+ if ( res )
+ return res;
+
+ list_for_each_entry(its_data, &host_its_list, entry)
+ {
+ its = its_data->dt_node;
+
+ res = fdt_begin_node(fdt, its->name);
+ if ( res )
+ return res;
+
+ res = fdt_property_string(fdt, "compatible", "arm,gic-v3-its");
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "msi-controller", NULL, 0);
+ if ( res )
+ return res;
+
+ if ( its->phandle )
+ {
+ res = fdt_property_cell(fdt, "phandle", its->phandle);
+ if ( res )
+ return res;
+ }
+
+ /* Use the same reg regions as the ITS node in host DTB. */
+ prop = dt_get_property(its, "reg", &len);
+ if ( !prop )
+ {
+ printk(XENLOG_ERR "GICv3: Can't find ITS reg property.\n");
+ res = -FDT_ERR_XEN(ENOENT);
+ return res;
+ }
+
+ res = fdt_property(fdt, "reg", prop, len);
+ if ( res )
+ return res;
+
+ fdt_end_node(fdt);
+ }
+
+ return res;
+}
+
/* Scan the DT for any ITS nodes and create a list of host ITSes out of it. */
void gicv3_its_dt_init(const struct dt_device_node *node)
{
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index d539d6c..c927306 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1172,8 +1172,10 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
res = fdt_property(fdt, "reg", new_cells, len);
xfree(new_cells);
+ if ( res )
+ return res;
- return res;
+ return gicv3_its_make_hwdom_dt_nodes(d, gic, fdt);
}
static const hw_irq_controller gicv3_host_irq_type = {
diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
index dbae6ee..d2a3e53 100644
--- a/xen/include/asm-arm/gic_v3_its.h
+++ b/xen/include/asm-arm/gic_v3_its.h
@@ -162,6 +162,11 @@ void vgic_v3_its_free_domain(struct domain *d);
int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr,
unsigned int devid_bits, unsigned int evid_bits);
+/* Create the appropriate DT nodes for a hardware domain. */
+int gicv3_its_make_hwdom_dt_nodes(const struct domain *d,
+ const struct dt_device_node *gic,
+ void *fdt);
+
/*
* Map a device on the host by allocating an ITT on the host (ITS).
* "nr_event" specifies how many events (interrupts) this device will need.
@@ -245,6 +250,13 @@ static inline void vgic_v3_its_free_domain(struct domain *d)
{
}
+static inline int gicv3_its_make_hwdom_dt_nodes(const struct domain *d,
+ const struct dt_device_node *gic,
+ void *fdt)
+{
+ return 0;
+}
+
#endif /* CONFIG_HAS_ITS */
#endif
--
2.8.2
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 82+ messages in thread
* Re: [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs
2017-04-12 0:44 ` [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs Andre Przywara
@ 2017-04-12 9:46 ` Andre Przywara
2017-04-12 16:49 ` Julien Grall
1 sibling, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 9:46 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 01:44, Andre Przywara wrote:
> When LPIs get unmapped by a guest, they might still be in some LR of
> some VCPU. Nevertheless we remove the corresponding pending_irq
> (possibly freeing it), and detect this case (irq_to_pending() returns
> NULL) when the LR gets cleaned up later.
> However a *new* LPI may get mapped with the same number while the old
> LPI is *still* in some LR. To avoid getting the wrong state, we mark
> every newly mapped LPI as PRISTINE, which means: has never been in an
> LR before. If we detect the LPI in an LR anyway, it must have been an
> older one, which we can simply retire.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic.c | 17 ++++++++++++++++-
> xen/arch/arm/vgic-v3-its.c | 5 +++++
> xen/include/asm-arm/vgic.h | 1 +
> 3 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index d752352..e8c3202 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -373,6 +373,8 @@ static inline void gic_set_lr(int lr, struct pending_irq *p,
> {
> ASSERT(!local_irq_is_enabled());
>
> + clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status);
> +
> gic_hw_ops->update_lr(lr, p, state);
>
> set_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
> @@ -510,7 +512,17 @@ static void gic_update_one_lr(struct vcpu *v, int i)
> }
> else if ( lr_val.state & GICH_LR_PENDING )
> {
> - int q __attribute__ ((unused)) = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
> + int q __attribute__ ((unused));
> +
> + if ( test_and_clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status) )
> + {
> + gic_hw_ops->clear_lr(i);
> + clear_bit(i, &this_cpu(lr_mask));
> +
> + return;
> + }
> +
> + q = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
> #ifdef GIC_DEBUG
> if ( q )
> gdprintk(XENLOG_DEBUG, "trying to inject irq=%d into d%dv%d, when it is already pending in LR%d\n",
> @@ -522,6 +534,9 @@ static void gic_update_one_lr(struct vcpu *v, int i)
> gic_hw_ops->clear_lr(i);
> clear_bit(i, &this_cpu(lr_mask));
>
> + if ( test_and_clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status) )
> + return;
> +
> if ( p->desc != NULL )
> clear_bit(_IRQ_INPROGRESS, &p->desc->status);
> clear_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index b7e61b2..0765810 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -618,6 +618,11 @@ static int its_handle_mapti(struct virt_its *its, uint64_t *cmdptr)
> goto out_remove_host_entry;
>
> pirq->lpi_vcpu_id = vcpu->vcpu_id;
> + /*
> + * Mark this LPI as new, so any older (now unmapped) LPI in any LR
> + * can be easily recognised as such.
> + */
> + set_bit(GIC_IRQ_GUEST_PRISTINE_LPI, pirq->status);
Arrgh, so I somehow managed to introduce that typo after testing, but
before sending (note to self: don't send stuff at 1:44 am), sorry for
that! Of course it must be "..., &pirq->status);"
Just compile-tested this series (with that typo fixed) again: every
commit compiles without warnings for arm64 (w/ and w/o ITS configured)
and arm32.
Also successfully boot-tested on the model with and without ITS support.
Cheers,
Andre.
> /*
> * Now insert the pending_irq into the domain's LPI tree, so that
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 02732db..b1a7525 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -66,6 +66,7 @@ struct pending_irq
> #define GIC_IRQ_GUEST_VISIBLE 2
> #define GIC_IRQ_GUEST_ENABLED 3
> #define GIC_IRQ_GUEST_MIGRATING 4
> +#define GIC_IRQ_GUEST_PRISTINE_LPI 5
> unsigned long status;
> struct irq_desc *desc; /* only set it the irq corresponds to a physical irq */
> unsigned int irq;
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 01/27] ARM: GICv3: propagate number of host LPIs to GICv3 guest
2017-04-12 0:44 ` [PATCH v8 01/27] ARM: GICv3: propagate number of host LPIs to GICv3 guest Andre Przywara
@ 2017-04-12 10:06 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 10:06 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> The host supports a certain number of LPI identifiers, as stored in
> the GICD_TYPER register.
> Store this number from the hardware register in vgic_v3_hw to allow
> injecting the very same number into a guest (Dom0).
DOM0 number should be derived from the limitation provided by the user
(i.e lpi_data.max_host_lpi_ids) and not directly from the hardware.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic-v3.c | 6 +++++-
> xen/arch/arm/vgic-v3.c | 7 ++++++-
> xen/include/asm-arm/domain.h | 1 +
> xen/include/asm-arm/vgic.h | 3 ++-
> 4 files changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index a559e5e..29c8964 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1579,6 +1579,7 @@ static int __init gicv3_init(void)
> {
> int res, i;
> uint32_t reg;
> + unsigned int intid_bits;
>
> if ( !cpu_has_gicv3 )
> {
> @@ -1622,8 +1623,11 @@ static int __init gicv3_init(void)
> i, r->base, r->base + r->size);
> }
>
> + reg = readl_relaxed(GICD + GICD_TYPER);
> + intid_bits = GICD_TYPE_ID_BITS(reg);
> +
> vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions,
> - gicv3.rdist_stride);
> + gicv3.rdist_stride, intid_bits);
This only works for DOM0, I would expect the number of the guest to come
up from the toolstack. You may want to have a look to
xen_arch_domain_config. This is not necessary for this version (*hint*
this should be listed in the list of TODO in the cover letter *hint*).
> gicv3_init_v2();
>
> spin_lock_init(&gicv3.lock);
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index d10757a..ebcca22 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -57,18 +57,21 @@ static struct {
> unsigned int nr_rdist_regions;
> const struct rdist_region *regions;
> uint32_t rdist_stride; /* Re-distributor stride */
> + unsigned int intid_bits; /* Number of interrupt ID bits */
> } vgic_v3_hw;
>
> void vgic_v3_setup_hw(paddr_t dbase,
> unsigned int nr_rdist_regions,
> const struct rdist_region *regions,
> - uint32_t rdist_stride)
> + uint32_t rdist_stride,
> + unsigned int intid_bits)
> {
> vgic_v3_hw.enabled = 1;
> vgic_v3_hw.dbase = dbase;
> vgic_v3_hw.nr_rdist_regions = nr_rdist_regions;
> vgic_v3_hw.regions = regions;
> vgic_v3_hw.rdist_stride = rdist_stride;
> + vgic_v3_hw.intid_bits = intid_bits;
> }
>
> static struct vcpu *vgic_v3_irouter_to_vcpu(struct domain *d, uint64_t irouter)
> @@ -1482,6 +1485,8 @@ static int vgic_v3_domain_init(struct domain *d)
>
> first_cpu += size / d->arch.vgic.rdist_stride;
> }
> +
> + d->arch.vgic.intid_bits = vgic_v3_hw.intid_bits;
So intid_bits is based on what the hardware report, however Xen may not
use ITS. So I would have expected this number to be derived from what is
actually exposed to the domain. For DOM0 it would be the number of LPIs
supported in Xen (e.g lpi_data.max_host_lpi_ids) if ITS is enabled.
Looking at the code (patch #12), you are only using intid_bits when ITS
is enabled which is really surprising as this value should be correct
even though LPIs are not used by DOM0.
> }
> else
> {
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 6de8082..7c3829d 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -111,6 +111,7 @@ struct arch_domain
> uint32_t rdist_stride; /* Re-Distributor stride */
> struct rb_root its_devices; /* Devices mapped to an ITS */
> spinlock_t its_devices_lock; /* Protects the its_devices tree */
> + unsigned int intid_bits;
> #endif
> } vgic;
>
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 544867a..df75064 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -346,7 +346,8 @@ struct rdist_region;
> void vgic_v3_setup_hw(paddr_t dbase,
> unsigned int nr_rdist_regions,
> const struct rdist_region *regions,
> - uint32_t rdist_stride);
> + uint32_t rdist_stride,
> + unsigned int intid_bits);
> #endif
>
> #endif /* __ASM_ARM_VGIC_H__ */
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 02/27] ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock
2017-04-12 0:44 ` [PATCH v8 02/27] ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock Andre Przywara
@ 2017-04-12 10:13 ` Julien Grall
2017-04-12 11:38 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 10:13 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> So far irq_to_pending() is just a convenience function to lookup
> in statically allocated arrays. This will change with LPIs, which are
> more dynamic.
> So move the irq_to_pending() call under the VGIC VCPU lock, so we
> only use this pointer while holding the lock.
That's a call for an ASSERT in irq_to_pending. And you would have notice
that not all irq_to_pending will then be protected by the vGIC lock (see
vgic_migrate_irq for instance).
Also, please explain why the vgic lock as technically irq_to_pending is
lock agnostic... And LPI structure will be per domain. So how do you
expect the locking to work?
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic.c | 5 ++++-
> xen/arch/arm/vgic.c | 8 +++++---
> 2 files changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index da19130..dcb1783 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -402,10 +402,13 @@ static inline void gic_add_to_lr_pending(struct vcpu *v, struct pending_irq *n)
>
> void gic_remove_from_queues(struct vcpu *v, unsigned int virtual_irq)
> {
> - struct pending_irq *p = irq_to_pending(v, virtual_irq);
> + struct pending_irq *p;
> unsigned long flags;
>
> spin_lock_irqsave(&v->arch.vgic.lock, flags);
> +
> + p = irq_to_pending(v, virtual_irq);
> +
> if ( !list_empty(&p->lr_queue) )
> list_del_init(&p->lr_queue);
> spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 83569b0..c953f13 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -466,14 +466,16 @@ void vgic_clear_pending_irqs(struct vcpu *v)
> void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
> {
> uint8_t priority;
> - struct pending_irq *iter, *n = irq_to_pending(v, virq);
> + struct pending_irq *iter, *n;
> unsigned long flags;
> bool running;
>
> - priority = vgic_get_virq_priority(v, virq);;
> -
> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>
> + n = irq_to_pending(v, virq);
> +
> + priority = vgic_get_virq_priority(v, virq);
The commit message only speak about moving "irq_to_pending. So why the
priority has been moved to?
This will introduce a lock ordering issue (rank lock vs vgic lock)
between vgic_vcpu_inject_irq and ITARGET/IROUTER emulation.
The former is taking vgic lock first and then rank whilst the latter is
doing the invert (see vgic_migrate_irq).
> +
> /* vcpu offline */
> if ( test_bit(_VPF_down, &v->pause_flags) )
> {
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's
2017-04-12 0:44 ` [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's Andre Przywara
@ 2017-04-12 10:25 ` Julien Grall
2017-04-12 14:51 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 10:25 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> For LPIs the struct pending_irq's are dynamically allocated and the
> pointers will be stored in a radix tree. Since an LPI can be "unmapped"
> at any time, teach the VGIC how to handle with irq_to_pending() returning
> a NULL pointer.
> We just do nothing in this case or clean up the LR if the virtual LPI
> number was still in an LR.
Why not all the irq_to_pending call are not protected (such as
vgic_irq_to_migrate)? This is a call to forget to check NULL if we
decide to use the function in the future.
Also, I would like a comment on top of irq_to_pending stating this can
return NULL as you change the semantic of the function.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic.c | 37 ++++++++++++++++++++++++++++++++-----
> xen/arch/arm/vgic.c | 6 ++++++
> 2 files changed, 38 insertions(+), 5 deletions(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index dcb1783..62ae3b8 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -408,9 +408,15 @@ void gic_remove_from_queues(struct vcpu *v, unsigned int virtual_irq)
> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>
> p = irq_to_pending(v, virtual_irq);
> -
> - if ( !list_empty(&p->lr_queue) )
> - list_del_init(&p->lr_queue);
> + /*
> + * If an LPIs has been removed meanwhile, it has been cleaned up
> + * already, so nothing to remove here.
> + */
> + if ( p )
> + {
> + if ( !list_empty(&p->lr_queue) )
> + list_del_init(&p->lr_queue);
> + }
This could be simplified with:
if ( p && !list_empty(&p->lr_queue) )
list_del_init(...);
Also, you probably want a likely(p).
> spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> }
>
> @@ -418,6 +424,10 @@ void gic_raise_inflight_irq(struct vcpu *v, unsigned int virtual_irq)
> {
> struct pending_irq *n = irq_to_pending(v, virtual_irq);
>
> + /* If an LPI has been removed meanwhile, there is nothing left to raise. */
> + if ( !n )
if ( unlikely(!n) )
> + return;
> +
> ASSERT(spin_is_locked(&v->arch.vgic.lock));
>
> if ( list_empty(&n->lr_queue) )
> @@ -437,6 +447,11 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
> {
> int i;
> unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
> + struct pending_irq *p = irq_to_pending(v, virtual_irq);
> +
> + if ( !p )
if ( unlikely(!p) )
> + /* An unmapped LPI does not need to be raised. */
> + return;
Please move this check after the ASSERT to keep the check on all the paths.
>
> ASSERT(spin_is_locked(&v->arch.vgic.lock));
>
> @@ -445,12 +460,12 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned int virtual_irq,
> i = find_first_zero_bit(&this_cpu(lr_mask), nr_lrs);
> if (i < nr_lrs) {
> set_bit(i, &this_cpu(lr_mask));
> - gic_set_lr(i, irq_to_pending(v, virtual_irq), GICH_LR_PENDING);
> + gic_set_lr(i, p, GICH_LR_PENDING);
> return;
> }
> }
>
> - gic_add_to_lr_pending(v, irq_to_pending(v, virtual_irq));
> + gic_add_to_lr_pending(v, p);
> }
>
> static void gic_update_one_lr(struct vcpu *v, int i)
> @@ -464,7 +479,19 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>
> gic_hw_ops->read_lr(i, &lr_val);
> irq = lr_val.virq;
> +
4th time I am saying this.... Spurious line.
> p = irq_to_pending(v, irq);
> + /* An LPI might have been unmapped, in which case we just clean up here. */
> + if ( !p )
unlikely(!p)
> + {
> + ASSERT(is_lpi(irq));
> +
> + gic_hw_ops->clear_lr(i);
> + clear_bit(i, &this_cpu(lr_mask));
> +
> + return;
> + }
> +
> if ( lr_val.state & GICH_LR_ACTIVE )
> {
> set_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index c953f13..b9fc837 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -473,6 +473,12 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>
> n = irq_to_pending(v, virq);
> + /* If an LPI has been removed, there is nothing to inject here. */
> + if ( !n )
unlikely(...)
> + {
> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> + return;
> + }
>
> priority = vgic_get_virq_priority(v, virq);
>
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 04/27] ARM: GICv3: introduce separate pending_irq structs for LPIs
2017-04-12 0:44 ` [PATCH v8 04/27] ARM: GICv3: introduce separate pending_irq structs for LPIs Andre Przywara
@ 2017-04-12 10:35 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 10:35 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> For the same reason that allocating a struct irq_desc for each
> possible LPI is not an option, having a struct pending_irq for each LPI
> is also not feasible. We only care about mapped LPIs, so we can get away
> with having struct pending_irq's only for them.
> Maintain a radix tree per domain where we drop the pointer to the
> respective pending_irq. The index used is the virtual LPI number.
> The memory for the actual structures has been allocated already per
> device at device mapping time.
> Teach the existing VGIC functions to find the right pointer when being
> given a virtual LPI number.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v2.c | 8 ++++++++
> xen/arch/arm/vgic-v3.c | 23 +++++++++++++++++++++++
> xen/arch/arm/vgic.c | 2 ++
> xen/include/asm-arm/domain.h | 2 ++
> xen/include/asm-arm/vgic.h | 2 ++
> 5 files changed, 37 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index dc9f95b..0587569 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -702,10 +702,18 @@ static void vgic_v2_domain_free(struct domain *d)
> /* Nothing to be cleanup for this driver */
> }
>
> +static struct pending_irq *vgic_v2_lpi_to_pending(struct domain *d,
> + unsigned int vlpi)
> +{
> + /* Dummy function, no LPIs on a VGICv2. */
> + BUG();
> +}
> +
> static const struct vgic_ops vgic_v2_ops = {
> .vcpu_init = vgic_v2_vcpu_init,
> .domain_init = vgic_v2_domain_init,
> .domain_free = vgic_v2_domain_free,
> + .lpi_to_pending = vgic_v2_lpi_to_pending,
> .max_vcpus = 8,
> };
>
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index ebcca22..f462610 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1454,6 +1454,9 @@ static int vgic_v3_domain_init(struct domain *d)
> d->arch.vgic.nr_regions = rdist_count;
> d->arch.vgic.rdist_regions = rdist_regions;
>
> + rwlock_init(&d->arch.vgic.pend_lpi_tree_lock);
> + radix_tree_init(&d->arch.vgic.pend_lpi_tree);
> +
> /*
> * Domain 0 gets the hardware address.
> * Guests get the virtual platform layout.
> @@ -1533,14 +1536,34 @@ static int vgic_v3_domain_init(struct domain *d)
> static void vgic_v3_domain_free(struct domain *d)
> {
> vgic_v3_its_free_domain(d);
> + radix_tree_destroy(&d->arch.vgic.pend_lpi_tree, NULL);
The question from Stefano was a call for more documentation in the code
who will free the pending_irqs...
> xfree(d->arch.vgic.rdist_regions);
> }
>
> +/*
> + * Looks up a virtual LPI number in our tree of mapped LPIs. This will return
> + * the corresponding struct pending_irq, which we also use to store the
> + * enabled and pending bit plus the priority.
> + * Returns NULL if an LPI cannot be found (or no LPIs are supported).
> + */
> +static struct pending_irq *vgic_v3_lpi_to_pending(struct domain *d,
> + unsigned int lpi)
> +{
> + struct pending_irq *pirq;
> +
> + read_lock(&d->arch.vgic.pend_lpi_tree_lock);
> + pirq = radix_tree_lookup(&d->arch.vgic.pend_lpi_tree, lpi);
> + read_unlock(&d->arch.vgic.pend_lpi_tree_lock);
> +
> + return pirq;
> +}
> +
> static const struct vgic_ops v3_ops = {
> .vcpu_init = vgic_v3_vcpu_init,
> .domain_init = vgic_v3_domain_init,
> .domain_free = vgic_v3_domain_free,
> .emulate_reg = vgic_v3_emulate_reg,
> + .lpi_to_pending = vgic_v3_lpi_to_pending,
> /*
> * We use both AFF1 and AFF0 in (v)MPIDR. Thus, the max number of CPU
> * that can be supported is up to 4096(==256*16) in theory.
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index b9fc837..c2bfdb1 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -439,6 +439,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
> * are used for SPIs; the rests are used for per cpu irqs */
> if ( irq < 32 )
> n = &v->arch.vgic.pending_irqs[irq];
> + else if ( is_lpi(irq) )
> + n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
> else
> n = &v->domain->arch.vgic.pending_irqs[irq - 32];
> return n;
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 7c3829d..3d8e84c 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -111,6 +111,8 @@ struct arch_domain
> uint32_t rdist_stride; /* Re-Distributor stride */
> struct rb_root its_devices; /* Devices mapped to an ITS */
> spinlock_t its_devices_lock; /* Protects the its_devices tree */
> + struct radix_tree_root pend_lpi_tree; /* Stores struct pending_irq's */
> + rwlock_t pend_lpi_tree_lock; /* Protects the pend_lpi_tree */
> unsigned int intid_bits;
> #endif
> } vgic;
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index df75064..c9075a9 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -134,6 +134,8 @@ struct vgic_ops {
> void (*domain_free)(struct domain *d);
> /* vGIC sysreg/cpregs emulate */
> bool (*emulate_reg)(struct cpu_user_regs *regs, union hsr hsr);
> + /* lookup the struct pending_irq for a given LPI interrupt */
> + struct pending_irq *(*lpi_to_pending)(struct domain *d, unsigned int vlpi);
> /* Maximum number of vCPU supported */
> const unsigned int max_vcpus;
> };
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-04-12 0:44 ` [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests Andre Przywara
@ 2017-04-12 10:44 ` Julien Grall
2017-04-12 17:26 ` Andre Przywara
2017-05-10 10:47 ` Andre Przywara
0 siblings, 2 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 10:44 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> Upon receiving an LPI on the host, we need to find the right VCPU and
> virtual IRQ number to get this IRQ injected.
> Iterate our two-level LPI table to find this information quickly when
> the host takes an LPI. Call the existing injection function to let the
> GIC emulation deal with this interrupt.
> Also we enhance struct pending_irq to cache the pending bit and the
> priority information for LPIs. Reading the information from there is
> faster than accessing the property table from guest memory. Also it
> use some padding area, so does not require more memory.
> This introduces a do_LPI() as a hardware gic_ops and a function to
> retrieve the (cached) priority value of an LPI and a vgic_ops.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic-v2.c | 7 ++++
> xen/arch/arm/gic-v3-lpi.c | 71 ++++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/gic-v3.c | 1 +
> xen/arch/arm/gic.c | 8 ++++-
> xen/arch/arm/vgic-v2.c | 7 ++++
> xen/arch/arm/vgic-v3.c | 12 +++++++
> xen/arch/arm/vgic.c | 7 +++-
> xen/include/asm-arm/domain.h | 3 +-
> xen/include/asm-arm/gic.h | 2 ++
> xen/include/asm-arm/gic_v3_its.h | 8 +++++
> xen/include/asm-arm/vgic.h | 2 ++
> 11 files changed, 125 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 270a136..ffbe47c 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -1217,6 +1217,12 @@ static int __init gicv2_init(void)
> return 0;
> }
>
> +static void gicv2_do_LPI(unsigned int lpi)
> +{
> + /* No LPIs in a GICv2 */
> + BUG();
> +}
> +
> const static struct gic_hw_operations gicv2_ops = {
> .info = &gicv2_info,
> .init = gicv2_init,
> @@ -1244,6 +1250,7 @@ const static struct gic_hw_operations gicv2_ops = {
> .make_hwdom_madt = gicv2_make_hwdom_madt,
> .map_hwdom_extra_mappings = gicv2_map_hwdown_extra_mappings,
> .iomem_deny_access = gicv2_iomem_deny_access,
> + .do_LPI = gicv2_do_LPI,
> };
>
> /* Set up the GIC */
> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
> index 292f2d0..44f6315 100644
> --- a/xen/arch/arm/gic-v3-lpi.c
> +++ b/xen/arch/arm/gic-v3-lpi.c
> @@ -136,6 +136,77 @@ uint64_t gicv3_get_redist_address(unsigned int cpu, bool use_pta)
> return per_cpu(lpi_redist, cpu).redist_id << 16;
> }
>
> +/*
> + * Handle incoming LPIs, which are a bit special, because they are potentially
> + * numerous and also only get injected into guests. Treat them specially here,
> + * by just looking up their target vCPU and virtual LPI number and hand it
> + * over to the injection function.
> + * Please note that LPIs are edge-triggered only, also have no active state,
> + * so spurious interrupts on the host side are no issue (we can just ignore
> + * them).
> + * Also a guest cannot expect that firing interrupts that haven't been
> + * fully configured yet will reach the CPU, so we don't need to care about
> + * this special case.
> + */
> +void gicv3_do_LPI(unsigned int lpi)
> +{
> + struct domain *d;
> + union host_lpi *hlpip, hlpi;
> + struct vcpu *vcpu;
> +
> + irq_enter();
> +
> + /* EOI the LPI already. */
> + WRITE_SYSREG32(lpi, ICC_EOIR1_EL1);
> +
> + /* Find out if a guest mapped something to this physical LPI. */
> + hlpip = gic_get_host_lpi(lpi);
> + if ( !hlpip )
> + goto out;
> +
> + hlpi.data = read_u64_atomic(&hlpip->data);
> +
> + /*
> + * Unmapped events are marked with an invalid LPI ID. We can safely
> + * ignore them, as they have no further state and no-one can expect
> + * to see them if they have not been mapped.
> + */
> + if ( hlpi.virt_lpi == INVALID_LPI )
> + goto out;
> +
> + d = rcu_lock_domain_by_id(hlpi.dom_id);
> + if ( !d )
> + goto out;
> +
> + /* Make sure we don't step beyond the vcpu array. */
> + if ( hlpi.vcpu_id >= d->max_vcpus )
> + {
> + rcu_unlock_domain(d);
> + goto out;
> + }
> +
> + vcpu = d->vcpu[hlpi.vcpu_id];
> +
> + /* Check if the VCPU is ready to receive LPIs. */
> + if ( vcpu->arch.vgic.flags & VGIC_V3_LPIS_ENABLED )
> + /*
> + * TODO: Investigate what to do here for potential interrupt storms.
> + * As we keep all host LPIs enabled, for disabling LPIs we would need
> + * to queue a ITS host command, which we avoid so far during a guest's
> + * runtime. Also re-enabling would trigger a host command upon the
> + * guest sending a command, which could be an attack vector for
> + * hogging the host command queue.
> + * See the thread around here for some background:
> + * https://lists.xen.org/archives/html/xen-devel/2016-12/msg00003.html
> + */
This TODO should have been listed in the cover letter.
> + vgic_vcpu_inject_irq(vcpu, hlpi.virt_lpi);
> +
> + rcu_unlock_domain(d);
> +
> +out:
> + irq_exit();
> +}
> +
> static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
> {
> uint64_t val;
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 29c8964..8140c5f 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1674,6 +1674,7 @@ static const struct gic_hw_operations gicv3_ops = {
> .make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
> .make_hwdom_madt = gicv3_make_hwdom_madt,
> .iomem_deny_access = gicv3_iomem_deny_access,
> + .do_LPI = gicv3_do_LPI,
> };
>
> static int __init gicv3_dt_preinit(struct dt_device_node *node, const void *data)
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 62ae3b8..d752352 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -735,7 +735,13 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
> do_IRQ(regs, irq, is_fiq);
> local_irq_disable();
> }
> - else if (unlikely(irq < 16))
> + else if ( is_lpi(irq) )
> + {
> + local_irq_enable();
> + gic_hw_ops->do_LPI(irq);
> + local_irq_disable();
> + }
> + else if ( unlikely(irq < 16) )
> {
> do_sgi(regs, irq);
> }
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index 0587569..df91940 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -709,11 +709,18 @@ static struct pending_irq *vgic_v2_lpi_to_pending(struct domain *d,
> BUG();
> }
>
> +static int vgic_v2_lpi_get_priority(struct domain *d, unsigned int vlpi)
> +{
> + /* Dummy function, no LPIs on a VGICv2. */
> + BUG();
> +}
> +
> static const struct vgic_ops vgic_v2_ops = {
> .vcpu_init = vgic_v2_vcpu_init,
> .domain_init = vgic_v2_domain_init,
> .domain_free = vgic_v2_domain_free,
> .lpi_to_pending = vgic_v2_lpi_to_pending,
> + .lpi_get_priority = vgic_v2_lpi_get_priority,
> .max_vcpus = 8,
> };
>
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index f462610..c059dbd 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1558,12 +1558,24 @@ static struct pending_irq *vgic_v3_lpi_to_pending(struct domain *d,
> return pirq;
> }
>
> +/* Retrieve the priority of an LPI from its struct pending_irq. */
> +static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
> +{
> + struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
> +
> + if ( !p )
> + return GIC_PRI_IRQ;
Why the check here? And why returning GIC_PRI_IRQ?
AFAICT, vgic_v3_lpi_to_pending should never return NULL when reading the
priority. Or else, you are in big trouble.
> +
> + return p->lpi_priority;
> +}
> +
> static const struct vgic_ops v3_ops = {
> .vcpu_init = vgic_v3_vcpu_init,
> .domain_init = vgic_v3_domain_init,
> .domain_free = vgic_v3_domain_free,
> .emulate_reg = vgic_v3_emulate_reg,
> .lpi_to_pending = vgic_v3_lpi_to_pending,
> + .lpi_get_priority = vgic_v3_lpi_get_priority,
> /*
> * We use both AFF1 and AFF0 in (v)MPIDR. Thus, the max number of CPU
> * that can be supported is up to 4096(==256*16) in theory.
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index c2bfdb1..b6fe34f 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -226,10 +226,15 @@ struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq)
>
> static int vgic_get_virq_priority(struct vcpu *v, unsigned int virq)
> {
> - struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
> + struct vgic_irq_rank *rank;
> unsigned long flags;
> int priority;
>
> + /* LPIs don't have a rank, also store their priority separately. */
> + if ( is_lpi(virq) )
> + return v->domain->arch.vgic.handler->lpi_get_priority(v->domain, virq);
> +
> + rank = vgic_rank_irq(v, virq);
> vgic_lock_rank(v, rank, flags);
> priority = rank->priority[virq & INTERRUPT_RANK_MASK];
> vgic_unlock_rank(v, rank, flags);
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 3d8e84c..ebaea35 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -260,7 +260,8 @@ struct arch_vcpu
>
> /* GICv3: redistributor base and flags for this vCPU */
> paddr_t rdist_base;
> -#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
> +#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
> +#define VGIC_V3_LPIS_ENABLED (1 << 1)
> uint8_t flags;
> } vgic;
>
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 836a103..42963c0 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -366,6 +366,8 @@ struct gic_hw_operations {
> int (*map_hwdom_extra_mappings)(struct domain *d);
> /* Deny access to GIC regions */
> int (*iomem_deny_access)(const struct domain *d);
> + /* Handle LPIs, which require special handling */
> + void (*do_LPI)(unsigned int lpi);
> };
>
> void register_gic_ops(const struct gic_hw_operations *ops);
> diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
> index 29559a3..7470779 100644
> --- a/xen/include/asm-arm/gic_v3_its.h
> +++ b/xen/include/asm-arm/gic_v3_its.h
> @@ -134,6 +134,8 @@ void gicv3_its_dt_init(const struct dt_device_node *node);
>
> bool gicv3_its_host_has_its(void);
>
> +void gicv3_do_LPI(unsigned int lpi);
> +
> int gicv3_lpi_init_rdist(void __iomem * rdist_base);
>
> /* Initialize the host structures for LPIs and the host ITSes. */
> @@ -175,6 +177,12 @@ static inline bool gicv3_its_host_has_its(void)
> return false;
> }
>
> +static inline void gicv3_do_LPI(unsigned int lpi)
> +{
> + /* We don't enable LPIs without an ITS. */
> + BUG();
> +}
> +
> static inline int gicv3_lpi_init_rdist(void __iomem * rdist_base)
> {
> return -ENODEV;
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index c9075a9..7efa164 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -72,6 +72,7 @@ struct pending_irq
> #define GIC_INVALID_LR (uint8_t)~0
> uint8_t lr;
> uint8_t priority;
> + uint8_t lpi_priority; /* Caches the priority if this is an LPI. */
> /* inflight is used to append instances of pending_irq to
> * vgic.inflight_irqs */
> struct list_head inflight;
> @@ -136,6 +137,7 @@ struct vgic_ops {
> bool (*emulate_reg)(struct cpu_user_regs *regs, union hsr hsr);
> /* lookup the struct pending_irq for a given LPI interrupt */
> struct pending_irq *(*lpi_to_pending)(struct domain *d, unsigned int vlpi);
> + int (*lpi_get_priority)(struct domain *d, uint32_t vlpi);
> /* Maximum number of vCPU supported */
> const unsigned int max_vcpus;
> };
>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables
2017-04-12 0:44 ` [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables Andre Przywara
@ 2017-04-12 10:55 ` Julien Grall
2017-04-12 13:12 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 10:55 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> Allow a guest to provide the address and size for the memory regions
> it has reserved for the GICv3 pending and property tables.
> We sanitise the various fields of the respective redistributor
> registers.
> The MMIO read and write accesses are protected by locks, to avoid any
> changing of the property or pending table address while a redistributor
> is live and also to protect the non-atomic vgic_reg64_extract() function
> on the MMIO read side.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> Reviewed-by: Julien Grall <julien.grall@arm.com>
Whilst I gave my reviewed-by it is very rude to ignore a comment.
It would have been nicer to answer even if it is just saying "I can add
a TODO and address it in a follow-up patch".
> ---
Please get use to mention all the changes (e.g the spin_*lock ->
spin_*lock_irq* change) you made in a patch. Mainly if you keep a
reviewed-by.
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index ebaea35..b2d98bb 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -109,11 +109,15 @@ struct arch_domain
> } *rdist_regions;
> int nr_regions; /* Number of rdist regions */
> uint32_t rdist_stride; /* Re-Distributor stride */
> + unsigned long int nr_lpis;
> + uint64_t rdist_propbase;
> struct rb_root its_devices; /* Devices mapped to an ITS */
> spinlock_t its_devices_lock; /* Protects the its_devices tree */
> struct radix_tree_root pend_lpi_tree; /* Stores struct pending_irq's */
> rwlock_t pend_lpi_tree_lock; /* Protects the pend_lpi_tree */
> unsigned int intid_bits;
> + bool rdists_enabled; /* Is any redistributor enabled? */
> + bool has_its;
The comment you ignore was the one about consolidating rdists_enabled
and has_its in a single field and use flags.
> #endif
> } vgic;
>
> @@ -260,6 +264,7 @@ struct arch_vcpu
>
> /* GICv3: redistributor base and flags for this vCPU */
> paddr_t rdist_base;
> + uint64_t rdist_pendbase;
> #define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
> #define VGIC_V3_LPIS_ENABLED (1 << 1)
> uint8_t flags;
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 02/27] ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock
2017-04-12 10:13 ` Julien Grall
@ 2017-04-12 11:38 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 11:38 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 11:13, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> So far irq_to_pending() is just a convenience function to lookup
>> in statically allocated arrays. This will change with LPIs, which are
>> more dynamic.
>> So move the irq_to_pending() call under the VGIC VCPU lock, so we
>> only use this pointer while holding the lock.
>
> That's a call for an ASSERT in irq_to_pending. And you would have notice
> that not all irq_to_pending will then be protected by the vGIC lock (see
> vgic_migrate_irq for instance).
>
> Also, please explain why the vgic lock as technically irq_to_pending is
> lock agnostic... And LPI structure will be per domain. So how do you
> expect the locking to work?
I thought a bit more about this and I think vgic_vcpu_inject_irq will
not be protected correctly for LPI.
Unlike SPIs, LPIs don't have active state in the GIC so theoretically a
new interrupt can come up right after handling the first one. Although,
it might have been possible that the vLPI was moved from vCPU A to vCPU
B and still in the LRs (not yet handled by the guest or not yet cleaned).
So there is a race between gic_update_one_lr and vgic_vcpu_inject, both
will take a different lock (resp. old and new) which may lead to a list
corruption.
I am not sure how to protect this case as this could happen because of a
"DISCARD -> MAPTI", "MOVI", "MOVALL"
>
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/gic.c | 5 ++++-
>> xen/arch/arm/vgic.c | 8 +++++---
>> 2 files changed, 9 insertions(+), 4 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index da19130..dcb1783 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -402,10 +402,13 @@ static inline void gic_add_to_lr_pending(struct
>> vcpu *v, struct pending_irq *n)
>>
>> void gic_remove_from_queues(struct vcpu *v, unsigned int virtual_irq)
>> {
>> - struct pending_irq *p = irq_to_pending(v, virtual_irq);
>> + struct pending_irq *p;
>> unsigned long flags;
>>
>> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>> +
>> + p = irq_to_pending(v, virtual_irq);
>> +
>> if ( !list_empty(&p->lr_queue) )
>> list_del_init(&p->lr_queue);
>> spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index 83569b0..c953f13 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -466,14 +466,16 @@ void vgic_clear_pending_irqs(struct vcpu *v)
>> void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
>> {
>> uint8_t priority;
>> - struct pending_irq *iter, *n = irq_to_pending(v, virq);
>> + struct pending_irq *iter, *n;
>> unsigned long flags;
>> bool running;
>>
>> - priority = vgic_get_virq_priority(v, virq);;
>
>> -
>> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>
>> + n = irq_to_pending(v, virq);
>> +
>> + priority = vgic_get_virq_priority(v, virq);
>
> The commit message only speak about moving "irq_to_pending. So why the
> priority has been moved to?
>
> This will introduce a lock ordering issue (rank lock vs vgic lock)
> between vgic_vcpu_inject_irq and ITARGET/IROUTER emulation.
>
> The former is taking vgic lock first and then rank whilst the latter is
> doing the invert (see vgic_migrate_irq).
>
>> +
>> /* vcpu offline */
>> if ( test_bit(_VPF_down, &v->pause_flags) )
>> {
>>
>
> Cheers,
>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 08/27] ARM: introduce vgic_access_guest_memory()
2017-04-12 0:44 ` [PATCH v8 08/27] ARM: introduce vgic_access_guest_memory() Andre Przywara
@ 2017-04-12 12:29 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 12:29 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> This function allows to copy a chunk of data from and to guest physical
> memory. It looks up the associated page from the guest's p2m tree
> and maps this page temporarily for the time of the access.
> This function was originally written by Vijaya as part of an earlier series:
> https://patchwork.kernel.org/patch/8177251
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/vgic.h | 3 +++
> 2 files changed, 53 insertions(+)
>
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index b6fe34f..e385b43 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -20,6 +20,7 @@
> #include <xen/bitops.h>
> #include <xen/lib.h>
> #include <xen/init.h>
> +#include <xen/domain_page.h>
> #include <xen/softirq.h>
> #include <xen/irq.h>
> #include <xen/sched.h>
> @@ -602,6 +603,55 @@ void vgic_free_virq(struct domain *d, unsigned int virq)
> }
>
> /*
> + * Temporarily map one physical guest page and copy data to or from it.
> + * The data to be copied cannot cross a page boundary.
> + */
> +int vgic_access_guest_memory(struct domain *d, paddr_t gpa, void *buf,
> + uint32_t size, bool_t is_write)
> +{
> + struct page_info *page;
> + uint64_t offset;
> + p2m_type_t p2mt;
> + int ret = 0;
> + void *p;
> +
> + page = get_page_from_gfn(d, paddr_to_pfn(gpa), &p2mt, P2M_ALLOC);
> + if ( !page )
> + {
> + printk(XENLOG_G_ERR "d%d: vITS: Failed to get table entry\n",
> + d->domain_id);
> + return -EINVAL;
> + }
> +
> + if ( !p2m_is_ram(p2mt) )
> + {
> + put_page(page);
> + printk(XENLOG_G_ERR "d%d: vITS: memory used by the ITS should be RAM.",
> + d->domain_id);
> + return -EINVAL;
> + }
> +
> + p = __map_domain_page(page);
> + /* Offset within the mapped page */
> + offset = gpa & ~PAGE_MASK;
> + /* Do not cross a page boundary. */
> + if ( size <= (PAGE_SIZE - offset) )
> + {
> + if ( is_write )
> + memcpy(p + offset, buf, size);
> + else
> + memcpy(buf, p + offset, size);
> + }
> + else
> + ret = -ENOSPC;
Why ENOSPC? The buffer still have space we are just not able to handle
the value, so it should be -EINVAL as I suggested on the previous version.
Also, I don't understand why you didn't move the check at the beginning
of the function. It would make the code more readable. By that I mean:
int vgic_access_....
{
offset = gpa & ~PAGE_MASK;
if ( size > (PAGE_SIZE - offset) )
return -EINVAL;
/* Rest of the code */
}
> +
> + unmap_domain_page(p);
> + put_page(page);
> +
> + return ret;
> +}
> +
> +/*
> * Local variables:
> * mode: C
> * c-file-style: "BSD"
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 7efa164..6b17802 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -313,6 +313,9 @@ extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
> int vgic_v2_init(struct domain *d, int *mmio_count);
> int vgic_v3_init(struct domain *d, int *mmio_count);
>
> +int vgic_access_guest_memory(struct domain *d, paddr_t gpa, void *buf,
> + uint32_t size, bool_t is_write);
> +
> extern int domain_vgic_register(struct domain *d, int *mmio_count);
> extern int vcpu_vgic_free(struct vcpu *v);
> extern bool vgic_to_sgi(struct vcpu *v, register_t sgir,
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 11/27] ARM: VGIC: add vcpu_id to struct pending_irq
2017-04-12 0:44 ` [PATCH v8 11/27] ARM: VGIC: add vcpu_id to struct pending_irq Andre Przywara
@ 2017-04-12 12:32 ` Julien Grall
2017-04-12 12:37 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 12:32 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> The target CPU for an LPI is encoded in the interrupt translation table
> entry, so can't be easily derived from just an LPI number (short of
> walking *all* tables and find the matching LPI).
> To avoid this in case we need to know the VCPU (for the INVALL command,
> for instance), put the VCPU ID in the struct pending_irq, so that it is
> easily accessible.
> We use the remaining 8 bits of padding space for that to avoid enlarging
> the size of struct pending_irq. The number of VCPUs is limited to 127
> at the moment anyway.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/include/asm-arm/vgic.h | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index e2111a5..02732db 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -73,6 +73,7 @@ struct pending_irq
> uint8_t lr;
> uint8_t priority;
> uint8_t lpi_priority; /* Caches the priority if this is an LPI. */
> + uint8_t lpi_vcpu_id; /* The VCPU for an LPI. */
You likely want a BUILD_BUG_ON in the code to check a vCPU ID will fit
in lpi_vcpu_id. See what we did in p2m_init.
> /* inflight is used to append instances of pending_irq to
> * vgic.inflight_irqs */
> struct list_head inflight;
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 11/27] ARM: VGIC: add vcpu_id to struct pending_irq
2017-04-12 12:32 ` Julien Grall
@ 2017-04-12 12:37 ` Andre Przywara
0 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 12:37 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 13:32, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> The target CPU for an LPI is encoded in the interrupt translation table
>> entry, so can't be easily derived from just an LPI number (short of
>> walking *all* tables and find the matching LPI).
>> To avoid this in case we need to know the VCPU (for the INVALL command,
>> for instance), put the VCPU ID in the struct pending_irq, so that it is
>> easily accessible.
>> We use the remaining 8 bits of padding space for that to avoid enlarging
>> the size of struct pending_irq. The number of VCPUs is limited to 127
>> at the moment anyway.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/include/asm-arm/vgic.h | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
>> index e2111a5..02732db 100644
>> --- a/xen/include/asm-arm/vgic.h
>> +++ b/xen/include/asm-arm/vgic.h
>> @@ -73,6 +73,7 @@ struct pending_irq
>> uint8_t lr;
>> uint8_t priority;
>> uint8_t lpi_priority; /* Caches the priority if this is an
>> LPI. */
>> + uint8_t lpi_vcpu_id; /* The VCPU for an LPI. */
>
> You likely want a BUILD_BUG_ON in the code to check a vCPU ID will fit
> in lpi_vcpu_id. See what we did in p2m_init.
Ah, MAX_VIRT_CPUS was the name I was looking for (but couldn't find)
when I had the very same idea yesterday ;-)
Thanks for the pointer!
Cheers,
Andre.
>
>> /* inflight is used to append instances of pending_irq to
>> * vgic.inflight_irqs */
>> struct list_head inflight;
>>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 12/27] ARM: vGIC: advertise LPI support
2017-04-12 0:44 ` [PATCH v8 12/27] ARM: vGIC: advertise LPI support Andre Przywara
@ 2017-04-12 12:38 ` Julien Grall
2017-04-12 12:48 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 12:38 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> To let a guest know about the availability of virtual LPIs, set the
> respective bits in the virtual GIC registers and let a guest control
> the LPI enable bit.
> Only report the LPI capability if the host has initialized at least
> one ITS.
> This removes a "TBD" comment, as we now populate the processor number
> in the GICR_TYPE register.
> Advertise 24 bits worth of LPIs to the guest.
This sentence is not valid anymore...
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 62 insertions(+), 5 deletions(-)
>
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 849a6f3..33cd5f7 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -170,8 +170,19 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> switch ( gicr_reg )
> {
> case VREG32(GICR_CTLR):
> - /* We have not implemented LPI's, read zero */
> - goto read_as_zero_32;
> + {
> + unsigned long flags;
> +
> + if ( !v->domain->arch.vgic.has_its )
> + goto read_as_zero_32;
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> +
> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
> + *r = vgic_reg32_extract(!!(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED),
> + info);
> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> + return 1;
> + }
>
> case VREG32(GICR_IIDR):
> if ( dabt.size != DABT_WORD ) goto bad_width;
> @@ -183,16 +194,20 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> uint64_t typer, aff;
>
> if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> - /* TBD: Update processor id in [23:8] when ITS support is added */
> aff = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 56 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
> typer = aff;
> + /* We use the VCPU ID as the redistributor ID in bits[23:8] */
> + typer |= (v->vcpu_id & 0xffff) << 8;
>
> if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
> typer |= GICR_TYPER_LAST;
>
> + if ( v->domain->arch.vgic.has_its )
> + typer |= GICR_TYPER_PLPIS;
> +
> *r = vgic_reg64_extract(typer, info);
>
> return 1;
> @@ -426,6 +441,25 @@ static uint64_t sanitize_pendbaser(uint64_t reg)
> return reg;
> }
>
> +static void vgic_vcpu_enable_lpis(struct vcpu *v)
> +{
> + uint64_t reg = v->domain->arch.vgic.rdist_propbase;
> + unsigned int nr_lpis = BIT((reg & 0x1f) + 1);
> +
> + if ( nr_lpis < LPI_OFFSET )
> + nr_lpis = 0;
> + else
> + nr_lpis -= LPI_OFFSET;
> +
> + if ( !v->domain->arch.vgic.rdists_enabled )
> + {
> + v->domain->arch.vgic.nr_lpis = nr_lpis;
> + v->domain->arch.vgic.rdists_enabled = true;
> + }
> +
> + v->arch.vgic.flags |= VGIC_V3_LPIS_ENABLED;
> +}
> +
> static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> uint32_t gicr_reg,
> register_t r)
> @@ -436,8 +470,26 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> switch ( gicr_reg )
> {
> case VREG32(GICR_CTLR):
> - /* LPI's not implemented */
> - goto write_ignore_32;
> + {
> + unsigned long flags;
> +
> + if ( !v->domain->arch.vgic.has_its )
> + goto write_ignore_32;
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> +
> + vgic_lock(v); /* protects rdists_enabled */
Again, can't you move this lock in vgic_vcpu_enable_lpis?
> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
> +
> + /* LPIs can only be enabled once, but never disabled again. */
> + if ( (r & GICR_CTLR_ENABLE_LPIS) &&
> + !(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED) )
> + vgic_vcpu_enable_lpis(v);
> +
> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> + vgic_unlock(v);
> +
> + return 1;
> + }
>
> case VREG32(GICR_IIDR):
> /* RO */
> @@ -1058,6 +1110,11 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
> typer = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
>
> + if ( v->domain->arch.vgic.has_its )
> + {
> + typer |= GICD_TYPE_LPIS;
> + irq_bits = v->domain->arch.vgic.intid_bits;
See my remark in patch #1. I would have expected the field intid_bits to
be used even if ITS is not present.
> + }
> typer |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
>
> *r = vgic_reg32_extract(typer, info);
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 12/27] ARM: vGIC: advertise LPI support
2017-04-12 12:38 ` Julien Grall
@ 2017-04-12 12:48 ` Andre Przywara
2017-04-12 13:04 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 12:48 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 13:38, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> To let a guest know about the availability of virtual LPIs, set the
>> respective bits in the virtual GIC registers and let a guest control
>> the LPI enable bit.
>> Only report the LPI capability if the host has initialized at least
>> one ITS.
>> This removes a "TBD" comment, as we now populate the processor number
>> in the GICR_TYPE register.
>> Advertise 24 bits worth of LPIs to the guest.
>
> This sentence is not valid anymore...
Indeed ....
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/vgic-v3.c | 67
>> ++++++++++++++++++++++++++++++++++++++++++++++----
>> 1 file changed, 62 insertions(+), 5 deletions(-)
>>
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index 849a6f3..33cd5f7 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -170,8 +170,19 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct
>> vcpu *v, mmio_info_t *info,
>> switch ( gicr_reg )
>> {
>> case VREG32(GICR_CTLR):
>> - /* We have not implemented LPI's, read zero */
>> - goto read_as_zero_32;
>> + {
>> + unsigned long flags;
>> +
>> + if ( !v->domain->arch.vgic.has_its )
>> + goto read_as_zero_32;
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> +
>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>> + *r = vgic_reg32_extract(!!(v->arch.vgic.flags &
>> VGIC_V3_LPIS_ENABLED),
>> + info);
>> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>> + return 1;
>> + }
>>
>> case VREG32(GICR_IIDR):
>> if ( dabt.size != DABT_WORD ) goto bad_width;
>> @@ -183,16 +194,20 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct
>> vcpu *v, mmio_info_t *info,
>> uint64_t typer, aff;
>>
>> if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
>> - /* TBD: Update processor id in [23:8] when ITS support is
>> added */
>> aff = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 56 |
>> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
>> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
>> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
>> typer = aff;
>> + /* We use the VCPU ID as the redistributor ID in bits[23:8] */
>> + typer |= (v->vcpu_id & 0xffff) << 8;
>>
>> if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
>> typer |= GICR_TYPER_LAST;
>>
>> + if ( v->domain->arch.vgic.has_its )
>> + typer |= GICR_TYPER_PLPIS;
>> +
>> *r = vgic_reg64_extract(typer, info);
>>
>> return 1;
>> @@ -426,6 +441,25 @@ static uint64_t sanitize_pendbaser(uint64_t reg)
>> return reg;
>> }
>>
>> +static void vgic_vcpu_enable_lpis(struct vcpu *v)
>> +{
>> + uint64_t reg = v->domain->arch.vgic.rdist_propbase;
>> + unsigned int nr_lpis = BIT((reg & 0x1f) + 1);
>> +
>> + if ( nr_lpis < LPI_OFFSET )
>> + nr_lpis = 0;
>> + else
>> + nr_lpis -= LPI_OFFSET;
>> +
>> + if ( !v->domain->arch.vgic.rdists_enabled )
>> + {
>> + v->domain->arch.vgic.nr_lpis = nr_lpis;
>> + v->domain->arch.vgic.rdists_enabled = true;
>> + }
>> +
>> + v->arch.vgic.flags |= VGIC_V3_LPIS_ENABLED;
>> +}
>> +
>> static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t
>> *info,
>> uint32_t gicr_reg,
>> register_t r)
>> @@ -436,8 +470,26 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct
>> vcpu *v, mmio_info_t *info,
>> switch ( gicr_reg )
>> {
>> case VREG32(GICR_CTLR):
>> - /* LPI's not implemented */
>> - goto write_ignore_32;
>> + {
>> + unsigned long flags;
>> +
>> + if ( !v->domain->arch.vgic.has_its )
>> + goto write_ignore_32;
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> +
>> + vgic_lock(v); /* protects rdists_enabled */
>
> Again, can't you move this lock in vgic_vcpu_enable_lpis?
But the domain VGIC lock must be taken before the VGIC VCPU lock, right?
And we need the VGIC VCPU lock before calling the function to check for
LPIS_ENABLED, isn't it?
>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>> +
>> + /* LPIs can only be enabled once, but never disabled again. */
>> + if ( (r & GICR_CTLR_ENABLE_LPIS) &&
>> + !(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED) )
>> + vgic_vcpu_enable_lpis(v);
>> +
>> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>> + vgic_unlock(v);
>> +
>> + return 1;
>> + }
>>
>> case VREG32(GICR_IIDR):
>> /* RO */
>> @@ -1058,6 +1110,11 @@ static int vgic_v3_distr_mmio_read(struct vcpu
>> *v, mmio_info_t *info,
>> typer = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
>> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
>>
>> + if ( v->domain->arch.vgic.has_its )
>> + {
>> + typer |= GICD_TYPE_LPIS;
>> + irq_bits = v->domain->arch.vgic.intid_bits;
>
> See my remark in patch #1. I would have expected the field intid_bits to
> be used even if ITS is not present.
Well, this is just mimicking current code, which just puts the value
covering the number of SPIs in here.
As we only support LPIs with an ITS, I don't see why we should present
any higher value in there than we do today already for a GICv3.
Cheers,
Andre.
>> + }
>> typer |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
>>
>> *r = vgic_reg32_extract(typer, info);
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 13/27] ARM: vITS: add command handling stub and MMIO emulation
2017-04-12 0:44 ` [PATCH v8 13/27] ARM: vITS: add command handling stub and MMIO emulation Andre Przywara
@ 2017-04-12 12:59 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 12:59 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> Emulate the memory mapped ITS registers and provide a stub to introduce
> the ITS command handling framework (but without actually emulating any
> commands at this time).
> This fixes a misnomer in our virtual ITS structure, where the spec is
> confusingly using ID_bits in GITS_TYPER to denote the number of event IDs
> (in contrast to GICD_TYPER, where it means number of LPIs).
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 523 ++++++++++++++++++++++++++++++++++++++-
> xen/include/asm-arm/gic_v3_its.h | 3 +
> 2 files changed, 525 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 065ffe2..a60f9b2 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -19,6 +19,16 @@
> * along with this program; If not, see <http://www.gnu.org/licenses/>.
> */
>
> +/*
> + * Locking order:
> + *
> + * its->vcmd_lock (protects the command queue)
> + * its->its_lock (protects the translation tables)
> + * d->its_devices_lock (protects the device RB tree)
> + * v->vgic.lock (protects the struct pending_irq)
> + * d->pend_lpi_tree_lock (protects the radix tree)
> + */
> +
> #include <xen/bitops.h>
> #include <xen/config.h>
> #include <xen/domain_page.h>
> @@ -43,7 +53,7 @@
> struct virt_its {
> struct domain *d;
> unsigned int devid_bits;
> - unsigned int intid_bits;
> + unsigned int evid_bits;
> spinlock_t vcmd_lock; /* Protects the virtual command buffer, which */
> uint64_t cwriter; /* consists of CWRITER and CREADR and those */
> uint64_t creadr; /* shadow variables cwriter and creadr. */
> @@ -53,6 +63,7 @@ struct virt_its {
> uint64_t baser_dev, baser_coll; /* BASER0 and BASER1 for the guest */
> unsigned int max_collections;
> unsigned int max_devices;
> + /* changing "enabled" requires to hold *both* the vcmd_lock and its_lock */
> bool enabled;
> };
>
> @@ -67,6 +78,9 @@ struct vits_itte
> uint16_t pad;
> };
>
> +#define GITS_BASER_RO_MASK (GITS_BASER_TYPE_MASK | \
> + (31UL << GITS_BASER_ENTRY_SIZE_SHIFT))
> +
> int vgic_v3_its_init_domain(struct domain *d)
> {
> spin_lock_init(&d->arch.vgic.its_devices_lock);
> @@ -80,6 +94,513 @@ void vgic_v3_its_free_domain(struct domain *d)
> ASSERT(RB_EMPTY_ROOT(&d->arch.vgic.its_devices));
> }
>
> +/**************************************
> + * Functions that handle ITS commands *
> + **************************************/
> +
> +static uint64_t its_cmd_mask_field(uint64_t *its_cmd, unsigned int word,
> + unsigned int shift, unsigned int size)
> +{
> + return (le64_to_cpu(its_cmd[word]) >> shift) & (BIT(size) - 1);
> +}
> +
> +#define its_cmd_get_command(cmd) its_cmd_mask_field(cmd, 0, 0, 8)
> +#define its_cmd_get_deviceid(cmd) its_cmd_mask_field(cmd, 0, 32, 32)
> +#define its_cmd_get_size(cmd) its_cmd_mask_field(cmd, 1, 0, 5)
> +#define its_cmd_get_id(cmd) its_cmd_mask_field(cmd, 1, 0, 32)
> +#define its_cmd_get_physical_id(cmd) its_cmd_mask_field(cmd, 1, 32, 32)
> +#define its_cmd_get_collection(cmd) its_cmd_mask_field(cmd, 2, 0, 16)
> +#define its_cmd_get_target_addr(cmd) its_cmd_mask_field(cmd, 2, 16, 32)
> +#define its_cmd_get_validbit(cmd) its_cmd_mask_field(cmd, 2, 63, 1)
> +#define its_cmd_get_ittaddr(cmd) (its_cmd_mask_field(cmd, 2, 8, 44) << 8)
> +
> +#define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> +#define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
> +
> +/*
> + * Must be called with the vcmd_lock held.
> + * TODO: Investigate whether we can be smarter here and don't need to hold
> + * the lock all of the time.
> + */
> +static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> +{
> + paddr_t addr = its->cbaser & GENMASK(51, 12);
> + uint64_t command[4];
> +
> + ASSERT(spin_is_locked(&its->vcmd_lock));
> +
> + if ( its->cwriter >= ITS_CMD_BUFFER_SIZE(its->cbaser) )
> + return -1;
> +
> + while ( its->creadr != its->cwriter )
> + {
> + int ret;
> +
> + ret = vgic_access_guest_memory(d, addr + its->creadr,
> + command, sizeof(command), false);
> + if ( ret )
> + return ret;
> +
> + switch ( its_cmd_get_command(command) )
> + {
> + case GITS_CMD_SYNC:
> + /* We handle ITS commands synchronously, so we ignore SYNC. */
> + break;
> + default:
> + gdprintk(XENLOG_WARNING, "ITS: unhandled ITS command %lu\n",
> + its_cmd_get_command(command));
Please stay consistent with the naming. It should be "vGITS:".
> + break;
> + }
> +
> + write_u64_atomic(&its->creadr, (its->creadr + ITS_CMD_SIZE) %
> + ITS_CMD_BUFFER_SIZE(its->cbaser));
> +
> + if ( ret )
> + gdprintk(XENLOG_WARNING,
> + "ITS: ITS command error %d while handling command %lu\n",
Same here: vGITS.
> + ret, its_cmd_get_command(command));
> + }
> +
> + return 0;
> +}
> +
> +/*****************************
> + * ITS registers read access *
> + *****************************/
> +
> +/* Identifying as an ARM IP, using "X" as the product ID. */
> +#define GITS_IIDR_VALUE 0x5800034c
> +
> +static int vgic_v3_its_mmio_read(struct vcpu *v, mmio_info_t *info,
> + register_t *r, void *priv)
> +{
> + struct virt_its *its = priv;
> + uint64_t reg;
> +
> + switch ( info->gpa & 0xffff )
> + {
> + case VREG32(GITS_CTLR):
> + {
> + /*
> + * We try to avoid waiting for the command queue lock and report
> + * non-quiescent if that lock is already taken.
> + */
> + bool have_cmd_lock;
> +
> + if ( info->dabt.size != DABT_WORD ) goto bad_width;
> +
> + have_cmd_lock = spin_trylock(&its->vcmd_lock);
> + spin_lock(&its->its_lock);
> + if ( its->enabled )
> + reg = GITS_CTLR_ENABLE;
> + else
> + reg = 0;
> +
> + if ( have_cmd_lock && its->cwriter == its->creadr )
> + reg |= GITS_CTLR_QUIESCENT;
> +
> + spin_unlock(&its->its_lock);
> + if ( have_cmd_lock )
> + spin_unlock(&its->vcmd_lock);
> +
> + *r = vgic_reg32_extract(reg, info);
> + break;
> + }
> + case VREG32(GITS_IIDR):
> + if ( info->dabt.size != DABT_WORD ) goto bad_width;
> + *r = vgic_reg32_extract(GITS_IIDR_VALUE, info);
> + break;
> + case VREG64(GITS_TYPER):
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> +
> + reg = GITS_TYPER_PHYSICAL;
> + reg |= (sizeof(struct vits_itte) - 1) << GITS_TYPER_ITT_SIZE_SHIFT;
> + reg |= (its->evid_bits - 1) << GITS_TYPER_IDBITS_SHIFT;
> + reg |= (its->devid_bits - 1) << GITS_TYPER_DEVIDS_SHIFT;
> + *r = vgic_reg64_extract(reg, info);
> + break;
> + case VRANGE32(0x0018, 0x001C):
> + goto read_reserved;
> + case VRANGE32(0x0020, 0x003C):
> + goto read_impl_defined;
> + case VRANGE32(0x0040, 0x007C):
> + goto read_reserved;
> + case VREG64(GITS_CBASER):
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> + spin_lock(&its->its_lock);
> + *r = vgic_reg64_extract(its->cbaser, info);
> + spin_unlock(&its->its_lock);
> + break;
> + case VREG64(GITS_CWRITER):
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> +
> + reg = its->cwriter;
> + *r = vgic_reg64_extract(reg, info);
> + break;
> + case VREG64(GITS_CREADR):
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> +
> + reg = its->creadr;
> + *r = vgic_reg64_extract(reg, info);
> + break;
> + case VRANGE64(0x0098, 0x00F8):
> + goto read_reserved;
> + case VREG64(GITS_BASER0): /* device table */
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> + spin_lock(&its->its_lock);
> + *r = vgic_reg64_extract(its->baser_dev, info);
> + spin_unlock(&its->its_lock);
> + break;
> + case VREG64(GITS_BASER1): /* collection table */
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> + spin_lock(&its->its_lock);
> + *r = vgic_reg64_extract(its->baser_coll, info);
> + spin_unlock(&its->its_lock);
> + break;
> + case VRANGE64(GITS_BASER2, GITS_BASER7):
> + goto read_as_zero_64;
> + case VRANGE32(0x0140, 0xBFFC):
> + goto read_reserved;
> + case VRANGE32(0xC000, 0xFFCC):
> + goto read_impl_defined;
> + case VRANGE32(0xFFD0, 0xFFE4):
> + goto read_impl_defined;
> + case VREG32(GITS_PIDR2):
> + if ( info->dabt.size != DABT_WORD ) goto bad_width;
> + *r = vgic_reg32_extract(GIC_PIDR2_ARCH_GICv3, info);
> + break;
> + case VRANGE32(0xFFEC, 0xFFFC):
> + goto read_impl_defined;
> + default:
> + printk(XENLOG_G_ERR
> + "%pv: vGITS: unhandled read r%d offset %#04lx\n",
> + v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> + return 0;
> + }
> +
> + return 1;
> +
> +read_as_zero_64:
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> + *r = 0;
> +
> + return 1;
> +
> +read_impl_defined:
> + printk(XENLOG_G_DEBUG
> + "%pv: vGITS: RAZ on implementation defined register offset %#04lx\n",
> + v, info->gpa & 0xffff);
> + *r = 0;
> + return 1;
> +
> +read_reserved:
> + printk(XENLOG_G_DEBUG
> + "%pv: vGITS: RAZ on reserved register offset %#04lx\n",
> + v, info->gpa & 0xffff);
> + *r = 0;
> + return 1;
> +
> +bad_width:
> + printk(XENLOG_G_ERR "vGIIS: bad read width %d r%d offset %#04lx\n",
s/vGIIS/vGITS/
> + info->dabt.size, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> + domain_crash_synchronous();
> +
> + return 0;
> +}
> +
> +/******************************
> + * ITS registers write access *
> + ******************************/
> +
> +static unsigned int its_baser_table_size(uint64_t baser)
> +{
> + unsigned int ret, page_size[4] = {SZ_4K, SZ_16K, SZ_64K, SZ_64K};
> +
> + ret = page_size[(baser >> GITS_BASER_PAGE_SIZE_SHIFT) & 3];
> +
> + return ret * ((baser & GITS_BASER_SIZE_MASK) + 1);
> +}
> +
> +static unsigned int its_baser_nr_entries(uint64_t baser)
> +{
> + unsigned int entry_size = GITS_BASER_ENTRY_SIZE(baser);
> +
> + return its_baser_table_size(baser) / entry_size;
> +}
> +
> +/* Must be called with the ITS lock held. */
> +static bool vgic_v3_verify_its_status(struct virt_its *its, bool status)
> +{
> + ASSERT(spin_is_locked(&its->its_lock));
> +
> + if ( !status )
> + return false;
> +
> + if ( !(its->cbaser & GITS_VALID_BIT) ||
> + !(its->baser_dev & GITS_VALID_BIT) ||
> + !(its->baser_coll & GITS_VALID_BIT) )
> + {
> + printk(XENLOG_G_WARNING "d%d tried to enable ITS without having the tables configured.\n",
> + its->d->domain_id);
> + return false;
> + }
> +
> + return true;
> +}
> +
> +static void sanitize_its_base_reg(uint64_t *reg)
> +{
> + uint64_t r = *reg;
> +
> + /* Avoid outer shareable. */
> + switch ( (r >> GITS_BASER_SHAREABILITY_SHIFT) & 0x03 )
> + {
> + case GIC_BASER_OuterShareable:
> + r &= ~GITS_BASER_SHAREABILITY_MASK;
> + r |= GIC_BASER_InnerShareable << GITS_BASER_SHAREABILITY_SHIFT;
> + break;
> + default:
> + break;
> + }
> +
> + /* Avoid any inner non-cacheable mapping. */
> + switch ( (r >> GITS_BASER_INNER_CACHEABILITY_SHIFT) & 0x07 )
> + {
> + case GIC_BASER_CACHE_nCnB:
> + case GIC_BASER_CACHE_nC:
> + r &= ~GITS_BASER_INNER_CACHEABILITY_MASK;
> + r |= GIC_BASER_CACHE_RaWb << GITS_BASER_INNER_CACHEABILITY_SHIFT;
> + break;
> + default:
> + break;
> + }
> +
> + /* Only allow non-cacheable or same-as-inner. */
> + switch ( (r >> GITS_BASER_OUTER_CACHEABILITY_SHIFT) & 0x07 )
> + {
> + case GIC_BASER_CACHE_SameAsInner:
> + case GIC_BASER_CACHE_nC:
> + break;
> + default:
> + r &= ~GITS_BASER_OUTER_CACHEABILITY_MASK;
> + r |= GIC_BASER_CACHE_nC << GITS_BASER_OUTER_CACHEABILITY_SHIFT;
> + break;
> + }
> +
> + *reg = r;
> +}
> +
> +static int vgic_v3_its_mmio_write(struct vcpu *v, mmio_info_t *info,
> + register_t r, void *priv)
> +{
> + struct domain *d = v->domain;
> + struct virt_its *its = priv;
> + uint64_t reg;
> + uint32_t reg32;
> +
> + switch ( info->gpa & 0xffff )
> + {
> + case VREG32(GITS_CTLR):
> + {
> + uint32_t ctlr;
> +
> + if ( info->dabt.size != DABT_WORD ) goto bad_width;
> +
> + /*
> + * We need to take the vcmd_lock to prevent a guest from disabling
> + * the ITS while commands are still processed.
> + */
> + spin_lock(&its->vcmd_lock);
> + spin_lock(&its->its_lock);
> + ctlr = its->enabled ? GITS_CTLR_ENABLE : 0;
> + reg32 = ctlr;
> + vgic_reg32_update(®32, r, info);
> +
> + if ( ctlr ^ reg32 )
> + its->enabled = vgic_v3_verify_its_status(its,
> + reg32 & GITS_CTLR_ENABLE);
> + spin_unlock(&its->its_lock);
> + spin_unlock(&its->vcmd_lock);
> + return 1;
> + }
> +
> + case VREG32(GITS_IIDR):
> + goto write_ignore_32;
> + case VREG32(GITS_TYPER):
> + goto write_ignore_32;
> + case VRANGE32(0x0018, 0x001C):
> + goto write_reserved;
> + case VRANGE32(0x0020, 0x003C):
> + goto write_impl_defined;
> + case VRANGE32(0x0040, 0x007C):
> + goto write_reserved;
> + case VREG64(GITS_CBASER):
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> +
> + spin_lock(&its->its_lock);
> + /* Changing base registers with the ITS enabled is UNPREDICTABLE. */
> + if ( its->enabled )
> + {
> + spin_unlock(&its->its_lock);
> + gdprintk(XENLOG_WARNING,
> + "ITS: tried to change CBASER with the ITS enabled.\n");
s/ITS:/vGITS/
> + return 1;
> + }
> +
> + reg = its->cbaser;
> + vgic_reg64_update(®, r, info);
> + sanitize_its_base_reg(®);
> +
> + its->cbaser = reg;
> + its->creadr = 0;
> + spin_unlock(&its->its_lock);
> +
> + return 1;
> +
> + case VREG64(GITS_CWRITER):
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> +
> + spin_lock(&its->vcmd_lock);
> + reg = ITS_CMD_OFFSET(its->cwriter);
> + vgic_reg64_update(®, r, info);
> + its->cwriter = ITS_CMD_OFFSET(reg);
> +
> + if ( its->enabled )
> + if ( vgic_its_handle_cmds(d, its) )
> + gdprintk(XENLOG_WARNING, "error handling ITS commands\n");
> +
> + spin_unlock(&its->vcmd_lock);
> +
> + return 1;
> +
> + case VREG64(GITS_CREADR):
> + goto write_ignore_64;
> +
> + case VRANGE32(0x0098, 0x00FC):
> + goto write_reserved;
> + case VREG64(GITS_BASER0): /* device table */
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> +
> + spin_lock(&its->its_lock);
> +
> + /*
> + * Changing base registers with the ITS enabled is UNPREDICTABLE,
> + * we choose to ignore it, but warn.
> + */
> + if ( its->enabled )
> + {
> + spin_unlock(&its->its_lock);
> + gdprintk(XENLOG_WARNING, "ITS: tried to change BASER with the ITS enabled.\n");
> +
> + return 1;
> + }
> +
> + reg = its->baser_dev;
> + vgic_reg64_update(®, r, info);
> +
> + /* We don't support indirect tables for now. */
> + reg &= ~(GITS_BASER_RO_MASK | GITS_BASER_INDIRECT);
> + reg |= (sizeof(uint64_t) - 1) << GITS_BASER_ENTRY_SIZE_SHIFT;
> + reg |= GITS_BASER_TYPE_DEVICE << GITS_BASER_TYPE_SHIFT;
> + sanitize_its_base_reg(®);
> +
> + if ( reg & GITS_VALID_BIT )
> + {
> + its->max_devices = its_baser_nr_entries(reg);
> + if ( its->max_devices > BIT(its->devid_bits) )
> + its->max_devices = BIT(its->devid_bits);
> + }
> + else
> + its->max_devices = 0;
> +
> + its->baser_dev = reg;
> + spin_unlock(&its->its_lock);
> + return 1;
> + case VREG64(GITS_BASER1): /* collection table */
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> +
> + spin_lock(&its->its_lock);
> + /*
> + * Changing base registers with the ITS enabled is UNPREDICTABLE,
> + * we choose to ignore it, but warn.
> + */
> + if ( its->enabled )
> + {
> + spin_unlock(&its->its_lock);
> + gdprintk(XENLOG_INFO, "ITS: tried to change BASER with the ITS enabled.\n");
> + return 1;
> + }
> +
> + reg = its->baser_coll;
> + vgic_reg64_update(®, r, info);
> + /* No indirect tables for the collection table. */
> + reg &= ~(GITS_BASER_RO_MASK | GITS_BASER_INDIRECT);
> + reg |= (sizeof(uint16_t) - 1) << GITS_BASER_ENTRY_SIZE_SHIFT;
> + reg |= GITS_BASER_TYPE_COLLECTION << GITS_BASER_TYPE_SHIFT;
> + sanitize_its_base_reg(®);
> +
> + if ( reg & GITS_VALID_BIT )
> + its->max_collections = its_baser_nr_entries(reg);
> + else
> + its->max_collections = 0;
> + its->baser_coll = reg;
> + spin_unlock(&its->its_lock);
> + return 1;
> + case VRANGE64(GITS_BASER2, GITS_BASER7):
> + goto write_ignore_64;
> + case VRANGE32(0x0140, 0xBFFC):
> + goto write_reserved;
> + case VRANGE32(0xC000, 0xFFCC):
> + goto write_impl_defined;
> + case VRANGE32(0xFFD0, 0xFFE4): /* IMPDEF identification registers */
> + goto write_impl_defined;
> + case VREG32(GITS_PIDR2):
> + goto write_ignore_32;
> + case VRANGE32(0xFFEC, 0xFFFC): /* IMPDEF identification registers */
> + goto write_impl_defined;
> + default:
> + printk(XENLOG_G_ERR
> + "%pv: vGITS: unhandled write r%d offset %#04lx\n",
> + v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> + return 0;
> + }
> +
> + return 1;
> +
> +write_ignore_64:
> + if ( !vgic_reg64_check_access(info->dabt) ) goto bad_width;
> + return 1;
> +
> +write_ignore_32:
> + if ( info->dabt.size != DABT_WORD ) goto bad_width;
> + return 1;
> +
> +write_impl_defined:
> + printk(XENLOG_G_DEBUG
> + "%pv: vGITS: WI on implementation defined register offset %#04lx\n",
> + v, info->gpa & 0xffff);
> + return 1;
> +
> +write_reserved:
> + printk(XENLOG_G_DEBUG
> + "%pv: vGITS: WI on implementation defined register offset %#04lx\n",
> + v, info->gpa & 0xffff);
> + return 1;
> +
> +bad_width:
> + printk(XENLOG_G_ERR "vGITS: bad write width %d r%d offset %#08lx\n",
> + info->dabt.size, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> +
> + domain_crash_synchronous();
> +
> + return 0;
> +}
> +
> +static const struct mmio_handler_ops vgic_its_mmio_handler = {
> + .read = vgic_v3_its_mmio_read,
> + .write = vgic_v3_its_mmio_write,
> +};
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
> index 7470779..40f4ef5 100644
> --- a/xen/include/asm-arm/gic_v3_its.h
> +++ b/xen/include/asm-arm/gic_v3_its.h
> @@ -35,6 +35,7 @@
> #define GITS_BASER5 0x128
> #define GITS_BASER6 0x130
> #define GITS_BASER7 0x138
> +#define GITS_PIDR2 GICR_PIDR2
>
> /* Register bits */
> #define GITS_VALID_BIT BIT(63)
> @@ -57,6 +58,7 @@
> #define GITS_TYPER_ITT_SIZE_MASK (0xfUL << GITS_TYPER_ITT_SIZE_SHIFT)
> #define GITS_TYPER_ITT_SIZE(r) ((((r) & GITS_TYPER_ITT_SIZE_MASK) >> \
> GITS_TYPER_ITT_SIZE_SHIFT) + 1)
> +#define GITS_TYPER_PHYSICAL (1U << 0)
>
> #define GITS_BASER_INDIRECT BIT(62)
> #define GITS_BASER_INNER_CACHEABILITY_SHIFT 59
> @@ -76,6 +78,7 @@
> (((reg >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
> #define GITS_BASER_SHAREABILITY_SHIFT 10
> #define GITS_BASER_PAGE_SIZE_SHIFT 8
> +#define GITS_BASER_SIZE_MASK 0xff
> #define GITS_BASER_SHAREABILITY_MASK (0x3ULL << GITS_BASER_SHAREABILITY_SHIFT)
> #define GITS_BASER_OUTER_CACHEABILITY_MASK (0x7ULL << GITS_BASER_OUTER_CACHEABILITY_SHIFT)
> #define GITS_BASER_INNER_CACHEABILITY_MASK (0x7ULL << GITS_BASER_INNER_CACHEABILITY_SHIFT)
>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 12/27] ARM: vGIC: advertise LPI support
2017-04-12 12:48 ` Andre Przywara
@ 2017-04-12 13:04 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 13:04 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 13:48, Andre Przywara wrote:
> On 12/04/17 13:38, Julien Grall wrote:
>> On 12/04/17 01:44, Andre Przywara wrote:
>>> +static void vgic_vcpu_enable_lpis(struct vcpu *v)
>>> +{
>>> + uint64_t reg = v->domain->arch.vgic.rdist_propbase;
>>> + unsigned int nr_lpis = BIT((reg & 0x1f) + 1);
>>> +
>>> + if ( nr_lpis < LPI_OFFSET )
>>> + nr_lpis = 0;
>>> + else
>>> + nr_lpis -= LPI_OFFSET;
>>> +
>>> + if ( !v->domain->arch.vgic.rdists_enabled )
>>> + {
>>> + v->domain->arch.vgic.nr_lpis = nr_lpis;
>>> + v->domain->arch.vgic.rdists_enabled = true;
>>> + }
>>> +
>>> + v->arch.vgic.flags |= VGIC_V3_LPIS_ENABLED;
>>> +}
>>> +
>>> static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t
>>> *info,
>>> uint32_t gicr_reg,
>>> register_t r)
>>> @@ -436,8 +470,26 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct
>>> vcpu *v, mmio_info_t *info,
>>> switch ( gicr_reg )
>>> {
>>> case VREG32(GICR_CTLR):
>>> - /* LPI's not implemented */
>>> - goto write_ignore_32;
>>> + {
>>> + unsigned long flags;
>>> +
>>> + if ( !v->domain->arch.vgic.has_its )
>>> + goto write_ignore_32;
>>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>>> +
>>> + vgic_lock(v); /* protects rdists_enabled */
>>
>> Again, can't you move this lock in vgic_vcpu_enable_lpis?
>
> But the domain VGIC lock must be taken before the VGIC VCPU lock, right?
> And we need the VGIC VCPU lock before calling the function to check for
> LPIS_ENABLED, isn't it?
Then add an ASSERT in vgic_vcpu_enable_lpis and explain the locking.
>
>>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>> +
>>> + /* LPIs can only be enabled once, but never disabled again. */
>>> + if ( (r & GICR_CTLR_ENABLE_LPIS) &&
>>> + !(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED) )
>>> + vgic_vcpu_enable_lpis(v);
>>> +
>>> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>>> + vgic_unlock(v);
>>> +
>>> + return 1;
>>> + }
>>>
>>> case VREG32(GICR_IIDR):
>>> /* RO */
>>> @@ -1058,6 +1110,11 @@ static int vgic_v3_distr_mmio_read(struct vcpu
>>> *v, mmio_info_t *info,
>>> typer = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
>>> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
>>>
>>> + if ( v->domain->arch.vgic.has_its )
>>> + {
>>> + typer |= GICD_TYPE_LPIS;
>>> + irq_bits = v->domain->arch.vgic.intid_bits;
>>
>> See my remark in patch #1. I would have expected the field intid_bits to
>> be used even if ITS is not present.
>
> Well, this is just mimicking current code, which just puts the value
> covering the number of SPIs in here.
> As we only support LPIs with an ITS, I don't see why we should present
> any higher value in there than we do today already for a GICv3.
You missed half of my point from patch #1. When I read "intid_bits", I
directly associate to the field GICD_TYPER.ID_bits and would expect the
both to be strictly identical.
So the field "intid_bits" should be correctly initialized in
vgic_v3_domain_init rather than having to wonder whether ITS is used or not.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables
2017-04-12 10:55 ` Julien Grall
@ 2017-04-12 13:12 ` Andre Przywara
2017-04-12 13:13 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 13:12 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 11:55, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> Allow a guest to provide the address and size for the memory regions
>> it has reserved for the GICv3 pending and property tables.
>> We sanitise the various fields of the respective redistributor
>> registers.
>> The MMIO read and write accesses are protected by locks, to avoid any
>> changing of the property or pending table address while a redistributor
>> is live and also to protect the non-atomic vgic_reg64_extract() function
>> on the MMIO read side.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> Reviewed-by: Julien Grall <julien.grall@arm.com>
>
> Whilst I gave my reviewed-by it is very rude to ignore a comment.
Yeah, sorry about that! I was unsure about that as well, so thought
about it and eventually forgot to answer.
> It would have been nicer to answer even if it is just saying "I can add
> a TODO and address it in a follow-up patch".
>
> Please get use to mention all the changes (e.g the spin_*lock ->
> spin_*lock_irq* change) you made in a patch. Mainly if you keep a
> reviewed-by.
I was really unsure about keeping or dropping it, but since you
complained about me dropping it last time I tried it the other way this
time ;-)
>> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
>> index ebaea35..b2d98bb 100644
>> --- a/xen/include/asm-arm/domain.h
>> +++ b/xen/include/asm-arm/domain.h
>> @@ -109,11 +109,15 @@ struct arch_domain
>> } *rdist_regions;
>> int nr_regions; /* Number of rdist
>> regions */
>> uint32_t rdist_stride; /* Re-Distributor stride */
>> + unsigned long int nr_lpis;
>> + uint64_t rdist_propbase;
>> struct rb_root its_devices; /* Devices mapped to an
>> ITS */
>> spinlock_t its_devices_lock; /* Protects the
>> its_devices tree */
>> struct radix_tree_root pend_lpi_tree; /* Stores struct
>> pending_irq's */
>> rwlock_t pend_lpi_tree_lock; /* Protects the
>> pend_lpi_tree */
>> unsigned int intid_bits;
>> + bool rdists_enabled; /* Is any redistributor
>> enabled? */
>> + bool has_its;
>
> The comment you ignore was the one about consolidating rdists_enabled
> and has_its in a single field and use flags.
Yes, I had the idea myself before, but decided against it as IMHO it
looks much nicer this way (compared to using a flags variable. which I
guess is what you mean).
Are you OK with that?
Cheers,
Andre.
>
>> #endif
>> } vgic;
>>
>> @@ -260,6 +264,7 @@ struct arch_vcpu
>>
>> /* GICv3: redistributor base and flags for this vCPU */
>> paddr_t rdist_base;
>> + uint64_t rdist_pendbase;
>> #define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the
>> rdist */
>> #define VGIC_V3_LPIS_ENABLED (1 << 1)
>> uint8_t flags;
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables
2017-04-12 13:12 ` Andre Przywara
@ 2017-04-12 13:13 ` Julien Grall
2017-05-11 17:54 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 13:13 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 14:12, Andre Przywara wrote:
> Hi,
Hi,
>
> On 12/04/17 11:55, Julien Grall wrote:
>> Hi Andre,
>>
>> On 12/04/17 01:44, Andre Przywara wrote:
>>> Allow a guest to provide the address and size for the memory regions
>>> it has reserved for the GICv3 pending and property tables.
>>> We sanitise the various fields of the respective redistributor
>>> registers.
>>> The MMIO read and write accesses are protected by locks, to avoid any
>>> changing of the property or pending table address while a redistributor
>>> is live and also to protect the non-atomic vgic_reg64_extract() function
>>> on the MMIO read side.
>>>
>>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>>> Reviewed-by: Julien Grall <julien.grall@arm.com>
>>
>> Whilst I gave my reviewed-by it is very rude to ignore a comment.
>
> Yeah, sorry about that! I was unsure about that as well, so thought
> about it and eventually forgot to answer.
>
>> It would have been nicer to answer even if it is just saying "I can add
>> a TODO and address it in a follow-up patch".
>>
>> Please get use to mention all the changes (e.g the spin_*lock ->
>> spin_*lock_irq* change) you made in a patch. Mainly if you keep a
>> reviewed-by.
>
> I was really unsure about keeping or dropping it, but since you
> complained about me dropping it last time I tried it the other way this
> time ;-)
>
>>> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
>>> index ebaea35..b2d98bb 100644
>>> --- a/xen/include/asm-arm/domain.h
>>> +++ b/xen/include/asm-arm/domain.h
>>> @@ -109,11 +109,15 @@ struct arch_domain
>>> } *rdist_regions;
>>> int nr_regions; /* Number of rdist
>>> regions */
>>> uint32_t rdist_stride; /* Re-Distributor stride */
>>> + unsigned long int nr_lpis;
>>> + uint64_t rdist_propbase;
>>> struct rb_root its_devices; /* Devices mapped to an
>>> ITS */
>>> spinlock_t its_devices_lock; /* Protects the
>>> its_devices tree */
>>> struct radix_tree_root pend_lpi_tree; /* Stores struct
>>> pending_irq's */
>>> rwlock_t pend_lpi_tree_lock; /* Protects the
>>> pend_lpi_tree */
>>> unsigned int intid_bits;
>>> + bool rdists_enabled; /* Is any redistributor
>>> enabled? */
>>> + bool has_its;
>>
>> The comment you ignore was the one about consolidating rdists_enabled
>> and has_its in a single field and use flags.
>
> Yes, I had the idea myself before, but decided against it as IMHO it
> looks much nicer this way (compared to using a flags variable. which I
> guess is what you mean).
>
> Are you OK with that?
Why is it nicer? You only need 1 bit to represent rdist_enabled and
another for has_its. This would save a bit a space in a resource limited
structure. I would be OK with a TODO so we know we can save space here
in the future...
Although, my main point was you not ignore comment and say no if you
disagree.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 14/27] ARM: vITS: introduce translation table walks
2017-04-12 0:44 ` [PATCH v8 14/27] ARM: vITS: introduce translation table walks Andre Przywara
@ 2017-04-12 13:22 ` Julien Grall
2017-04-12 13:36 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 13:22 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 01:44, Andre Przywara wrote:
> The ITS stores the target (v)CPU and the (virtual) LPI number in tables.
> Introduce functions to walk those tables and translate an device ID -
> event ID pair into a pair of virtual LPI and vCPU.
> We map those tables on demand - which is cheap on arm64 - and copy the
> respective entries before using them, to avoid the guest tampering with
> them meanwhile.
>
> To allow compiling without warnings, we declare two functions as
> non-static for the moment, which two later patches will fix.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 183 +++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 183 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index a60f9b2..632ab84 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -78,6 +78,7 @@ struct vits_itte
> uint16_t pad;
> };
>
> +#define UNMAPPED_COLLECTION ((uint16_t)~0)
On v5 (patch #28), I asked to minimize the use of uint16_t (resp.
uint64_t) when it refers to collection (resp. ITTE). But it sounds like
you missed again that point.
> #define GITS_BASER_RO_MASK (GITS_BASER_TYPE_MASK | \
> (31UL << GITS_BASER_ENTRY_SIZE_SHIFT))
>
> @@ -94,6 +95,188 @@ void vgic_v3_its_free_domain(struct domain *d)
> ASSERT(RB_EMPTY_ROOT(&d->arch.vgic.its_devices));
> }
>
> +/*
> + * The physical address is encoded slightly differently depending on
> + * the used page size: the highest four bits are stored in the lowest
> + * four bits of the field for 64K pages.
> + */
> +static paddr_t get_baser_phys_addr(uint64_t reg)
> +{
> + if ( reg & BIT(9) )
> + return (reg & GENMASK(47, 16)) |
> + ((reg & GENMASK(15, 12)) << 36);
> + else
> + return reg & GENMASK(47, 12);
> +}
> +
> +/*
> + * Our collection table encoding:
> + * Just contains the 16-bit VCPU ID of the respective vCPU.
> + */
> +#define COLL_TABLE_ENTRY_SIZE 2UL
> +
> +/* Must be called with the ITS lock held. */
> +static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
> + uint16_t collid)
> +{
> + paddr_t addr = get_baser_phys_addr(its->baser_coll);
> + uint16_t vcpu_id;
I don't see any BUILD_BUG_ON making sure you can store a vCPU ID in
uint16_t.
> + int ret;
> +
> + ASSERT(spin_is_locked(&its->its_lock));
> +
> + if ( collid >= its->max_collections )
> + return NULL;
> +
> + ret = vgic_access_guest_memory(its->d, addr + collid * sizeof(uint16_t),
> + &vcpu_id, sizeof(vcpu_id), false);
Minimizing the use of uint16_t will likely make this code less confusing
to read as it mixes sizeof(vcpu_id) and sizeof(uint16_t).
> + if ( ret )
> + return NULL;
> +
> + if ( vcpu_id == UNMAPPED_COLLECTION || vcpu_id >= its->d->max_vcpus )
> + return NULL;
> +
> + return its->d->vcpu[vcpu_id];
> +}
> +
> +/*
> + * Our device table encodings:
> + * Contains the guest physical address of the Interrupt Translation Table in
> + * bits [51:8], and the size of it is encoded as the number of bits minus one
> + * in the lowest 5 bits of the word.
> + */
> +#define DEV_TABLE_ENTRY_SIZE 8UL
> +#define DEV_TABLE_ITT_ADDR(x) ((x) & GENMASK(51, 8))
> +#define DEV_TABLE_ITT_SIZE(x) (BIT(((x) & GENMASK(4, 0)) + 1))
> +#define DEV_TABLE_ENTRY(addr, bits) \
> + (((addr) & GENMASK(51, 8)) | (((bits) - 1) & GENMASK(4, 0)))
> +
> +/*
> + * Lookup the address of the Interrupt Translation Table associated with
> + * that device ID.
> + * TODO: add support for walking indirect tables.
> + */
> +static int its_get_itt(struct virt_its *its, uint32_t devid,
> + uint64_t *itt)
> +{
> + paddr_t addr = get_baser_phys_addr(its->baser_dev);
> +
> + if ( devid >= its->max_devices )
> + return -EINVAL;
> +
> + return vgic_access_guest_memory(its->d, addr + devid * sizeof(uint64_t),
> + itt, sizeof(*itt), false);
See my remark above for uint64_t.
> +}
> +
> +/*
> + * Lookup the address of the Interrupt Translation Table associated with
> + * a device ID and return the address of the ITTE belonging to the event ID
> + * (which is an index into that table).
> + */
> +static paddr_t its_get_itte_address(struct virt_its *its,
> + uint32_t devid, uint32_t evid)
> +{
> + uint64_t itt;
> + int ret;
> +
> + ret = its_get_itt(its, devid, &itt);
> + if ( ret )
> + return INVALID_PADDR;
> +
> + if ( evid >= DEV_TABLE_ITT_SIZE(itt) ||
> + DEV_TABLE_ITT_ADDR(itt) == INVALID_PADDR )
> + return INVALID_PADDR;
> +
> + return DEV_TABLE_ITT_ADDR(itt) + evid * sizeof(struct vits_itte);
> +}
> +
> +/*
> + * Queries the collection and device tables to get the vCPU and virtual
> + * LPI number for a given guest event. This first accesses the guest memory
> + * to resolve the address of the ITTE, then reads the ITTE entry at this
> + * address and puts the result in vcpu_ptr and vlpi_ptr.
> + * Must be called with the ITS lock held.
> + */
> +static bool read_itte_locked(struct virt_its *its, uint32_t devid,
> + uint32_t evid, struct vcpu **vcpu_ptr,
> + uint32_t *vlpi_ptr)
> +{
> + paddr_t addr;
> + struct vits_itte itte;
> + struct vcpu *vcpu;
> +
> + ASSERT(spin_is_locked(&its->its_lock));
> +
> + addr = its_get_itte_address(its, devid, evid);
> + if ( addr == INVALID_PADDR )
> + return false;
> +
> + if ( vgic_access_guest_memory(its->d, addr, &itte, sizeof(itte), false) )
> + return false;
> +
> + vcpu = get_vcpu_from_collection(its, itte.collection);
> + if ( !vcpu )
> + return false;
> +
> + *vcpu_ptr = vcpu;
> + *vlpi_ptr = itte.vlpi;
> + return true;
> +}
> +
> +/*
> + * This function takes care of the locking by taking the its_lock itself, so
> + * a caller shall not hold this. Before returning, the lock is dropped again.
> + */
> +bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
> + struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
> +{
> + bool ret;
> +
> + spin_lock(&its->its_lock);
> + ret = read_itte_locked(its, devid, evid, vcpu_ptr, vlpi_ptr);
> + spin_unlock(&its->its_lock);
> +
> + return ret;
> +}
> +
> +/*
> + * Queries the collection and device tables to translate the device ID and
> + * event ID and find the appropriate ITTE. The given collection ID and the
> + * virtual LPI number are then stored into that entry.
> + * If vcpu_ptr is provided, returns the VCPU belonging to that collection.
> + * Must be called with the ITS lock held.
> + */
> +bool write_itte_locked(struct virt_its *its, uint32_t devid,
> + uint32_t evid, uint32_t collid, uint32_t vlpi,
> + struct vcpu **vcpu_ptr)
> +{
> + paddr_t addr;
> + struct vits_itte itte;
> +
> + ASSERT(spin_is_locked(&its->its_lock));
> +
> + if ( collid >= its->max_collections )
> + return false;
> +
> + if ( vlpi >= its->d->arch.vgic.nr_lpis )
> + return false;
> +
> + addr = its_get_itte_address(its, devid, evid);
> + if ( addr == INVALID_PADDR )
> + return false;
> +
> + itte.collection = collid;
> + itte.vlpi = vlpi;
> +
> + if ( vgic_access_guest_memory(its->d, addr, &itte, sizeof(itte), true) )
> + return false;
> +
> + if ( vcpu_ptr )
> + *vcpu_ptr = get_vcpu_from_collection(its, collid);
> +
> + return true;
> +}
> +
> /**************************************
> * Functions that handle ITS commands *
> **************************************/
>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 14/27] ARM: vITS: introduce translation table walks
2017-04-12 13:22 ` Julien Grall
@ 2017-04-12 13:36 ` Andre Przywara
2017-04-12 13:37 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 13:36 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 14:22, Julien Grall wrote:
>
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> The ITS stores the target (v)CPU and the (virtual) LPI number in tables.
>> Introduce functions to walk those tables and translate an device ID -
>> event ID pair into a pair of virtual LPI and vCPU.
>> We map those tables on demand - which is cheap on arm64 - and copy the
>> respective entries before using them, to avoid the guest tampering with
>> them meanwhile.
>>
>> To allow compiling without warnings, we declare two functions as
>> non-static for the moment, which two later patches will fix.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/vgic-v3-its.c | 183
>> +++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 183 insertions(+)
>>
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index a60f9b2..632ab84 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -78,6 +78,7 @@ struct vits_itte
>> uint16_t pad;
>> };
>>
>> +#define UNMAPPED_COLLECTION ((uint16_t)~0)
>
> On v5 (patch #28), I asked to minimize the use of uint16_t (resp.
> uint64_t) when it refers to collection (resp. ITTE). But it sounds like
> you missed again that point.
Well, I introduced that below, just failed to use it *everywhere*.
...
>
>> #define GITS_BASER_RO_MASK (GITS_BASER_TYPE_MASK | \
>> (31UL << GITS_BASER_ENTRY_SIZE_SHIFT))
>>
>> @@ -94,6 +95,188 @@ void vgic_v3_its_free_domain(struct domain *d)
>> ASSERT(RB_EMPTY_ROOT(&d->arch.vgic.its_devices));
>> }
>>
>> +/*
>> + * The physical address is encoded slightly differently depending on
>> + * the used page size: the highest four bits are stored in the lowest
>> + * four bits of the field for 64K pages.
>> + */
>> +static paddr_t get_baser_phys_addr(uint64_t reg)
>> +{
>> + if ( reg & BIT(9) )
>> + return (reg & GENMASK(47, 16)) |
>> + ((reg & GENMASK(15, 12)) << 36);
>> + else
>> + return reg & GENMASK(47, 12);
>> +}
>> +
>> +/*
>> + * Our collection table encoding:
>> + * Just contains the 16-bit VCPU ID of the respective vCPU.
>> + */
>> +#define COLL_TABLE_ENTRY_SIZE 2UL
... (here)
>> +
>> +/* Must be called with the ITS lock held. */
>> +static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
>> + uint16_t collid)
>> +{
>> + paddr_t addr = get_baser_phys_addr(its->baser_coll);
>> + uint16_t vcpu_id;
>
> I don't see any BUILD_BUG_ON making sure you can store a vCPU ID in
> uint16_t.
>
>> + int ret;
>> +
>> + ASSERT(spin_is_locked(&its->its_lock));
>> +
>> + if ( collid >= its->max_collections )
>> + return NULL;
>> +
>> + ret = vgic_access_guest_memory(its->d, addr + collid *
>> sizeof(uint16_t),
>> + &vcpu_id, sizeof(vcpu_id), false);
>
> Minimizing the use of uint16_t will likely make this code less confusing
> to read as it mixes sizeof(vcpu_id) and sizeof(uint16_t).
The idea here is to use the actual type of the variable used instead of
putting an explicit type in here, since this can get out of sync.
So would a typedef meet your expectations?
typedef uint16_t coll_table_entry_t;
typedef uint64_t dev_table_entry_t;
And then use that throughout the code?
Cheers,
Andre.
>
>> + if ( ret )
>> + return NULL;
>> +
>> + if ( vcpu_id == UNMAPPED_COLLECTION || vcpu_id >=
>> its->d->max_vcpus )
>> + return NULL;
>> +
>> + return its->d->vcpu[vcpu_id];
>> +}
>> +
>> +/*
>> + * Our device table encodings:
>> + * Contains the guest physical address of the Interrupt Translation
>> Table in
>> + * bits [51:8], and the size of it is encoded as the number of bits
>> minus one
>> + * in the lowest 5 bits of the word.
>> + */
>> +#define DEV_TABLE_ENTRY_SIZE 8UL
>> +#define DEV_TABLE_ITT_ADDR(x) ((x) & GENMASK(51, 8))
>> +#define DEV_TABLE_ITT_SIZE(x) (BIT(((x) & GENMASK(4, 0)) + 1))
>> +#define DEV_TABLE_ENTRY(addr, bits) \
>> + (((addr) & GENMASK(51, 8)) | (((bits) - 1) & GENMASK(4, 0)))
>> +
>> +/*
>> + * Lookup the address of the Interrupt Translation Table associated with
>> + * that device ID.
>> + * TODO: add support for walking indirect tables.
>> + */
>> +static int its_get_itt(struct virt_its *its, uint32_t devid,
>> + uint64_t *itt)
>> +{
>> + paddr_t addr = get_baser_phys_addr(its->baser_dev);
>> +
>> + if ( devid >= its->max_devices )
>> + return -EINVAL;
>> +
>> + return vgic_access_guest_memory(its->d, addr + devid *
>> sizeof(uint64_t),
>> + itt, sizeof(*itt), false);
>
> See my remark above for uint64_t.
>
>> +}
>> +
>> +/*
>> + * Lookup the address of the Interrupt Translation Table associated with
>> + * a device ID and return the address of the ITTE belonging to the
>> event ID
>> + * (which is an index into that table).
>> + */
>> +static paddr_t its_get_itte_address(struct virt_its *its,
>> + uint32_t devid, uint32_t evid)
>> +{
>> + uint64_t itt;
>> + int ret;
>> +
>> + ret = its_get_itt(its, devid, &itt);
>> + if ( ret )
>> + return INVALID_PADDR;
>> +
>> + if ( evid >= DEV_TABLE_ITT_SIZE(itt) ||
>> + DEV_TABLE_ITT_ADDR(itt) == INVALID_PADDR )
>> + return INVALID_PADDR;
>> +
>> + return DEV_TABLE_ITT_ADDR(itt) + evid * sizeof(struct vits_itte);
>> +}
>> +
>> +/*
>> + * Queries the collection and device tables to get the vCPU and virtual
>> + * LPI number for a given guest event. This first accesses the guest
>> memory
>> + * to resolve the address of the ITTE, then reads the ITTE entry at this
>> + * address and puts the result in vcpu_ptr and vlpi_ptr.
>> + * Must be called with the ITS lock held.
>> + */
>> +static bool read_itte_locked(struct virt_its *its, uint32_t devid,
>> + uint32_t evid, struct vcpu **vcpu_ptr,
>> + uint32_t *vlpi_ptr)
>> +{
>> + paddr_t addr;
>> + struct vits_itte itte;
>> + struct vcpu *vcpu;
>> +
>> + ASSERT(spin_is_locked(&its->its_lock));
>> +
>> + addr = its_get_itte_address(its, devid, evid);
>> + if ( addr == INVALID_PADDR )
>> + return false;
>> +
>> + if ( vgic_access_guest_memory(its->d, addr, &itte, sizeof(itte),
>> false) )
>> + return false;
>> +
>> + vcpu = get_vcpu_from_collection(its, itte.collection);
>> + if ( !vcpu )
>> + return false;
>> +
>> + *vcpu_ptr = vcpu;
>> + *vlpi_ptr = itte.vlpi;
>> + return true;
>> +}
>> +
>> +/*
>> + * This function takes care of the locking by taking the its_lock
>> itself, so
>> + * a caller shall not hold this. Before returning, the lock is
>> dropped again.
>> + */
>> +bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
>> + struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
>> +{
>> + bool ret;
>> +
>> + spin_lock(&its->its_lock);
>> + ret = read_itte_locked(its, devid, evid, vcpu_ptr, vlpi_ptr);
>> + spin_unlock(&its->its_lock);
>> +
>> + return ret;
>> +}
>> +
>> +/*
>> + * Queries the collection and device tables to translate the device
>> ID and
>> + * event ID and find the appropriate ITTE. The given collection ID
>> and the
>> + * virtual LPI number are then stored into that entry.
>> + * If vcpu_ptr is provided, returns the VCPU belonging to that
>> collection.
>> + * Must be called with the ITS lock held.
>> + */
>> +bool write_itte_locked(struct virt_its *its, uint32_t devid,
>> + uint32_t evid, uint32_t collid, uint32_t vlpi,
>> + struct vcpu **vcpu_ptr)
>> +{
>> + paddr_t addr;
>> + struct vits_itte itte;
>> +
>> + ASSERT(spin_is_locked(&its->its_lock));
>> +
>> + if ( collid >= its->max_collections )
>> + return false;
>> +
>> + if ( vlpi >= its->d->arch.vgic.nr_lpis )
>> + return false;
>> +
>> + addr = its_get_itte_address(its, devid, evid);
>> + if ( addr == INVALID_PADDR )
>> + return false;
>> +
>> + itte.collection = collid;
>> + itte.vlpi = vlpi;
>> +
>> + if ( vgic_access_guest_memory(its->d, addr, &itte, sizeof(itte),
>> true) )
>> + return false;
>> +
>> + if ( vcpu_ptr )
>> + *vcpu_ptr = get_vcpu_from_collection(its, collid);
>> +
>> + return true;
>> +}
>> +
>> /**************************************
>> * Functions that handle ITS commands *
>> **************************************/
>>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 14/27] ARM: vITS: introduce translation table walks
2017-04-12 13:36 ` Andre Przywara
@ 2017-04-12 13:37 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 13:37 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 14:36, Andre Przywara wrote:
> Hi,
>
> On 12/04/17 14:22, Julien Grall wrote:
>>
>>
>> On 12/04/17 01:44, Andre Przywara wrote:
>>> The ITS stores the target (v)CPU and the (virtual) LPI number in tables.
>>> Introduce functions to walk those tables and translate an device ID -
>>> event ID pair into a pair of virtual LPI and vCPU.
>>> We map those tables on demand - which is cheap on arm64 - and copy the
>>> respective entries before using them, to avoid the guest tampering with
>>> them meanwhile.
>>>
>>> To allow compiling without warnings, we declare two functions as
>>> non-static for the moment, which two later patches will fix.
>>>
>>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>>> ---
>>> xen/arch/arm/vgic-v3-its.c | 183
>>> +++++++++++++++++++++++++++++++++++++++++++++
>>> 1 file changed, 183 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>>> index a60f9b2..632ab84 100644
>>> --- a/xen/arch/arm/vgic-v3-its.c
>>> +++ b/xen/arch/arm/vgic-v3-its.c
>>> @@ -78,6 +78,7 @@ struct vits_itte
>>> uint16_t pad;
>>> };
>>>
>>> +#define UNMAPPED_COLLECTION ((uint16_t)~0)
>>
>> On v5 (patch #28), I asked to minimize the use of uint16_t (resp.
>> uint64_t) when it refers to collection (resp. ITTE). But it sounds like
>> you missed again that point.
>
> Well, I introduced that below, just failed to use it *everywhere*.
>
> ...
>
>>
>>> #define GITS_BASER_RO_MASK (GITS_BASER_TYPE_MASK | \
>>> (31UL << GITS_BASER_ENTRY_SIZE_SHIFT))
>>>
>>> @@ -94,6 +95,188 @@ void vgic_v3_its_free_domain(struct domain *d)
>>> ASSERT(RB_EMPTY_ROOT(&d->arch.vgic.its_devices));
>>> }
>>>
>>> +/*
>>> + * The physical address is encoded slightly differently depending on
>>> + * the used page size: the highest four bits are stored in the lowest
>>> + * four bits of the field for 64K pages.
>>> + */
>>> +static paddr_t get_baser_phys_addr(uint64_t reg)
>>> +{
>>> + if ( reg & BIT(9) )
>>> + return (reg & GENMASK(47, 16)) |
>>> + ((reg & GENMASK(15, 12)) << 36);
>>> + else
>>> + return reg & GENMASK(47, 12);
>>> +}
>>> +
>>> +/*
>>> + * Our collection table encoding:
>>> + * Just contains the 16-bit VCPU ID of the respective vCPU.
>>> + */
>>> +#define COLL_TABLE_ENTRY_SIZE 2UL
>
> ... (here)
>
>>> +
>>> +/* Must be called with the ITS lock held. */
>>> +static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
>>> + uint16_t collid)
>>> +{
>>> + paddr_t addr = get_baser_phys_addr(its->baser_coll);
>>> + uint16_t vcpu_id;
>>
>> I don't see any BUILD_BUG_ON making sure you can store a vCPU ID in
>> uint16_t.
>>
>>> + int ret;
>>> +
>>> + ASSERT(spin_is_locked(&its->its_lock));
>>> +
>>> + if ( collid >= its->max_collections )
>>> + return NULL;
>>> +
>>> + ret = vgic_access_guest_memory(its->d, addr + collid *
>>> sizeof(uint16_t),
>>> + &vcpu_id, sizeof(vcpu_id), false);
>>
>> Minimizing the use of uint16_t will likely make this code less confusing
>> to read as it mixes sizeof(vcpu_id) and sizeof(uint16_t).
>
> The idea here is to use the actual type of the variable used instead of
> putting an explicit type in here, since this can get out of sync.
>
> So would a typedef meet your expectations?
>
> typedef uint16_t coll_table_entry_t;
> typedef uint64_t dev_table_entry_t;
>
> And then use that throughout the code?
Yes please.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 15/27] ARM: vITS: handle CLEAR command
2017-04-12 0:44 ` [PATCH v8 15/27] ARM: vITS: handle CLEAR command Andre Przywara
@ 2017-04-12 14:10 ` Julien Grall
2017-04-12 14:29 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 14:10 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> This introduces the ITS command handler for the CLEAR command, which
> clears the pending state of an LPI.
> This removes a not-yet injected, but already queued IRQ from a VCPU.
> As read_itte() is now eventually used, we add the static keyword.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 52 ++++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 50 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 632ab84..e24ab60 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -227,8 +227,8 @@ static bool read_itte_locked(struct virt_its *its, uint32_t devid,
> * This function takes care of the locking by taking the its_lock itself, so
> * a caller shall not hold this. Before returning, the lock is dropped again.
> */
> -bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
> - struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
> +static bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
> + struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
> {
> bool ret;
>
> @@ -297,6 +297,51 @@ static uint64_t its_cmd_mask_field(uint64_t *its_cmd, unsigned int word,
> #define its_cmd_get_validbit(cmd) its_cmd_mask_field(cmd, 2, 63, 1)
> #define its_cmd_get_ittaddr(cmd) (its_cmd_mask_field(cmd, 2, 8, 44) << 8)
>
> +/*
> + * CLEAR removes the pending state from an LPI. */
> +static int its_handle_clear(struct virt_its *its, uint64_t *cmdptr)
> +{
> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
> + uint32_t eventid = its_cmd_get_id(cmdptr);
> + struct pending_irq *p;
> + struct vcpu *vcpu;
> + uint32_t vlpi;
> + unsigned long flags;
> +
> + /* Translate the DevID/EvID pair into a vCPU/vLPI pair. */
> + if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
So the vCPU ID is retrieved from guest memory, therefore you cannot
trust the value for taking a lock.
Indeed, a malicious guest could rewrite the value with another vCPU ID
and you would take the wrong vCPU vGIC lock.
What is the plan to fix that?
> + return -1;
> +
> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
I don't think this lock will protect correctly the pending_irq because
if you do a MOVI before hand you will use the new vCPU ID and not the
old one (see my explanation on patch #2).
> +
> + p = its->d->arch.vgic.handler->lpi_to_pending(its->d, vlpi);
> + if ( !p )
Please detail what means NULL here.
> + {
> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
> + return -1;
> + }
> +
> + /*
> + * If the LPI is already visible on the guest, it is too late to
> + * clear the pending state. However this is a benign race that can
> + * happen on real hardware, too: If the LPI has already been forwarded
> + * to a CPU interface, a CLEAR request reaching the redistributor has
> + * no effect on that LPI anymore. Since LPIs are edge triggered and
> + * have no active state, we don't need to care about this here.
> + */
> + if ( !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> + {
> + /* Remove a pending, but not yet injected guest IRQ. */
> + clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
> + list_del_init(&p->inflight);
> + list_del_init(&p->lr_queue);
I am not sure to understand why you open-code gic_remove_from_queues
rather than reworking it.
> + }
> +
> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
> +
> + return 0;
> +}
> +
> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>
> @@ -326,6 +371,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
>
> switch ( its_cmd_get_command(command) )
> {
> + case GITS_CMD_CLEAR:
> + ret = its_handle_clear(its, command);
> + break;
> case GITS_CMD_SYNC:
> /* We handle ITS commands synchronously, so we ignore SYNC. */
> break;
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 00/27] arm64: Dom0 ITS emulation
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
` (26 preceding siblings ...)
2017-04-12 0:44 ` [PATCH v8 27/27] ARM: vITS: create ITS subnodes for Dom0 DT Andre Przywara
@ 2017-04-12 14:13 ` Julien Grall
27 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 14:13 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 01:44, Andre Przywara wrote:
> Hi,
>
> this is a reworked version of the second part of the ITS emulation series,
> where the first part has been merged already.
> It extends the concept of letting Xen deal with irq_to_pending()
> potentially returning NULL, by making sure the retrieval and usage
> of the pending_irq pointer is always happening under the VCPU VGIC lock
> (patch 02 and 03). This allows us to free the memory for the pending_irqs
> when a device gets unmapped.
> Patch 20 contains a relatively easy solution to some LPI unmap/map corner
> case (DISCARD;MAPTI sequence using the same virtual LPI number while the
> VLPI is pending in some LR).
> On top of these changes I addressed the remaining comments from v5 and v6.
> For a detailed changelog see below.
>
> Cheers,
> Andre
>
> ----------------------------------
> This series adds support for emulation of an ARM GICv3 ITS interrupt
> controller. For hardware which relies on the ITS to provide interrupts for
> its peripherals this code is needed to get a machine booted into Dom0 at
> all. ITS emulation for DomUs is only really useful with PCI passthrough,
> which is not yet available for ARM. It is expected that this feature
> will be co-developed with the ITS DomU code. However this code drop here
> considered DomU emulation already, to keep later architectural changes
> to a minimum.
>
> This is technical preview version to allow early testing of the feature.
> Things not (properly) addressed in this release:
> - The MOVALL command is not emulated. In our case there is really nothing
> to do here. We might need to revisit this in the future for DomU support.
> - The INVALL command might need some rework to be more efficient. Currently
> we iterate over all mapped LPIs, which might take a bit longer.
> - Indirect tables are not supported. This affects both the host and the
> virtual side.
> - The command queue locking is currently suboptimal and should be made more
> fine-grained in the future, if possible.
> - We don't move the LPIs on the host to the pCPU where the target VCPU
> is currently running on. Doing so would involve sending ITS commands to the
> host. We should investigate whether this is feasible during scheduling.
> - MOVI doesn't move pending IRQs.
> - We need to properly investigate the possible interaction when devices get
> removed. This requires to properly clean up and remove any associated
> resources like pending or in-flight LPIs, for instance.
It looks like you missed some items not (properly) addressed:
- SMMU support with ITS
- Prevent interrupt storm
- DomU support
* Export nr_lpis through arch_domain_config
* Protection of memory provided by the guest (e.g Device Table,
Collection Table...)
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 15/27] ARM: vITS: handle CLEAR command
2017-04-12 14:10 ` Julien Grall
@ 2017-04-12 14:29 ` Andre Przywara
2017-04-12 14:49 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 14:29 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 15:10, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> This introduces the ITS command handler for the CLEAR command, which
>> clears the pending state of an LPI.
>> This removes a not-yet injected, but already queued IRQ from a VCPU.
>> As read_itte() is now eventually used, we add the static keyword.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/vgic-v3-its.c | 52
>> ++++++++++++++++++++++++++++++++++++++++++++--
>> 1 file changed, 50 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index 632ab84..e24ab60 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -227,8 +227,8 @@ static bool read_itte_locked(struct virt_its *its,
>> uint32_t devid,
>> * This function takes care of the locking by taking the its_lock
>> itself, so
>> * a caller shall not hold this. Before returning, the lock is
>> dropped again.
>> */
>> -bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
>> - struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
>> +static bool read_itte(struct virt_its *its, uint32_t devid, uint32_t
>> evid,
>> + struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
>> {
>> bool ret;
>>
>> @@ -297,6 +297,51 @@ static uint64_t its_cmd_mask_field(uint64_t
>> *its_cmd, unsigned int word,
>> #define its_cmd_get_validbit(cmd) its_cmd_mask_field(cmd, 2,
>> 63, 1)
>> #define its_cmd_get_ittaddr(cmd) (its_cmd_mask_field(cmd, 2,
>> 8, 44) << 8)
>>
>> +/*
>> + * CLEAR removes the pending state from an LPI. */
>> +static int its_handle_clear(struct virt_its *its, uint64_t *cmdptr)
>> +{
>> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
>> + uint32_t eventid = its_cmd_get_id(cmdptr);
>> + struct pending_irq *p;
>> + struct vcpu *vcpu;
>> + uint32_t vlpi;
>> + unsigned long flags;
>> +
>> + /* Translate the DevID/EvID pair into a vCPU/vLPI pair. */
>> + if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
>
> So the vCPU ID is retrieved from guest memory, therefore you cannot
> trust the value for taking a lock.
>
> Indeed, a malicious guest could rewrite the value with another vCPU ID
> and you would take the wrong vCPU vGIC lock.
>
> What is the plan to fix that?
To (write-)protect the tables. I will mention that in the cover letter.
>
>> + return -1;
>> +
>> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
>
> I don't think this lock will protect correctly the pending_irq because
> if you do a MOVI before hand you will use the new vCPU ID and not the
> old one (see my explanation on patch #2).
>
>> +
>> + p = its->d->arch.vgic.handler->lpi_to_pending(its->d, vlpi);
>> + if ( !p )
>
> Please detail what means NULL here.
>
>> + {
>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>> + return -1;
>> + }
>> +
>> + /*
>> + * If the LPI is already visible on the guest, it is too late to
>> + * clear the pending state. However this is a benign race that can
>> + * happen on real hardware, too: If the LPI has already been
>> forwarded
>> + * to a CPU interface, a CLEAR request reaching the redistributor
>> has
>> + * no effect on that LPI anymore. Since LPIs are edge triggered and
>> + * have no active state, we don't need to care about this here.
>> + */
>> + if ( !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
>> + {
>> + /* Remove a pending, but not yet injected guest IRQ. */
>> + clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
>> + list_del_init(&p->inflight);
>> + list_del_init(&p->lr_queue);
>
> I am not sure to understand why you open-code gic_remove_from_queues
> rather than reworking it.
Well, actually all that gic_remove_from_queues does is:
list_del_init(&p->lr_queue);
under the VGIC lock with getting the pending_irq first.
I have the struct pending_irq already, also the lock. So there is no
point in calling that function (which would block anyway).
And since I wanted to keep changes to the existing code to a minimum, I
decided to just not touch this function, instead just put that *one*
line in here, next to the other list_del_init().
Does that sound sensible?
Cheers,
Andre.
>> + }
>> +
>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>> +
>> + return 0;
>> +}
>> +
>> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
>> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>>
>> @@ -326,6 +371,9 @@ static int vgic_its_handle_cmds(struct domain *d,
>> struct virt_its *its)
>>
>> switch ( its_cmd_get_command(command) )
>> {
>> + case GITS_CMD_CLEAR:
>> + ret = its_handle_clear(its, command);
>> + break;
>> case GITS_CMD_SYNC:
>> /* We handle ITS commands synchronously, so we ignore
>> SYNC. */
>> break;
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 15/27] ARM: vITS: handle CLEAR command
2017-04-12 14:29 ` Andre Przywara
@ 2017-04-12 14:49 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 14:49 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 15:29, Andre Przywara wrote:
> Hi,
>
> On 12/04/17 15:10, Julien Grall wrote:
>> Hi Andre,
>>
>> On 12/04/17 01:44, Andre Przywara wrote:
>>> This introduces the ITS command handler for the CLEAR command, which
>>> clears the pending state of an LPI.
>>> This removes a not-yet injected, but already queued IRQ from a VCPU.
>>> As read_itte() is now eventually used, we add the static keyword.
>>>
>>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>>> ---
>>> xen/arch/arm/vgic-v3-its.c | 52
>>> ++++++++++++++++++++++++++++++++++++++++++++--
>>> 1 file changed, 50 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>>> index 632ab84..e24ab60 100644
>>> --- a/xen/arch/arm/vgic-v3-its.c
>>> +++ b/xen/arch/arm/vgic-v3-its.c
>>> @@ -227,8 +227,8 @@ static bool read_itte_locked(struct virt_its *its,
>>> uint32_t devid,
>>> * This function takes care of the locking by taking the its_lock
>>> itself, so
>>> * a caller shall not hold this. Before returning, the lock is
>>> dropped again.
>>> */
>>> -bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
>>> - struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
>>> +static bool read_itte(struct virt_its *its, uint32_t devid, uint32_t
>>> evid,
>>> + struct vcpu **vcpu_ptr, uint32_t *vlpi_ptr)
>>> {
>>> bool ret;
>>>
>>> @@ -297,6 +297,51 @@ static uint64_t its_cmd_mask_field(uint64_t
>>> *its_cmd, unsigned int word,
>>> #define its_cmd_get_validbit(cmd) its_cmd_mask_field(cmd, 2,
>>> 63, 1)
>>> #define its_cmd_get_ittaddr(cmd) (its_cmd_mask_field(cmd, 2,
>>> 8, 44) << 8)
>>>
>>> +/*
>>> + * CLEAR removes the pending state from an LPI. */
>>> +static int its_handle_clear(struct virt_its *its, uint64_t *cmdptr)
>>> +{
>>> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
>>> + uint32_t eventid = its_cmd_get_id(cmdptr);
>>> + struct pending_irq *p;
>>> + struct vcpu *vcpu;
>>> + uint32_t vlpi;
>>> + unsigned long flags;
>>> +
>>> + /* Translate the DevID/EvID pair into a vCPU/vLPI pair. */
>>> + if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
>>
>> So the vCPU ID is retrieved from guest memory, therefore you cannot
>> trust the value for taking a lock.
>>
>> Indeed, a malicious guest could rewrite the value with another vCPU ID
>> and you would take the wrong vCPU vGIC lock.
>>
>> What is the plan to fix that?
>
> To (write-)protect the tables. I will mention that in the cover letter.
>
>>
>>> + return -1;
>>> +
>>> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
>>
>> I don't think this lock will protect correctly the pending_irq because
>> if you do a MOVI before hand you will use the new vCPU ID and not the
>> old one (see my explanation on patch #2).
>>
>>> +
>>> + p = its->d->arch.vgic.handler->lpi_to_pending(its->d, vlpi);
>>> + if ( !p )
>>
>> Please detail what means NULL here.
>>
>>> + {
>>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>>> + return -1;
>>> + }
>>> +
>>> + /*
>>> + * If the LPI is already visible on the guest, it is too late to
>>> + * clear the pending state. However this is a benign race that can
>>> + * happen on real hardware, too: If the LPI has already been
>>> forwarded
>>> + * to a CPU interface, a CLEAR request reaching the redistributor
>>> has
>>> + * no effect on that LPI anymore. Since LPIs are edge triggered and
>>> + * have no active state, we don't need to care about this here.
>>> + */
>>> + if ( !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
>>> + {
>>> + /* Remove a pending, but not yet injected guest IRQ. */
>>> + clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
>>> + list_del_init(&p->inflight);
>>> + list_del_init(&p->lr_queue);
>>
>> I am not sure to understand why you open-code gic_remove_from_queues
>> rather than reworking it.
>
> Well, actually all that gic_remove_from_queues does is:
> list_del_init(&p->lr_queue);
> under the VGIC lock with getting the pending_irq first.
> I have the struct pending_irq already, also the lock. So there is no
> point in calling that function (which would block anyway).
>
> And since I wanted to keep changes to the existing code to a minimum, I
> decided to just not touch this function, instead just put that *one*
> line in here, next to the other list_del_init().
> Does that sound sensible?
I am sorry but it does not. If one day someone decides to update
gic_remove_from_queues, he will have to remember that we open-coded in
MOVI because you didn't want to touch common code.
Common code is not set in stone, the goal is to abstract all the issues
to make easier to propagate change. I am always in favor of reworking
the common code.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 16/27] ARM: vITS: handle INT command
2017-04-12 0:44 ` [PATCH v8 16/27] ARM: vITS: handle INT command Andre Przywara
@ 2017-04-12 14:50 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 14:50 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 01:44, Andre Przywara wrote:
> The INT command sets a given LPI identified by a DeviceID/EventID pair
> as pending and thus triggers it to be injected.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 21 +++++++++++++++++++++
> 1 file changed, 21 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index e24ab60..eeb1306 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -342,6 +342,24 @@ static int its_handle_clear(struct virt_its *its, uint64_t *cmdptr)
> return 0;
> }
>
> +static int its_handle_int(struct virt_its *its, uint64_t *cmdptr)
> +{
> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
> + uint32_t eventid = its_cmd_get_id(cmdptr);
> + struct vcpu *vcpu;
> + uint32_t vlpi;
> +
> + if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
> + return -1;
Same problem as we patch #2, the vcpu may have changed by MOVI and
pending_irq would be protected by the wrong lock.
> +
> + if ( vlpi == INVALID_LPI )
> + return -1;
> +
> + vgic_vcpu_inject_irq(vcpu, vlpi);
> +
> + return 0;
> +}
> +
> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>
> @@ -374,6 +392,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_CLEAR:
> ret = its_handle_clear(its, command);
> break;
> + case GITS_CMD_INT:
> + ret = its_handle_int(its, command);
> + break;
> case GITS_CMD_SYNC:
> /* We handle ITS commands synchronously, so we ignore SYNC. */
> break;
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 17/27] ARM: vITS: handle MAPC command
2017-04-12 0:44 ` [PATCH v8 17/27] ARM: vITS: handle MAPC command Andre Przywara
@ 2017-04-12 14:51 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 14:51 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 01:44, Andre Przywara wrote:
> The MAPC command associates a given collection ID with a given
> redistributor, thus mapping collections to VCPUs.
> We just store the vcpu_id in the collection table for that.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Julien Grall <julien.grall@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 41 +++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 41 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index eeb1306..6e505cb 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -116,6 +116,21 @@ static paddr_t get_baser_phys_addr(uint64_t reg)
> #define COLL_TABLE_ENTRY_SIZE 2UL
>
> /* Must be called with the ITS lock held. */
> +static int its_set_collection(struct virt_its *its, uint16_t collid,
> + uint16_t vcpu_id)
> +{
> + paddr_t addr = get_baser_phys_addr(its->baser_coll);
> +
> + ASSERT(spin_is_locked(&its->its_lock));
> +
> + if ( collid >= its->max_collections )
> + return -ENOENT;
> +
> + return vgic_access_guest_memory(its->d, addr + collid * sizeof(uint16_t),
> + &vcpu_id, sizeof(vcpu_id), true);
> +}
> +
> +/* Must be called with the ITS lock held. */
> static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
> uint16_t collid)
> {
> @@ -360,6 +375,29 @@ static int its_handle_int(struct virt_its *its, uint64_t *cmdptr)
> return 0;
> }
>
> +static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
> +{
> + uint32_t collid = its_cmd_get_collection(cmdptr);
> + uint64_t rdbase = its_cmd_mask_field(cmdptr, 2, 16, 44);
> +
> + if ( collid >= its->max_collections )
> + return -1;
> +
> + if ( rdbase >= its->d->max_vcpus )
> + return -1;
> +
> + spin_lock(&its->its_lock);
> +
> + if ( its_cmd_get_validbit(cmdptr) )
> + its_set_collection(its, collid, rdbase);
> + else
> + its_set_collection(its, collid, UNMAPPED_COLLECTION);
> +
> + spin_unlock(&its->its_lock);
> +
> + return 0;
> +}
> +
> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>
> @@ -395,6 +433,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_INT:
> ret = its_handle_int(its, command);
> break;
> + case GITS_CMD_MAPC:
> + ret = its_handle_mapc(its, command);
> + break;
> case GITS_CMD_SYNC:
> /* We handle ITS commands synchronously, so we ignore SYNC. */
> break;
>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's
2017-04-12 10:25 ` Julien Grall
@ 2017-04-12 14:51 ` Andre Przywara
2017-04-12 14:52 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 14:51 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 11:25, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> For LPIs the struct pending_irq's are dynamically allocated and the
>> pointers will be stored in a radix tree. Since an LPI can be "unmapped"
>> at any time, teach the VGIC how to handle with irq_to_pending() returning
>> a NULL pointer.
>> We just do nothing in this case or clean up the LR if the virtual LPI
>> number was still in an LR.
>
> Why not all the irq_to_pending call are not protected (such as
> vgic_irq_to_migrate)?
Some of them are never called by LPIs.
Is it worth the put ASSERTs in there everywhere?
I can copy the table with all call sites and their evaluations from that
other email into this commit message, if that sounds good.
Fixed the rest.
Cheers,
Andre.
> This is a call to forget to check NULL if we
> decide to use the function in the future.
>
> Also, I would like a comment on top of irq_to_pending stating this can
> return NULL as you change the semantic of the function.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/gic.c | 37 ++++++++++++++++++++++++++++++++-----
>> xen/arch/arm/vgic.c | 6 ++++++
>> 2 files changed, 38 insertions(+), 5 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index dcb1783..62ae3b8 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -408,9 +408,15 @@ void gic_remove_from_queues(struct vcpu *v,
>> unsigned int virtual_irq)
>> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>
>> p = irq_to_pending(v, virtual_irq);
>> -
>> - if ( !list_empty(&p->lr_queue) )
>> - list_del_init(&p->lr_queue);
>> + /*
>> + * If an LPIs has been removed meanwhile, it has been cleaned up
>> + * already, so nothing to remove here.
>> + */
>> + if ( p )
>> + {
>> + if ( !list_empty(&p->lr_queue) )
>> + list_del_init(&p->lr_queue);
>> + }
>
> This could be simplified with:
>
> if ( p && !list_empty(&p->lr_queue) )
> list_del_init(...);
>
> Also, you probably want a likely(p).
>
>> spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>> }
>>
>> @@ -418,6 +424,10 @@ void gic_raise_inflight_irq(struct vcpu *v,
>> unsigned int virtual_irq)
>> {
>> struct pending_irq *n = irq_to_pending(v, virtual_irq);
>>
>> + /* If an LPI has been removed meanwhile, there is nothing left to
>> raise. */
>> + if ( !n )
>
> if ( unlikely(!n) )
>
>> + return;
>> +
>> ASSERT(spin_is_locked(&v->arch.vgic.lock));
>>
>> if ( list_empty(&n->lr_queue) )
>> @@ -437,6 +447,11 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned
>> int virtual_irq,
>> {
>> int i;
>> unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
>> + struct pending_irq *p = irq_to_pending(v, virtual_irq);
>> +
>> + if ( !p )
>
> if ( unlikely(!p) )
>
>> + /* An unmapped LPI does not need to be raised. */
>> + return;
>
> Please move this check after the ASSERT to keep the check on all the paths.
>
>>
>> ASSERT(spin_is_locked(&v->arch.vgic.lock));
>>
>> @@ -445,12 +460,12 @@ void gic_raise_guest_irq(struct vcpu *v,
>> unsigned int virtual_irq,
>> i = find_first_zero_bit(&this_cpu(lr_mask), nr_lrs);
>> if (i < nr_lrs) {
>> set_bit(i, &this_cpu(lr_mask));
>> - gic_set_lr(i, irq_to_pending(v, virtual_irq),
>> GICH_LR_PENDING);
>> + gic_set_lr(i, p, GICH_LR_PENDING);
>> return;
>> }
>> }
>>
>> - gic_add_to_lr_pending(v, irq_to_pending(v, virtual_irq));
>> + gic_add_to_lr_pending(v, p);
>> }
>>
>> static void gic_update_one_lr(struct vcpu *v, int i)
>> @@ -464,7 +479,19 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>>
>> gic_hw_ops->read_lr(i, &lr_val);
>> irq = lr_val.virq;
>> +
>
> 4th time I am saying this.... Spurious line.
>
>> p = irq_to_pending(v, irq);
>> + /* An LPI might have been unmapped, in which case we just clean
>> up here. */
>> + if ( !p )
>
> unlikely(!p)
>
>> + {
>> + ASSERT(is_lpi(irq));
>> +
>> + gic_hw_ops->clear_lr(i);
>> + clear_bit(i, &this_cpu(lr_mask));
>> +
>> + return;
>> + }
>> +
>> if ( lr_val.state & GICH_LR_ACTIVE )
>> {
>> set_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index c953f13..b9fc837 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -473,6 +473,12 @@ void vgic_vcpu_inject_irq(struct vcpu *v,
>> unsigned int virq)
>> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>
>> n = irq_to_pending(v, virq);
>> + /* If an LPI has been removed, there is nothing to inject here. */
>> + if ( !n )
>
> unlikely(...)
>
>> + {
>> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>> + return;
>> + }
>>
>> priority = vgic_get_virq_priority(v, virq);
>>
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's
2017-04-12 14:51 ` Andre Przywara
@ 2017-04-12 14:52 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 14:52 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 15:51, Andre Przywara wrote:
> Hi,
>
> On 12/04/17 11:25, Julien Grall wrote:
>> Hi Andre,
>>
>> On 12/04/17 01:44, Andre Przywara wrote:
>>> For LPIs the struct pending_irq's are dynamically allocated and the
>>> pointers will be stored in a radix tree. Since an LPI can be "unmapped"
>>> at any time, teach the VGIC how to handle with irq_to_pending() returning
>>> a NULL pointer.
>>> We just do nothing in this case or clean up the LR if the virtual LPI
>>> number was still in an LR.
>>
>> Why not all the irq_to_pending call are not protected (such as
>> vgic_irq_to_migrate)?
>
> Some of them are never called by LPIs.
> Is it worth the put ASSERTs in there everywhere?
Yes.
> I can copy the table with all call sites and their evaluations from that
> other email into this commit message, if that sounds good.
And yes. A commit message should contain all the details saving us time
to look in the e-mails...
>
> Fixed the rest.
>
> Cheers,
> Andre.
>
>> This is a call to forget to check NULL if we
>> decide to use the function in the future.
>>
>> Also, I would like a comment on top of irq_to_pending stating this can
>> return NULL as you change the semantic of the function.
>>>
>>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>>> ---
>>> xen/arch/arm/gic.c | 37 ++++++++++++++++++++++++++++++++-----
>>> xen/arch/arm/vgic.c | 6 ++++++
>>> 2 files changed, 38 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>> index dcb1783..62ae3b8 100644
>>> --- a/xen/arch/arm/gic.c
>>> +++ b/xen/arch/arm/gic.c
>>> @@ -408,9 +408,15 @@ void gic_remove_from_queues(struct vcpu *v,
>>> unsigned int virtual_irq)
>>> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>>
>>> p = irq_to_pending(v, virtual_irq);
>>> -
>>> - if ( !list_empty(&p->lr_queue) )
>>> - list_del_init(&p->lr_queue);
>>> + /*
>>> + * If an LPIs has been removed meanwhile, it has been cleaned up
>>> + * already, so nothing to remove here.
>>> + */
>>> + if ( p )
>>> + {
>>> + if ( !list_empty(&p->lr_queue) )
>>> + list_del_init(&p->lr_queue);
>>> + }
>>
>> This could be simplified with:
>>
>> if ( p && !list_empty(&p->lr_queue) )
>> list_del_init(...);
>>
>> Also, you probably want a likely(p).
>>
>>> spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>>> }
>>>
>>> @@ -418,6 +424,10 @@ void gic_raise_inflight_irq(struct vcpu *v,
>>> unsigned int virtual_irq)
>>> {
>>> struct pending_irq *n = irq_to_pending(v, virtual_irq);
>>>
>>> + /* If an LPI has been removed meanwhile, there is nothing left to
>>> raise. */
>>> + if ( !n )
>>
>> if ( unlikely(!n) )
>>
>>> + return;
>>> +
>>> ASSERT(spin_is_locked(&v->arch.vgic.lock));
>>>
>>> if ( list_empty(&n->lr_queue) )
>>> @@ -437,6 +447,11 @@ void gic_raise_guest_irq(struct vcpu *v, unsigned
>>> int virtual_irq,
>>> {
>>> int i;
>>> unsigned int nr_lrs = gic_hw_ops->info->nr_lrs;
>>> + struct pending_irq *p = irq_to_pending(v, virtual_irq);
>>> +
>>> + if ( !p )
>>
>> if ( unlikely(!p) )
>>
>>> + /* An unmapped LPI does not need to be raised. */
>>> + return;
>>
>> Please move this check after the ASSERT to keep the check on all the paths.
>>
>>>
>>> ASSERT(spin_is_locked(&v->arch.vgic.lock));
>>>
>>> @@ -445,12 +460,12 @@ void gic_raise_guest_irq(struct vcpu *v,
>>> unsigned int virtual_irq,
>>> i = find_first_zero_bit(&this_cpu(lr_mask), nr_lrs);
>>> if (i < nr_lrs) {
>>> set_bit(i, &this_cpu(lr_mask));
>>> - gic_set_lr(i, irq_to_pending(v, virtual_irq),
>>> GICH_LR_PENDING);
>>> + gic_set_lr(i, p, GICH_LR_PENDING);
>>> return;
>>> }
>>> }
>>>
>>> - gic_add_to_lr_pending(v, irq_to_pending(v, virtual_irq));
>>> + gic_add_to_lr_pending(v, p);
>>> }
>>>
>>> static void gic_update_one_lr(struct vcpu *v, int i)
>>> @@ -464,7 +479,19 @@ static void gic_update_one_lr(struct vcpu *v, int i)
>>>
>>> gic_hw_ops->read_lr(i, &lr_val);
>>> irq = lr_val.virq;
>>> +
>>
>> 4th time I am saying this.... Spurious line.
>>
>>> p = irq_to_pending(v, irq);
>>> + /* An LPI might have been unmapped, in which case we just clean
>>> up here. */
>>> + if ( !p )
>>
>> unlikely(!p)
>>
>>> + {
>>> + ASSERT(is_lpi(irq));
>>> +
>>> + gic_hw_ops->clear_lr(i);
>>> + clear_bit(i, &this_cpu(lr_mask));
>>> +
>>> + return;
>>> + }
>>> +
>>> if ( lr_val.state & GICH_LR_ACTIVE )
>>> {
>>> set_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>> index c953f13..b9fc837 100644
>>> --- a/xen/arch/arm/vgic.c
>>> +++ b/xen/arch/arm/vgic.c
>>> @@ -473,6 +473,12 @@ void vgic_vcpu_inject_irq(struct vcpu *v,
>>> unsigned int virq)
>>> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>>
>>> n = irq_to_pending(v, virq);
>>> + /* If an LPI has been removed, there is nothing to inject here. */
>>> + if ( !n )
>>
>> unlikely(...)
>>
>>> + {
>>> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>>> + return;
>>> + }
>>>
>>> priority = vgic_get_virq_priority(v, virq);
>>>
>>>
>>
>> Cheers,
>>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-04-12 0:44 ` [PATCH v8 18/27] ARM: vITS: handle MAPD command Andre Przywara
@ 2017-04-12 15:21 ` Julien Grall
2017-04-12 17:03 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 15:21 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> The MAPD command maps a device by associating a memory region for
> storing ITEs with a certain device ID. Since it features a valid bit,
> MAPD also covers the "unmap" functionality, which we also cover here.
> We store the given guest physical address in the device table, and, if
> this command comes from Dom0, tell the host ITS driver about this new
> mapping, so it can issue the corresponding host MAPD command and create
> the required tables. We take care of rolling back actions should one
> step fail.
> Upon unmapping a device we make sure we clean up all associated
> resources and release the memory again.
> We use our existing guest memory access function to find the right ITT
> entry and store the mapping there (in guest memory).
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic-v3-its.c | 60 +++++++++++++++++
> xen/arch/arm/gic-v3-lpi.c | 18 +++++
> xen/arch/arm/vgic-v3-its.c | 137 +++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic_v3_its.h | 6 ++
> 4 files changed, 221 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index aebc257..900c9d1 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -800,6 +800,66 @@ out:
> return ret;
> }
>
> +/* Must be called with the its_device_lock held. */
> +static struct its_device *get_its_device(struct domain *d, paddr_t vdoorbell,
> + uint32_t vdevid)
> +{
> + struct rb_node *node = d->arch.vgic.its_devices.rb_node;
> + struct its_device *dev;
> +
> + ASSERT(spin_is_locked(&d->arch.vgic.its_devices_lock));
> +
> + while (node)
> + {
> + int cmp;
> +
> + dev = rb_entry(node, struct its_device, rbnode);
> + cmp = compare_its_guest_devices(dev, vdoorbell, vdevid);
> +
> + if ( !cmp )
> + return dev;
> +
> + if ( cmp > 0 )
> + node = node->rb_left;
> + else
> + node = node->rb_right;
> + }
> +
> + return NULL;
> +}
> +
> +static uint32_t get_host_lpi(struct its_device *dev, uint32_t eventid)
> +{
> + uint32_t host_lpi = INVALID_LPI;
> +
> + if ( dev && (eventid < dev->eventids) )
> + host_lpi = dev->host_lpi_blocks[eventid / LPI_BLOCK] +
> + (eventid % LPI_BLOCK);
> +
> + return host_lpi;
> +}
> +
> +int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
> + uint32_t vdevid, uint32_t veventid)
> +{
> + struct its_device *dev;
> + uint32_t host_lpi = INVALID_LPI;
> +
> + spin_lock(&d->arch.vgic.its_devices_lock);
> + dev = get_its_device(d, vdoorbell_address, vdevid);
> + if ( dev && veventid <= dev->eventids )
> + host_lpi = get_host_lpi(dev, veventid);
> + spin_unlock(&d->arch.vgic.its_devices_lock);
> +
> + if ( host_lpi == INVALID_LPI )
> + return -EINVAL;
> +
> + gicv3_lpi_update_host_entry(host_lpi, d->domain_id,
> + INVALID_VCPU_ID, INVALID_LPI);
> +
> + return 0;
> +}
> +
> /* Scan the DT for any ITS nodes and create a list of host ITSes out of it. */
> void gicv3_its_dt_init(const struct dt_device_node *node)
> {
> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
> index 44f6315..d427539 100644
> --- a/xen/arch/arm/gic-v3-lpi.c
> +++ b/xen/arch/arm/gic-v3-lpi.c
> @@ -207,6 +207,24 @@ out:
> irq_exit();
> }
>
> +void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
> + unsigned int vcpu_id, uint32_t virt_lpi)
> +{
> + union host_lpi *hlpip, hlpi;
> +
> + ASSERT(host_lpi >= LPI_OFFSET);
> +
> + host_lpi -= LPI_OFFSET;
> +
> + hlpip = &lpi_data.host_lpis[host_lpi / HOST_LPIS_PER_PAGE][host_lpi % HOST_LPIS_PER_PAGE];
> +
> + hlpi.virt_lpi = virt_lpi;
> + hlpi.dom_id = domain_id;
> + hlpi.vcpu_id = vcpu_id;
> +
> + write_u64_atomic(&hlpip->data, hlpi.data);
> +}
> +
> static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
> {
> uint64_t val;
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 6e505cb..104017e 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -52,6 +52,7 @@
> */
> struct virt_its {
> struct domain *d;
> + paddr_t doorbell_address;
> unsigned int devid_bits;
> unsigned int evid_bits;
> spinlock_t vcmd_lock; /* Protects the virtual command buffer, which */
> @@ -166,6 +167,20 @@ static struct vcpu *get_vcpu_from_collection(struct virt_its *its,
> #define DEV_TABLE_ENTRY(addr, bits) \
> (((addr) & GENMASK(51, 8)) | (((bits) - 1) & GENMASK(4, 0)))
>
> +/* Set the address of an ITT for a given device ID. */
> +static int its_set_itt_address(struct virt_its *its, uint32_t devid,
> + paddr_t itt_address, uint32_t nr_bits)
> +{
> + paddr_t addr = get_baser_phys_addr(its->baser_dev);
> + uint64_t itt_entry = DEV_TABLE_ENTRY(itt_address, nr_bits);
> +
> + if ( devid >= its->max_devices )
> + return -ENOENT;
> +
> + return vgic_access_guest_memory(its->d, addr + devid * sizeof(uint64_t),
> + &itt_entry, sizeof(itt_entry), true);
> +}
> +
> /*
> * Lookup the address of the Interrupt Translation Table associated with
> * that device ID.
> @@ -398,6 +413,125 @@ static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
> return 0;
> }
>
> +/* Must be called with the ITS lock held. */
> +static int its_discard_event(struct virt_its *its,
> + uint32_t vdevid, uint32_t vevid)
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> + struct vcpu *vcpu;
> + uint32_t vlpi;
> +
> + ASSERT(spin_is_locked(&its->its_lock));
> +
> + if ( !read_itte_locked(its, vdevid, vevid, &vcpu, &vlpi) )
> + return -ENOENT;
> +
> + if ( vlpi == INVALID_LPI )
> + return -ENOENT;
> +
> + /* Lock this VCPU's VGIC to make sure nobody is using the pending_irq. */
> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
There is an interesting issue happening with this code. You don't check
the content of the memory provided by the guest. So a malicious guest
could craft the memory in order to setup mapping with known vlpi and a
different vCPU.
This would lead to use the wrong lock here and corrupt the list.
> +
> + /* Remove the pending_irq from the tree. */
> + write_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
> + p = radix_tree_delete(&its->d->arch.vgic.pend_lpi_tree, vlpi);
> + write_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
> +
> + if ( !p )
> + {
> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
> +
> + return -ENOENT;
> + }
> +
> + /* Cleanup the pending_irq, to be reusable later. */
> + list_del_init(&p->inflight);
> + list_del_init(&p->lr_queue);
> + p->status = 0;
> + p->lr = GIC_INVALID_LR;
> + p->irq = INVALID_LPI;
Can't we have a generic helper to do that?
> +
> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
> +
> + /* Remove the corresponding host LPI entry */
> + return gicv3_remove_guest_event(its->d, its->doorbell_address,
> + vdevid, vevid);
> +}
> +
> +static int its_unmap_device(struct virt_its *its, uint32_t devid)
> +{
> + uint64_t itt, evid;
> + int ret;
> +
> + spin_lock(&its->its_lock);
> +
> + ret = its_get_itt(its, devid, &itt);
> + if ( ret )
> + {
> + spin_unlock(&its->its_lock);
> + return ret;
> + }
> +
> + for ( evid = 0; evid < DEV_TABLE_ITT_SIZE(itt); evid++ )
> + /* Don't care about errors here, clean up as much as possible. */
> + its_discard_event(its, devid, evid);
> +
> + spin_unlock(&its->its_lock);
This code can be long to execute as the number of event can be huge. How
do you plan to handle that?
*hint* this likely means a comment + ASSERT + TODO in the cover letter
*hint*
> +
> + return 0;
> +}
> +
> +static int its_handle_mapd(struct virt_its *its, uint64_t *cmdptr)
> +{
> + /* size and devid get validated by the functions called below. */
> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
> + unsigned int size = its_cmd_get_size(cmdptr) + 1;
> + bool valid = its_cmd_get_validbit(cmdptr);
> + paddr_t itt_addr = its_cmd_get_ittaddr(cmdptr);
> + int ret;
> +
> + /* Sanitize the number of events. */
> + if ( valid && (size > its->evid_bits) )
> + return -1;
> +
> + /*
> + * There is no easy and clean way for Xen to know the ITS device ID of a
> + * particular (PCI) device, so we have to rely on the guest telling
> + * us about it. For *now* we are just using the device ID *Dom0* uses,
> + * because the driver there has the actual knowledge.
> + * Eventually this will be replaced with a dedicated hypercall to
> + * announce pass-through of devices.
> + */
> + if ( is_hardware_domain(its->d) )
> + {
> + if ( !valid )
> + /* Discard all events and remove pending LPIs. */
> + its_unmap_device(its, devid);
Likely, this call will have to be done for all the guests.
> +
> + /*
> + * Dom0's ITSes are mapped 1:1, so both addresses are the same.
> + * Also the device IDs are equal.
> + */
> + ret = gicv3_its_map_guest_device(its->d, its->doorbell_address, devid,
> + its->doorbell_address, devid,
> + BIT(size), valid);
> + if ( ret && valid )
> + return ret;
> + }
> +
> + spin_lock(&its->its_lock);
> +
> + if ( valid )
> + ret = its_set_itt_address(its, devid, itt_addr, size);
> + else
> + ret = its_set_itt_address(its, devid, INVALID_PADDR, 1);
> +
> + spin_unlock(&its->its_lock);
> +
> + return ret;
> +}
> +
> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>
> @@ -436,6 +570,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_MAPC:
> ret = its_handle_mapc(its, command);
> break;
> + case GITS_CMD_MAPD:
> + ret = its_handle_mapd(its, command);
> + break;
> case GITS_CMD_SYNC:
> /* We handle ITS commands synchronously, so we ignore SYNC. */
> break;
> diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
> index 40f4ef5..60ffdb6 100644
> --- a/xen/include/asm-arm/gic_v3_its.h
> +++ b/xen/include/asm-arm/gic_v3_its.h
> @@ -169,6 +169,12 @@ int gicv3_its_map_guest_device(struct domain *d,
> int gicv3_allocate_host_lpi_block(struct domain *d, uint32_t *first_lpi);
> void gicv3_free_host_lpi_block(uint32_t first_lpi);
>
> +int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
> + uint32_t vdevid, uint32_t veventid);
> +
> +void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
> + unsigned int vcpu_id, uint32_t virt_lpi);
> +
> #else
>
> static inline void gicv3_its_dt_init(const struct dt_device_node *node)
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 19/27] ARM: vITS: handle MAPTI command
2017-04-12 0:44 ` [PATCH v8 19/27] ARM: vITS: handle MAPTI command Andre Przywara
@ 2017-04-12 16:18 ` Julien Grall
2017-04-12 16:27 ` Andre Przywara
2017-04-12 17:16 ` Julien Grall
2017-04-12 17:25 ` Julien Grall
2 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 16:18 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> The MAPTI commands associates a DeviceID/EventID pair with a LPI/CPU
> pair and actually instantiates LPI interrupts.
> We connect the already allocated host LPI to this virtual LPI, so that
> any triggering LPI on the host can be quickly forwarded to a guest.
> Beside entering the VCPU and the virtual LPI number in the respective
> host LPI entry, we also initialize and add the already allocated
> struct pending_irq to our radix tree, so that we can now easily find it
> by its virtual LPI number.
> We also read the property table to update the enabled bit and the
> priority for our new LPI, as we might have missed this during an earlier
> INVALL call (which only checks mapped LPIs).
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic-v3-its.c | 35 ++++++++++++
> xen/arch/arm/vgic-v3-its.c | 114 +++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic_v3_its.h | 3 ++
> 3 files changed, 152 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 900c9d1..fa1f2d5 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -860,6 +860,41 @@ int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
> return 0;
> }
>
> +/*
> + * Connects the event ID for an already assigned device to the given VCPU/vLPI
> + * pair. The corresponding physical LPI is already mapped on the host side
> + * (when assigning the physical device to the guest), so we just connect the
> + * target VCPU/vLPI pair to that interrupt to inject it properly if it fires.
> + * Returns a pointer to the already allocated struct pending_irq that is
> + * meant to be used by that event.
> + */
> +struct pending_irq *gicv3_assign_guest_event(struct domain *d,
> + paddr_t vdoorbell_address,
> + uint32_t vdevid, uint32_t veventid,
> + struct vcpu *v, uint32_t virt_lpi)
> +{
> + struct its_device *dev;
> + struct pending_irq *pirq = NULL;
> + uint32_t host_lpi = 0;
> +
> + spin_lock(&d->arch.vgic.its_devices_lock);
> + dev = get_its_device(d, vdoorbell_address, vdevid);
> + if ( dev )
> + {
> + host_lpi = get_host_lpi(dev, veventid);
> + pirq = &dev->pend_irqs[veventid];
> + }
> + spin_unlock(&d->arch.vgic.its_devices_lock);
> +
> + if ( !host_lpi || !pirq )
Again, how pirq could be NULL if host_lpi is set?
> + return NULL;
> +
> + gicv3_lpi_update_host_entry(host_lpi, d->domain_id,
> + v ? v->vcpu_id : INVALID_VCPU_ID, virt_lpi);
> +
> + return pirq;
> +}
> +
> /* Scan the DT for any ITS nodes and create a list of host ITSes out of it. */
> void gicv3_its_dt_init(const struct dt_device_node *node)
> {
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 104017e..b7e61b2 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -390,6 +390,34 @@ static int its_handle_int(struct virt_its *its, uint64_t *cmdptr)
> return 0;
> }
>
> +/*
> + * For a given virtual LPI read the enabled bit and priority from the virtual
> + * property table and update the virtual IRQ's state in the given pending_irq.
> + * Must be called with the respective VGIC VCPU lock held.
> + */
> +static int update_lpi_property(struct domain *d, uint32_t vlpi,
> + struct pending_irq *p)
> +{
> + paddr_t addr;
> + uint8_t property;
> + int ret;
> +
> + addr = d->arch.vgic.rdist_propbase & GENMASK(51, 12);
> +
> + ret = vgic_access_guest_memory(d, addr + vlpi - LPI_OFFSET,
> + &property, sizeof(property), false);
> + if ( ret )
> + return ret;
> +
> + p->lpi_priority = property & LPI_PROP_PRIO_MASK;
> + if ( property & LPI_PROP_ENABLED )
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + else
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + return 0;
> +}
> +
> static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
> {
> uint32_t collid = its_cmd_get_collection(cmdptr);
> @@ -532,6 +560,88 @@ static int its_handle_mapd(struct virt_its *its, uint64_t *cmdptr)
> return ret;
> }
>
> +static int its_handle_mapti(struct virt_its *its, uint64_t *cmdptr)
> +{
> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
> + uint32_t eventid = its_cmd_get_id(cmdptr);
> + uint32_t intid = its_cmd_get_physical_id(cmdptr), _intid;
> + uint16_t collid = its_cmd_get_collection(cmdptr);
> + struct pending_irq *pirq;
> + struct vcpu *vcpu = NULL;
> + int ret = -1;
> +
> + if ( its_cmd_get_command(cmdptr) == GITS_CMD_MAPI )
> + intid = eventid;
> +
> + spin_lock(&its->its_lock);
> + /*
> + * Check whether there is a valid existing mapping. If yes, behavior is
> + * unpredictable, we choose to ignore this command here.
> + * This makes sure we start with a pristine pending_irq below.
> + */
> + if ( read_itte_locked(its, devid, eventid, &vcpu, &_intid) &&
> + _intid != INVALID_LPI )
> + {
> + spin_unlock(&its->its_lock);
> + return -1;
> + }
> +
> + /* Enter the mapping in our virtual ITS tables. */
> + if ( !write_itte_locked(its, devid, eventid, collid, intid, &vcpu) )
> + {
> + spin_unlock(&its->its_lock);
> + return -1;
> + }
> +
> + spin_unlock(&its->its_lock);
> +
> + /*
> + * Connect this virtual LPI to the corresponding host LPI, which is
> + * determined by the same device ID and event ID on the host side.
> + * This returns us the corresponding, still unused pending_irq.
> + */
> + pirq = gicv3_assign_guest_event(its->d, its->doorbell_address,
> + devid, eventid, vcpu, intid);
> + if ( !pirq )
> + goto out_remove_mapping;
> +
> + vgic_init_pending_irq(pirq, intid);
> +
> + /*
> + * Now read the guest's property table to initialize our cached state.
> + * It can't fire at this time, because it is not known to the host yet.
> + * We don't need the VGIC VCPU lock here, because the pending_irq isn't
> + * in the radix tree yet.
> + */
> + ret = update_lpi_property(its->d, intid, pirq);
> + if ( ret )
> + goto out_remove_host_entry;
> +
> + pirq->lpi_vcpu_id = vcpu->vcpu_id;
> +
> + /*
> + * Now insert the pending_irq into the domain's LPI tree, so that
> + * it becomes live.
> + */
> + write_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
> + ret = radix_tree_insert(&its->d->arch.vgic.pend_lpi_tree, intid, pirq);
> + write_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
> +
> + if ( !ret )
> + return 0;
> +
> +out_remove_host_entry:
> + gicv3_remove_guest_event(its->d, its->doorbell_address, devid, eventid);
> +
> +out_remove_mapping:
> + spin_lock(&its->its_lock);
> + write_itte_locked(its, devid, eventid,
> + UNMAPPED_COLLECTION, INVALID_LPI, NULL);
> + spin_unlock(&its->its_lock);
> +
> + return ret;
> +}
> +
> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>
> @@ -573,6 +683,10 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_MAPD:
> ret = its_handle_mapd(its, command);
> break;
> + case GITS_CMD_MAPI:
> + case GITS_CMD_MAPTI:
> + ret = its_handle_mapti(its, command);
> + break;
> case GITS_CMD_SYNC:
> /* We handle ITS commands synchronously, so we ignore SYNC. */
> break;
> diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
> index 60ffdb6..7b16aeb 100644
> --- a/xen/include/asm-arm/gic_v3_its.h
> +++ b/xen/include/asm-arm/gic_v3_its.h
> @@ -172,6 +172,9 @@ void gicv3_free_host_lpi_block(uint32_t first_lpi);
> int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
> uint32_t vdevid, uint32_t veventid);
>
> +struct pending_irq *gicv3_assign_guest_event(struct domain *d, paddr_t doorbell,
> + uint32_t devid, uint32_t eventid,
> + struct vcpu *v, uint32_t virt_lpi);
> void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
> unsigned int vcpu_id, uint32_t virt_lpi);
>
>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 19/27] ARM: vITS: handle MAPTI command
2017-04-12 16:18 ` Julien Grall
@ 2017-04-12 16:27 ` Andre Przywara
0 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 16:27 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 17:18, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> The MAPTI commands associates a DeviceID/EventID pair with a LPI/CPU
>> pair and actually instantiates LPI interrupts.
>> We connect the already allocated host LPI to this virtual LPI, so that
>> any triggering LPI on the host can be quickly forwarded to a guest.
>> Beside entering the VCPU and the virtual LPI number in the respective
>> host LPI entry, we also initialize and add the already allocated
>> struct pending_irq to our radix tree, so that we can now easily find it
>> by its virtual LPI number.
>> We also read the property table to update the enabled bit and the
>> priority for our new LPI, as we might have missed this during an earlier
>> INVALL call (which only checks mapped LPIs).
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/gic-v3-its.c | 35 ++++++++++++
>> xen/arch/arm/vgic-v3-its.c | 114
>> +++++++++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/gic_v3_its.h | 3 ++
>> 3 files changed, 152 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> index 900c9d1..fa1f2d5 100644
>> --- a/xen/arch/arm/gic-v3-its.c
>> +++ b/xen/arch/arm/gic-v3-its.c
>> @@ -860,6 +860,41 @@ int gicv3_remove_guest_event(struct domain *d,
>> paddr_t vdoorbell_address,
>> return 0;
>> }
>>
>> +/*
>> + * Connects the event ID for an already assigned device to the given
>> VCPU/vLPI
>> + * pair. The corresponding physical LPI is already mapped on the host
>> side
>> + * (when assigning the physical device to the guest), so we just
>> connect the
>> + * target VCPU/vLPI pair to that interrupt to inject it properly if
>> it fires.
>> + * Returns a pointer to the already allocated struct pending_irq that is
>> + * meant to be used by that event.
>> + */
>> +struct pending_irq *gicv3_assign_guest_event(struct domain *d,
>> + paddr_t vdoorbell_address,
>> + uint32_t vdevid,
>> uint32_t veventid,
>> + struct vcpu *v, uint32_t
>> virt_lpi)
>> +{
>> + struct its_device *dev;
>> + struct pending_irq *pirq = NULL;
>> + uint32_t host_lpi = 0;
>> +
>> + spin_lock(&d->arch.vgic.its_devices_lock);
>> + dev = get_its_device(d, vdoorbell_address, vdevid);
>> + if ( dev )
>> + {
>> + host_lpi = get_host_lpi(dev, veventid);
>> + pirq = &dev->pend_irqs[veventid];
>> + }
>> + spin_unlock(&d->arch.vgic.its_devices_lock);
>> +
>> + if ( !host_lpi || !pirq )
>
> Again, how pirq could be NULL if host_lpi is set?
Aaargh, I missed that again. Originally I wanted to be super sure, but
it's really not necessary. Sorry, consider this fixed.
Cheers,
Andre.
>> + return NULL;
>> +
>> + gicv3_lpi_update_host_entry(host_lpi, d->domain_id,
>> + v ? v->vcpu_id : INVALID_VCPU_ID,
>> virt_lpi);
>> +
>> + return pirq;
>> +}
>> +
>> /* Scan the DT for any ITS nodes and create a list of host ITSes out
>> of it. */
>> void gicv3_its_dt_init(const struct dt_device_node *node)
>> {
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index 104017e..b7e61b2 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -390,6 +390,34 @@ static int its_handle_int(struct virt_its *its,
>> uint64_t *cmdptr)
>> return 0;
>> }
>>
>> +/*
>> + * For a given virtual LPI read the enabled bit and priority from the
>> virtual
>> + * property table and update the virtual IRQ's state in the given
>> pending_irq.
>> + * Must be called with the respective VGIC VCPU lock held.
>> + */
>> +static int update_lpi_property(struct domain *d, uint32_t vlpi,
>> + struct pending_irq *p)
>> +{
>> + paddr_t addr;
>> + uint8_t property;
>> + int ret;
>> +
>> + addr = d->arch.vgic.rdist_propbase & GENMASK(51, 12);
>> +
>> + ret = vgic_access_guest_memory(d, addr + vlpi - LPI_OFFSET,
>> + &property, sizeof(property), false);
>> + if ( ret )
>> + return ret;
>> +
>> + p->lpi_priority = property & LPI_PROP_PRIO_MASK;
>> + if ( property & LPI_PROP_ENABLED )
>> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>> + else
>> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>> +
>> + return 0;
>> +}
>> +
>> static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
>> {
>> uint32_t collid = its_cmd_get_collection(cmdptr);
>> @@ -532,6 +560,88 @@ static int its_handle_mapd(struct virt_its *its,
>> uint64_t *cmdptr)
>> return ret;
>> }
>>
>> +static int its_handle_mapti(struct virt_its *its, uint64_t *cmdptr)
>> +{
>> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
>> + uint32_t eventid = its_cmd_get_id(cmdptr);
>> + uint32_t intid = its_cmd_get_physical_id(cmdptr), _intid;
>> + uint16_t collid = its_cmd_get_collection(cmdptr);
>> + struct pending_irq *pirq;
>> + struct vcpu *vcpu = NULL;
>> + int ret = -1;
>> +
>> + if ( its_cmd_get_command(cmdptr) == GITS_CMD_MAPI )
>> + intid = eventid;
>> +
>> + spin_lock(&its->its_lock);
>> + /*
>> + * Check whether there is a valid existing mapping. If yes,
>> behavior is
>> + * unpredictable, we choose to ignore this command here.
>> + * This makes sure we start with a pristine pending_irq below.
>> + */
>> + if ( read_itte_locked(its, devid, eventid, &vcpu, &_intid) &&
>> + _intid != INVALID_LPI )
>> + {
>> + spin_unlock(&its->its_lock);
>> + return -1;
>> + }
>> +
>> + /* Enter the mapping in our virtual ITS tables. */
>> + if ( !write_itte_locked(its, devid, eventid, collid, intid, &vcpu) )
>> + {
>> + spin_unlock(&its->its_lock);
>> + return -1;
>> + }
>> +
>> + spin_unlock(&its->its_lock);
>> +
>> + /*
>> + * Connect this virtual LPI to the corresponding host LPI, which is
>> + * determined by the same device ID and event ID on the host side.
>> + * This returns us the corresponding, still unused pending_irq.
>> + */
>> + pirq = gicv3_assign_guest_event(its->d, its->doorbell_address,
>> + devid, eventid, vcpu, intid);
>> + if ( !pirq )
>> + goto out_remove_mapping;
>> +
>> + vgic_init_pending_irq(pirq, intid);
>> +
>> + /*
>> + * Now read the guest's property table to initialize our cached
>> state.
>> + * It can't fire at this time, because it is not known to the
>> host yet.
>> + * We don't need the VGIC VCPU lock here, because the pending_irq
>> isn't
>> + * in the radix tree yet.
>> + */
>> + ret = update_lpi_property(its->d, intid, pirq);
>> + if ( ret )
>> + goto out_remove_host_entry;
>> +
>> + pirq->lpi_vcpu_id = vcpu->vcpu_id;
>> +
>> + /*
>> + * Now insert the pending_irq into the domain's LPI tree, so that
>> + * it becomes live.
>> + */
>> + write_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
>> + ret = radix_tree_insert(&its->d->arch.vgic.pend_lpi_tree, intid,
>> pirq);
>> + write_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
>> +
>> + if ( !ret )
>> + return 0;
>> +
>> +out_remove_host_entry:
>> + gicv3_remove_guest_event(its->d, its->doorbell_address, devid,
>> eventid);
>> +
>> +out_remove_mapping:
>> + spin_lock(&its->its_lock);
>> + write_itte_locked(its, devid, eventid,
>> + UNMAPPED_COLLECTION, INVALID_LPI, NULL);
>> + spin_unlock(&its->its_lock);
>> +
>> + return ret;
>> +}
>> +
>> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
>> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>>
>> @@ -573,6 +683,10 @@ static int vgic_its_handle_cmds(struct domain *d,
>> struct virt_its *its)
>> case GITS_CMD_MAPD:
>> ret = its_handle_mapd(its, command);
>> break;
>> + case GITS_CMD_MAPI:
>> + case GITS_CMD_MAPTI:
>> + ret = its_handle_mapti(its, command);
>> + break;
>> case GITS_CMD_SYNC:
>> /* We handle ITS commands synchronously, so we ignore
>> SYNC. */
>> break;
>> diff --git a/xen/include/asm-arm/gic_v3_its.h
>> b/xen/include/asm-arm/gic_v3_its.h
>> index 60ffdb6..7b16aeb 100644
>> --- a/xen/include/asm-arm/gic_v3_its.h
>> +++ b/xen/include/asm-arm/gic_v3_its.h
>> @@ -172,6 +172,9 @@ void gicv3_free_host_lpi_block(uint32_t first_lpi);
>> int gicv3_remove_guest_event(struct domain *d, paddr_t
>> vdoorbell_address,
>> uint32_t vdevid, uint32_t
>> veventid);
>>
>> +struct pending_irq *gicv3_assign_guest_event(struct domain *d,
>> paddr_t doorbell,
>> + uint32_t devid, uint32_t
>> eventid,
>> + struct vcpu *v, uint32_t
>> virt_lpi);
>> void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
>> unsigned int vcpu_id, uint32_t
>> virt_lpi);
>>
>>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs
2017-04-12 0:44 ` [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs Andre Przywara
2017-04-12 9:46 ` Andre Przywara
@ 2017-04-12 16:49 ` Julien Grall
1 sibling, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 16:49 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
TBH, I would have expected this patch to be split in two:
- Introduction of the flag before patch #19
- Set flag in patch #19
This would have make easier to review the implementation of MAPTI.
On 12/04/17 01:44, Andre Przywara wrote:
> When LPIs get unmapped by a guest, they might still be in some LR of
> some VCPU. Nevertheless we remove the corresponding pending_irq
> (possibly freeing it), and detect this case (irq_to_pending() returns
> NULL) when the LR gets cleaned up later.
> However a *new* LPI may get mapped with the same number while the old
> LPI is *still* in some LR. To avoid getting the wrong state, we mark
> every newly mapped LPI as PRISTINE, which means: has never been in an
> LR before. If we detect the LPI in an LR anyway, it must have been an
> older one, which we can simply retire.
From the GICv3 spec (8.4.6 in ARM IHI 0069C):
"Behavior is UNPREDICTABLE if two or more List Registers specify the
same vINTID when
- ICH_LR<n>_EL2.State == 01.
- ICH_LR<n>_EL2.State == 10.
- ICH_LR<n>_EL2.State == 11.
"
The LPI does not have active bit and we remap the vLPI to another pLPI.
So I think we might end up to have the same vLPI twice in the LR if the
new and old vLPI was routed to the same vCPU which is UNPREDICABLE.
Another case if both pending_irq are targeting a different vCPU. In this
case, the interrupt may be injected simultaneously to different vCPU. I
am not sure if it is valid...
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic.c | 17 ++++++++++++++++-
> xen/arch/arm/vgic-v3-its.c | 5 +++++
> xen/include/asm-arm/vgic.h | 1 +
> 3 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index d752352..e8c3202 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -373,6 +373,8 @@ static inline void gic_set_lr(int lr, struct pending_irq *p,
> {
> ASSERT(!local_irq_is_enabled());
>
> + clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status);
> +
This placement looks wrong, gic_set_lr is called by vgic_vcpu_inject_irq
but you don't know if the LRs have been cleared yet so you could end up
with the vIRQ in 2 different LRs which is UNPREDICTABLE.
> gic_hw_ops->update_lr(lr, p, state);
>
> set_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
> @@ -510,7 +512,17 @@ static void gic_update_one_lr(struct vcpu *v, int i)
> }
> else if ( lr_val.state & GICH_LR_PENDING )
> {
> - int q __attribute__ ((unused)) = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
> + int q __attribute__ ((unused));
> +
> + if ( test_and_clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status) )
> + {
> + gic_hw_ops->clear_lr(i);
> + clear_bit(i, &this_cpu(lr_mask));
> +
> + return;
> + }
> +
> + q = test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status);
> #ifdef GIC_DEBUG
> if ( q )
> gdprintk(XENLOG_DEBUG, "trying to inject irq=%d into d%dv%d, when it is already pending in LR%d\n",
> @@ -522,6 +534,9 @@ static void gic_update_one_lr(struct vcpu *v, int i)
> gic_hw_ops->clear_lr(i);
> clear_bit(i, &this_cpu(lr_mask));
>
> + if ( test_and_clear_bit(GIC_IRQ_GUEST_PRISTINE_LPI, &p->status) )
> + return;
> +
> if ( p->desc != NULL )
> clear_bit(_IRQ_INPROGRESS, &p->desc->status);
> clear_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index b7e61b2..0765810 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -618,6 +618,11 @@ static int its_handle_mapti(struct virt_its *its, uint64_t *cmdptr)
> goto out_remove_host_entry;
>
> pirq->lpi_vcpu_id = vcpu->vcpu_id;
> + /*
> + * Mark this LPI as new, so any older (now unmapped) LPI in any LR
> + * can be easily recognised as such.
> + */
> + set_bit(GIC_IRQ_GUEST_PRISTINE_LPI, pirq->status);
>
> /*
> * Now insert the pending_irq into the domain's LPI tree, so that
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 02732db..b1a7525 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -66,6 +66,7 @@ struct pending_irq
> #define GIC_IRQ_GUEST_VISIBLE 2
> #define GIC_IRQ_GUEST_ENABLED 3
> #define GIC_IRQ_GUEST_MIGRATING 4
> +#define GIC_IRQ_GUEST_PRISTINE_LPI 5
Please document this new flag...
> unsigned long status;
> struct irq_desc *desc; /* only set it the irq corresponds to a physical irq */
> unsigned int irq;
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 21/27] ARM: vITS: handle MOVI command
2017-04-12 0:44 ` [PATCH v8 21/27] ARM: vITS: handle MOVI command Andre Przywara
@ 2017-04-12 16:59 ` Julien Grall
2017-05-10 10:34 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 16:59 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> The MOVI command moves the interrupt affinity from one redistributor
> (read: VCPU) to another.
> For now migration of "live" LPIs is not yet implemented, but we store
> the changed affinity in the host LPI structure and in our virtual ITTE.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/gic-v3-its.c | 24 +++++++++++++++++
> xen/arch/arm/gic-v3-lpi.c | 15 +++++++++++
> xen/arch/arm/vgic-v3-its.c | 57 ++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic_v3_its.h | 4 +++
> 4 files changed, 100 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index fa1f2d5..1a08d43 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -895,6 +895,30 @@ struct pending_irq *gicv3_assign_guest_event(struct domain *d,
> return pirq;
> }
>
> +/* Changes the target VCPU for a given host LPI assigned to a domain. */
> +int gicv3_lpi_change_vcpu(struct domain *d, paddr_t vdoorbell,
> + uint32_t vdevid, uint32_t veventid,
> + unsigned int vcpu_id)
> +{
> + uint32_t host_lpi;
> + struct its_device *dev;
> +
> + spin_lock(&d->arch.vgic.its_devices_lock);
> + dev = get_its_device(d, vdoorbell, vdevid);
> + if ( dev )
> + host_lpi = get_host_lpi(dev, veventid);
> + else
> + host_lpi = 0;
> + spin_unlock(&d->arch.vgic.its_devices_lock);
> +
> + if ( !host_lpi )
> + return -ENOENT;
> +
> + gicv3_lpi_update_host_vcpuid(host_lpi, vcpu_id);
In v5, Stefano suggested to add a TODO here:
TODO: we do not change physical irq affinity, in response to a virtual
movi command. In other words, the physical LPI will still be delivered
to the same pcpu.
He was already suggested to print a warning (with gdprintk).
I see none of them, where are they?
> +
> + return 0;
> +}
> +
> /* Scan the DT for any ITS nodes and create a list of host ITSes out of it. */
> void gicv3_its_dt_init(const struct dt_device_node *node)
> {
> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
> index d427539..6af5ad9 100644
> --- a/xen/arch/arm/gic-v3-lpi.c
> +++ b/xen/arch/arm/gic-v3-lpi.c
> @@ -225,6 +225,21 @@ void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
> write_u64_atomic(&hlpip->data, hlpi.data);
> }
>
> +int gicv3_lpi_update_host_vcpuid(uint32_t host_lpi, unsigned int vcpu_id)
> +{
> + union host_lpi *hlpip;
> +
> + ASSERT(host_lpi >= LPI_OFFSET);
> +
> + host_lpi -= LPI_OFFSET;
> +
> + hlpip = &lpi_data.host_lpis[host_lpi / HOST_LPIS_PER_PAGE][host_lpi % HOST_LPIS_PER_PAGE];
> +
> + write_u16_atomic(&hlpip->vcpu_id, vcpu_id);
> +
> + return 0;
> +}
> +
> static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
> {
> uint64_t val;
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 0765810..be9de08 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -647,6 +647,57 @@ out_remove_mapping:
> return ret;
> }
>
> +static int its_handle_movi(struct virt_its *its, uint64_t *cmdptr)
> +{
> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
> + uint32_t eventid = its_cmd_get_id(cmdptr);
> + int collid = its_cmd_get_collection(cmdptr);
why collid is signed? From the spec, this should be uint16_t.
> + unsigned long flags;
> + struct pending_irq *p;
> + struct vcpu *ovcpu, *nvcpu;
> + uint32_t vlpi;
> + int ret = -1;
> +
> + spin_lock(&its->its_lock);
> + /* Check for a mapped LPI and get the LPI number. */
> + if ( !read_itte_locked(its, devid, eventid, &ovcpu, &vlpi) )
> + goto out_unlock;
> +
> + if ( vlpi == INVALID_LPI )
> + {
> + spin_unlock(&its->its_lock);
> + return -1;
> + }
> +
> + /* Check the new collection ID and get the new VCPU pointer */
> + nvcpu = get_vcpu_from_collection(its, collid);
> + if ( !nvcpu )
> + goto out_unlock;
> +
> + spin_lock_irqsave(&ovcpu->arch.vgic.lock, flags);
> +
> + /* Update our cached vcpu_id in the pending_irq. */
> + p = its->d->arch.vgic.handler->lpi_to_pending(its->d, vlpi);
So the table could be crafted by the guest before enable the LPI in
order to show a valid LPI. This would lead to returning NULL here as it
is not mapped. And crash Xen.
> + p->lpi_vcpu_id = nvcpu->vcpu_id;
> +
> + /* Now store the new collection in the translation table. */
> + if ( !write_itte_locked(its, devid, eventid, collid, vlpi, &nvcpu) )
> + goto out_unlock;
> +
> + spin_unlock_irqrestore(&ovcpu->arch.vgic.lock, flags);
> + spin_unlock(&its->its_lock);
> +
> + /* TODO: lookup currently-in-guest virtual IRQs and migrate them? */
> +
> + return gicv3_lpi_change_vcpu(its->d, its->doorbell_address,
> + devid, eventid, nvcpu->vcpu_id);
> +
> +out_unlock:
> + spin_unlock(&its->its_lock);
> +
> + return ret;
> +}
> +
> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>
> @@ -692,6 +743,12 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_MAPTI:
> ret = its_handle_mapti(its, command);
> break;
> + case GITS_CMD_MOVALL:
> + gdprintk(XENLOG_G_INFO, "ITS: ignoring MOVALL command\n");
Again, I'd like some explanation in the commit message why MOVALL is not
implemented and a TODO in the code.
> + break;
> + case GITS_CMD_MOVI:
> + ret = its_handle_movi(its, command);
> + break;
> case GITS_CMD_SYNC:
> /* We handle ITS commands synchronously, so we ignore SYNC. */
> break;
> diff --git a/xen/include/asm-arm/gic_v3_its.h b/xen/include/asm-arm/gic_v3_its.h
> index 7b16aeb..ee69e9b 100644
> --- a/xen/include/asm-arm/gic_v3_its.h
> +++ b/xen/include/asm-arm/gic_v3_its.h
> @@ -175,8 +175,12 @@ int gicv3_remove_guest_event(struct domain *d, paddr_t vdoorbell_address,
> struct pending_irq *gicv3_assign_guest_event(struct domain *d, paddr_t doorbell,
> uint32_t devid, uint32_t eventid,
> struct vcpu *v, uint32_t virt_lpi);
> +int gicv3_lpi_change_vcpu(struct domain *d, paddr_t doorbell,
> + uint32_t devid, uint32_t eventid,
> + unsigned int vcpu_id);
> void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
> unsigned int vcpu_id, uint32_t virt_lpi);
> +int gicv3_lpi_update_host_vcpuid(uint32_t host_lpi, unsigned int vcpu_id);
>
> #else
>
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-04-12 15:21 ` Julien Grall
@ 2017-04-12 17:03 ` Andre Przywara
2017-04-12 17:05 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 17:03 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 16:21, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> The MAPD command maps a device by associating a memory region for
>> storing ITEs with a certain device ID. Since it features a valid bit,
>> MAPD also covers the "unmap" functionality, which we also cover here.
>> We store the given guest physical address in the device table, and, if
>> this command comes from Dom0, tell the host ITS driver about this new
>> mapping, so it can issue the corresponding host MAPD command and create
>> the required tables. We take care of rolling back actions should one
>> step fail.
>> Upon unmapping a device we make sure we clean up all associated
>> resources and release the memory again.
>> We use our existing guest memory access function to find the right ITT
>> entry and store the mapping there (in guest memory).
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/gic-v3-its.c | 60 +++++++++++++++++
>> xen/arch/arm/gic-v3-lpi.c | 18 +++++
>> xen/arch/arm/vgic-v3-its.c | 137
>> +++++++++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/gic_v3_its.h | 6 ++
>> 4 files changed, 221 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> index aebc257..900c9d1 100644
>> --- a/xen/arch/arm/gic-v3-its.c
>> +++ b/xen/arch/arm/gic-v3-its.c
>> @@ -800,6 +800,66 @@ out:
>> return ret;
>> }
>>
>> +/* Must be called with the its_device_lock held. */
>> +static struct its_device *get_its_device(struct domain *d, paddr_t
>> vdoorbell,
>> + uint32_t vdevid)
>> +{
>> + struct rb_node *node = d->arch.vgic.its_devices.rb_node;
>> + struct its_device *dev;
>> +
>> + ASSERT(spin_is_locked(&d->arch.vgic.its_devices_lock));
>> +
>> + while (node)
>> + {
>> + int cmp;
>> +
>> + dev = rb_entry(node, struct its_device, rbnode);
>> + cmp = compare_its_guest_devices(dev, vdoorbell, vdevid);
>> +
>> + if ( !cmp )
>> + return dev;
>> +
>> + if ( cmp > 0 )
>> + node = node->rb_left;
>> + else
>> + node = node->rb_right;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +static uint32_t get_host_lpi(struct its_device *dev, uint32_t eventid)
>> +{
>> + uint32_t host_lpi = INVALID_LPI;
>> +
>> + if ( dev && (eventid < dev->eventids) )
>> + host_lpi = dev->host_lpi_blocks[eventid / LPI_BLOCK] +
>> + (eventid % LPI_BLOCK);
>> +
>> + return host_lpi;
>> +}
>> +
>> +int gicv3_remove_guest_event(struct domain *d, paddr_t
>> vdoorbell_address,
>> + uint32_t vdevid, uint32_t veventid)
>> +{
>> + struct its_device *dev;
>> + uint32_t host_lpi = INVALID_LPI;
>> +
>> + spin_lock(&d->arch.vgic.its_devices_lock);
>> + dev = get_its_device(d, vdoorbell_address, vdevid);
>> + if ( dev && veventid <= dev->eventids )
>> + host_lpi = get_host_lpi(dev, veventid);
>> + spin_unlock(&d->arch.vgic.its_devices_lock);
>> +
>> + if ( host_lpi == INVALID_LPI )
>> + return -EINVAL;
>> +
>> + gicv3_lpi_update_host_entry(host_lpi, d->domain_id,
>> + INVALID_VCPU_ID, INVALID_LPI);
>> +
>> + return 0;
>> +}
>> +
>> /* Scan the DT for any ITS nodes and create a list of host ITSes out
>> of it. */
>> void gicv3_its_dt_init(const struct dt_device_node *node)
>> {
>> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
>> index 44f6315..d427539 100644
>> --- a/xen/arch/arm/gic-v3-lpi.c
>> +++ b/xen/arch/arm/gic-v3-lpi.c
>> @@ -207,6 +207,24 @@ out:
>> irq_exit();
>> }
>>
>> +void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
>> + unsigned int vcpu_id, uint32_t
>> virt_lpi)
>> +{
>> + union host_lpi *hlpip, hlpi;
>> +
>> + ASSERT(host_lpi >= LPI_OFFSET);
>> +
>> + host_lpi -= LPI_OFFSET;
>> +
>> + hlpip = &lpi_data.host_lpis[host_lpi /
>> HOST_LPIS_PER_PAGE][host_lpi % HOST_LPIS_PER_PAGE];
>> +
>> + hlpi.virt_lpi = virt_lpi;
>> + hlpi.dom_id = domain_id;
>> + hlpi.vcpu_id = vcpu_id;
>> +
>> + write_u64_atomic(&hlpip->data, hlpi.data);
>> +}
>> +
>> static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
>> {
>> uint64_t val;
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index 6e505cb..104017e 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -52,6 +52,7 @@
>> */
>> struct virt_its {
>> struct domain *d;
>> + paddr_t doorbell_address;
>> unsigned int devid_bits;
>> unsigned int evid_bits;
>> spinlock_t vcmd_lock; /* Protects the virtual command
>> buffer, which */
>> @@ -166,6 +167,20 @@ static struct vcpu
>> *get_vcpu_from_collection(struct virt_its *its,
>> #define DEV_TABLE_ENTRY(addr, bits) \
>> (((addr) & GENMASK(51, 8)) | (((bits) - 1) & GENMASK(4, 0)))
>>
>> +/* Set the address of an ITT for a given device ID. */
>> +static int its_set_itt_address(struct virt_its *its, uint32_t devid,
>> + paddr_t itt_address, uint32_t nr_bits)
>> +{
>> + paddr_t addr = get_baser_phys_addr(its->baser_dev);
>> + uint64_t itt_entry = DEV_TABLE_ENTRY(itt_address, nr_bits);
>> +
>> + if ( devid >= its->max_devices )
>> + return -ENOENT;
>> +
>> + return vgic_access_guest_memory(its->d, addr + devid *
>> sizeof(uint64_t),
>> + &itt_entry, sizeof(itt_entry),
>> true);
>> +}
>> +
>> /*
>> * Lookup the address of the Interrupt Translation Table associated with
>> * that device ID.
>> @@ -398,6 +413,125 @@ static int its_handle_mapc(struct virt_its *its,
>> uint64_t *cmdptr)
>> return 0;
>> }
>>
>> +/* Must be called with the ITS lock held. */
>> +static int its_discard_event(struct virt_its *its,
>> + uint32_t vdevid, uint32_t vevid)
>> +{
>> + struct pending_irq *p;
>> + unsigned long flags;
>> + struct vcpu *vcpu;
>> + uint32_t vlpi;
>> +
>> + ASSERT(spin_is_locked(&its->its_lock));
>> +
>> + if ( !read_itte_locked(its, vdevid, vevid, &vcpu, &vlpi) )
>> + return -ENOENT;
>> +
>> + if ( vlpi == INVALID_LPI )
>> + return -ENOENT;
>> +
>> + /* Lock this VCPU's VGIC to make sure nobody is using the
>> pending_irq. */
>> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
>
> There is an interesting issue happening with this code. You don't check
> the content of the memory provided by the guest. So a malicious guest
> could craft the memory in order to setup mapping with known vlpi and a
> different vCPU.
>
> This would lead to use the wrong lock here and corrupt the list.
But this is true for basically most of the ITS table lookups. I added an
item to the cover letter to note this.
>> +
>> + /* Remove the pending_irq from the tree. */
>> + write_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
>> + p = radix_tree_delete(&its->d->arch.vgic.pend_lpi_tree, vlpi);
>> + write_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
>> +
>> + if ( !p )
>> + {
>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>> +
>> + return -ENOENT;
>> + }
>> +
>> + /* Cleanup the pending_irq, to be reusable later. */
>> + list_del_init(&p->inflight);
>> + list_del_init(&p->lr_queue);
>> + p->status = 0;
>> + p->lr = GIC_INVALID_LR;
>> + p->irq = INVALID_LPI;
>
> Can't we have a generic helper to do that?
Added vgic_clear_pending_irq() next to vgic_init_pending_irq() in
vgic.c, if that is what you meant...
>> +
>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>> +
>> + /* Remove the corresponding host LPI entry */
>> + return gicv3_remove_guest_event(its->d, its->doorbell_address,
>> + vdevid, vevid);
>> +}
>> +
>> +static int its_unmap_device(struct virt_its *its, uint32_t devid)
>> +{
>> + uint64_t itt, evid;
>> + int ret;
>> +
>> + spin_lock(&its->its_lock);
>> +
>> + ret = its_get_itt(its, devid, &itt);
>> + if ( ret )
>> + {
>> + spin_unlock(&its->its_lock);
>> + return ret;
>> + }
>> +
>> + for ( evid = 0; evid < DEV_TABLE_ITT_SIZE(itt); evid++ )
>> + /* Don't care about errors here, clean up as much as
>> possible. */
>> + its_discard_event(its, devid, evid);
>> +
>> + spin_unlock(&its->its_lock);
>
> This code can be long to execute as the number of event can be huge. How
> do you plan to handle that?
It is assumed that DomUs get only a *very* limited and controlled number
of ITS resources, so the number of LPIs, number of devices and their
number of events will be very small, probably just enough as really
needed (event ID bits being 4 or so, for instance).
> *hint* this likely means a comment + ASSERT + TODO in the cover letter
> *hint*
Added.
>> +
>> + return 0;
>> +}
>> +
>> +static int its_handle_mapd(struct virt_its *its, uint64_t *cmdptr)
>> +{
>> + /* size and devid get validated by the functions called below. */
>> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
>> + unsigned int size = its_cmd_get_size(cmdptr) + 1;
>> + bool valid = its_cmd_get_validbit(cmdptr);
>> + paddr_t itt_addr = its_cmd_get_ittaddr(cmdptr);
>> + int ret;
>> +
>> + /* Sanitize the number of events. */
>> + if ( valid && (size > its->evid_bits) )
>> + return -1;
>> +
>> + /*
>> + * There is no easy and clean way for Xen to know the ITS device
>> ID of a
>> + * particular (PCI) device, so we have to rely on the guest telling
>> + * us about it. For *now* we are just using the device ID *Dom0*
>> uses,
>> + * because the driver there has the actual knowledge.
>> + * Eventually this will be replaced with a dedicated hypercall to
>> + * announce pass-through of devices.
>> + */
>> + if ( is_hardware_domain(its->d) )
>> + {
>> + if ( !valid )
>> + /* Discard all events and remove pending LPIs. */
>> + its_unmap_device(its, devid);
>
> Likely, this call will have to be done for all the guests.
True.
>> +
>> + /*
>> + * Dom0's ITSes are mapped 1:1, so both addresses are the same.
>> + * Also the device IDs are equal.
>> + */
>> + ret = gicv3_its_map_guest_device(its->d,
>> its->doorbell_address, devid,
>> + its->doorbell_address, devid,
>> + BIT(size), valid);
>> + if ( ret && valid )
>> + return ret;
>> + }
>> +
>> + spin_lock(&its->its_lock);
>> +
>> + if ( valid )
>> + ret = its_set_itt_address(its, devid, itt_addr, size);
>> + else
>> + ret = its_set_itt_address(its, devid, INVALID_PADDR, 1);
>> +
>> + spin_unlock(&its->its_lock);
>> +
>> + return ret;
>> +}
>> +
>> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
>> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>>
>> @@ -436,6 +570,9 @@ static int vgic_its_handle_cmds(struct domain *d,
>> struct virt_its *its)
>> case GITS_CMD_MAPC:
>> ret = its_handle_mapc(its, command);
>> break;
>> + case GITS_CMD_MAPD:
>> + ret = its_handle_mapd(its, command);
>> + break;
>> case GITS_CMD_SYNC:
>> /* We handle ITS commands synchronously, so we ignore
>> SYNC. */
>> break;
>> diff --git a/xen/include/asm-arm/gic_v3_its.h
>> b/xen/include/asm-arm/gic_v3_its.h
>> index 40f4ef5..60ffdb6 100644
>> --- a/xen/include/asm-arm/gic_v3_its.h
>> +++ b/xen/include/asm-arm/gic_v3_its.h
>> @@ -169,6 +169,12 @@ int gicv3_its_map_guest_device(struct domain *d,
>> int gicv3_allocate_host_lpi_block(struct domain *d, uint32_t
>> *first_lpi);
>> void gicv3_free_host_lpi_block(uint32_t first_lpi);
>>
>> +int gicv3_remove_guest_event(struct domain *d, paddr_t
>> vdoorbell_address,
>> + uint32_t vdevid, uint32_t
>> veventid);
>> +
>> +void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
>> + unsigned int vcpu_id, uint32_t
>> virt_lpi);
>> +
>> #else
>>
>> static inline void gicv3_its_dt_init(const struct dt_device_node *node)
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-04-12 17:03 ` Andre Przywara
@ 2017-04-12 17:05 ` Julien Grall
2017-04-12 17:24 ` Andrew Cooper
2017-05-10 10:42 ` Andre Przywara
0 siblings, 2 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 17:05 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 18:03, Andre Przywara wrote:
> On 12/04/17 16:21, Julien Grall wrote:
>> On 12/04/17 01:44, Andre Przywara wrote:
>>> +
>>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>>> +
>>> + /* Remove the corresponding host LPI entry */
>>> + return gicv3_remove_guest_event(its->d, its->doorbell_address,
>>> + vdevid, vevid);
>>> +}
>>> +
>>> +static int its_unmap_device(struct virt_its *its, uint32_t devid)
>>> +{
>>> + uint64_t itt, evid;
>>> + int ret;
>>> +
>>> + spin_lock(&its->its_lock);
>>> +
>>> + ret = its_get_itt(its, devid, &itt);
>>> + if ( ret )
>>> + {
>>> + spin_unlock(&its->its_lock);
>>> + return ret;
>>> + }
>>> +
>>> + for ( evid = 0; evid < DEV_TABLE_ITT_SIZE(itt); evid++ )
>>> + /* Don't care about errors here, clean up as much as
>>> possible. */
>>> + its_discard_event(its, devid, evid);
>>> +
>>> + spin_unlock(&its->its_lock);
>>
>> This code can be long to execute as the number of event can be huge. How
>> do you plan to handle that?
>
> It is assumed that DomUs get only a *very* limited and controlled number
> of ITS resources, so the number of LPIs, number of devices and their
> number of events will be very small, probably just enough as really
> needed (event ID bits being 4 or so, for instance).
I really doubt this. I would not be surprised to see PCI device
passthrough with hundreds of event. And then we will have a beloved XSA
to handle...
>
>> *hint* this likely means a comment + ASSERT + TODO in the cover letter
>> *hint*
>
> Added.
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 22/27] ARM: vITS: handle DISCARD command
2017-04-12 0:44 ` [PATCH v8 22/27] ARM: vITS: handle DISCARD command Andre Przywara
@ 2017-04-12 17:06 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 17:06 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> The DISCARD command drops the connection between a DeviceID/EventID
> and an LPI/collection pair.
> We mark the respective structure entries as not allocated and make
> sure that any queued IRQs are removed.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 30 +++++++++++++++++++++++++++---
> 1 file changed, 27 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index be9de08..09cb3af 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -276,9 +276,9 @@ static bool read_itte(struct virt_its *its, uint32_t devid, uint32_t evid,
> * If vcpu_ptr is provided, returns the VCPU belonging to that collection.
> * Must be called with the ITS lock held.
> */
> -bool write_itte_locked(struct virt_its *its, uint32_t devid,
> - uint32_t evid, uint32_t collid, uint32_t vlpi,
> - struct vcpu **vcpu_ptr)
> +static bool write_itte_locked(struct virt_its *its, uint32_t devid,
> + uint32_t evid, uint32_t collid, uint32_t vlpi,
> + struct vcpu **vcpu_ptr)
Why the static has been added here and not in the first patch (patch #19
I think) who call it? Also this should be explained in the commit message...
> {
> paddr_t addr;
> struct vits_itte itte;
> @@ -698,6 +698,27 @@ out_unlock:
> return ret;
> }
>
> +static int its_handle_discard(struct virt_its *its, uint64_t *cmdptr)
> +{
> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
> + uint32_t eventid = its_cmd_get_id(cmdptr);
> + int ret;
> +
> + spin_lock(&its->its_lock);
> +
> + /* Remove from the radix tree and remove the host entry. */
> + ret = its_discard_event(its, devid, eventid);
> +
> + /* Remove from the guest's ITTE. */
> + if ( ret || write_itte_locked(its, devid, eventid,
> + UNMAPPED_COLLECTION, INVALID_LPI, NULL) )
> + ret = -1;
> +
> + spin_unlock(&its->its_lock);
> +
> + return ret;
> +}
> +
> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>
> @@ -730,6 +751,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_CLEAR:
> ret = its_handle_clear(its, command);
> break;
> + case GITS_CMD_DISCARD:
> + ret = its_handle_discard(its, command);
> + break;
> case GITS_CMD_INT:
> ret = its_handle_int(its, command);
> break;
>
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 19/27] ARM: vITS: handle MAPTI command
2017-04-12 0:44 ` [PATCH v8 19/27] ARM: vITS: handle MAPTI command Andre Przywara
2017-04-12 16:18 ` Julien Grall
@ 2017-04-12 17:16 ` Julien Grall
2017-04-12 17:25 ` Julien Grall
2 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 17:16 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> +/*
> + * For a given virtual LPI read the enabled bit and priority from the virtual
> + * property table and update the virtual IRQ's state in the given pending_irq.
> + * Must be called with the respective VGIC VCPU lock held.
> + */
> +static int update_lpi_property(struct domain *d, uint32_t vlpi,
> + struct pending_irq *p)
> +{
> + paddr_t addr;
> + uint8_t property;
> + int ret;
> +
> + addr = d->arch.vgic.rdist_propbase & GENMASK(51, 12);
> +
> + ret = vgic_access_guest_memory(d, addr + vlpi - LPI_OFFSET,
> + &property, sizeof(property), false);
> + if ( ret )
> + return ret;
> +
> + p->lpi_priority = property & LPI_PROP_PRIO_MASK;
Again, I don't think this will update lpi_priority atomically.
> + if ( property & LPI_PROP_ENABLED )
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + else
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + return 0;
> +}
> +
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 23/27] ARM: vITS: handle INV command
2017-04-12 0:44 ` [PATCH v8 23/27] ARM: vITS: handle INV command Andre Przywara
@ 2017-04-12 17:20 ` Julien Grall
2017-05-10 15:11 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-04-12 17:20 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Andre,
On 12/04/17 01:44, Andre Przywara wrote:
> The INV command instructs the ITS to update the configuration data for
> a given LPI by re-reading its entry from the property table.
> We don't need to care so much about the priority value, but enabling
> or disabling an LPI has some effect: We remove or push virtual LPIs
> to their VCPUs, also check the virtual pending bit if an LPI gets enabled.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 65 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 09cb3af..f2789c5 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -418,6 +418,68 @@ static int update_lpi_property(struct domain *d, uint32_t vlpi,
> return 0;
> }
>
> +/*
> + * Checks whether an LPI that got enabled or disabled needs to change
> + * something in the VGIC (added or removed from the LR or queues).
> + * Must be called with the VCPU VGIC lock held.
> + */
> +static void update_lpi_vgic_status(struct vcpu *v, struct pending_irq *p,
> + uint32_t vlpi)
p->irq should be equal to vlpi. No?
> +{
> + ASSERT(spin_is_locked(&v->arch.vgic.lock));
The locking is likely to wrong here too (see patch #2). For instance
with a MOVI then INV on interrupt enabled.
> +
> + if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> + {
> + if ( !list_empty(&p->inflight) &&
> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> + gic_raise_guest_irq(v, vlpi, p->lpi_priority);
> + }
> + else
> + {
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + list_del_init(&p->lr_queue);
> + }
> +}
> +
> +static int its_handle_inv(struct virt_its *its, uint64_t *cmdptr)
> +{
> + struct domain *d = its->d;
> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
> + uint32_t eventid = its_cmd_get_id(cmdptr);
> + struct pending_irq *p;
> + unsigned long flags;
> + struct vcpu *vcpu;
> + uint32_t vlpi;
> + int ret = -1;
> +
> + /* Translate the event into a vCPU/vLPI pair. */
> + if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
> + return -1;
> +
> + if ( vlpi == INVALID_LPI )
> + return -1;
> +
> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
> +
> + p = d->arch.vgic.handler->lpi_to_pending(d, vlpi);
> + if ( !p )
> + goto out_unlock;
As said on v5, this could be simpler and use the pending_irqs in the
device. That would be an improvement though. So a would be good.
> +
> + /* Read the property table and update our cached status. */
> + if ( update_lpi_property(d, vlpi, p) )
> + goto out_unlock;
> +
> + /* Check whether the LPI needs to go on a VCPU. */
> + update_lpi_vgic_status(vcpu, p, vlpi);
> +
> + ret = 0;
> +
> +out_unlock:
> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
> +
> + return ret;
> +}
> +
> static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
> {
> uint32_t collid = its_cmd_get_collection(cmdptr);
> @@ -757,6 +819,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_INT:
> ret = its_handle_int(its, command);
> break;
> + case GITS_CMD_INV:
> + ret = its_handle_inv(its, command);
> + break;
> case GITS_CMD_MAPC:
> ret = its_handle_mapc(its, command);
> break;
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-04-12 17:05 ` Julien Grall
@ 2017-04-12 17:24 ` Andrew Cooper
2017-04-12 18:18 ` Wei Liu
2017-05-10 10:42 ` Andre Przywara
1 sibling, 1 reply; 82+ messages in thread
From: Andrew Cooper @ 2017-04-12 17:24 UTC (permalink / raw)
To: Julien Grall, Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Shanker Donthineni, Vijay Kilari
On 12/04/17 18:05, Julien Grall wrote:
> I really doubt this. I would not be surprised to see PCI device
> passthrough with hundreds of event. And then we will have a beloved
> XSA to handle...
You and I clearly have a different idea of what beloved means :)
~Andrew (with a security team hat on)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 19/27] ARM: vITS: handle MAPTI command
2017-04-12 0:44 ` [PATCH v8 19/27] ARM: vITS: handle MAPTI command Andre Przywara
2017-04-12 16:18 ` Julien Grall
2017-04-12 17:16 ` Julien Grall
@ 2017-04-12 17:25 ` Julien Grall
2 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 17:25 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 12/04/17 01:44, Andre Przywara wrote:
> +/*
> + * For a given virtual LPI read the enabled bit and priority from the virtual
> + * property table and update the virtual IRQ's state in the given pending_irq.
> + * Must be called with the respective VGIC VCPU lock held.
> + */
> +static int update_lpi_property(struct domain *d, uint32_t vlpi,
> + struct pending_irq *p)
Technically p->irq should be equal to the vlpi. If no, there is an issue
somewhere else...
> +{
> + paddr_t addr;
> + uint8_t property;
> + int ret;
> +
> + addr = d->arch.vgic.rdist_propbase & GENMASK(51, 12);
> +
> + ret = vgic_access_guest_memory(d, addr + vlpi - LPI_OFFSET,
> + &property, sizeof(property), false);
> + if ( ret )
> + return ret;
> +
> + p->lpi_priority = property & LPI_PROP_PRIO_MASK;
> + if ( property & LPI_PROP_ENABLED )
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + else
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + return 0;
> +}
> +
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-04-12 10:44 ` Julien Grall
@ 2017-04-12 17:26 ` Andre Przywara
2017-05-10 10:47 ` Andre Przywara
1 sibling, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-04-12 17:26 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 11:44, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> Upon receiving an LPI on the host, we need to find the right VCPU and
>> virtual IRQ number to get this IRQ injected.
>> Iterate our two-level LPI table to find this information quickly when
>> the host takes an LPI. Call the existing injection function to let the
>> GIC emulation deal with this interrupt.
>> Also we enhance struct pending_irq to cache the pending bit and the
>> priority information for LPIs. Reading the information from there is
>> faster than accessing the property table from guest memory. Also it
>> use some padding area, so does not require more memory.
>> This introduces a do_LPI() as a hardware gic_ops and a function to
>> retrieve the (cached) priority value of an LPI and a vgic_ops.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/gic-v2.c | 7 ++++
>> xen/arch/arm/gic-v3-lpi.c | 71
>> ++++++++++++++++++++++++++++++++++++++++
>> xen/arch/arm/gic-v3.c | 1 +
>> xen/arch/arm/gic.c | 8 ++++-
>> xen/arch/arm/vgic-v2.c | 7 ++++
>> xen/arch/arm/vgic-v3.c | 12 +++++++
>> xen/arch/arm/vgic.c | 7 +++-
>> xen/include/asm-arm/domain.h | 3 +-
>> xen/include/asm-arm/gic.h | 2 ++
>> xen/include/asm-arm/gic_v3_its.h | 8 +++++
>> xen/include/asm-arm/vgic.h | 2 ++
>> 11 files changed, 125 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
>> index 270a136..ffbe47c 100644
>> --- a/xen/arch/arm/gic-v2.c
>> +++ b/xen/arch/arm/gic-v2.c
>> @@ -1217,6 +1217,12 @@ static int __init gicv2_init(void)
>> return 0;
>> }
>>
>> +static void gicv2_do_LPI(unsigned int lpi)
>> +{
>> + /* No LPIs in a GICv2 */
>> + BUG();
>> +}
>> +
>> const static struct gic_hw_operations gicv2_ops = {
>> .info = &gicv2_info,
>> .init = gicv2_init,
>> @@ -1244,6 +1250,7 @@ const static struct gic_hw_operations gicv2_ops = {
>> .make_hwdom_madt = gicv2_make_hwdom_madt,
>> .map_hwdom_extra_mappings = gicv2_map_hwdown_extra_mappings,
>> .iomem_deny_access = gicv2_iomem_deny_access,
>> + .do_LPI = gicv2_do_LPI,
>> };
>>
>> /* Set up the GIC */
>> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
>> index 292f2d0..44f6315 100644
>> --- a/xen/arch/arm/gic-v3-lpi.c
>> +++ b/xen/arch/arm/gic-v3-lpi.c
>> @@ -136,6 +136,77 @@ uint64_t gicv3_get_redist_address(unsigned int
>> cpu, bool use_pta)
>> return per_cpu(lpi_redist, cpu).redist_id << 16;
>> }
>>
>> +/*
>> + * Handle incoming LPIs, which are a bit special, because they are
>> potentially
>> + * numerous and also only get injected into guests. Treat them
>> specially here,
>> + * by just looking up their target vCPU and virtual LPI number and
>> hand it
>> + * over to the injection function.
>> + * Please note that LPIs are edge-triggered only, also have no active
>> state,
>> + * so spurious interrupts on the host side are no issue (we can just
>> ignore
>> + * them).
>> + * Also a guest cannot expect that firing interrupts that haven't been
>> + * fully configured yet will reach the CPU, so we don't need to care
>> about
>> + * this special case.
>> + */
>> +void gicv3_do_LPI(unsigned int lpi)
>> +{
>> + struct domain *d;
>> + union host_lpi *hlpip, hlpi;
>> + struct vcpu *vcpu;
>> +
>> + irq_enter();
>> +
>> + /* EOI the LPI already. */
>> + WRITE_SYSREG32(lpi, ICC_EOIR1_EL1);
>> +
>> + /* Find out if a guest mapped something to this physical LPI. */
>> + hlpip = gic_get_host_lpi(lpi);
>> + if ( !hlpip )
>> + goto out;
>> +
>> + hlpi.data = read_u64_atomic(&hlpip->data);
>> +
>> + /*
>> + * Unmapped events are marked with an invalid LPI ID. We can safely
>> + * ignore them, as they have no further state and no-one can expect
>> + * to see them if they have not been mapped.
>> + */
>> + if ( hlpi.virt_lpi == INVALID_LPI )
>> + goto out;
>> +
>> + d = rcu_lock_domain_by_id(hlpi.dom_id);
>> + if ( !d )
>> + goto out;
>> +
>> + /* Make sure we don't step beyond the vcpu array. */
>> + if ( hlpi.vcpu_id >= d->max_vcpus )
>> + {
>> + rcu_unlock_domain(d);
>> + goto out;
>> + }
>> +
>> + vcpu = d->vcpu[hlpi.vcpu_id];
>> +
>> + /* Check if the VCPU is ready to receive LPIs. */
>> + if ( vcpu->arch.vgic.flags & VGIC_V3_LPIS_ENABLED )
>> + /*
>> + * TODO: Investigate what to do here for potential interrupt
>> storms.
>> + * As we keep all host LPIs enabled, for disabling LPIs we
>> would need
>> + * to queue a ITS host command, which we avoid so far during
>> a guest's
>> + * runtime. Also re-enabling would trigger a host command
>> upon the
>> + * guest sending a command, which could be an attack vector for
>> + * hogging the host command queue.
>> + * See the thread around here for some background:
>> + *
>> https://lists.xen.org/archives/html/xen-devel/2016-12/msg00003.html
>> + */
>
> This TODO should have been listed in the cover letter.
Done.
>> + vgic_vcpu_inject_irq(vcpu, hlpi.virt_lpi);
>> +
>> + rcu_unlock_domain(d);
>> +
>> +out:
>> + irq_exit();
>> +}
>> +
>> static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
>> {
>> uint64_t val;
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index 29c8964..8140c5f 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -1674,6 +1674,7 @@ static const struct gic_hw_operations gicv3_ops = {
>> .make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
>> .make_hwdom_madt = gicv3_make_hwdom_madt,
>> .iomem_deny_access = gicv3_iomem_deny_access,
>> + .do_LPI = gicv3_do_LPI,
>> };
>>
>> static int __init gicv3_dt_preinit(struct dt_device_node *node, const
>> void *data)
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index 62ae3b8..d752352 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -735,7 +735,13 @@ void gic_interrupt(struct cpu_user_regs *regs,
>> int is_fiq)
>> do_IRQ(regs, irq, is_fiq);
>> local_irq_disable();
>> }
>> - else if (unlikely(irq < 16))
>> + else if ( is_lpi(irq) )
>> + {
>> + local_irq_enable();
>> + gic_hw_ops->do_LPI(irq);
>> + local_irq_disable();
>> + }
>> + else if ( unlikely(irq < 16) )
>> {
>> do_sgi(regs, irq);
>> }
>> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
>> index 0587569..df91940 100644
>> --- a/xen/arch/arm/vgic-v2.c
>> +++ b/xen/arch/arm/vgic-v2.c
>> @@ -709,11 +709,18 @@ static struct pending_irq
>> *vgic_v2_lpi_to_pending(struct domain *d,
>> BUG();
>> }
>>
>> +static int vgic_v2_lpi_get_priority(struct domain *d, unsigned int vlpi)
>> +{
>> + /* Dummy function, no LPIs on a VGICv2. */
>> + BUG();
>> +}
>> +
>> static const struct vgic_ops vgic_v2_ops = {
>> .vcpu_init = vgic_v2_vcpu_init,
>> .domain_init = vgic_v2_domain_init,
>> .domain_free = vgic_v2_domain_free,
>> .lpi_to_pending = vgic_v2_lpi_to_pending,
>> + .lpi_get_priority = vgic_v2_lpi_get_priority,
>> .max_vcpus = 8,
>> };
>>
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index f462610..c059dbd 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -1558,12 +1558,24 @@ static struct pending_irq
>> *vgic_v3_lpi_to_pending(struct domain *d,
>> return pirq;
>> }
>>
>> +/* Retrieve the priority of an LPI from its struct pending_irq. */
>> +static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
>> +{
>> + struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
>> +
>> + if ( !p )
>> + return GIC_PRI_IRQ;
>
> Why the check here? And why returning GIC_PRI_IRQ?
>
> AFAICT, vgic_v3_lpi_to_pending should never return NULL when reading the
> priority. Or else, you are in big trouble.
Now (with one change I made after your comment on 02/27) we call
vgic_get_virq_priority() as the very first thing in
vgic_vcpu_inject_irq(). So lpi_to_pending() could very well returning
NULL, which we would handle properly later.
As we can't move the vgic_get_virq_priority() call due to the locking
order rules, we just deal with this, returning *some* value here.
Shall I make the return value zero or 0xff in this case?
Cheers,
Andre.
>> +
>> + return p->lpi_priority;
>> +}
>> +
>> static const struct vgic_ops v3_ops = {
>> .vcpu_init = vgic_v3_vcpu_init,
>> .domain_init = vgic_v3_domain_init,
>> .domain_free = vgic_v3_domain_free,
>> .emulate_reg = vgic_v3_emulate_reg,
>> .lpi_to_pending = vgic_v3_lpi_to_pending,
>> + .lpi_get_priority = vgic_v3_lpi_get_priority,
>> /*
>> * We use both AFF1 and AFF0 in (v)MPIDR. Thus, the max number of
>> CPU
>> * that can be supported is up to 4096(==256*16) in theory.
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index c2bfdb1..b6fe34f 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -226,10 +226,15 @@ struct vcpu *vgic_get_target_vcpu(struct vcpu
>> *v, unsigned int virq)
>>
>> static int vgic_get_virq_priority(struct vcpu *v, unsigned int virq)
>> {
>> - struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
>> + struct vgic_irq_rank *rank;
>> unsigned long flags;
>> int priority;
>>
>> + /* LPIs don't have a rank, also store their priority separately. */
>> + if ( is_lpi(virq) )
>> + return
>> v->domain->arch.vgic.handler->lpi_get_priority(v->domain, virq);
>> +
>> + rank = vgic_rank_irq(v, virq);
>> vgic_lock_rank(v, rank, flags);
>> priority = rank->priority[virq & INTERRUPT_RANK_MASK];
>> vgic_unlock_rank(v, rank, flags);
>> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
>> index 3d8e84c..ebaea35 100644
>> --- a/xen/include/asm-arm/domain.h
>> +++ b/xen/include/asm-arm/domain.h
>> @@ -260,7 +260,8 @@ struct arch_vcpu
>>
>> /* GICv3: redistributor base and flags for this vCPU */
>> paddr_t rdist_base;
>> -#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
>> +#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the
>> rdist */
>> +#define VGIC_V3_LPIS_ENABLED (1 << 1)
>> uint8_t flags;
>> } vgic;
>>
>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>> index 836a103..42963c0 100644
>> --- a/xen/include/asm-arm/gic.h
>> +++ b/xen/include/asm-arm/gic.h
>> @@ -366,6 +366,8 @@ struct gic_hw_operations {
>> int (*map_hwdom_extra_mappings)(struct domain *d);
>> /* Deny access to GIC regions */
>> int (*iomem_deny_access)(const struct domain *d);
>> + /* Handle LPIs, which require special handling */
>> + void (*do_LPI)(unsigned int lpi);
>> };
>>
>> void register_gic_ops(const struct gic_hw_operations *ops);
>> diff --git a/xen/include/asm-arm/gic_v3_its.h
>> b/xen/include/asm-arm/gic_v3_its.h
>> index 29559a3..7470779 100644
>> --- a/xen/include/asm-arm/gic_v3_its.h
>> +++ b/xen/include/asm-arm/gic_v3_its.h
>> @@ -134,6 +134,8 @@ void gicv3_its_dt_init(const struct dt_device_node
>> *node);
>>
>> bool gicv3_its_host_has_its(void);
>>
>> +void gicv3_do_LPI(unsigned int lpi);
>> +
>> int gicv3_lpi_init_rdist(void __iomem * rdist_base);
>>
>> /* Initialize the host structures for LPIs and the host ITSes. */
>> @@ -175,6 +177,12 @@ static inline bool gicv3_its_host_has_its(void)
>> return false;
>> }
>>
>> +static inline void gicv3_do_LPI(unsigned int lpi)
>> +{
>> + /* We don't enable LPIs without an ITS. */
>> + BUG();
>> +}
>> +
>> static inline int gicv3_lpi_init_rdist(void __iomem * rdist_base)
>> {
>> return -ENODEV;
>> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
>> index c9075a9..7efa164 100644
>> --- a/xen/include/asm-arm/vgic.h
>> +++ b/xen/include/asm-arm/vgic.h
>> @@ -72,6 +72,7 @@ struct pending_irq
>> #define GIC_INVALID_LR (uint8_t)~0
>> uint8_t lr;
>> uint8_t priority;
>> + uint8_t lpi_priority; /* Caches the priority if this is an
>> LPI. */
>> /* inflight is used to append instances of pending_irq to
>> * vgic.inflight_irqs */
>> struct list_head inflight;
>> @@ -136,6 +137,7 @@ struct vgic_ops {
>> bool (*emulate_reg)(struct cpu_user_regs *regs, union hsr hsr);
>> /* lookup the struct pending_irq for a given LPI interrupt */
>> struct pending_irq *(*lpi_to_pending)(struct domain *d, unsigned
>> int vlpi);
>> + int (*lpi_get_priority)(struct domain *d, uint32_t vlpi);
>> /* Maximum number of vCPU supported */
>> const unsigned int max_vcpus;
>> };
>>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 24/27] ARM: vITS: handle INVALL command
2017-04-12 0:44 ` [PATCH v8 24/27] ARM: vITS: handle INVALL command Andre Przywara
@ 2017-04-12 17:26 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-04-12 17:26 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 01:44, Andre Przywara wrote:
> The INVALL command instructs an ITS to invalidate the configuration
> data for all LPIs associated with a given redistributor (read: VCPU).
> This is nasty to emulate exactly with our architecture, so we just
> iterate over all mapped LPIs and filter for those from that particular
> VCPU.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 66 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index f2789c5..9b5032b 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -480,6 +480,69 @@ out_unlock:
> return ret;
> }
>
> +/*
> + * INVALL updates the per-LPI configuration status for every LPI mapped to
> + * a particular redistributor.
> + * We iterate over all mapped LPIs in our radix tree and update those.
> + */
> +static int its_handle_invall(struct virt_its *its, uint64_t *cmdptr)
> +{
> + uint32_t collid = its_cmd_get_collection(cmdptr);
> + struct vcpu *vcpu;
> + struct pending_irq *pirqs[16];
> + uint64_t vlpi = 0; /* 64-bit to catch overflows */
> + unsigned int nr_lpis, i;
> + unsigned long flags;
> + int ret = 0;
> +
> + /*
> + * As this implementation walks over all mapped LPIs, it might take
> + * too long for a real guest, so we might want to revisit this
> + * implementation for DomUs.
> + * However this command is very rare, also we don't expect many
> + * LPIs to be actually mapped, so it's fine for Dom0 to use.
> + */
> + ASSERT(is_hardware_domain(its->d));
> +
> + spin_lock(&its->its_lock);
> + vcpu = get_vcpu_from_collection(its, collid);
> + spin_unlock(&its->its_lock);
> +
> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
See my remark earlier for the locking.
> + read_lock(&its->d->arch.vgic.pend_lpi_tree_lock);
> +
> + do
> + {
> + nr_lpis = radix_tree_gang_lookup(&its->d->arch.vgic.pend_lpi_tree,
> + (void **)pirqs, vlpi,
> + ARRAY_SIZE(pirqs));
> +
> + for ( i = 0; i < nr_lpis; i++ )
> + {
> + /* We only care about LPIs on our VCPU. */
> + if ( pirqs[i]->lpi_vcpu_id != vcpu->vcpu_id )
> + continue;
> +
> + vlpi = pirqs[i]->irq;
> + /* If that fails for a single LPI, carry on to handle the rest. */
> + ret = update_lpi_property(its->d, vlpi, pirqs[i]);
It is a bit weird that update_lpi_property take vlpi then pending_irq
and ...
> + if ( !ret )
> + update_lpi_vgic_status(vcpu, pirqs[i], vlpi);
update_lpi_vgic_status pending_irq then vlpi.
> + }
> + /*
> + * Loop over the next gang of pending_irqs until we reached the end of
> + * a (fully populated) tree or the lookup function returns less LPIs than
> + * it has been asked for.
> + */
> + } while ( (++vlpi < its->d->arch.vgic.nr_lpis) &&
> + (nr_lpis == ARRAY_SIZE(pirqs)) );
> +
> + read_unlock(&its->d->arch.vgic.pend_lpi_tree_lock);
> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
> +
> + return ret;
> +}
> +
> static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
> {
> uint32_t collid = its_cmd_get_collection(cmdptr);
> @@ -822,6 +885,9 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its)
> case GITS_CMD_INV:
> ret = its_handle_inv(its, command);
> break;
> + case GITS_CMD_INVALL:
> + ret = its_handle_invall(its, command);
> + break;
> case GITS_CMD_MAPC:
> ret = its_handle_mapc(its, command);
> break;
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-04-12 17:24 ` Andrew Cooper
@ 2017-04-12 18:18 ` Wei Liu
0 siblings, 0 replies; 82+ messages in thread
From: Wei Liu @ 2017-04-12 18:18 UTC (permalink / raw)
To: Andrew Cooper
Cc: Stefano Stabellini, Wei Liu, Vijay Kilari, Andre Przywara,
Vijaya Kumar K, Julien Grall, xen-devel, Shanker Donthineni
On Wed, Apr 12, 2017 at 06:24:58PM +0100, Andrew Cooper wrote:
> On 12/04/17 18:05, Julien Grall wrote:
> > I really doubt this. I would not be surprised to see PCI device
> > passthrough with hundreds of event. And then we will have a beloved
> > XSA to handle...
>
> You and I clearly have a different idea of what beloved means :)
>
El Reg and co. love them, no? ;-)
/me ducks
> ~Andrew (with a security team hat on)
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 21/27] ARM: vITS: handle MOVI command
2017-04-12 16:59 ` Julien Grall
@ 2017-05-10 10:34 ` Andre Przywara
0 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-05-10 10:34 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 17:59, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> The MOVI command moves the interrupt affinity from one redistributor
>> (read: VCPU) to another.
>> For now migration of "live" LPIs is not yet implemented, but we store
>> the changed affinity in the host LPI structure and in our virtual ITTE.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/gic-v3-its.c | 24 +++++++++++++++++
>> xen/arch/arm/gic-v3-lpi.c | 15 +++++++++++
>> xen/arch/arm/vgic-v3-its.c | 57
>> ++++++++++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/gic_v3_its.h | 4 +++
>> 4 files changed, 100 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> index fa1f2d5..1a08d43 100644
>> --- a/xen/arch/arm/gic-v3-its.c
>> +++ b/xen/arch/arm/gic-v3-its.c
>> @@ -895,6 +895,30 @@ struct pending_irq
>> *gicv3_assign_guest_event(struct domain *d,
>> return pirq;
>> }
>>
>> +/* Changes the target VCPU for a given host LPI assigned to a domain. */
>> +int gicv3_lpi_change_vcpu(struct domain *d, paddr_t vdoorbell,
>> + uint32_t vdevid, uint32_t veventid,
>> + unsigned int vcpu_id)
>> +{
>> + uint32_t host_lpi;
>> + struct its_device *dev;
>> +
>> + spin_lock(&d->arch.vgic.its_devices_lock);
>> + dev = get_its_device(d, vdoorbell, vdevid);
>> + if ( dev )
>> + host_lpi = get_host_lpi(dev, veventid);
>> + else
>> + host_lpi = 0;
>> + spin_unlock(&d->arch.vgic.its_devices_lock);
>> +
>> + if ( !host_lpi )
>> + return -ENOENT;
>> +
>> + gicv3_lpi_update_host_vcpuid(host_lpi, vcpu_id);
>
> In v5, Stefano suggested to add a TODO here:
>
> TODO: we do not change physical irq affinity, in response to a virtual
> movi command. In other words, the physical LPI will still be delivered
> to the same pcpu.
>
> He was already suggested to print a warning (with gdprintk).
I added a TODO, but don't think a warning is a good idea. There is no
*requirement* to move the physical affinity along with the virtual one,
so printing a warning for something we know will happen seems a bit odd.
Cheers,
Andre.
> I see none of them, where are they?
>
>> +
>> + return 0;
>> +}
>> +
>> /* Scan the DT for any ITS nodes and create a list of host ITSes out
>> of it. */
>> void gicv3_its_dt_init(const struct dt_device_node *node)
>> {
>> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
>> index d427539..6af5ad9 100644
>> --- a/xen/arch/arm/gic-v3-lpi.c
>> +++ b/xen/arch/arm/gic-v3-lpi.c
>> @@ -225,6 +225,21 @@ void gicv3_lpi_update_host_entry(uint32_t
>> host_lpi, int domain_id,
>> write_u64_atomic(&hlpip->data, hlpi.data);
>> }
>>
>> +int gicv3_lpi_update_host_vcpuid(uint32_t host_lpi, unsigned int
>> vcpu_id)
>> +{
>> + union host_lpi *hlpip;
>> +
>> + ASSERT(host_lpi >= LPI_OFFSET);
>> +
>> + host_lpi -= LPI_OFFSET;
>> +
>> + hlpip = &lpi_data.host_lpis[host_lpi /
>> HOST_LPIS_PER_PAGE][host_lpi % HOST_LPIS_PER_PAGE];
>> +
>> + write_u16_atomic(&hlpip->vcpu_id, vcpu_id);
>> +
>> + return 0;
>> +}
>> +
>> static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
>> {
>> uint64_t val;
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index 0765810..be9de08 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -647,6 +647,57 @@ out_remove_mapping:
>> return ret;
>> }
>>
>> +static int its_handle_movi(struct virt_its *its, uint64_t *cmdptr)
>> +{
>> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
>> + uint32_t eventid = its_cmd_get_id(cmdptr);
>> + int collid = its_cmd_get_collection(cmdptr);
>
> why collid is signed? From the spec, this should be uint16_t.
>
>> + unsigned long flags;
>> + struct pending_irq *p;
>> + struct vcpu *ovcpu, *nvcpu;
>> + uint32_t vlpi;
>> + int ret = -1;
>> +
>> + spin_lock(&its->its_lock);
>> + /* Check for a mapped LPI and get the LPI number. */
>> + if ( !read_itte_locked(its, devid, eventid, &ovcpu, &vlpi) )
>> + goto out_unlock;
>> +
>> + if ( vlpi == INVALID_LPI )
>> + {
>> + spin_unlock(&its->its_lock);
>> + return -1;
>> + }
>> +
>> + /* Check the new collection ID and get the new VCPU pointer */
>> + nvcpu = get_vcpu_from_collection(its, collid);
>> + if ( !nvcpu )
>> + goto out_unlock;
>> +
>> + spin_lock_irqsave(&ovcpu->arch.vgic.lock, flags);
>> +
>> + /* Update our cached vcpu_id in the pending_irq. */
>> + p = its->d->arch.vgic.handler->lpi_to_pending(its->d, vlpi);
>
> So the table could be crafted by the guest before enable the LPI in
> order to show a valid LPI. This would lead to returning NULL here as it
> is not mapped. And crash Xen.
>
>> + p->lpi_vcpu_id = nvcpu->vcpu_id;
>> +
>> + /* Now store the new collection in the translation table. */
>> + if ( !write_itte_locked(its, devid, eventid, collid, vlpi, &nvcpu) )
>> + goto out_unlock;
>> +
>> + spin_unlock_irqrestore(&ovcpu->arch.vgic.lock, flags);
>> + spin_unlock(&its->its_lock);
>> +
>> + /* TODO: lookup currently-in-guest virtual IRQs and migrate them? */
>> +
>> + return gicv3_lpi_change_vcpu(its->d, its->doorbell_address,
>> + devid, eventid, nvcpu->vcpu_id);
>> +
>> +out_unlock:
>> + spin_unlock(&its->its_lock);
>> +
>> + return ret;
>> +}
>> +
>> #define ITS_CMD_BUFFER_SIZE(baser) ((((baser) & 0xff) + 1) << 12)
>> #define ITS_CMD_OFFSET(reg) ((reg) & GENMASK(19, 5))
>>
>> @@ -692,6 +743,12 @@ static int vgic_its_handle_cmds(struct domain *d,
>> struct virt_its *its)
>> case GITS_CMD_MAPTI:
>> ret = its_handle_mapti(its, command);
>> break;
>> + case GITS_CMD_MOVALL:
>> + gdprintk(XENLOG_G_INFO, "ITS: ignoring MOVALL command\n");
>
> Again, I'd like some explanation in the commit message why MOVALL is not
> implemented and a TODO in the code.
>
>> + break;
>> + case GITS_CMD_MOVI:
>> + ret = its_handle_movi(its, command);
>> + break;
>> case GITS_CMD_SYNC:
>> /* We handle ITS commands synchronously, so we ignore
>> SYNC. */
>> break;
>> diff --git a/xen/include/asm-arm/gic_v3_its.h
>> b/xen/include/asm-arm/gic_v3_its.h
>> index 7b16aeb..ee69e9b 100644
>> --- a/xen/include/asm-arm/gic_v3_its.h
>> +++ b/xen/include/asm-arm/gic_v3_its.h
>> @@ -175,8 +175,12 @@ int gicv3_remove_guest_event(struct domain *d,
>> paddr_t vdoorbell_address,
>> struct pending_irq *gicv3_assign_guest_event(struct domain *d,
>> paddr_t doorbell,
>> uint32_t devid, uint32_t
>> eventid,
>> struct vcpu *v, uint32_t
>> virt_lpi);
>> +int gicv3_lpi_change_vcpu(struct domain *d, paddr_t doorbell,
>> + uint32_t devid, uint32_t eventid,
>> + unsigned int vcpu_id);
>> void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id,
>> unsigned int vcpu_id, uint32_t
>> virt_lpi);
>> +int gicv3_lpi_update_host_vcpuid(uint32_t host_lpi, unsigned int
>> vcpu_id);
>>
>> #else
>>
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-04-12 17:05 ` Julien Grall
2017-04-12 17:24 ` Andrew Cooper
@ 2017-05-10 10:42 ` Andre Przywara
2017-05-10 11:30 ` Julien Grall
1 sibling, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-05-10 10:42 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 18:05, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 18:03, Andre Przywara wrote:
>> On 12/04/17 16:21, Julien Grall wrote:
>>> On 12/04/17 01:44, Andre Przywara wrote:
>>>> +
>>>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>>>> +
>>>> + /* Remove the corresponding host LPI entry */
>>>> + return gicv3_remove_guest_event(its->d, its->doorbell_address,
>>>> + vdevid, vevid);
>>>> +}
>>>> +
>>>> +static int its_unmap_device(struct virt_its *its, uint32_t devid)
>>>> +{
>>>> + uint64_t itt, evid;
>>>> + int ret;
>>>> +
>>>> + spin_lock(&its->its_lock);
>>>> +
>>>> + ret = its_get_itt(its, devid, &itt);
>>>> + if ( ret )
>>>> + {
>>>> + spin_unlock(&its->its_lock);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + for ( evid = 0; evid < DEV_TABLE_ITT_SIZE(itt); evid++ )
>>>> + /* Don't care about errors here, clean up as much as
>>>> possible. */
>>>> + its_discard_event(its, devid, evid);
>>>> +
>>>> + spin_unlock(&its->its_lock);
>>>
>>> This code can be long to execute as the number of event can be huge. How
>>> do you plan to handle that?
>>
>> It is assumed that DomUs get only a *very* limited and controlled number
>> of ITS resources, so the number of LPIs, number of devices and their
>> number of events will be very small, probably just enough as really
>> needed (event ID bits being 4 or so, for instance).
>
> I really doubt this. I would not be surprised to see PCI device
> passthrough with hundreds of event.
But that's not *huge*. Chasing down the calls from here I don't see
anything which takes a long time to execute. So the execution time
becomes only an issue if we reach into the tens of thousands of events,
but I'd rely on the responsibility of a sysadmin when passing through
devices with that large number of MSI. I'd hope that PCI passthrough
would help here with providing some means of policy limits if users care
about this.
Cheers,
Andre.
> And then we will have a beloved XSA
> to handle...
>
>>
>>> *hint* this likely means a comment + ASSERT + TODO in the cover letter
>>> *hint*
>>
>> Added.
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-04-12 10:44 ` Julien Grall
2017-04-12 17:26 ` Andre Przywara
@ 2017-05-10 10:47 ` Andre Przywara
2017-05-10 11:07 ` Julien Grall
1 sibling, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-05-10 10:47 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 11:44, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> Upon receiving an LPI on the host, we need to find the right VCPU and
>> virtual IRQ number to get this IRQ injected.
>> Iterate our two-level LPI table to find this information quickly when
>> the host takes an LPI. Call the existing injection function to let the
>> GIC emulation deal with this interrupt.
>> Also we enhance struct pending_irq to cache the pending bit and the
>> priority information for LPIs. Reading the information from there is
>> faster than accessing the property table from guest memory. Also it
>> use some padding area, so does not require more memory.
>> This introduces a do_LPI() as a hardware gic_ops and a function to
>> retrieve the (cached) priority value of an LPI and a vgic_ops.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/gic-v2.c | 7 ++++
>> xen/arch/arm/gic-v3-lpi.c | 71
>> ++++++++++++++++++++++++++++++++++++++++
>> xen/arch/arm/gic-v3.c | 1 +
>> xen/arch/arm/gic.c | 8 ++++-
>> xen/arch/arm/vgic-v2.c | 7 ++++
>> xen/arch/arm/vgic-v3.c | 12 +++++++
>> xen/arch/arm/vgic.c | 7 +++-
>> xen/include/asm-arm/domain.h | 3 +-
>> xen/include/asm-arm/gic.h | 2 ++
>> xen/include/asm-arm/gic_v3_its.h | 8 +++++
>> xen/include/asm-arm/vgic.h | 2 ++
>> 11 files changed, 125 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
>> index 270a136..ffbe47c 100644
>> --- a/xen/arch/arm/gic-v2.c
>> +++ b/xen/arch/arm/gic-v2.c
>> @@ -1217,6 +1217,12 @@ static int __init gicv2_init(void)
>> return 0;
>> }
>>
>> +static void gicv2_do_LPI(unsigned int lpi)
>> +{
>> + /* No LPIs in a GICv2 */
>> + BUG();
>> +}
>> +
>> const static struct gic_hw_operations gicv2_ops = {
>> .info = &gicv2_info,
>> .init = gicv2_init,
>> @@ -1244,6 +1250,7 @@ const static struct gic_hw_operations gicv2_ops = {
>> .make_hwdom_madt = gicv2_make_hwdom_madt,
>> .map_hwdom_extra_mappings = gicv2_map_hwdown_extra_mappings,
>> .iomem_deny_access = gicv2_iomem_deny_access,
>> + .do_LPI = gicv2_do_LPI,
>> };
>>
>> /* Set up the GIC */
>> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
>> index 292f2d0..44f6315 100644
>> --- a/xen/arch/arm/gic-v3-lpi.c
>> +++ b/xen/arch/arm/gic-v3-lpi.c
>> @@ -136,6 +136,77 @@ uint64_t gicv3_get_redist_address(unsigned int
>> cpu, bool use_pta)
>> return per_cpu(lpi_redist, cpu).redist_id << 16;
>> }
>>
>> +/*
>> + * Handle incoming LPIs, which are a bit special, because they are
>> potentially
>> + * numerous and also only get injected into guests. Treat them
>> specially here,
>> + * by just looking up their target vCPU and virtual LPI number and
>> hand it
>> + * over to the injection function.
>> + * Please note that LPIs are edge-triggered only, also have no active
>> state,
>> + * so spurious interrupts on the host side are no issue (we can just
>> ignore
>> + * them).
>> + * Also a guest cannot expect that firing interrupts that haven't been
>> + * fully configured yet will reach the CPU, so we don't need to care
>> about
>> + * this special case.
>> + */
>> +void gicv3_do_LPI(unsigned int lpi)
>> +{
>> + struct domain *d;
>> + union host_lpi *hlpip, hlpi;
>> + struct vcpu *vcpu;
>> +
>> + irq_enter();
>> +
>> + /* EOI the LPI already. */
>> + WRITE_SYSREG32(lpi, ICC_EOIR1_EL1);
>> +
>> + /* Find out if a guest mapped something to this physical LPI. */
>> + hlpip = gic_get_host_lpi(lpi);
>> + if ( !hlpip )
>> + goto out;
>> +
>> + hlpi.data = read_u64_atomic(&hlpip->data);
>> +
>> + /*
>> + * Unmapped events are marked with an invalid LPI ID. We can safely
>> + * ignore them, as they have no further state and no-one can expect
>> + * to see them if they have not been mapped.
>> + */
>> + if ( hlpi.virt_lpi == INVALID_LPI )
>> + goto out;
>> +
>> + d = rcu_lock_domain_by_id(hlpi.dom_id);
>> + if ( !d )
>> + goto out;
>> +
>> + /* Make sure we don't step beyond the vcpu array. */
>> + if ( hlpi.vcpu_id >= d->max_vcpus )
>> + {
>> + rcu_unlock_domain(d);
>> + goto out;
>> + }
>> +
>> + vcpu = d->vcpu[hlpi.vcpu_id];
>> +
>> + /* Check if the VCPU is ready to receive LPIs. */
>> + if ( vcpu->arch.vgic.flags & VGIC_V3_LPIS_ENABLED )
>> + /*
>> + * TODO: Investigate what to do here for potential interrupt
>> storms.
>> + * As we keep all host LPIs enabled, for disabling LPIs we
>> would need
>> + * to queue a ITS host command, which we avoid so far during
>> a guest's
>> + * runtime. Also re-enabling would trigger a host command
>> upon the
>> + * guest sending a command, which could be an attack vector for
>> + * hogging the host command queue.
>> + * See the thread around here for some background:
>> + *
>> https://lists.xen.org/archives/html/xen-devel/2016-12/msg00003.html
>> + */
>
> This TODO should have been listed in the cover letter.
OK.
I extended the TODO/limitations section accordingly.
>> + vgic_vcpu_inject_irq(vcpu, hlpi.virt_lpi);
>> +
>> + rcu_unlock_domain(d);
>> +
>> +out:
>> + irq_exit();
>> +}
>> +
>> static int gicv3_lpi_allocate_pendtable(uint64_t *reg)
>> {
>> uint64_t val;
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index 29c8964..8140c5f 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -1674,6 +1674,7 @@ static const struct gic_hw_operations gicv3_ops = {
>> .make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
>> .make_hwdom_madt = gicv3_make_hwdom_madt,
>> .iomem_deny_access = gicv3_iomem_deny_access,
>> + .do_LPI = gicv3_do_LPI,
>> };
>>
>> static int __init gicv3_dt_preinit(struct dt_device_node *node, const
>> void *data)
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index 62ae3b8..d752352 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -735,7 +735,13 @@ void gic_interrupt(struct cpu_user_regs *regs,
>> int is_fiq)
>> do_IRQ(regs, irq, is_fiq);
>> local_irq_disable();
>> }
>> - else if (unlikely(irq < 16))
>> + else if ( is_lpi(irq) )
>> + {
>> + local_irq_enable();
>> + gic_hw_ops->do_LPI(irq);
>> + local_irq_disable();
>> + }
>> + else if ( unlikely(irq < 16) )
>> {
>> do_sgi(regs, irq);
>> }
>> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
>> index 0587569..df91940 100644
>> --- a/xen/arch/arm/vgic-v2.c
>> +++ b/xen/arch/arm/vgic-v2.c
>> @@ -709,11 +709,18 @@ static struct pending_irq
>> *vgic_v2_lpi_to_pending(struct domain *d,
>> BUG();
>> }
>>
>> +static int vgic_v2_lpi_get_priority(struct domain *d, unsigned int vlpi)
>> +{
>> + /* Dummy function, no LPIs on a VGICv2. */
>> + BUG();
>> +}
>> +
>> static const struct vgic_ops vgic_v2_ops = {
>> .vcpu_init = vgic_v2_vcpu_init,
>> .domain_init = vgic_v2_domain_init,
>> .domain_free = vgic_v2_domain_free,
>> .lpi_to_pending = vgic_v2_lpi_to_pending,
>> + .lpi_get_priority = vgic_v2_lpi_get_priority,
>> .max_vcpus = 8,
>> };
>>
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index f462610..c059dbd 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -1558,12 +1558,24 @@ static struct pending_irq
>> *vgic_v3_lpi_to_pending(struct domain *d,
>> return pirq;
>> }
>>
>> +/* Retrieve the priority of an LPI from its struct pending_irq. */
>> +static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
>> +{
>> + struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
>> +
>> + if ( !p )
>> + return GIC_PRI_IRQ;
>
> Why the check here? And why returning GIC_PRI_IRQ?
Because you surely want to avoid dereferencing NULL?
I changed the return value to 0xff, which is the lowest priority.
Frankly I think we could just return anything, as we will stop handling
this LPI anyway a bit later in the code if p is NULL here.
> AFAICT, vgic_v3_lpi_to_pending should never return NULL when reading the
> priority. Or else, you are in big trouble.
That depends on where and when you call it, which I don't want to make
any assumptions about.
In my latest version the vgic_get_virq_priority() call now stays at the
very beginning of vgic_vcpu_inject_irq(), so at this point the LPI could
have been unmapped meanwhile.
Surely you will bail out handling this LPI later in the code if it
returns NULL here, but for the sake of this function I think we need the
check.
To me it looks a bit like having this abstraction here is a bit
pointless and complicates things, but well ....
Cheers,
Andre.
>
>> +
>> + return p->lpi_priority;
>> +}
>> +
>> static const struct vgic_ops v3_ops = {
>> .vcpu_init = vgic_v3_vcpu_init,
>> .domain_init = vgic_v3_domain_init,
>> .domain_free = vgic_v3_domain_free,
>> .emulate_reg = vgic_v3_emulate_reg,
>> .lpi_to_pending = vgic_v3_lpi_to_pending,
>> + .lpi_get_priority = vgic_v3_lpi_get_priority,
>> /*
>> * We use both AFF1 and AFF0 in (v)MPIDR. Thus, the max number of
>> CPU
>> * that can be supported is up to 4096(==256*16) in theory.
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index c2bfdb1..b6fe34f 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -226,10 +226,15 @@ struct vcpu *vgic_get_target_vcpu(struct vcpu
>> *v, unsigned int virq)
>>
>> static int vgic_get_virq_priority(struct vcpu *v, unsigned int virq)
>> {
>> - struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
>> + struct vgic_irq_rank *rank;
>> unsigned long flags;
>> int priority;
>>
>> + /* LPIs don't have a rank, also store their priority separately. */
>> + if ( is_lpi(virq) )
>> + return
>> v->domain->arch.vgic.handler->lpi_get_priority(v->domain, virq);
>> +
>> + rank = vgic_rank_irq(v, virq);
>> vgic_lock_rank(v, rank, flags);
>> priority = rank->priority[virq & INTERRUPT_RANK_MASK];
>> vgic_unlock_rank(v, rank, flags);
>> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
>> index 3d8e84c..ebaea35 100644
>> --- a/xen/include/asm-arm/domain.h
>> +++ b/xen/include/asm-arm/domain.h
>> @@ -260,7 +260,8 @@ struct arch_vcpu
>>
>> /* GICv3: redistributor base and flags for this vCPU */
>> paddr_t rdist_base;
>> -#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
>> +#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the
>> rdist */
>> +#define VGIC_V3_LPIS_ENABLED (1 << 1)
>> uint8_t flags;
>> } vgic;
>>
>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>> index 836a103..42963c0 100644
>> --- a/xen/include/asm-arm/gic.h
>> +++ b/xen/include/asm-arm/gic.h
>> @@ -366,6 +366,8 @@ struct gic_hw_operations {
>> int (*map_hwdom_extra_mappings)(struct domain *d);
>> /* Deny access to GIC regions */
>> int (*iomem_deny_access)(const struct domain *d);
>> + /* Handle LPIs, which require special handling */
>> + void (*do_LPI)(unsigned int lpi);
>> };
>>
>> void register_gic_ops(const struct gic_hw_operations *ops);
>> diff --git a/xen/include/asm-arm/gic_v3_its.h
>> b/xen/include/asm-arm/gic_v3_its.h
>> index 29559a3..7470779 100644
>> --- a/xen/include/asm-arm/gic_v3_its.h
>> +++ b/xen/include/asm-arm/gic_v3_its.h
>> @@ -134,6 +134,8 @@ void gicv3_its_dt_init(const struct dt_device_node
>> *node);
>>
>> bool gicv3_its_host_has_its(void);
>>
>> +void gicv3_do_LPI(unsigned int lpi);
>> +
>> int gicv3_lpi_init_rdist(void __iomem * rdist_base);
>>
>> /* Initialize the host structures for LPIs and the host ITSes. */
>> @@ -175,6 +177,12 @@ static inline bool gicv3_its_host_has_its(void)
>> return false;
>> }
>>
>> +static inline void gicv3_do_LPI(unsigned int lpi)
>> +{
>> + /* We don't enable LPIs without an ITS. */
>> + BUG();
>> +}
>> +
>> static inline int gicv3_lpi_init_rdist(void __iomem * rdist_base)
>> {
>> return -ENODEV;
>> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
>> index c9075a9..7efa164 100644
>> --- a/xen/include/asm-arm/vgic.h
>> +++ b/xen/include/asm-arm/vgic.h
>> @@ -72,6 +72,7 @@ struct pending_irq
>> #define GIC_INVALID_LR (uint8_t)~0
>> uint8_t lr;
>> uint8_t priority;
>> + uint8_t lpi_priority; /* Caches the priority if this is an
>> LPI. */
>> /* inflight is used to append instances of pending_irq to
>> * vgic.inflight_irqs */
>> struct list_head inflight;
>> @@ -136,6 +137,7 @@ struct vgic_ops {
>> bool (*emulate_reg)(struct cpu_user_regs *regs, union hsr hsr);
>> /* lookup the struct pending_irq for a given LPI interrupt */
>> struct pending_irq *(*lpi_to_pending)(struct domain *d, unsigned
>> int vlpi);
>> + int (*lpi_get_priority)(struct domain *d, uint32_t vlpi);
>> /* Maximum number of vCPU supported */
>> const unsigned int max_vcpus;
>> };
>>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-05-10 10:47 ` Andre Przywara
@ 2017-05-10 11:07 ` Julien Grall
2017-05-10 17:14 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-05-10 11:07 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 05/10/2017 11:47 AM, Andre Przywara wrote:
> Hi,
Hi Andre,
> On 12/04/17 11:44, Julien Grall wrote:
>> On 12/04/17 01:44, Andre Przywara wrote:
>>> +/* Retrieve the priority of an LPI from its struct pending_irq. */
>>> +static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
>>> +{
>>> + struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
>>> +
>>> + if ( !p )
>>> + return GIC_PRI_IRQ;
>>
>> Why the check here? And why returning GIC_PRI_IRQ?
>
> Because you surely want to avoid dereferencing NULL?
> I changed the return value to 0xff, which is the lowest priority.
> Frankly I think we could just return anything, as we will stop handling
> this LPI anyway a bit later in the code if p is NULL here.
I agree that you want to prevent NULL. But we also want to avoid return
fake value because there was a caller that didn't bother to check
whether the interrupt is valid at first hand.
If you ever have NULL here then there is a latent BUG in your code
somewhere else. Ignoring the NULL and return a fake value is likely not
the right solution for development.
I can see two solutions for this:
- ASSERT(p)
- if ( !p )
{
ASSERT_UNREACHABLE();
return 0xff;
}
The later would still return a dumb value but at least we would catch
programming mistake during development.
>
>> AFAICT, vgic_v3_lpi_to_pending should never return NULL when reading the
>> priority. Or else, you are in big trouble.
>
> That depends on where and when you call it, which I don't want to make
> any assumptions about.
> In my latest version the vgic_get_virq_priority() call now stays at the
> very beginning of vgic_vcpu_inject_irq(), so at this point the LPI could
> have been unmapped meanwhile.
> Surely you will bail out handling this LPI later in the code if it
> returns NULL here, but for the sake of this function I think we need the
> check.
> To me it looks a bit like having this abstraction here is a bit
> pointless and complicates things, but well ....
Well, I am not against very defensive programming. I am more against
returning a fake value that may impact the rest of the vGIC (not even
mentioning the lack of comment explain why). After all, the priority is
controlled by the guest and not the hypervisor.
I suggested few way above to catch those errors.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 18/27] ARM: vITS: handle MAPD command
2017-05-10 10:42 ` Andre Przywara
@ 2017-05-10 11:30 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-05-10 11:30 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 05/10/2017 11:42 AM, Andre Przywara wrote:
> Hi,
Hi Andre,
> On 12/04/17 18:05, Julien Grall wrote:
>> Hi Andre,
>>
>> On 12/04/17 18:03, Andre Przywara wrote:
>>> On 12/04/17 16:21, Julien Grall wrote:
>>>> On 12/04/17 01:44, Andre Przywara wrote:
>>>>> +
>>>>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>>>>> +
>>>>> + /* Remove the corresponding host LPI entry */
>>>>> + return gicv3_remove_guest_event(its->d, its->doorbell_address,
>>>>> + vdevid, vevid);
>>>>> +}
>>>>> +
>>>>> +static int its_unmap_device(struct virt_its *its, uint32_t devid)
>>>>> +{
>>>>> + uint64_t itt, evid;
>>>>> + int ret;
>>>>> +
>>>>> + spin_lock(&its->its_lock);
>>>>> +
>>>>> + ret = its_get_itt(its, devid, &itt);
>>>>> + if ( ret )
>>>>> + {
>>>>> + spin_unlock(&its->its_lock);
>>>>> + return ret;
>>>>> + }
>>>>> +
>>>>> + for ( evid = 0; evid < DEV_TABLE_ITT_SIZE(itt); evid++ )
>>>>> + /* Don't care about errors here, clean up as much as
>>>>> possible. */
>>>>> + its_discard_event(its, devid, evid);
>>>>> +
>>>>> + spin_unlock(&its->its_lock);
>>>>
>>>> This code can be long to execute as the number of event can be huge. How
>>>> do you plan to handle that?
>>>
>>> It is assumed that DomUs get only a *very* limited and controlled number
>>> of ITS resources, so the number of LPIs, number of devices and their
>>> number of events will be very small, probably just enough as really
>>> needed (event ID bits being 4 or so, for instance).
>>
>> I really doubt this. I would not be surprised to see PCI device
>> passthrough with hundreds of event.
>
> But that's not *huge*. Chasing down the calls from here I don't see
> anything which takes a long time to execute. So the execution time
> becomes only an issue if we reach into the tens of thousands of events,
> but I'd rely on the responsibility of a sysadmin when passing through
> devices with that large number of MSI. I'd hope that PCI passthrough
> would help here with providing some means of policy limits if users care
> about this.
I am getting annoyed to repeat that again and again. Xen has to deal
with *any* number of MSIs as long as the spec says it is possible and
not expose a security issue. The problem can appear easily because of
command time may be multiplied * 32K if the guest decides to queue 32K
command.
How a sysadmin, which very likely does not know the internal Xen, would
know that he should not passing through devices with that large number
of MSI? Surely he will not be able to guess that the ITS code is not
able to cope with any number without any documentation.
I really don't want to have to deal with XSAs because you knowingly hide
a potential issue thinking someone else will fix for you.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 23/27] ARM: vITS: handle INV command
2017-04-12 17:20 ` Julien Grall
@ 2017-05-10 15:11 ` Andre Przywara
2017-05-11 10:43 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-05-10 15:11 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 18:20, Julien Grall wrote:
> Hi Andre,
>
> On 12/04/17 01:44, Andre Przywara wrote:
>> The INV command instructs the ITS to update the configuration data for
>> a given LPI by re-reading its entry from the property table.
>> We don't need to care so much about the priority value, but enabling
>> or disabling an LPI has some effect: We remove or push virtual LPIs
>> to their VCPUs, also check the virtual pending bit if an LPI gets
>> enabled.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>> xen/arch/arm/vgic-v3-its.c | 65
>> ++++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 65 insertions(+)
>>
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index 09cb3af..f2789c5 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -418,6 +418,68 @@ static int update_lpi_property(struct domain *d,
>> uint32_t vlpi,
>> return 0;
>> }
>>
>> +/*
>> + * Checks whether an LPI that got enabled or disabled needs to change
>> + * something in the VGIC (added or removed from the LR or queues).
>> + * Must be called with the VCPU VGIC lock held.
>> + */
>> +static void update_lpi_vgic_status(struct vcpu *v, struct pending_irq
>> *p,
>> + uint32_t vlpi)
>
> p->irq should be equal to vlpi. No?
It is, by I liked the idea of having logically separate parameters
expressed as such. But I removed vlpi now and am using p->irq instead.
>
>> +{
>> + ASSERT(spin_is_locked(&v->arch.vgic.lock));
>
> The locking is likely to wrong here too (see patch #2). For instance
> with a MOVI then INV on interrupt enabled.
>
>> +
>> + if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>> + {
>> + if ( !list_empty(&p->inflight) &&
>> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
>> + gic_raise_guest_irq(v, vlpi, p->lpi_priority);
>> + }
>> + else
>> + {
>> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>> + list_del_init(&p->lr_queue);
>> + }
>> +}
>> +
>> +static int its_handle_inv(struct virt_its *its, uint64_t *cmdptr)
>> +{
>> + struct domain *d = its->d;
>> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
>> + uint32_t eventid = its_cmd_get_id(cmdptr);
>> + struct pending_irq *p;
>> + unsigned long flags;
>> + struct vcpu *vcpu;
>> + uint32_t vlpi;
>> + int ret = -1;
>> +
>> + /* Translate the event into a vCPU/vLPI pair. */
>> + if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
>> + return -1;
>> +
>> + if ( vlpi == INVALID_LPI )
>> + return -1;
>> +
>> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
>> +
>> + p = d->arch.vgic.handler->lpi_to_pending(d, vlpi);
>> + if ( !p )
>> + goto out_unlock;
>
> As said on v5, this could be simpler and use the pending_irqs in the
> device. That would be an improvement though. So a would be good.
Originally I found it more straight-forward to use the one existing
interface (the rbtree) we also use in the VGIC part, which would allow
us to handle locking or ref-counting in one central place.
But indeed the ITS command handling has all the data we need to find the
pending_irq directly from the virtual device.
So I replaced all lpi_to_pending() calls in those handlers with a new
function gicv3_its_get_event_pending_irq(), which looks up the struct
from an ITS/device/event triple.
I take and keep the its->lock for the runtime of these functions, so
those events and their memory will not vanish meanwhile.
Does that make sense?
Cheers,
Andre.
>> +
>> + /* Read the property table and update our cached status. */
>> + if ( update_lpi_property(d, vlpi, p) )
>> + goto out_unlock;
>> +
>> + /* Check whether the LPI needs to go on a VCPU. */
>> + update_lpi_vgic_status(vcpu, p, vlpi);
>> +
>> + ret = 0;
>> +
>> +out_unlock:
>> + spin_unlock_irqrestore(&vcpu->arch.vgic.lock, flags);
>> +
>> + return ret;
>> +}
>> +
>> static int its_handle_mapc(struct virt_its *its, uint64_t *cmdptr)
>> {
>> uint32_t collid = its_cmd_get_collection(cmdptr);
>> @@ -757,6 +819,9 @@ static int vgic_its_handle_cmds(struct domain *d,
>> struct virt_its *its)
>> case GITS_CMD_INT:
>> ret = its_handle_int(its, command);
>> break;
>> + case GITS_CMD_INV:
>> + ret = its_handle_inv(its, command);
>> + break;
>> case GITS_CMD_MAPC:
>> ret = its_handle_mapc(its, command);
>> break;
>>
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-05-10 11:07 ` Julien Grall
@ 2017-05-10 17:14 ` Andre Przywara
2017-05-10 17:17 ` Julien Grall
0 siblings, 1 reply; 82+ messages in thread
From: Andre Przywara @ 2017-05-10 17:14 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 10/05/17 12:07, Julien Grall wrote:
>
>
> On 05/10/2017 11:47 AM, Andre Przywara wrote:
>> Hi,
>
> Hi Andre,
>
>> On 12/04/17 11:44, Julien Grall wrote:
>>> On 12/04/17 01:44, Andre Przywara wrote:
>>>> +/* Retrieve the priority of an LPI from its struct pending_irq. */
>>>> +static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
>>>> +{
>>>> + struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
>>>> +
>>>> + if ( !p )
>>>> + return GIC_PRI_IRQ;
>>>
>>> Why the check here? And why returning GIC_PRI_IRQ?
>>
>> Because you surely want to avoid dereferencing NULL?
>> I changed the return value to 0xff, which is the lowest priority.
>> Frankly I think we could just return anything, as we will stop handling
>> this LPI anyway a bit later in the code if p is NULL here.
>
> I agree that you want to prevent NULL. But we also want to avoid return
> fake value because there was a caller that didn't bother to check
> whether the interrupt is valid at first hand.
Well, I changed the sequence in vgic_vcpu_inject_irq() back to be:
priority = vgic_get_virq_priority(v, virq);
spin_lock_irqsave(&v->arch.vgic.lock, flags);
n = irq_to_pending(v, virq);
mostly to prevent the locking order (rank vs. VCPU lock) issue you
mentioned. We read the latest priority value upfront, but only use it
later if the pending_irq is valid. I don't see how this should create
problems. Eventually this will be solved properly by the pending_irq lock.
> If you ever have NULL here then there is a latent BUG in your code
> somewhere else.
Not in this case.
> Ignoring the NULL and return a fake value is likely not
> the right solution for development.
>
> I can see two solutions for this:
> - ASSERT(p)
> - if ( !p )
> {
> ASSERT_UNREACHABLE();
> return 0xff;
> }
>
> The later would still return a dumb value but at least we would catch
> programming mistake during development.
I think this solution asks for the ASSERT to trigger in corner cases: If
the LPI fired on the host, but got unmapped shortly afterwards. In this
case vgic_vcpu_inject_irq() can be reached with an invalid LPI number,
and we handle this properly when irq_to_pending() returns NULL.
But in this case get_priority() will be called with the same invalid
LPI, so should be able to cope with that as well.
Again this will eventually be solved properly with the per-IRQ lock.
Cheers,
Andre.
>>
>>> AFAICT, vgic_v3_lpi_to_pending should never return NULL when reading the
>>> priority. Or else, you are in big trouble.
>>
>> That depends on where and when you call it, which I don't want to make
>> any assumptions about.
>> In my latest version the vgic_get_virq_priority() call now stays at the
>> very beginning of vgic_vcpu_inject_irq(), so at this point the LPI could
>> have been unmapped meanwhile.
>> Surely you will bail out handling this LPI later in the code if it
>> returns NULL here, but for the sake of this function I think we need the
>> check.
>> To me it looks a bit like having this abstraction here is a bit
>> pointless and complicates things, but well ....
>
> Well, I am not against very defensive programming. I am more against
> returning a fake value that may impact the rest of the vGIC (not even
> mentioning the lack of comment explain why). After all, the priority is
> controlled by the guest and not the hypervisor.
>
> I suggested few way above to catch those errors.
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-05-10 17:14 ` Andre Przywara
@ 2017-05-10 17:17 ` Julien Grall
2017-05-11 17:55 ` Andre Przywara
0 siblings, 1 reply; 82+ messages in thread
From: Julien Grall @ 2017-05-10 17:17 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 05/10/2017 06:14 PM, Andre Przywara wrote:
> Hi,
>
> On 10/05/17 12:07, Julien Grall wrote:
>>
>>
>> On 05/10/2017 11:47 AM, Andre Przywara wrote:
>>> Hi,
>>
>> Hi Andre,
>>
>>> On 12/04/17 11:44, Julien Grall wrote:
>>>> On 12/04/17 01:44, Andre Przywara wrote:
>>>>> +/* Retrieve the priority of an LPI from its struct pending_irq. */
>>>>> +static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
>>>>> +{
>>>>> + struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
>>>>> +
>>>>> + if ( !p )
>>>>> + return GIC_PRI_IRQ;
>>>>
>>>> Why the check here? And why returning GIC_PRI_IRQ?
>>>
>>> Because you surely want to avoid dereferencing NULL?
>>> I changed the return value to 0xff, which is the lowest priority.
>>> Frankly I think we could just return anything, as we will stop handling
>>> this LPI anyway a bit later in the code if p is NULL here.
>>
>> I agree that you want to prevent NULL. But we also want to avoid return
>> fake value because there was a caller that didn't bother to check
>> whether the interrupt is valid at first hand.
>
> Well, I changed the sequence in vgic_vcpu_inject_irq() back to be:
>
> priority = vgic_get_virq_priority(v, virq);
>
> spin_lock_irqsave(&v->arch.vgic.lock, flags);
> n = irq_to_pending(v, virq);
>
> mostly to prevent the locking order (rank vs. VCPU lock) issue you
> mentioned. We read the latest priority value upfront, but only use it
> later if the pending_irq is valid. I don't see how this should create
> problems. Eventually this will be solved properly by the pending_irq lock.
>
>> If you ever have NULL here then there is a latent BUG in your code
>> somewhere else.
>
> Not in this case.
Because of the locking issue? I know there are locking issue, but it
does not mean we should introduce bad code just for workaround them for
the time being...
>
>> Ignoring the NULL and return a fake value is likely not
>> the right solution for development.
>>
>> I can see two solutions for this:
>> - ASSERT(p)
>> - if ( !p )
>> {
>> ASSERT_UNREACHABLE();
>> return 0xff;
>> }
>>
>> The later would still return a dumb value but at least we would catch
>> programming mistake during development.
>
> I think this solution asks for the ASSERT to trigger in corner cases: If
> the LPI fired on the host, but got unmapped shortly afterwards. In this
> case vgic_vcpu_inject_irq() can be reached with an invalid LPI number,
> and we handle this properly when irq_to_pending() returns NULL.
> But in this case get_priority() will be called with the same invalid
> LPI, so should be able to cope with that as well.
> Again this will eventually be solved properly with the per-IRQ lock.
See above. I still prefer to see the ASSERT firing time to time than bad
code going in staging.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 23/27] ARM: vITS: handle INV command
2017-05-10 15:11 ` Andre Przywara
@ 2017-05-11 10:43 ` Julien Grall
0 siblings, 0 replies; 82+ messages in thread
From: Julien Grall @ 2017-05-11 10:43 UTC (permalink / raw)
To: Andre Przywara, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
On 10/05/17 16:11, Andre Przywara wrote:
> Hi,
Hi Andre,
> On 12/04/17 18:20, Julien Grall wrote:
>> On 12/04/17 01:44, Andre Przywara wrote:
>>
>>> +{
>>> + ASSERT(spin_is_locked(&v->arch.vgic.lock));
>>
>> The locking is likely to wrong here too (see patch #2). For instance
>> with a MOVI then INV on interrupt enabled.
>>
>>> +
>>> + if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>>> + {
>>> + if ( !list_empty(&p->inflight) &&
>>> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
>>> + gic_raise_guest_irq(v, vlpi, p->lpi_priority);
>>> + }
>>> + else
>>> + {
>>> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>>> + list_del_init(&p->lr_queue);
>>> + }
>>> +}
>>> +
>>> +static int its_handle_inv(struct virt_its *its, uint64_t *cmdptr)
>>> +{
>>> + struct domain *d = its->d;
>>> + uint32_t devid = its_cmd_get_deviceid(cmdptr);
>>> + uint32_t eventid = its_cmd_get_id(cmdptr);
>>> + struct pending_irq *p;
>>> + unsigned long flags;
>>> + struct vcpu *vcpu;
>>> + uint32_t vlpi;
>>> + int ret = -1;
>>> +
>>> + /* Translate the event into a vCPU/vLPI pair. */
>>> + if ( !read_itte(its, devid, eventid, &vcpu, &vlpi) )
>>> + return -1;
>>> +
>>> + if ( vlpi == INVALID_LPI )
>>> + return -1;
>>> +
>>> + spin_lock_irqsave(&vcpu->arch.vgic.lock, flags);
>>> +
>>> + p = d->arch.vgic.handler->lpi_to_pending(d, vlpi);
>>> + if ( !p )
>>> + goto out_unlock;
>>
>> As said on v5, this could be simpler and use the pending_irqs in the
>> device. That would be an improvement though. So a would be good.
>
> Originally I found it more straight-forward to use the one existing
> interface (the rbtree) we also use in the VGIC part, which would allow
> us to handle locking or ref-counting in one central place.
> But indeed the ITS command handling has all the data we need to find the
> pending_irq directly from the virtual device.
> So I replaced all lpi_to_pending() calls in those handlers with a new
> function gicv3_its_get_event_pending_irq(), which looks up the struct
> from an ITS/device/event triple.
> I take and keep the its->lock for the runtime of these functions, so
> those events and their memory will not vanish meanwhile.
>
> Does that make sense?
It makes sense to keep the ref-counting in one central place. But it is
better to avoid reading guest memory and therefore avoid most of
checking and overhead to translate the IPA to PA.
That's why I suggested to use pending_irqs :).
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables
2017-04-12 13:13 ` Julien Grall
@ 2017-05-11 17:54 ` Andre Przywara
0 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-05-11 17:54 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi,
On 12/04/17 14:13, Julien Grall wrote:
>
>
> On 12/04/17 14:12, Andre Przywara wrote:
>> Hi,
>
> Hi,
>
>>
>> On 12/04/17 11:55, Julien Grall wrote:
>>> Hi Andre,
>>>
>>> On 12/04/17 01:44, Andre Przywara wrote:
>>>> Allow a guest to provide the address and size for the memory regions
>>>> it has reserved for the GICv3 pending and property tables.
>>>> We sanitise the various fields of the respective redistributor
>>>> registers.
>>>> The MMIO read and write accesses are protected by locks, to avoid any
>>>> changing of the property or pending table address while a redistributor
>>>> is live and also to protect the non-atomic vgic_reg64_extract()
>>>> function
>>>> on the MMIO read side.
>>>>
>>>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>>>> Reviewed-by: Julien Grall <julien.grall@arm.com>
>>>
>>> Whilst I gave my reviewed-by it is very rude to ignore a comment.
>>
>> Yeah, sorry about that! I was unsure about that as well, so thought
>> about it and eventually forgot to answer.
>>
>>> It would have been nicer to answer even if it is just saying "I can add
>>> a TODO and address it in a follow-up patch".
>>>
>>> Please get use to mention all the changes (e.g the spin_*lock ->
>>> spin_*lock_irq* change) you made in a patch. Mainly if you keep a
>>> reviewed-by.
>>
>> I was really unsure about keeping or dropping it, but since you
>> complained about me dropping it last time I tried it the other way this
>> time ;-)
>>
>>>> diff --git a/xen/include/asm-arm/domain.h
>>>> b/xen/include/asm-arm/domain.h
>>>> index ebaea35..b2d98bb 100644
>>>> --- a/xen/include/asm-arm/domain.h
>>>> +++ b/xen/include/asm-arm/domain.h
>>>> @@ -109,11 +109,15 @@ struct arch_domain
>>>> } *rdist_regions;
>>>> int nr_regions; /* Number of rdist
>>>> regions */
>>>> uint32_t rdist_stride; /* Re-Distributor
>>>> stride */
>>>> + unsigned long int nr_lpis;
>>>> + uint64_t rdist_propbase;
>>>> struct rb_root its_devices; /* Devices mapped to an
>>>> ITS */
>>>> spinlock_t its_devices_lock; /* Protects the
>>>> its_devices tree */
>>>> struct radix_tree_root pend_lpi_tree; /* Stores struct
>>>> pending_irq's */
>>>> rwlock_t pend_lpi_tree_lock; /* Protects the
>>>> pend_lpi_tree */
>>>> unsigned int intid_bits;
>>>> + bool rdists_enabled; /* Is any redistributor
>>>> enabled? */
>>>> + bool has_its;
>>>
>>> The comment you ignore was the one about consolidating rdists_enabled
>>> and has_its in a single field and use flags.
>>
>> Yes, I had the idea myself before, but decided against it as IMHO it
>> looks much nicer this way (compared to using a flags variable. which I
>> guess is what you mean).
>>
>> Are you OK with that?
>
> Why is it nicer?
Because I think it's more readable to have: "if ( has_its )" than
"if ( flags & HAS_ITS )".
> You only need 1 bit to represent rdist_enabled and
> another for has_its. This would save a bit a space in a resource limited
> structure.
But because of the padding it wouldn't, so I'd keep it as it is.
Cheers,
Andre.
> I would be OK with a TODO so we know we can save space here
> in the future...
>
> Although, my main point was you not ignore comment and say no if you
> disagree.
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests
2017-05-10 17:17 ` Julien Grall
@ 2017-05-11 17:55 ` Andre Przywara
0 siblings, 0 replies; 82+ messages in thread
From: Andre Przywara @ 2017-05-11 17:55 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Vijaya Kumar K, Vijay Kilari, Shanker Donthineni
Hi Julien,
On 10/05/17 18:17, Julien Grall wrote:
>
>
> On 05/10/2017 06:14 PM, Andre Przywara wrote:
>> Hi,
>>
>> On 10/05/17 12:07, Julien Grall wrote:
>>>
>>>
>>> On 05/10/2017 11:47 AM, Andre Przywara wrote:
>>>> Hi,
>>>
>>> Hi Andre,
>>>
>>>> On 12/04/17 11:44, Julien Grall wrote:
>>>>> On 12/04/17 01:44, Andre Przywara wrote:
>>>>>> +/* Retrieve the priority of an LPI from its struct pending_irq. */
>>>>>> +static int vgic_v3_lpi_get_priority(struct domain *d, uint32_t vlpi)
>>>>>> +{
>>>>>> + struct pending_irq *p = vgic_v3_lpi_to_pending(d, vlpi);
>>>>>> +
>>>>>> + if ( !p )
>>>>>> + return GIC_PRI_IRQ;
>>>>>
>>>>> Why the check here? And why returning GIC_PRI_IRQ?
>>>>
>>>> Because you surely want to avoid dereferencing NULL?
>>>> I changed the return value to 0xff, which is the lowest priority.
>>>> Frankly I think we could just return anything, as we will stop handling
>>>> this LPI anyway a bit later in the code if p is NULL here.
>>>
>>> I agree that you want to prevent NULL. But we also want to avoid return
>>> fake value because there was a caller that didn't bother to check
>>> whether the interrupt is valid at first hand.
>>
>> Well, I changed the sequence in vgic_vcpu_inject_irq() back to be:
>>
>> priority = vgic_get_virq_priority(v, virq);
>>
>> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>> n = irq_to_pending(v, virq);
>>
>> mostly to prevent the locking order (rank vs. VCPU lock) issue you
>> mentioned. We read the latest priority value upfront, but only use it
>> later if the pending_irq is valid. I don't see how this should create
>> problems. Eventually this will be solved properly by the pending_irq
>> lock.
>>
>>> If you ever have NULL here then there is a latent BUG in your code
>>> somewhere else.
>>
>> Not in this case.
>
> Because of the locking issue? I know there are locking issue, but it
> does not mean we should introduce bad code just for workaround them for
> the time being...
No, because we now (as before) call vgic_get_virq_priority() without any
locks, so anything could happen. And in the spirit of checking *every*
irq_to_pending() call I'd rather protect this case properly (without
trading NULL pointer exceptions for ASSERTs).
Please look at the new code and tell me if you still don't like it.
Cheers,
Andre.
>>
>>> Ignoring the NULL and return a fake value is likely not
>>> the right solution for development.
>>>
>>> I can see two solutions for this:
>>> - ASSERT(p)
>>> - if ( !p )
>>> {
>>> ASSERT_UNREACHABLE();
>>> return 0xff;
>>> }
>>>
>>> The later would still return a dumb value but at least we would catch
>>> programming mistake during development.
>>
>> I think this solution asks for the ASSERT to trigger in corner cases: If
>> the LPI fired on the host, but got unmapped shortly afterwards. In this
>> case vgic_vcpu_inject_irq() can be reached with an invalid LPI number,
>> and we handle this properly when irq_to_pending() returns NULL.
>> But in this case get_priority() will be called with the same invalid
>> LPI, so should be able to cope with that as well.
>> Again this will eventually be solved properly with the per-IRQ lock.
>
> See above. I still prefer to see the ASSERT firing time to time than bad
> code going in staging.
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 82+ messages in thread
end of thread, other threads:[~2017-05-11 17:53 UTC | newest]
Thread overview: 82+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-12 0:44 [PATCH v8 00/27] arm64: Dom0 ITS emulation Andre Przywara
2017-04-12 0:44 ` [PATCH v8 01/27] ARM: GICv3: propagate number of host LPIs to GICv3 guest Andre Przywara
2017-04-12 10:06 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 02/27] ARM: VGIC: move irq_to_pending() calls under the VGIC VCPU lock Andre Przywara
2017-04-12 10:13 ` Julien Grall
2017-04-12 11:38 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 03/27] ARM: GIC: Add checks for NULL pointer pending_irq's Andre Przywara
2017-04-12 10:25 ` Julien Grall
2017-04-12 14:51 ` Andre Przywara
2017-04-12 14:52 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 04/27] ARM: GICv3: introduce separate pending_irq structs for LPIs Andre Przywara
2017-04-12 10:35 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 05/27] ARM: GICv3: forward pending LPIs to guests Andre Przywara
2017-04-12 10:44 ` Julien Grall
2017-04-12 17:26 ` Andre Przywara
2017-05-10 10:47 ` Andre Przywara
2017-05-10 11:07 ` Julien Grall
2017-05-10 17:14 ` Andre Przywara
2017-05-10 17:17 ` Julien Grall
2017-05-11 17:55 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 06/27] ARM: GICv3: enable ITS and LPIs on the host Andre Przywara
2017-04-12 0:44 ` [PATCH v8 07/27] ARM: vGICv3: handle virtual LPI pending and property tables Andre Przywara
2017-04-12 10:55 ` Julien Grall
2017-04-12 13:12 ` Andre Przywara
2017-04-12 13:13 ` Julien Grall
2017-05-11 17:54 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 08/27] ARM: introduce vgic_access_guest_memory() Andre Przywara
2017-04-12 12:29 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 09/27] ARM: vGICv3: re-use vgic_reg64_check_access Andre Przywara
2017-04-12 0:44 ` [PATCH v8 10/27] ARM: GIC: export vgic_init_pending_irq() Andre Przywara
2017-04-12 0:44 ` [PATCH v8 11/27] ARM: VGIC: add vcpu_id to struct pending_irq Andre Przywara
2017-04-12 12:32 ` Julien Grall
2017-04-12 12:37 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 12/27] ARM: vGIC: advertise LPI support Andre Przywara
2017-04-12 12:38 ` Julien Grall
2017-04-12 12:48 ` Andre Przywara
2017-04-12 13:04 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 13/27] ARM: vITS: add command handling stub and MMIO emulation Andre Przywara
2017-04-12 12:59 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 14/27] ARM: vITS: introduce translation table walks Andre Przywara
2017-04-12 13:22 ` Julien Grall
2017-04-12 13:36 ` Andre Przywara
2017-04-12 13:37 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 15/27] ARM: vITS: handle CLEAR command Andre Przywara
2017-04-12 14:10 ` Julien Grall
2017-04-12 14:29 ` Andre Przywara
2017-04-12 14:49 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 16/27] ARM: vITS: handle INT command Andre Przywara
2017-04-12 14:50 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 17/27] ARM: vITS: handle MAPC command Andre Przywara
2017-04-12 14:51 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 18/27] ARM: vITS: handle MAPD command Andre Przywara
2017-04-12 15:21 ` Julien Grall
2017-04-12 17:03 ` Andre Przywara
2017-04-12 17:05 ` Julien Grall
2017-04-12 17:24 ` Andrew Cooper
2017-04-12 18:18 ` Wei Liu
2017-05-10 10:42 ` Andre Przywara
2017-05-10 11:30 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 19/27] ARM: vITS: handle MAPTI command Andre Przywara
2017-04-12 16:18 ` Julien Grall
2017-04-12 16:27 ` Andre Przywara
2017-04-12 17:16 ` Julien Grall
2017-04-12 17:25 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 20/27] ARM: GICv3: handle unmapped LPIs Andre Przywara
2017-04-12 9:46 ` Andre Przywara
2017-04-12 16:49 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 21/27] ARM: vITS: handle MOVI command Andre Przywara
2017-04-12 16:59 ` Julien Grall
2017-05-10 10:34 ` Andre Przywara
2017-04-12 0:44 ` [PATCH v8 22/27] ARM: vITS: handle DISCARD command Andre Przywara
2017-04-12 17:06 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 23/27] ARM: vITS: handle INV command Andre Przywara
2017-04-12 17:20 ` Julien Grall
2017-05-10 15:11 ` Andre Przywara
2017-05-11 10:43 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 24/27] ARM: vITS: handle INVALL command Andre Przywara
2017-04-12 17:26 ` Julien Grall
2017-04-12 0:44 ` [PATCH v8 25/27] ARM: vITS: increase mmio_count for each ITS Andre Przywara
2017-04-12 0:44 ` [PATCH v8 26/27] ARM: vITS: create and initialize virtual ITSes for Dom0 Andre Przywara
2017-04-12 0:44 ` [PATCH v8 27/27] ARM: vITS: create ITS subnodes for Dom0 DT Andre Przywara
2017-04-12 14:13 ` [PATCH v8 00/27] arm64: Dom0 ITS emulation Julien Grall
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.