* [PATCH v7 00/28] Add ITS support
@ 2015-09-18 13:08 vijay.kilari
2015-09-18 13:08 ` [PATCH v7 01/28] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
` (28 more replies)
0 siblings, 29 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
This is based on DraftG version
http://xenbits.xen.org/people/ianc/vits/draftG.pdf
Following major features are supported
- GICv3 ITS support for arm64 platform
- Only Dom0 is supported. For DomU pci passthrough feature
is required.
Changes in v7:
- Rebased to latest staging branch.
- Compiled all the patches individually for both arm32 and arm64
- Patch xen-arm-ITS-implement-hw_irq_controller-for-LPIs.patch
split into two patches.
- xen-arm-ITS-implement-hw_irq_controller-for-LPIs.patch and
- xen-arm-ITS-Plumbing-hw_irq_controller-for-LPIs.patch.
- Moved patch xen-arm-ITS-Add-GITS-registers-emulation.patch
before vITS compilation.
- Merged xen-arm-ITS-Enable-virtual-ITS-driver.patch with
patch xen-arm-ITS-Export-ITS-info-to-Virtual-ITS.patch.
- Dropped patch xen-arm-ITS-Add-32-bit-access-to-GICR_TYPER.patch.
This is only minor change. Hence merged with
xen-arm-ITS-Add-GICR-register-emulation.patch.
- Dropped patch xen-arm-ITS-Introduce-helper-to-get-number-of-event-.patch
- Dropped gic_lpi_supported helper function.
- Changed LPI property table usage and handling.
- Added support to pass nr_lpis from bootargs.
- Added support to enable/disable its support from bootargs.
- Fixed issues around freeing of its_device on error.
- Introduced vits.h file
- Dropped xen-dt-Handle-correctly-node-with-interrupt-map-in-d.patch
from this series as it is already merged.
Could not share code via Github. Network is very slow.
Please, let me know if required.
Changes in v6:
- Rebased to latest staging branch.
- Compiled all the patches individually for both arm32 and arm64
- Split the patch "xen/arm: ITS: Allocate irq descriptors for LPIs" into two.
One for allocating LPI irq_desc and other patch for allocating pending_irq desc
for LPIs
- Following new patches are introduced
1) xen/arm: Rename NR_IRQs and vgic_num_irqs helper function
2) xen/arm: ITS: Introduce msi_desc for LPIs
3) xen/arm: Move vgic locking inside get_irq_priority callback
4) xen/arm: ITS: Store LPIs allocated and IRQ ID bits per domain
5) xen/arm: ITS: Introduce helper to get number of event IDs
6) xen/arm: ITS: Add virtual ITS availability check helper
7) xen/arm: ITS: Add 32-bit access to GICR_TYPER)
8) xen/arm: ITS: Allocate pending_lpi descriptors for LPIs
- Based on below patch set
http://lists.xen.org/archives/html/xen-devel/2015-08/msg00168.html
Some Major TODOs:
1) Avoid making vits_process_cmd() static in later point of time
2) How to handle LPI that does not have LPI config table entry.
3) Enable/disable ITS to Dom0
Changes in v5:
- Added following new patches
0001-xen-arm-Return-success-if-dt-node-does-not-have-irq-.patch
0004-xen-arm-Set-nr_cpu_ids-to-available-number-of-cpus.patch
0009-xen-arm-ITS-Export-ITS-info-to-Virtual-ITS.patch
0013-xen-arm-ITS-Implement-gic_is_lpi-helper-function.patch
- Split patch #12 into two patches #14 & #16
0014-xen-arm-ITS-Allocate-irq-descriptors-for-LPIs.patch
0016-xen-arm-ITS-Route-LPIs.patch
- Introduce new API to route LPI (route_lpi_to_guest() )
- Moved patch #8 in v4 as patch #19
- irq_descritors for LPIs are allocated dynamically
- Removed RB-tree for managing vitual its devices. Will be
introduced when pci-passthrough is implemented
- its_add_device() api now takes nr_ites and DT its node as parameters
- some function are kept as non-static when introduced in a patch for
compilation purpose and eventually made static when used.
- Tested compilation for arm32
Changes in v4:
- Patch for rate limiting of error message is removed.
- Patch #4 and #5 in v3 is merged
- Merged #13 and #16 as one patch
- hw_irq_controller is implemented for LPIs
- GITS and GICR emulation for LPIs in separate patches
- Removed build functions for ITS command in physical ITS driver
- Added new patch to add and assign devices from platform file
- Enable compilation of vits and pits driver in separate patch
- Replace msi-parent property in all pci dt nodes to single
ITS node generated by Xen for Dom0
Vijaya Kumar K (28):
xen/arm: Add bitmap_find_next_zero_area helper function
xen: Add log2 functionality
xen/arm: Set nr_cpu_ids to available number of cpus
xen/arm: Rename NR_IRQs and vgic_num_irqs helper function
xen/arm: ITS: Port ITS driver to Xen
xen/arm: ITS: Add helper functions to manage its_devices
xen/arm: ITS: Introduce msi_desc for LPIs
xen/arm: ITS: Add APIs to add and assign device
xen/arm: ITS: Introduce gic_is_lpi helper function
xen/arm: ITS: Implement hw_irq_controller for LPIs
xen/arm: ITS: Enable compilation of physical ITS driver
xen/arm: ITS: Plumbing hw_irq_controller for LPIs
xen/arm: Move vgic rank locking inside get_irq_priority
xen/arm: ITS: Initialize physical ITS and export lpi support
xen/arm: ITS: Add virtual ITS driver
xen/arm: ITS: Add virtual ITS commands support
xen/arm: ITS: Add GITS registers emulation
xen/arm: ITS: Export ITS info to Virtual ITS
xen/arm: ITS: Store LPIs allocated per domain
xen/arm: ITS: Add virtual ITS availability check helper
xen/arm: ITS: Add GICR register emulation
xen/arm: ITS: Allocate irq descriptors for LPIs
xen/arm: ITS: Allocate pending_lpi descriptors for LPIs
xen/arm: ITS: Route LPIs
xen/arm: ITS: Add domain specific ITS initialization
xen/arm: ITS: Map ITS translation space
xen/arm: ITS: Generate ITS node for Dom0
xen/arm: ITS: Add pci devices in ThunderX
xen/arch/arm/Makefile | 2 +
xen/arch/arm/domain_build.c | 14 +
xen/arch/arm/gic-hip04.c | 17 +-
xen/arch/arm/gic-v2.c | 17 +-
xen/arch/arm/gic-v3-its.c | 1652 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 127 ++-
xen/arch/arm/gic.c | 66 +-
xen/arch/arm/irq.c | 183 +++-
xen/arch/arm/platforms/Makefile | 1 +
xen/arch/arm/platforms/thunderx.c | 200 +++++
xen/arch/arm/setup.c | 2 +
xen/arch/arm/vgic-v2.c | 6 +-
xen/arch/arm/vgic-v3-its.c | 951 +++++++++++++++++++++
xen/arch/arm/vgic-v3.c | 406 ++++++++-
xen/arch/arm/vgic.c | 146 +++-
xen/common/bitmap.c | 39 +
xen/include/asm-arm/domain.h | 10 +
xen/include/asm-arm/gic-its.h | 313 +++++++
xen/include/asm-arm/gic.h | 21 +-
xen/include/asm-arm/gic_v3_defs.h | 51 +-
xen/include/asm-arm/irq.h | 24 +-
xen/include/asm-arm/vgic.h | 14 +-
xen/include/asm-arm/vits.h | 100 +++
xen/include/xen/bitmap.h | 16 +
xen/include/xen/lib.h | 2 +
xen/include/xen/log2.h | 167 ++++
26 files changed, 4464 insertions(+), 83 deletions(-)
create mode 100644 xen/arch/arm/gic-v3-its.c
create mode 100644 xen/arch/arm/platforms/thunderx.c
create mode 100644 xen/arch/arm/vgic-v3-its.c
create mode 100644 xen/include/asm-arm/gic-its.h
create mode 100644 xen/include/asm-arm/vits.h
create mode 100644 xen/include/xen/log2.h
--
1.7.9.5
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v7 01/28] xen/arm: Add bitmap_find_next_zero_area helper function
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-18 13:08 ` [PATCH v7 02/28] xen: Add log2 functionality vijay.kilari
` (27 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Keir Fraser, Ian Campbell, vijay.kilari, Prasun.Kapoor,
Vijaya Kumar K, Ian Jackson, Jan Beulich, manish.jaggi
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
bitmap_find_next_zero_area helper function will be used
by physical ITS driver. This is imported from linux 4.2
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Keir Fraser <keir@xen.org>
CC: Tim Deegan <tim@xen.org>
---
v5: Ported from Linux 4.2.
Added bitmap_find_next_zero_area_off().
v4: Removed spaces and added tabs
Moved ALIGN macro to lib.h
v3: Moved changes to xen/common/bitmap.c and
xen/include/xen/bitmap.h
---
xen/common/bitmap.c | 39 +++++++++++++++++++++++++++++++++++++++
xen/include/xen/bitmap.h | 16 ++++++++++++++++
xen/include/xen/lib.h | 2 ++
3 files changed, 57 insertions(+)
diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index 61d1ea4..ad665d1 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -489,6 +489,45 @@ int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
}
EXPORT_SYMBOL(bitmap_allocate_region);
+/*
+ * bitmap_find_next_zero_area_off - find a contiguous aligned zero area
+ * @map: The address to base the search on
+ * @size: The bitmap size in bits
+ * @start: The bitnumber to start searching at
+ * @nr: The number of zeroed bits we're looking for
+ * @align_mask: Alignment mask for zero area
+ * @align_offset: Alignment offset for zero area.
+ *
+ * The @align_mask should be one less than a power of 2; the effect is that
+ * the bit offset of all zero areas this function finds plus @align_offset
+ * is multiple of that power of 2.
+ */
+unsigned long bitmap_find_next_zero_area_off(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask,
+ unsigned long align_offset)
+{
+ unsigned long index, end, i;
+again:
+ index = find_next_zero_bit(map, size, start);
+
+ /* Align allocation */
+ index = ALIGN_MASK(index + align_offset, align_mask) - align_offset;
+
+ end = index + nr;
+ if (end > size)
+ return end;
+ i = find_next_bit(map, end, index);
+ if (i < end) {
+ start = i + 1;
+ goto again;
+ }
+ return index;
+}
+EXPORT_SYMBOL(bitmap_find_next_zero_area_off)
+
#ifdef __BIG_ENDIAN
void bitmap_long_to_byte(uint8_t *bp, const unsigned long *lp, int nbits)
diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
index e2a3686..161f990 100644
--- a/xen/include/xen/bitmap.h
+++ b/xen/include/xen/bitmap.h
@@ -101,6 +101,22 @@ extern int bitmap_scnlistprintf(char *buf, unsigned int len,
extern int bitmap_find_free_region(unsigned long *bitmap, int bits, int order);
extern void bitmap_release_region(unsigned long *bitmap, int pos, int order);
extern int bitmap_allocate_region(unsigned long *bitmap, int pos, int order);
+extern unsigned long bitmap_find_next_zero_area_off(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask,
+ unsigned long align_offset);
+
+static inline unsigned long bitmap_find_next_zero_area(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask)
+{
+ return bitmap_find_next_zero_area_off(map, size, start, nr,
+ align_mask, 0);
+}
#define BITMAP_LAST_WORD_MASK(nbits) \
( \
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 4258912..e7d9d95 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -55,6 +55,8 @@
#define ROUNDUP(x, a) (((x) + (a) - 1) & ~((a) - 1))
+#define ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
+
#define reserve_bootmem(_p,_l) ((void)0)
struct domain;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 02/28] xen: Add log2 functionality
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
2015-09-18 13:08 ` [PATCH v7 01/28] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-18 13:08 ` [PATCH v7 03/28] xen/arm: Set nr_cpu_ids to available number of cpus vijay.kilari
` (26 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Keir Fraser, Ian Campbell, vijay.kilari, Prasun.Kapoor,
Vijaya Kumar K, Ian Jackson, Jan Beulich, manish.jaggi
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
log2 helper apis are ported from linux from
commit 13c07b0286d340275f2d97adf085cecda37ede37
(linux/log2.h: Fix rounddown_pow_of_two(1))
Changes made for xen are:
- Only required functionality is retained
- Replace fls_long with flsl
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Keir Fraser <keir@xen.org>
CC: Tim Deegan <tim@xen.org>
---
v4: - Only retained required functionality
- Replaced fls_long with flsl
- Removed fls_long implementation in bitops.h in v3 version
---
xen/include/xen/log2.h | 167 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 167 insertions(+)
diff --git a/xen/include/xen/log2.h b/xen/include/xen/log2.h
new file mode 100644
index 0000000..86bd861
--- /dev/null
+++ b/xen/include/xen/log2.h
@@ -0,0 +1,167 @@
+/*
+ * Copyright (C) 2006 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _XEN_LOG2_H
+#define _XEN_LOG2_H
+
+#include <xen/types.h>
+#include <xen/bitops.h>
+
+/*
+ * deal with unrepresentable constant logarithms
+ */
+extern __attribute__((const))
+int ____ilog2_NaN(void);
+
+/*
+ * non-constant log of base 2 calculators
+ * - the arch may override these in asm/bitops.h if they can be implemented
+ * more efficiently than using fls() and fls64()
+ * - the arch is not required to handle n==0 if implementing the fallback
+ */
+static inline __attribute__((const))
+int __ilog2_u32(u32 n)
+{
+ return fls(n) - 1;
+}
+
+static inline __attribute__((const))
+int __ilog2_u64(u64 n)
+{
+ return flsl(n) - 1;
+}
+
+/*
+ * round up to nearest power of two
+ */
+static inline __attribute__((const))
+unsigned long __roundup_pow_of_two(unsigned long n)
+{
+ return 1UL << flsl(n - 1);
+}
+
+/**
+ * ilog2 - log of base 2 of 32-bit or a 64-bit unsigned value
+ * @n - parameter
+ *
+ * constant-capable log of base 2 calculation
+ * - this can be used to initialise global variables from constant data, hence
+ * the massive ternary operator construction
+ *
+ * selects the appropriately-sized optimised version depending on sizeof(n)
+ */
+#define ilog2(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n) < 1 ? ____ilog2_NaN() : \
+ (n) & (1ULL << 63) ? 63 : \
+ (n) & (1ULL << 62) ? 62 : \
+ (n) & (1ULL << 61) ? 61 : \
+ (n) & (1ULL << 60) ? 60 : \
+ (n) & (1ULL << 59) ? 59 : \
+ (n) & (1ULL << 58) ? 58 : \
+ (n) & (1ULL << 57) ? 57 : \
+ (n) & (1ULL << 56) ? 56 : \
+ (n) & (1ULL << 55) ? 55 : \
+ (n) & (1ULL << 54) ? 54 : \
+ (n) & (1ULL << 53) ? 53 : \
+ (n) & (1ULL << 52) ? 52 : \
+ (n) & (1ULL << 51) ? 51 : \
+ (n) & (1ULL << 50) ? 50 : \
+ (n) & (1ULL << 49) ? 49 : \
+ (n) & (1ULL << 48) ? 48 : \
+ (n) & (1ULL << 47) ? 47 : \
+ (n) & (1ULL << 46) ? 46 : \
+ (n) & (1ULL << 45) ? 45 : \
+ (n) & (1ULL << 44) ? 44 : \
+ (n) & (1ULL << 43) ? 43 : \
+ (n) & (1ULL << 42) ? 42 : \
+ (n) & (1ULL << 41) ? 41 : \
+ (n) & (1ULL << 40) ? 40 : \
+ (n) & (1ULL << 39) ? 39 : \
+ (n) & (1ULL << 38) ? 38 : \
+ (n) & (1ULL << 37) ? 37 : \
+ (n) & (1ULL << 36) ? 36 : \
+ (n) & (1ULL << 35) ? 35 : \
+ (n) & (1ULL << 34) ? 34 : \
+ (n) & (1ULL << 33) ? 33 : \
+ (n) & (1ULL << 32) ? 32 : \
+ (n) & (1ULL << 31) ? 31 : \
+ (n) & (1ULL << 30) ? 30 : \
+ (n) & (1ULL << 29) ? 29 : \
+ (n) & (1ULL << 28) ? 28 : \
+ (n) & (1ULL << 27) ? 27 : \
+ (n) & (1ULL << 26) ? 26 : \
+ (n) & (1ULL << 25) ? 25 : \
+ (n) & (1ULL << 24) ? 24 : \
+ (n) & (1ULL << 23) ? 23 : \
+ (n) & (1ULL << 22) ? 22 : \
+ (n) & (1ULL << 21) ? 21 : \
+ (n) & (1ULL << 20) ? 20 : \
+ (n) & (1ULL << 19) ? 19 : \
+ (n) & (1ULL << 18) ? 18 : \
+ (n) & (1ULL << 17) ? 17 : \
+ (n) & (1ULL << 16) ? 16 : \
+ (n) & (1ULL << 15) ? 15 : \
+ (n) & (1ULL << 14) ? 14 : \
+ (n) & (1ULL << 13) ? 13 : \
+ (n) & (1ULL << 12) ? 12 : \
+ (n) & (1ULL << 11) ? 11 : \
+ (n) & (1ULL << 10) ? 10 : \
+ (n) & (1ULL << 9) ? 9 : \
+ (n) & (1ULL << 8) ? 8 : \
+ (n) & (1ULL << 7) ? 7 : \
+ (n) & (1ULL << 6) ? 6 : \
+ (n) & (1ULL << 5) ? 5 : \
+ (n) & (1ULL << 4) ? 4 : \
+ (n) & (1ULL << 3) ? 3 : \
+ (n) & (1ULL << 2) ? 2 : \
+ (n) & (1ULL << 1) ? 1 : \
+ (n) & (1ULL << 0) ? 0 : \
+ ____ilog2_NaN() \
+ ) : \
+ (sizeof(n) <= 4) ? \
+ __ilog2_u32(n) : \
+ __ilog2_u64(n) \
+ )
+
+/**
+ * roundup_pow_of_two - round the given value up to nearest power of two
+ * @n - parameter
+ *
+ * round the given value up to the nearest power of two
+ * - the result is undefined when n == 0
+ * - this can be used to initialise global variables from constant data
+ */
+#define roundup_pow_of_two(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n == 1) ? 1 : \
+ (1UL << (ilog2((n) - 1) + 1)) \
+ ) : \
+ __roundup_pow_of_two(n) \
+ )
+
+/**
+ * order_base_2 - calculate the (rounded up) base 2 order of the argument
+ * @n: parameter
+ *
+ * The first few values calculated by this routine:
+ * ob2(0) = 0
+ * ob2(1) = 0
+ * ob2(2) = 1
+ * ob2(3) = 2
+ * ob2(4) = 2
+ * ob2(5) = 3
+ * ... and so on.
+ */
+
+#define order_base_2(n) ilog2(roundup_pow_of_two(n))
+#endif /* _XEN_LOG2_H */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 03/28] xen/arm: Set nr_cpu_ids to available number of cpus
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
2015-09-18 13:08 ` [PATCH v7 01/28] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
2015-09-18 13:08 ` [PATCH v7 02/28] xen: Add log2 functionality vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-18 13:08 ` [PATCH v7 04/28] xen/arm: Rename NR_IRQs and vgic_num_irqs helper function vijay.kilari
` (25 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
nr_cpu_ids for arm platforms is incorrectly set to NR_CPUS
irrespective of the number of cpus supported by platform.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
---
v6: - Updated nr_cpu_ids in setup.c instead of creating
a helper function
---
xen/arch/arm/setup.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 48f734f..2bb6274 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -771,6 +771,8 @@ void __init start_xen(unsigned long boot_phys_offset,
smp_init_cpus();
cpus = smp_get_max_cpus();
+ printk(XENLOG_INFO "SMP: Allowing %u CPUs\n", cpus);
+ nr_cpu_ids = cpus;
init_xen_time();
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 04/28] xen/arm: Rename NR_IRQs and vgic_num_irqs helper function
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (2 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 03/28] xen/arm: Set nr_cpu_ids to available number of cpus vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-21 10:40 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
` (24 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
NR_IRQS represents the number of lines (i.e SGIs, PPIs and SPIs).
With the introduction of LPIs, NR_IRQs is renamed to NR_ITLINES.
Similarly vgic_num_irqs() is renamed as vgic_num_irq_lines().
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Renamed NR_LINE_IRQS as NR_ITLINES and vgic_num_line_irqs()
as vgic_num_irq_lines()
---
xen/arch/arm/gic.c | 2 +-
xen/arch/arm/irq.c | 10 +++++-----
xen/arch/arm/vgic-v3.c | 2 +-
xen/arch/arm/vgic.c | 10 +++++-----
xen/include/asm-arm/irq.h | 9 +++++----
xen/include/asm-arm/vgic.h | 2 +-
6 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 1757193..1d94e5e 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -140,7 +140,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
ASSERT(spin_is_locked(&desc->lock));
/* Caller has already checked that the IRQ is an SPI */
ASSERT(virq >= 32);
- ASSERT(virq < vgic_num_irqs(d));
+ ASSERT(virq < vgic_num_irq_lines(d));
vgic_lock_rank(v_target, rank, flags);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 1f38605..9b48d9c 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -55,7 +55,7 @@ hw_irq_controller no_irq_type = {
.end = end_none
};
-static irq_desc_t irq_desc[NR_IRQS];
+static irq_desc_t irq_desc[NR_ITLINES];
static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
irq_desc_t *__irq_to_desc(int irq)
@@ -75,7 +75,7 @@ static int __init init_irq_data(void)
{
int irq;
- for (irq = NR_LOCAL_IRQS; irq < NR_IRQS; irq++) {
+ for (irq = NR_LOCAL_IRQS; irq < NR_ITLINES; irq++) {
struct irq_desc *desc = irq_to_desc(irq);
init_one_irq_desc(desc);
desc->irq = irq;
@@ -407,11 +407,11 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned long flags;
int retval = 0;
- if ( virq >= vgic_num_irqs(d) )
+ if ( virq >= vgic_num_irq_lines(d) )
{
printk(XENLOG_G_ERR
"the vIRQ number %u is too high for domain %u (max = %u)\n",
- irq, d->domain_id, vgic_num_irqs(d));
+ irq, d->domain_id, vgic_num_irq_lines(d));
return -EINVAL;
}
@@ -523,7 +523,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
int ret;
/* Only SPIs are supported */
- if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
+ if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irq_lines(d) )
return -EINVAL;
p = spi_to_pending(d, virq);
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index daa510a..598f634 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -721,7 +721,7 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
* Number of interrupt identifier bits supported by the GIC
* Stream Protocol Interface
*/
- unsigned int irq_bits = get_count_order(vgic_num_irqs(v->domain));
+ unsigned int irq_bits = get_count_order(vgic_num_irq_lines(v->domain));
/*
* Number of processors that may be used as interrupt targets when ARE
* bit is zero. The maximum is 8.
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index a6835a8..80a8f4e 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -122,7 +122,7 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
return ret;
d->arch.vgic.allocated_irqs =
- xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
+ xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irq_lines(d)));
if ( !d->arch.vgic.allocated_irqs )
return -ENOMEM;
@@ -254,7 +254,7 @@ void arch_move_irqs(struct vcpu *v)
struct vcpu *v_target;
int i;
- for ( i = 32; i < vgic_num_irqs(d); i++ )
+ for ( i = 32; i < vgic_num_irq_lines(d); i++ )
{
v_target = vgic_get_target_vcpu(v, i);
p = irq_to_pending(v_target, i);
@@ -465,7 +465,7 @@ void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
struct vcpu *v;
/* the IRQ needs to be an SPI */
- ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
+ ASSERT(virq >= 32 && virq <= vgic_num_irq_lines(d));
v = vgic_get_target_vcpu(d->vcpu[0], virq);
vgic_vcpu_inject_irq(v, virq);
@@ -487,7 +487,7 @@ int vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
bool_t vgic_reserve_virq(struct domain *d, unsigned int virq)
{
- if ( virq >= vgic_num_irqs(d) )
+ if ( virq >= vgic_num_irq_lines(d) )
return 0;
return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
@@ -507,7 +507,7 @@ int vgic_allocate_virq(struct domain *d, bool_t spi)
else
{
first = 32;
- end = vgic_num_irqs(d);
+ end = vgic_num_irq_lines(d);
}
/*
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index f33c331..9be83b4 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -19,11 +19,12 @@ struct arch_irq_desc {
};
#define NR_LOCAL_IRQS 32
-#define NR_IRQS 1024
+/* Number of SGIs + PPIs + SPIs */
+#define NR_ITLINES 1024
-#define nr_irqs NR_IRQS
-#define nr_static_irqs NR_IRQS
-#define arch_hwdom_irqs(domid) NR_IRQS
+#define nr_irqs NR_ITLINES
+#define nr_static_irqs NR_ITLINES
+#define arch_hwdom_irqs(domid) NR_ITLINES
struct irq_desc;
struct irqaction;
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 62d24b6..cf5a790 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -307,7 +307,7 @@ enum gic_sgi_mode;
*/
#define REG_RANK_INDEX(b, n, s) ((((n) >> s) & ((b)-1)) % 32)
-#define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
+#define vgic_num_irq_lines(d) ((d)->arch.vgic.nr_spis + 32)
extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
extern void domain_vgic_free(struct domain *d);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (3 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 04/28] xen/arm: Rename NR_IRQs and vgic_num_irqs helper function vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-21 11:23 ` Julien Grall
2015-09-21 13:08 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
` (23 subsequent siblings)
28 siblings, 2 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
The linux driver is based on 4.1 with below commit id
591e5bec13f15feb13fc445b6c9c59954711c4ac
Only following code from Linux ITS driver is ported
and compiled
- LPI initialization
- ITS configuration code
- Physical command queue management
- ITS command building
Also redistributor information is split into rdist and
rdist_prop structures.
The rdist_prop struct holds the redistributor common
information for all re-distributor and rdist struct
holds the per-cpu specific information.
This per-cpu rdist is defined as global and shared with
physical ITS driver.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7:
- Introduced back its_send_inv()
- Fix coding styles
v6:
- made dump_cmd() static and const parameter
- Include commit 591e5bec13f15feb13fc445b6c9c59954711c4ac
"irqchip/gicv3-its: Fix mapping of LPIs to collections".
- reodered its_alloc_lpi_tables() and its_lpi_init()
v5:
- dump_cmd is called from its_flush_cmd
- Added its_lpi_alloc_chunks, lpi_free, its_send_inv, its_send_mapd,
its_send_mapvi to this patch as these functions are ported from
linux and more logical to be in this patch.
For now these functions are non-static. Will be made static
when used.
- Used this_cpu instead of per_cpu
- Moved GITS_* definitions to git-its.h
v4: Major changes
- Redistributor refactoring patch is merged
- Fixed comments from v3 related to coding style and
removing duplicate code.
- Target address is stored from bits[48:16] to avoid
shifting of target address while building ITS commands
- Removed non-static functions
- Removed usage of command builder functions
- Changed its_cmd_block union to include mix of bit and unsigned
variable types to define ITS command structure
v3:
- Only required changes from Linux ITS driver is ported
- Xen coding style is followed.
---
xen/arch/arm/gic-v3-its.c | 1025 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 15 +-
xen/include/asm-arm/gic-its.h | 282 ++++++++++
xen/include/asm-arm/gic.h | 1 +
xen/include/asm-arm/gic_v3_defs.h | 42 +-
5 files changed, 1358 insertions(+), 7 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
new file mode 100644
index 0000000..4a6c8a3
--- /dev/null
+++ b/xen/arch/arm/gic-v3-its.c
@@ -0,0 +1,1025 @@
+/*
+ * Copyright (C) 2013, 2014 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Xen changes:
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ * Copyright (C) 2014, 2015 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/config.h>
+#include <xen/bitops.h>
+#include <xen/init.h>
+#include <xen/mm.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/errno.h>
+#include <xen/delay.h>
+#include <xen/list.h>
+#include <xen/sizes.h>
+#include <xen/vmap.h>
+#include <asm/p2m.h>
+#include <asm/domain.h>
+#include <asm/io.h>
+#include <asm/device.h>
+#include <asm/gic.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
+#include <xen/log2.h>
+
+#define its_print(lvl, fmt, ...) \
+ printk(lvl "GIC-ITS:" fmt, ## __VA_ARGS__)
+
+#define its_err(fmt, ...) its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
+/* TODO: ratelimit for Xen messages */
+#define its_err_ratelimited(fmt, ...) \
+ its_print(XENLOG_G_ERR, fmt, ## __VA_ARGS__)
+
+#define its_dbg(fmt, ...) \
+ its_print(XENLOG_DEBUG, fmt, ## __VA_ARGS__)
+
+#define its_info(fmt, ...) \
+ its_print(XENLOG_INFO, fmt, ## __VA_ARGS__)
+
+#define its_warn(fmt, ...) \
+ its_print(XENLOG_WARNING, fmt, ## __VA_ARGS__)
+
+//#define DEBUG_GIC_ITS
+
+#ifdef DEBUG_GIC_ITS
+# define DPRINTK(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1 << 0)
+#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
+
+/*
+ * The ITS structure - contains most of the infrastructure, with the
+ * msi_controller, the command queue, the collections, and the list of
+ * devices writing to it.
+ */
+struct its_node {
+ spinlock_t lock;
+ struct list_head entry;
+ void __iomem *base;
+ paddr_t phys_base;
+ paddr_t phys_size;
+ its_cmd_block *cmd_base;
+ its_cmd_block *cmd_write;
+ void *tables[GITS_BASER_NR_REGS];
+ u32 order[GITS_BASER_NR_REGS];
+ struct its_collection *collections;
+ u64 flags;
+ u32 ite_size;
+ struct dt_device_node *dt_node;
+};
+
+#define ITS_ITT_ALIGN SZ_256
+
+static LIST_HEAD(its_nodes);
+static DEFINE_SPINLOCK(its_lock);
+static struct rdist_prop *gic_rdists;
+
+#define gic_data_rdist() (this_cpu(rdist))
+
+#ifdef DEBUG_GIC_ITS
+static void dump_cmd(const its_cmd_block *cmd)
+{
+ printk(XENLOG_DEBUG
+ "ITS: CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
+}
+#else
+static void dump_cmd(const its_cmd_block *cmd) { }
+#endif
+
+static struct its_collection *dev_event_to_col(struct its_device *dev,
+ u32 event)
+{
+ struct its_node *its = dev->its;
+
+ return its->collections + dev->event_map.col_map[event];
+}
+
+#define ITS_CMD_QUEUE_SZ SZ_64K
+#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
+
+static u64 its_cmd_ptr_to_offset(struct its_node *its, its_cmd_block *ptr)
+{
+ return (ptr - its->cmd_base) * sizeof(*ptr);
+}
+
+static int its_queue_full(struct its_node *its)
+{
+ int widx;
+ int ridx;
+
+ widx = its->cmd_write - its->cmd_base;
+ ridx = readl_relaxed(its->base + GITS_CREADR) / sizeof(its_cmd_block);
+
+ /* This is incredibly unlikely to happen, unless the ITS locks up. */
+ if ( ((widx + 1) % ITS_CMD_QUEUE_NR_ENTRIES) == ridx )
+ return 1;
+
+ return 0;
+}
+
+static its_cmd_block *its_allocate_entry(struct its_node *its)
+{
+ its_cmd_block *cmd;
+ u32 count = 1000000; /* 1s! */
+
+ while ( its_queue_full(its) )
+ {
+ count--;
+ if ( !count )
+ {
+ its_err_ratelimited("ITS queue not draining\n");
+ return NULL;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+
+ cmd = its->cmd_write++;
+
+ /* Handle queue wrapping */
+ if (its->cmd_write == (its->cmd_base + ITS_CMD_QUEUE_NR_ENTRIES))
+ its->cmd_write = its->cmd_base;
+
+ return cmd;
+}
+
+static its_cmd_block *its_post_commands(struct its_node *its)
+{
+ u64 wr = its_cmd_ptr_to_offset(its, its->cmd_write);
+
+ writel_relaxed(wr, its->base + GITS_CWRITER);
+
+ return its->cmd_write;
+}
+
+static void its_flush_cmd(struct its_node *its, its_cmd_block *cmd)
+{
+ /*
+ * Make sure the commands written to memory are observable by
+ * the ITS.
+ */
+ if ( its->flags & ITS_FLAGS_CMDQ_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cmd, sizeof(*cmd));
+ else
+ dsb(ishst);
+
+ dump_cmd(cmd);
+}
+
+static void its_wait_for_range_completion(struct its_node *its,
+ its_cmd_block *from,
+ its_cmd_block *to)
+{
+ u64 rd_idx, from_idx, to_idx;
+ u32 count = 1000000; /* 1s! */
+
+ from_idx = its_cmd_ptr_to_offset(its, from);
+ to_idx = its_cmd_ptr_to_offset(its, to);
+
+ while ( 1 )
+ {
+ rd_idx = readl_relaxed(its->base + GITS_CREADR);
+ if ( rd_idx >= to_idx || rd_idx < from_idx )
+ break;
+
+ count--;
+ if ( !count )
+ {
+ its_err_ratelimited("ITS queue timeout\n");
+ return;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static void its_send_single_command(struct its_node *its,
+ its_cmd_block *src,
+ struct its_collection *sync_col)
+{
+ its_cmd_block *cmd, *sync_cmd, *next_cmd;
+ unsigned long flags;
+
+ BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
+
+ spin_lock_irqsave(&its->lock, flags);
+
+ cmd = its_allocate_entry(its);
+ if ( !cmd )
+ { /* We're soooooo screewed... */
+ its_err_ratelimited("ITS can't allocate, dropping command\n");
+ spin_unlock_irqrestore(&its->lock, flags);
+ return;
+ }
+
+ memcpy(cmd, src, sizeof(its_cmd_block));
+ its_flush_cmd(its, cmd);
+
+ if ( sync_col )
+ {
+ sync_cmd = its_allocate_entry(its);
+ if ( !sync_cmd )
+ {
+ its_err_ratelimited("ITS can't SYNC, skipping\n");
+ goto post;
+ }
+ sync_cmd->sync.cmd = GITS_CMD_SYNC;
+ sync_cmd->sync.ta = sync_col->target_address;
+
+ its_flush_cmd(its, sync_cmd);
+ }
+
+post:
+ next_cmd = its_post_commands(its);
+ spin_unlock_irqrestore(&its->lock, flags);
+
+ its_wait_for_range_completion(its, cmd, next_cmd);
+}
+
+static void its_send_inv(struct its_device *dev, u32 event)
+{
+ its_cmd_block cmd;
+ struct its_collection *col = dev_event_to_col(dev, event);
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.inv.cmd = GITS_CMD_INV;
+ cmd.inv.devid = dev->device_id;
+ cmd.inv.event = event;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
+static void its_send_mapd(struct its_device *dev, int valid)
+{
+ its_cmd_block cmd;
+ unsigned long itt_addr;
+ u8 size;
+
+ size = ilog2(dev->event_map.nr_lpis);
+ itt_addr = __pa(dev->itt);
+ itt_addr = ROUNDUP(itt_addr, ITS_ITT_ALIGN);
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapd.cmd = GITS_CMD_MAPD;
+ cmd.mapd.devid = dev->device_id;
+ cmd.mapd.size = size - 1;
+ /*
+ * ITT address field of MAPD command holds bit[48:8] of
+ * itt address. Hence shift by 8.
+ */
+ cmd.mapd.itt = itt_addr >> 8;
+ cmd.mapd.valid = !!valid;
+
+ its_send_single_command(dev->its, &cmd, NULL);
+}
+
+static void its_send_mapc(struct its_node *its, struct its_collection *col,
+ int valid)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapc.cmd = GITS_CMD_MAPC;
+ cmd.mapc.col = col->col_id;
+ cmd.mapc.ta = col->target_address;
+ cmd.mapc.valid = !!valid;
+
+ its_send_single_command(its, &cmd, col);
+}
+
+static void its_send_mapvi(struct its_device *dev, u32 phys_id, u32 event)
+{
+ its_cmd_block cmd;
+ struct its_collection *col = dev_event_to_col(dev, event);
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapvi.cmd = GITS_CMD_MAPVI;
+ cmd.mapvi.devid = dev->device_id;
+ cmd.mapvi.event = event;
+ cmd.mapvi.phy_id = phys_id;
+ cmd.mapvi.col = col->col_id;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
+static void its_send_invall(struct its_node *its, struct its_collection *col)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.invall.cmd = GITS_CMD_INVALL;
+ cmd.invall.col = col->col_id;
+
+ its_send_single_command(its, &cmd, NULL);
+}
+
+static void its_send_discard(struct its_device *dev, u32 event)
+{
+ its_cmd_block cmd;
+ struct its_collection *col = dev_event_to_col(dev, event);
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.discard.devid = dev->device_id;
+ cmd.discard.event = event;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
+/*
+ * How we allocate LPIs:
+ *
+ * The GIC has id_bits bits for interrupt identifiers. From there, we
+ * must subtract 8192 which are reserved for SGIs/PPIs/SPIs. Then, as
+ * we allocate LPIs by chunks of 32, we can shift the whole thing by 5
+ * bits to the right.
+ *
+ * This gives us (((1UL << id_bits) - 8192) >> 5) possible allocations.
+ */
+#define IRQS_PER_CHUNK_SHIFT 5
+#define IRQS_PER_CHUNK (1 << IRQS_PER_CHUNK_SHIFT)
+
+static unsigned long *lpi_bitmap;
+static u32 lpi_chunks;
+static DEFINE_SPINLOCK(lpi_lock);
+
+static int its_lpi_to_chunk(int lpi)
+{
+ return (lpi - 8192) >> IRQS_PER_CHUNK_SHIFT;
+}
+
+static int its_chunk_to_lpi(int chunk)
+{
+ return (chunk << IRQS_PER_CHUNK_SHIFT) + 8192;
+}
+
+static int its_lpi_init(u32 id_bits)
+{
+ lpi_chunks = its_lpi_to_chunk(1UL << id_bits);
+
+ lpi_bitmap = xzalloc_bytes(BITS_TO_LONGS(lpi_chunks) * sizeof(long));
+ if ( !lpi_bitmap )
+ {
+ lpi_chunks = 0;
+ return -ENOMEM;
+ }
+
+ its_info("ITS: Allocated %d chunks for LPIs\n", (int)lpi_chunks);
+
+ return 0;
+}
+
+static unsigned long *its_lpi_alloc_chunks(int nirqs, int *base)
+{
+ unsigned long *bitmap = NULL;
+ int chunk_id, nr_chunks, nr_ids, i;
+
+ nr_chunks = DIV_ROUND_UP(nirqs, IRQS_PER_CHUNK);
+
+ spin_lock(&lpi_lock);
+
+ do {
+ chunk_id = bitmap_find_next_zero_area(lpi_bitmap, lpi_chunks,
+ 0, nr_chunks, 0);
+ if ( chunk_id < lpi_chunks )
+ break;
+
+ nr_chunks--;
+ } while ( nr_chunks > 0 );
+
+ if ( !nr_chunks )
+ goto out;
+
+ bitmap = xzalloc_bytes(BITS_TO_LONGS(nr_chunks * IRQS_PER_CHUNK) *
+ sizeof (long));
+ if ( !bitmap )
+ goto out;
+
+ for ( i = 0; i < nr_chunks; i++ )
+ set_bit(chunk_id + i, lpi_bitmap);
+
+ *base = its_chunk_to_lpi(chunk_id);
+ nr_ids = nr_chunks * IRQS_PER_CHUNK;
+
+ if ( nr_ids < nirqs )
+ {
+ xfree(bitmap);
+ bitmap = NULL;
+ }
+
+out:
+ spin_unlock(&lpi_lock);
+
+ return bitmap;
+}
+
+static void its_lpi_free(struct its_device *dev)
+{
+ int lpi;
+
+ spin_lock(&lpi_lock);
+
+ for ( lpi = dev->event_map.lpi_base;
+ lpi < (dev->event_map.lpi_base + dev->event_map.nr_lpis);
+ lpi += IRQS_PER_CHUNK )
+ {
+ int chunk = its_lpi_to_chunk(lpi);
+
+ if (chunk > lpi_chunks)
+ its_err("Bad LPI chunk %d\n", chunk);
+ if ( test_bit(chunk, lpi_bitmap) )
+ clear_bit(chunk, lpi_bitmap);
+ }
+
+ spin_unlock(&lpi_lock);
+
+ xfree(dev->event_map.lpi_map);
+}
+
+/*
+ * We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
+ * deal with (one configuration byte per interrupt). PENDBASE has to
+ * be 64kB aligned (one bit per LPI, plus 8192 bits for SPI/PPI/SGI).
+ */
+#define LPI_PROPBASE_SZ SZ_64K
+#define LPI_PENDBASE_SZ (LPI_PROPBASE_SZ / 8 + SZ_1K)
+
+/*
+ * This is how many bits of ID we need, including the useless ones.
+ */
+#define LPI_NRBITS ilog2(LPI_PROPBASE_SZ + SZ_8K)
+
+static int __init its_alloc_lpi_tables(void)
+{
+ paddr_t paddr;
+
+ gic_rdists->prop_page =
+ alloc_xenheap_pages(get_order_from_bytes(LPI_PROPBASE_SZ), 0);
+
+ if ( !gic_rdists->prop_page )
+ {
+ its_err("Failed to allocate PROPBASE\n");
+ return -ENOMEM;
+ }
+
+ paddr = __pa(gic_rdists->prop_page);
+ its_info("GIC: using LPI property table @%pa\n", &paddr);
+
+ /* Set LPI priority and Group-1, but disabled */
+ memset(gic_rdists->prop_page,
+ GIC_PRI_IRQ | LPI_PROP_GROUP1,
+ LPI_PROPBASE_SZ);
+
+ /* Make sure the GIC will observe the written configuration */
+ clean_and_invalidate_dcache_va_range(gic_rdists->prop_page,
+ LPI_PROPBASE_SZ);
+
+ return 0;
+}
+
+static const char *its_base_type_string[] = {
+ [GITS_BASER_TYPE_DEVICE] = "Devices",
+ [GITS_BASER_TYPE_VCPU] = "Virtual CPUs",
+ [GITS_BASER_TYPE_CPU] = "Physical CPUs",
+ [GITS_BASER_TYPE_COLLECTION] = "Interrupt Collections",
+ [GITS_BASER_TYPE_RESERVED5] = "Reserved (5)",
+ [GITS_BASER_TYPE_RESERVED6] = "Reserved (6)",
+ [GITS_BASER_TYPE_RESERVED7] = "Reserved (7)",
+};
+
+static void its_free_tables(struct its_node *its)
+{
+ int i;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ if ( its->tables[i] )
+ {
+ free_xenheap_pages(its->tables[i], its->order[i]);
+ its->tables[i] = NULL;
+ its->order[i] = 0;
+ }
+ }
+}
+
+static int its_alloc_tables(struct its_node *its)
+{
+ int err;
+ int i;
+ int psz = SZ_64K;
+ u64 shr = GITS_BASER_InnerShareable;
+ u64 cache = GITS_BASER_WaWb;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
+ u64 type = GITS_BASER_TYPE(val);
+ u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
+ unsigned int order = get_order_from_bytes(psz);
+ int alloc_size;
+ u64 tmp;
+ void *base;
+
+ if ( type == GITS_BASER_TYPE_NONE )
+ continue;
+
+ /*
+ * Allocate as many entries as required to fit the
+ * range of device IDs that the ITS can grok... The ID
+ * space being incredibly sparse, this results in a
+ * massive waste of memory.
+ *
+ * For other tables, only allocate a single page.
+ */
+ if ( type == GITS_BASER_TYPE_DEVICE )
+ {
+ u64 typer = readq_relaxed(its->base + GITS_TYPER);
+ u32 ids = GITS_TYPER_DEVBITS(typer);
+
+ order = max(get_order_from_bytes((1UL << ids) * entry_size), order);
+ if (order >= MAX_ORDER)
+ {
+ order = MAX_ORDER - 1;
+ its_warn("Device Table too large,reduce its page order to %u\n",
+ order);
+ }
+ }
+
+ alloc_size = (1 << order) * PAGE_SIZE;
+ base = alloc_xenheap_pages(order, 0);
+ if ( !base )
+ {
+ err = -ENOMEM;
+ goto out_free;
+ }
+ memset(base, 0, alloc_size);
+ its->tables[i] = base;
+ its->order[i] = order;
+
+retry_baser:
+ val = (__pa(base) |
+ (type << GITS_BASER_TYPE_SHIFT) |
+ ((entry_size - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
+ cache |
+ shr |
+ GITS_BASER_VALID);
+
+ switch (psz) {
+ case SZ_4K:
+ val |= GITS_BASER_PAGE_SIZE_4K;
+ break;
+ case SZ_16K:
+ val |= GITS_BASER_PAGE_SIZE_16K;
+ break;
+ case SZ_64K:
+ val |= GITS_BASER_PAGE_SIZE_64K;
+ break;
+ }
+
+ val |= (alloc_size / psz) - 1;
+
+ writeq_relaxed(val, its->base + GITS_BASER + i * 8);
+ tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
+
+ if ( (val ^ tmp) & GITS_BASER_SHAREABILITY_MASK )
+ {
+ /*
+ * Shareability didn't stick. Just use
+ * whatever the read reported, which is likely
+ * to be the only thing this redistributor
+ * supports.
+ */
+ shr = tmp & GITS_BASER_SHAREABILITY_MASK;
+ if ( !shr )
+ cache = GITS_BASER_nC;
+ goto retry_baser;
+ }
+
+ if ( (val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK )
+ {
+ /*
+ * Page size didn't stick. Let's try a smaller
+ * size and retry. If we reach 4K, then
+ * something is horribly wrong...
+ */
+ switch (psz) {
+ case SZ_16K:
+ psz = SZ_4K;
+ goto retry_baser;
+ case SZ_64K:
+ psz = SZ_16K;
+ goto retry_baser;
+ }
+ }
+
+ if ( val != tmp )
+ {
+ its_err("ITS: GITS_BASER%d doesn't stick: %lx %lx\n",
+ i, (unsigned long) val, (unsigned long) tmp);
+ err = -ENXIO;
+ goto out_free;
+ }
+
+ its_info("ITS: allocated %d %s @%lx (psz %dK, shr %d)\n",
+ (int)(alloc_size / entry_size),
+ its_base_type_string[type],
+ (unsigned long)__pa(base),
+ psz / SZ_1K, (int)shr >> GITS_BASER_SHAREABILITY_SHIFT);
+ }
+
+ return 0;
+
+out_free:
+ its_free_tables(its);
+
+ return err;
+}
+
+static int its_alloc_collections(struct its_node *its)
+{
+ its->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
+ if ( !its->collections )
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void its_cpu_init_lpis(void)
+{
+ void __iomem *rbase = gic_data_rdist().rbase;
+ void *pend_page;
+ u64 val, tmp;
+
+ /* If we didn't allocate the pending table yet, do it now */
+ pend_page = gic_data_rdist().pend_page;
+ if ( !pend_page )
+ {
+ paddr_t paddr;
+ u32 order;
+
+ /*
+ * The pending pages have to be at least 64kB aligned,
+ * hence the 'max(LPI_PENDBASE_SZ, SZ_64K)' below.
+ */
+ order = get_order_from_bytes(max(LPI_PENDBASE_SZ, SZ_64K));
+ pend_page = alloc_xenheap_pages(order, 0);
+ if ( !pend_page )
+ {
+ its_err("Failed to allocate PENDBASE for CPU%d with order %d\n",
+ smp_processor_id(), order);
+ return;
+ }
+
+ memset(pend_page, 0, max(LPI_PENDBASE_SZ, SZ_64K));
+ /* Make sure the GIC will observe the zero-ed page */
+ clean_and_invalidate_dcache_va_range(pend_page, LPI_PENDBASE_SZ);
+
+ paddr = __pa(pend_page);
+
+ its_info("CPU%d: using LPI pending table @%pa\n",
+ smp_processor_id(), &paddr);
+
+ gic_data_rdist().pend_page = pend_page;
+ }
+
+ /* Disable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val &= ~GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /*
+ * Make sure any change to the table is observable by the GIC.
+ */
+ dsb(sy);
+
+ /* set PROPBASE */
+ val = (__pa(gic_rdists->prop_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb |
+ ((LPI_NRBITS - 1) & GICR_PROPBASER_IDBITS_MASK));
+
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ tmp = readq_relaxed(rbase + GICR_PROPBASER);
+
+ if ( (tmp ^ val) & GICR_PROPBASER_SHAREABILITY_MASK )
+ {
+ if ( !(tmp & GICR_PROPBASER_SHAREABILITY_MASK) )
+ {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as well.
+ */
+ val &= ~(GICR_PROPBASER_SHAREABILITY_MASK |
+ GICR_PROPBASER_CACHEABILITY_MASK);
+ val |= GICR_PROPBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ }
+
+ its_info("GIC: using cache flushing for LPI property table\n");
+ gic_rdists->flags |= RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING;
+ }
+
+ /* set PENDBASE */
+ val = (__pa(pend_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb);
+
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+ tmp = readq_relaxed(rbase + GICR_PENDBASER);
+
+ if ( !(tmp & GICR_PENDBASER_SHAREABILITY_MASK) )
+ {
+ /*
+ * The HW reports non-shareable, we must remove the
+ * cacheability attributes as well.
+ */
+ val &= ~(GICR_PENDBASER_SHAREABILITY_MASK |
+ GICR_PENDBASER_CACHEABILITY_MASK);
+ val |= GICR_PENDBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+ }
+
+ /* Enable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val |= GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /* Make sure the GIC has seen the above */
+ dsb(sy);
+}
+
+static void its_cpu_init_collection(void)
+{
+ struct its_node *its;
+ int cpu;
+
+ spin_lock(&its_lock);
+ cpu = smp_processor_id();
+
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ u64 target;
+
+ /*
+ * We now have to bind each collection to its target
+ * redistributor.
+ */
+ if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
+ {
+ target = gic_data_rdist().phys_base;
+ /* ITS command considers only [48:16] of the GICR address */
+ target >>= 16;
+ }
+ else
+ {
+ /*
+ * This ITS wants a linear CPU number.
+ */
+ target = readq_relaxed(gic_data_rdist().rbase + GICR_TYPER);
+ target = GICR_TYPER_CPU_NUMBER(target);
+ }
+
+ /* Perform collection mapping */
+ its->collections[cpu].col_id = cpu;
+ its->collections[cpu].target_address = target;
+
+ its_send_mapc(its, &its->collections[cpu], 1);
+ its_send_invall(its, &its->collections[cpu]);
+ }
+
+ spin_unlock(&its_lock);
+}
+
+static int its_force_quiescent(void __iomem *base)
+{
+ u32 count = 1000000; /* 1s */
+ u32 val;
+
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ /* Disable the generation of all interrupts to this ITS */
+ val &= ~GITS_CTLR_ENABLE;
+ writel_relaxed(val, base + GITS_CTLR);
+
+ /* Poll GITS_CTLR and wait until ITS becomes quiescent */
+ while ( 1 )
+ {
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ count--;
+ if ( !count )
+ return -EBUSY;
+
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static int its_probe(struct dt_device_node *node)
+{
+ paddr_t its_addr, its_size;
+ struct its_node *its;
+ void __iomem *its_base;
+ u32 val, typer;
+ u64 baser, tmp;
+ int err;
+
+ if ( !dt_get_property(node, "msi-controller", NULL) )
+ {
+ its_warn("%s: not a msi-controller\n", node->full_name);
+ return -ENXIO;
+ }
+
+ err = dt_device_get_address(node, 0, &its_addr, &its_size);
+ if ( err )
+ {
+ its_warn("%s: no regs?\n", node->full_name);
+ return -ENXIO;
+ }
+
+ its_base = ioremap_nocache(its_addr, its_size);
+ if ( !its_base )
+ {
+ its_warn("%s: unable to map registers\n", node->full_name);
+ return -ENOMEM;
+ }
+
+ val = readl_relaxed(its_base + GITS_PIDR2) & GIC_PIDR2_ARCH_REV_MASK;
+ if ( val != 0x30 && val != 0x40 )
+ {
+ its_warn("%s: no ITS detected, giving up\n", node->full_name);
+ err = -ENODEV;
+ goto out_unmap;
+ }
+
+ err = its_force_quiescent(its_base);
+ if ( err )
+ {
+ its_warn("%s: failed to quiesce, giving up\n",
+ node->full_name);
+ goto out_unmap;
+ }
+
+ its_info("ITS: %s\n", node->full_name);
+
+ its = xzalloc(struct its_node);
+ if ( !its )
+ {
+ err = -ENOMEM;
+ goto out_unmap;
+ }
+
+ spin_lock_init(&its->lock);
+ INIT_LIST_HEAD(&its->entry);
+ its->dt_node = node;
+ its->base = its_base;
+ its->phys_base = its_addr;
+ its->phys_size = its_size;
+ typer = readl_relaxed(its_base + GITS_TYPER);
+ its->ite_size = ((typer >> 4) & 0xf) + 1;
+
+ its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
+ if ( !its->cmd_base )
+ {
+ err = -ENOMEM;
+ goto out_free_its;
+ }
+ its->cmd_write = its->cmd_base;
+
+ err = its_alloc_tables(its);
+ if ( err )
+ goto out_free_cmd;
+
+ err = its_alloc_collections(its);
+ if ( err )
+ goto out_free_tables;
+
+ baser = (__pa(its->cmd_base) |
+ GITS_CBASER_WaWb |
+ GITS_CBASER_InnerShareable |
+ (ITS_CMD_QUEUE_SZ / SZ_4K - 1) |
+ GITS_CBASER_VALID);
+
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ tmp = readq_relaxed(its->base + GITS_CBASER);
+ if ( (tmp ^ baser) & GITS_CBASER_SHAREABILITY_MASK )
+ {
+ if (!(tmp & GITS_CBASER_SHAREABILITY_MASK))
+ {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as
+ * well.
+ */
+ baser &= ~(GITS_CBASER_SHAREABILITY_MASK |
+ GITS_CBASER_CACHEABILITY_MASK);
+ baser |= GITS_CBASER_nC;
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ }
+
+ its_info("ITS: using cache flushing for cmd queue\n");
+ its->flags |= ITS_FLAGS_CMDQ_NEEDS_FLUSHING;
+ }
+
+ writeq_relaxed(0, its->base + GITS_CWRITER);
+ writel_relaxed(GITS_CTLR_ENABLE, its->base + GITS_CTLR);
+
+ spin_lock(&its_lock);
+ list_add(&its->entry, &its_nodes);
+ spin_unlock(&its_lock);
+
+ return 0;
+
+out_free_tables:
+ its_free_tables(its);
+out_free_cmd:
+ xfree(its->cmd_base);
+out_free_its:
+ xfree(its);
+out_unmap:
+ iounmap(its_base);
+ its_err("ITS: failed probing %s (%d)\n", node->full_name, err);
+ return err;
+}
+
+static bool gic_rdists_supports_plpis(void)
+{
+ return !!(readl_relaxed(gic_data_rdist().rbase + GICR_TYPER) &
+ GICR_TYPER_PLPIS);
+}
+
+int its_cpu_init(void)
+{
+ if ( !list_empty(&its_nodes) )
+ {
+ if ( !gic_rdists_supports_plpis() )
+ {
+ its_info("CPU%d: LPIs not supported\n", smp_processor_id());
+ return -ENXIO;
+ }
+ its_cpu_init_lpis();
+ its_cpu_init_collection();
+ }
+
+ return 0;
+}
+
+int __init its_init(struct rdist_prop *rdists)
+{
+ struct dt_device_node *np = NULL;
+
+ static const struct dt_device_match its_device_ids[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+
+ for ( np = dt_find_matching_node(NULL, its_device_ids); np;
+ np = dt_find_matching_node(np, its_device_ids) )
+ its_probe(np);
+
+ if ( list_empty(&its_nodes) )
+ {
+ its_warn("ITS: No ITS available, not enabling LPIs\n");
+ return -ENXIO;
+ }
+
+ gic_rdists = rdists;
+ its_alloc_lpi_tables();
+ its_lpi_init(rdists->id_bits);
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index d1db1ce..5076706 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -45,6 +45,7 @@
/* Global state */
static struct {
void __iomem *map_dbase; /* Mapped address of distributor registers */
+ struct rdist_prop rdist_data;
struct rdist_region *rdist_regions;
uint32_t rdist_stride;
unsigned int rdist_count; /* Number of rdist regions count */
@@ -55,10 +56,10 @@ static struct {
static struct gic_info gicv3_info;
/* per-cpu re-distributor base */
-static DEFINE_PER_CPU(void __iomem*, rbase);
+DEFINE_PER_CPU(struct rdist, rdist);
#define GICD (gicv3.map_dbase)
-#define GICD_RDIST_BASE (this_cpu(rbase))
+#define GICD_RDIST_BASE (this_cpu(rdist).rbase)
#define GICD_RDIST_SGI_BASE (GICD_RDIST_BASE + SZ_64K)
/*
@@ -618,6 +619,7 @@ static int __init gicv3_populate_rdist(void)
uint32_t aff;
uint32_t reg;
uint64_t typer;
+ uint64_t offset;
uint64_t mpidr = cpu_logical_map(smp_processor_id());
/*
@@ -653,9 +655,12 @@ static int __init gicv3_populate_rdist(void)
if ( (typer >> 32) == aff )
{
- this_cpu(rbase) = ptr;
- printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
- smp_processor_id(), i, ptr);
+ offset = ptr - gicv3.rdist_regions[i].map_base;
+ this_cpu(rdist).rbase = ptr;
+ this_cpu(rdist).phys_base = gicv3.rdist_regions[i].base + offset;
+ printk("GICv3: CPU%d: Found redistributor in region %d @%"PRIpaddr"\n",
+ smp_processor_id(), i,
+ this_cpu(rdist).phys_base);
return 0;
}
if ( gicv3.rdist_stride )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
new file mode 100644
index 0000000..5cc567c
--- /dev/null
+++ b/xen/include/asm-arm/gic-its.h
@@ -0,0 +1,282 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_ARM_GIC_ITS_H__
+#define __ASM_ARM_GIC_ITS_H__
+
+#include <asm/gic_v3_defs.h>
+
+/*
+ * ITS registers, offsets from ITS_base
+ */
+#define GITS_CTLR 0x0000
+#define GITS_IIDR 0x0004
+#define GITS_TYPER 0x0008
+#define GITS_CBASER 0x0080
+#define GITS_CWRITER 0x0088
+#define GITS_CREADR 0x0090
+#define GITS_BASER0 0x0100
+#define GITS_BASER1 0x0108
+#define GITS_BASER 0x0100
+#define GITS_BASERN 0x013c
+#define GITS_PIDR0 GICR_PIDR0
+#define GITS_PIDR1 GICR_PIDR1
+#define GITS_PIDR2 GICR_PIDR2
+#define GITS_PIDR3 GICR_PIDR3
+#define GITS_PIDR4 GICR_PIDR4
+#define GITS_PIDR5 GICR_PIDR5
+#define GITS_PIDR7 GICR_PIDR7
+
+#define GITS_TRANSLATER 0x10040
+#define GITS_CTLR_QUIESCENT (1U << 31)
+#define GITS_CTLR_ENABLE (1U << 0)
+
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_DEVBITS(r) ((((r) >> GITS_TYPER_DEVBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_IDBITS_SHIFT 8
+#define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_PTA (1UL << 19)
+#define GITS_TYPER_HCC_SHIFT (24)
+
+#define GITS_CBASER_VALID (1UL << 63)
+#define GITS_CBASER_nC (1UL << 59)
+#define GITS_CBASER_WaWb (5UL << 59)
+#define GITS_CBASER_InnerShareable (1UL << 10)
+#define GITS_CBASER_SHAREABILITY_MASK (3UL << 10)
+#define GITS_CBASER_CACHEABILITY_MASK (7UL << 59)
+
+#define GITS_BASER_NR_REGS 8
+
+#define GITS_BASER_VALID (1UL << 63)
+#define GITS_BASER_nC (1UL << 59)
+#define GITS_BASER_WaWb (5UL << 59)
+#define GITS_BASER_TYPE_SHIFT (56)
+#define GITS_BASER_TYPE_MASK (0x7)
+#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
+#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0xff) + 1)
+#define GITS_BASER_InnerShareable (1UL << 10)
+#define GITS_BASER_SHAREABILITY_SHIFT (10)
+#define GITS_BASER_SHAREABILITY_MASK (3UL << GITS_BASER_SHAREABILITY_SHIFT)
+#define GITS_BASER_PAGE_SIZE_SHIFT (8)
+#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_TYPE_NONE 0
+#define GITS_BASER_TYPE_DEVICE 1
+#define GITS_BASER_TYPE_VCPU 2
+#define GITS_BASER_TYPE_CPU 3
+#define GITS_BASER_TYPE_COLLECTION 4
+#define GITS_BASER_TYPE_RESERVED5 5
+#define GITS_BASER_TYPE_RESERVED6 6
+#define GITS_BASER_TYPE_RESERVED7 7
+
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD 0x08
+#define GITS_CMD_MAPC 0x09
+#define GITS_CMD_MAPVI 0x0a
+#define GITS_CMD_MAPI 0x0b
+#define GITS_CMD_MOVI 0x01
+#define GITS_CMD_DISCARD 0x0f
+#define GITS_CMD_INV 0x0c
+#define GITS_CMD_MOVALL 0x0e
+#define GITS_CMD_INVALL 0x0d
+#define GITS_CMD_INT 0x03
+#define GITS_CMD_CLEAR 0x04
+#define GITS_CMD_SYNC 0x05
+
+#define LPI_PROP_ENABLED (1 << 0)
+#define LPI_PROP_GROUP1 (1 << 1)
+
+/*
+ * Collection structure - just an ID, and a redistributor address to
+ * ping. We use one per CPU as collection of interrupts assigned to this
+ * CPU.
+ */
+struct its_collection {
+ u64 target_address;
+ u16 col_id;
+};
+
+/* ITS command structure */
+typedef union {
+ u64 bits[4];
+ struct __packed {
+ uint8_t cmd;
+ uint8_t pad[7];
+ } hdr;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u64 size:5;
+ u64 res2:59;
+ /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
+ u64 res3:8;
+ u64 itt:40;
+ u64 res4:15;
+ u64 valid:1;
+ u64 res5;
+ } mapd;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 col;
+ u32 ta;
+ u16 res3:15;
+ u16 valid:1;
+ u64 res4;
+ } mapc;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } mapi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 phy_id;
+ u16 col;
+ u8 res2[6];
+ u64 res3;
+ } mapvi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } movi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } discard;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } inv;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 res3;
+ u32 ta1;
+ u16 res4;
+ u16 res5;
+ u32 ta2;
+ u16 res6;
+ } movall;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } invall;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } int_cmd;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } clear;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 res3;
+ u32 ta;
+ u16 res4;
+ u64 res5;
+ } sync;
+} its_cmd_block;
+
+struct event_lpi_map {
+ /* Physical LPI map */
+ unsigned long *lpi_map;
+ /* Collection map */
+ u16 *col_map;
+ /* First Physical LPI number assigned */
+ u32 lpi_base;
+ /* Number of ITES/Physical LPIs assigned */
+ u32 nr_lpis;
+};
+
+/*
+ * The ITS view of a device.
+ */
+struct its_device {
+ /* Physical ITS */
+ struct its_node *its;
+ /* Device ITT address */
+ void *itt;
+ /* Device ITT size */
+ unsigned long itt_size;
+ /* LPI and event mapping */
+ struct event_lpi_map event_map;
+ /* Physical Device id */
+ u32 device_id;
+};
+
+int its_init(struct rdist_prop *rdists);
+int its_cpu_init(void);
+
+#endif /* __ASM_ARM_GIC_ITS_H__ */
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index d343abf..e330fe3 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -161,6 +161,7 @@
DT_MATCH_COMPATIBLE("arm,gic-400")
#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
+#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
#ifdef HAS_GICV3
/*
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index bf7b239..2c322da 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -48,6 +48,7 @@
/* Additional bits in GICD_TYPER defined by GICv3 */
#define GICD_TYPE_ID_BITS_SHIFT 19
+#define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
#define GICD_CTLR_RWP (1UL << 31)
#define GICD_CTLR_ARE_NS (1U << 4)
#define GICD_CTLR_ENABLE_G1A (1U << 1)
@@ -59,11 +60,12 @@
#define GICR_WAKER_ProcessorSleep (1U << 1)
#define GICR_WAKER_ChildrenAsleep (1U << 2)
-#define GICD_PIDR2_ARCH_REV_MASK (0xf0)
+#define GIC_PIDR2_ARCH_REV_MASK (0xf0)
+#define GICD_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICD_PIDR2_ARCH_REV_SHIFT (0x4)
#define GICD_PIDR2_ARCH_GICV3 (0x3)
-#define GICR_PIDR2_ARCH_REV_MASK GICD_PIDR2_ARCH_REV_MASK
+#define GICR_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICR_PIDR2_ARCH_REV_SHIFT GICD_PIDR2_ARCH_REV_SHIFT
#define GICR_PIDR2_ARCH_GICV3 GICD_PIDR2_ARCH_GICV3
@@ -113,10 +115,24 @@
#define GICR_ICFGR1 (0x0C04)
#define GICR_NSACR (0x0E00)
+#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
+#define GICR_TYPER_CPU_NUMBER(r) (((r) >> 8) & 0xffff)
+
+#define GICR_PROPBASER_InnerShareable (1U << 10)
+#define GICR_PROPBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PROPBASER_nC (1U << 7)
+#define GICR_PROPBASER_WaWb (5U << 7)
+#define GICR_PROPBASER_CACHEABILITY_MASK (7U << 7)
+#define GICR_PROPBASER_IDBITS_MASK (0x1f)
#define GICR_TYPER_PLPIS (1U << 0)
#define GICR_TYPER_VLPIS (1U << 1)
#define GICR_TYPER_LAST (1U << 4)
+#define GICR_PENDBASER_InnerShareable (1U << 10)
+#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PENDBASER_nC (1U << 7)
+#define GICR_PENDBASER_CACHEABILITY_MASK (7U << 7)
+
#define DEFAULT_PMR_VALUE 0xff
#define GICH_VMCR_EOI (1 << 9)
@@ -161,6 +177,28 @@ struct rdist_region {
void __iomem *map_base;
};
+/* Re-distributor per-cpu information */
+struct rdist {
+ /* CPU Re-distributor address */
+ void __iomem *rbase;
+ /* CPU LPI pending table */
+ void *pend_page;
+ /* CPU Re-distributor physical address */
+ paddr_t phys_base;
+};
+
+/* Common Re-distributor information */
+struct rdist_prop {
+ /* CPUs LPI configuration table */
+ void *prop_page;
+ /* Number of ID bits */
+ int id_bits;
+ /* Flags to store rdist attributes */
+ uint64_t flags;
+};
+
+DECLARE_PER_CPU(struct rdist, rdist);
+
#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (4 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-18 14:15 ` Julien Grall
2015-09-21 11:26 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 07/28] xen/arm: ITS: Introduce msi_desc for LPIs vijay.kilari
` (22 subsequent siblings)
28 siblings, 2 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Helper functions to manage its devices using RB-tree
are introduced in physical ITS driver.
This is global list of all the devices.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
---
v7: - Introduce its_remove_device api to remove device
from rb-tree
v5: - Added assert on spinlock
v4: - Remove passing of root node as parameter
- Declare prototype in header file
- Rename find_its_device to its_find_device
---
xen/arch/arm/gic-v3-its.c | 59 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 3 +++
2 files changed, 62 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 4a6c8a3..b983297 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -93,6 +93,8 @@ struct its_node {
static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
+static struct rb_root rb_its_dev;
+static DEFINE_SPINLOCK(rb_its_dev_lock);
#define gic_data_rdist() (this_cpu(rdist))
@@ -115,6 +117,61 @@ static struct its_collection *dev_event_to_col(struct its_device *dev,
return its->collections + dev->event_map.col_map[event];
}
+/* RB-tree helpers for its_device */
+static struct its_device *its_find_device(u32 devid)
+{
+ struct rb_node *node = rb_its_dev.rb_node;
+
+ ASSERT(spin_is_locked(&rb_its_dev_lock));
+ while ( node )
+ {
+ struct its_device *dev;
+
+ dev = container_of(node, struct its_device, node);
+ if ( devid < dev->device_id )
+ node = node->rb_left;
+ else if ( devid > dev->device_id )
+ node = node->rb_right;
+ else
+ return dev;
+ }
+
+ return NULL;
+}
+
+static int its_insert_device(struct its_device *dev)
+{
+ struct rb_node **new, *parent;
+
+ ASSERT(spin_is_locked(&rb_its_dev_lock));
+ new = &rb_its_dev.rb_node;
+ parent = NULL;
+ while ( *new )
+ {
+ struct its_device *this;
+
+ this = container_of(*new, struct its_device, node);
+ parent = *new;
+ if ( dev->device_id < this->device_id )
+ new = &((*new)->rb_left);
+ else if ( dev->device_id > this->device_id )
+ new = &((*new)->rb_right);
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&dev->node, parent, new);
+ rb_insert_color(&dev->node, &rb_its_dev);
+
+ return 0;
+}
+
+static void its_remove_device(struct its_device *dev)
+{
+ if ( dev )
+ rb_erase(&dev->node, &rb_its_dev);
+}
+
#define ITS_CMD_QUEUE_SZ SZ_64K
#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
@@ -952,6 +1009,8 @@ static int its_probe(struct dt_device_node *node)
list_add(&its->entry, &its_nodes);
spin_unlock(&its_lock);
+ rb_its_dev = RB_ROOT;
+
return 0;
out_free_tables:
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 5cc567c..8b84ae2 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -19,6 +19,7 @@
#define __ASM_ARM_GIC_ITS_H__
#include <asm/gic_v3_defs.h>
+#include <xen/rbtree.h>
/*
* ITS registers, offsets from ITS_base
@@ -266,6 +267,8 @@ struct its_device {
struct event_lpi_map event_map;
/* Physical Device id */
u32 device_id;
+ /* RB-tree entry */
+ struct rb_node node;
};
int its_init(struct rdist_prop *rdists);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 07/28] xen/arm: ITS: Introduce msi_desc for LPIs
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (5 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-18 13:08 ` [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
` (21 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Define msi_desc structure for arm and introduce
helper functions to access msi_desc member variables.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
---
xen/arch/arm/gic-v3-its.c | 28 ++++++++++++++++++++++++++++
xen/arch/arm/irq.c | 12 ++++++++++++
xen/include/asm-arm/gic-its.h | 4 ++++
xen/include/asm-arm/irq.h | 9 +++++++++
4 files changed, 53 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index b983297..4b7d9ed 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -109,6 +109,34 @@ static void dump_cmd(const its_cmd_block *cmd)
static void dump_cmd(const its_cmd_block *cmd) { }
#endif
+void irqdesc_set_lpi_event(struct irq_desc *desc, unsigned id)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ irq_get_msi_desc(desc)->eventID = id;
+}
+
+unsigned int irqdesc_get_lpi_event(struct irq_desc *desc)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ return irq_get_msi_desc(desc)->eventID;
+}
+
+struct its_device *irqdesc_get_its_device(struct irq_desc *desc)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ return irq_get_msi_desc(desc)->dev;
+}
+
+void irqdesc_set_its_device(struct irq_desc *desc, struct its_device *dev)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ irq_get_msi_desc(desc)->dev = dev;
+}
+
static struct its_collection *dev_event_to_col(struct its_device *dev,
u32 event)
{
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 9b48d9c..d582f57 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -143,6 +143,18 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
return irq_get_guest_info(desc)->d;
}
+void irq_set_msi_desc(struct irq_desc *desc, struct msi_desc *msi)
+{
+ desc->msi_desc = msi;
+}
+
+struct msi_desc *irq_get_msi_desc(struct irq_desc *desc)
+{
+ ASSERT(desc->msi_desc != NULL);
+
+ return desc->msi_desc;
+}
+
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
{
if ( desc != NULL )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 8b84ae2..c632854 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -271,6 +271,10 @@ struct its_device {
struct rb_node node;
};
+void irqdesc_set_lpi_event(struct irq_desc *desc, unsigned id);
+unsigned int irqdesc_get_lpi_event(struct irq_desc *desc);
+struct its_device *irqdesc_get_its_device(struct irq_desc *desc);
+void irqdesc_set_its_device(struct irq_desc *desc, struct its_device *dev);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 9be83b4..bafcd30 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -18,6 +18,13 @@ struct arch_irq_desc {
unsigned int type;
};
+struct msi_desc {
+#ifdef HAS_GICV3
+ unsigned int eventID;
+ struct its_device *dev;
+#endif
+};
+
#define NR_LOCAL_IRQS 32
/* Number of SGIs + PPIs + SPIs */
#define NR_ITLINES 1024
@@ -56,6 +63,8 @@ int irq_set_spi_type(unsigned int spi, unsigned int type);
int platform_get_irq(const struct dt_device_node *device, int index);
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask);
+void irq_set_msi_desc(struct irq_desc *desc, struct msi_desc *msi);
+struct msi_desc *irq_get_msi_desc(struct irq_desc *desc);
#endif /* _ASM_HW_IRQ_H */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (6 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 07/28] xen/arm: ITS: Introduce msi_desc for LPIs vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-21 13:47 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function vijay.kilari
` (20 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add APIs to add devices to RB-tree, assign and remove
devices to domain.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Added check for domain in its_assign_device() to avoid
assigning device to DomU
- Added comments whereever requested
- Called its_remove_device to remove from rb-tree
when failed to add device
- Changed dprintk to printk
- Store domain pointer instead of domain id in its_device struct
- Free msi_desc and reset irq_desc when lpis are discarded
- Add function its_free_msi_descs() to free msi_desc
v6: - Moved this patch #19 to patch #8
- Used col_map to store collection id
- Use helper functions to update msi_desc members
v5: - Removed its_detach_device API
- Pass nr_ites as parameter to its_add_device
v4: - Introduced helper to populate its_device struct
- Fixed freeing of its_device memory
- its_device struct holds domain id
---
xen/arch/arm/gic-v3-its.c | 261 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 6 +
2 files changed, 267 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 4b7d9ed..bc3b73c 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -145,6 +145,19 @@ static struct its_collection *dev_event_to_col(struct its_device *dev,
return its->collections + dev->event_map.col_map[event];
}
+static struct its_node *its_get_phys_node(struct dt_device_node *dt)
+{
+ struct its_node *its;
+
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ if ( its->dt_node == dt )
+ return its;
+ }
+
+ return NULL;
+}
+
/* RB-tree helpers for its_device */
static struct its_device *its_find_device(u32 devid)
{
@@ -537,8 +550,256 @@ static void its_lpi_free(struct its_device *dev)
}
spin_unlock(&lpi_lock);
+}
+
+static void its_discard_lpis(struct its_device *dev, u32 ids)
+{
+ int i;
+
+ for ( i = 0; i < ids; i++ )
+ its_send_discard(dev, i);
+}
+
+static void its_free_msi_descs(struct its_device *dev, int count)
+{
+ int i, irq;
+ struct irq_desc *desc;
+
+ for ( i = 0; i < count; i++ )
+ {
+ irq = dev->event_map.lpi_base + i;
+ desc = irq_to_desc(irq);
+
+ spin_lock(&desc->lock);
+ irqdesc_set_lpi_event(desc, 0);
+ irqdesc_set_its_device(desc, NULL);
+ xfree(irq_get_msi_desc(desc));
+ irq_set_msi_desc(desc, NULL);
+ spin_unlock(&desc->lock);
+ }
+}
+
+static inline u32 its_get_plpi(struct its_device *dev, u32 event)
+{
+ return dev->event_map.lpi_base + event;
+}
+
+static int its_alloc_device_irq(struct its_device *dev, u32 *hwirq)
+{
+ int idx;
+
+ idx = find_first_zero_bit(dev->event_map.lpi_map, dev->event_map.nr_lpis);
+ if ( idx == dev->event_map.nr_lpis )
+ return -ENOSPC;
+
+ *hwirq = its_get_plpi(dev, idx);
+ set_bit(idx, dev->event_map.lpi_map);
+ return 0;
+}
+
+static void its_free_device(struct its_device *dev)
+{
+ xfree(dev->itt);
+ its_lpi_free(dev);
xfree(dev->event_map.lpi_map);
+ xfree(dev->event_map.col_map);
+ xfree(dev);
+}
+
+static struct its_device *its_alloc_device(u32 devid, u32 nr_ites,
+ struct dt_device_node *dt_its)
+{
+ struct its_device *dev;
+ unsigned long *lpi_map;
+ int lpi_base, sz;
+ u16 *col_map = NULL;
+
+ dev = xzalloc(struct its_device);
+ if ( dev == NULL )
+ return NULL;
+
+ dev->its = its_get_phys_node(dt_its);
+ if (dev->its == NULL)
+ {
+ printk(XENLOG_G_ERR
+ "ITS: Failed to find ITS node for devid 0x%"PRIx32"\n", devid);
+ goto err;
+ }
+
+ sz = nr_ites * dev->its->ite_size;
+ sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
+ dev->itt = xzalloc_bytes(sz);
+ if ( !dev->itt )
+ goto err;
+
+ lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base);
+ if ( !lpi_map )
+ goto lpi_err;
+
+ col_map = xzalloc_bytes(sizeof(*col_map) * nr_ites);
+ if ( !col_map )
+ goto col_err;
+ dev->event_map.lpi_map = lpi_map;
+ dev->event_map.lpi_base = lpi_base;
+ dev->event_map.col_map = col_map;
+ dev->event_map.nr_lpis = nr_ites;
+ dev->device_id = devid;
+
+ return dev;
+
+col_err:
+ its_free_device(dev);
+ return NULL;
+lpi_err:
+ xfree(dev->itt);
+err:
+ xfree(dev);
+
+ return NULL;
+}
+
+/* Device assignment */
+int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its)
+{
+ struct its_device *dev;
+ u32 i, plpi = 0;
+ struct its_collection *col;
+ struct irq_desc *desc;
+ struct msi_desc *msi = NULL;
+ int res = 0;
+
+ spin_lock(&rb_its_dev_lock);
+ dev = its_find_device(devid);
+ if ( dev )
+ {
+ printk(XENLOG_G_ERR "ITS: Device already exists 0x%"PRIx32"\n",
+ dev->device_id);
+ res = -EEXIST;
+ goto err_unlock;
+ }
+
+ dev = its_alloc_device(devid, nr_ites, dt_its);
+ if ( !dev )
+ {
+ res = -ENOMEM;
+ goto err_unlock;
+ }
+
+ if ( its_insert_device(dev) )
+ {
+ its_free_device(dev);
+ printk(XENLOG_G_ERR "ITS: Failed to insert device 0x%"PRIx32"\n", devid);
+ res = -EINVAL;
+ goto err_unlock;
+ }
+ /* Assign device to dom0 by default */
+ dev->domain = hardware_domain;
+ spin_unlock(&rb_its_dev_lock);
+
+ DPRINTK("ITS:Add Device 0x%"PRIx32" lpis %"PRIu32" base 0x%"PRIx32"\n",
+ dev->device_id, dev->event_map.nr_lpis, dev->event_map.lpi_base);
+
+ /* Map device to ITS ITT */
+ its_send_mapd(dev, 1);
+
+ for ( i = 0; i < dev->event_map.nr_lpis; i++ )
+ {
+ msi = xzalloc(struct msi_desc);
+ if ( its_alloc_device_irq(dev, &plpi) || !msi )
+ {
+ /* Discard LPIs and free device on failure to allocate pLPI */
+ its_discard_lpis(dev, i);
+ its_free_msi_descs(dev, i);
+ its_send_mapd(dev, 0);
+
+ spin_lock(&rb_its_dev_lock);
+ its_remove_device(dev);
+ spin_unlock(&rb_its_dev_lock);
+
+ its_free_device(dev);
+
+ printk(XENLOG_G_ERR "ITS: Cannot add device 0x%"PRIx32"\n", devid);
+ res = -ENOSPC;
+ goto err;
+ }
+
+ /*
+ * Each Collection is mapped to one physical CPU and
+ * each pLPI allocated to this device is mapped one collection
+ * in a roundrobin fashion. Hence all pLPIs are distributed
+ * across all processors in the system.
+ */
+ col = &dev->its->collections[(i % nr_cpu_ids)];
+ desc = irq_to_desc(plpi);
+
+ spin_lock(&desc->lock);
+ dev->event_map.col_map[i] = col->col_id;
+ irq_set_msi_desc(desc, msi);
+ irqdesc_set_lpi_event(desc, i);
+ irqdesc_set_its_device(desc, dev);
+ spin_unlock(&desc->lock);
+
+ /* For each pLPI send MAPVI command */
+ its_send_mapvi(dev, plpi, i);
+ }
+
+ return 0;
+
+err_unlock:
+ spin_unlock(&rb_its_dev_lock);
+err:
+ return res;
+}
+
+int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
+{
+ struct its_device *pdev;
+ u32 plpi, i;
+
+ DPRINTK("ITS: Assign request for dev 0x%"PRIx32" to domain %"PRIu16"\n",
+ vdevid, d->domain_id);
+
+ spin_lock(&rb_its_dev_lock);
+ pdev = its_find_device(pdevid);
+ if ( !pdev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ return -ENODEV;
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ /*
+ * TODO: For pass-through following has to be implemented
+ * 1) Allow device to be assigned to other domains (Dom0 -> DomU).
+ * 2) Allow device to be re-assigned to Dom0 (DomU -> Dom0).
+ * Implement separate function to handle this or rework this function.
+ * For now do not allow assigning devices other than Dom0.
+ *
+ * By default all devices are assigned to Dom0.
+ * Device should be with Dom0 before assigning to other domain.
+ */
+ if ( !is_hardware_domain(d) || !is_hardware_domain(pdev->domain) )
+ {
+ printk(XENLOG_ERR
+ "ITS: PCI-Passthrough not supported!! to assign from d%d to d%d",
+ pdev->domain->domain_id, d->domain_id);
+ return -ENXIO;
+ }
+
+ pdev->domain = d;
+ pdev->virt_device_id = vdevid;
+
+ DPRINTK("ITS: Assign pdevid 0x%"PRIx32" lpis %"PRIu32" for dom %"PRIu16"\n",
+ pdevid, pdev->event_map.nr_lpis, d->domain_id);
+
+ for ( i = 0; i < pdev->event_map.nr_lpis; i++ )
+ {
+ plpi = its_get_plpi(pdev, i);
+ /* TODO: Route lpi */
+ }
+
+ return 0;
}
/*
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index c632854..f5fba49 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -267,6 +267,10 @@ struct its_device {
struct event_lpi_map event_map;
/* Physical Device id */
u32 device_id;
+ /* Virtual Device id */
+ u32 virt_device_id;
+ /* Domain assigned */
+ struct domain *domain;
/* RB-tree entry */
struct rb_node node;
};
@@ -277,6 +281,8 @@ struct its_device *irqdesc_get_its_device(struct irq_desc *desc);
void irqdesc_set_its_device(struct irq_desc *desc, struct its_device *dev);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
+int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its);
+int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (7 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-21 14:20 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 10/28] xen/arm: ITS: Implement hw_irq_controller for LPIs vijay.kilari
` (19 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, Zoltan Kiss, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Helper function gic_is_lpi() is used to find
if irq is lpi or not. For GICv2 platforms this function
returns number of irq ids which represents only number of line irqs.
For GICv3 platform irq ids are calculated from nr_lpis if
HW supports LPIs
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
CC: Zoltan Kiss <zoltan.kiss@huawei.com>
---
v7: - Rename gic_info.nr_id_bits as gic_info.nr_irq_ids
- gic_is_lpi() is common for both ARM64/32
- Introduce global variable nr_lpis from patch #17
- Reset nr_lpis to 0 when HW does not support LPIs
- pass nr_lpis as boot args to Xen. Set minimum to 8192
v6: - Added gic_info.nr_id_bits to hold number of SPI/LPIs
supported
- Dropped is_lpi() callback
---
xen/arch/arm/gic-hip04.c | 3 ++-
xen/arch/arm/gic-v2.c | 3 ++-
xen/arch/arm/gic-v3.c | 20 ++++++++++++++++++++
xen/arch/arm/gic.c | 10 ++++++++++
xen/arch/arm/irq.c | 15 +++++++++++++++
xen/include/asm-arm/gic.h | 5 +++++
xen/include/asm-arm/irq.h | 3 +++
7 files changed, 57 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/gic-hip04.c b/xen/arch/arm/gic-hip04.c
index c5ed545..398881b 100644
--- a/xen/arch/arm/gic-hip04.c
+++ b/xen/arch/arm/gic-hip04.c
@@ -301,7 +301,8 @@ static void __init hip04gic_dist_init(void)
/* Only 1020 interrupts are supported */
gicv2_info.nr_lines = min(1020U, nr_lines);
-
+ /* Number of IRQ ids supported */
+ gicv2_info.nr_irq_ids = gicv2_info.nr_lines;
/* Turn on the distributor */
writel_gicd(GICD_CTL_ENABLE, GICD_CTLR);
}
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 596126d..6673f29 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -291,7 +291,8 @@ static void __init gicv2_dist_init(void)
/* Only 1020 interrupts are supported */
gicv2_info.nr_lines = min(1020U, nr_lines);
-
+ /* Number of IRQ ids supported */
+ gicv2_info.nr_irq_ids = nr_lines;
/* Turn on the distributor */
writel_gicd(GICD_CTL_ENABLE, GICD_CTLR);
}
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 5076706..6680bd2 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -529,6 +529,11 @@ static void gicv3_set_irq_properties(struct irq_desc *desc,
spin_unlock(&gicv3.lock);
}
+static int gicv3_dist_supports_lpis(void)
+{
+ return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
+}
+
static void __init gicv3_dist_init(void)
{
uint32_t type;
@@ -578,6 +583,21 @@ static void __init gicv3_dist_init(void)
/* Only 1020 interrupts are supported */
gicv3_info.nr_lines = min(1020U, nr_lines);
+
+ /*
+ * Number of IRQ ids supported.
+ * Here we override HW supported number of LPIs and
+ * limit to to LPIs specified in nr_lpis.
+ */
+ if ( gicv3_dist_supports_lpis() )
+ gicv3_info.nr_irq_ids = nr_lpis + FIRST_GIC_LPI;
+ else
+ {
+ gicv3_info.nr_irq_ids = gicv3_info.nr_lines;
+ /* LPIs are not supported by HW. Reset to 0 */
+ nr_lpis = 0;
+ }
+
}
static int gicv3_enable_redist(void)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 1d94e5e..a0ed7df 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -62,6 +62,16 @@ enum gic_version gic_hw_version(void)
return gic_hw_ops->info->hw_version;
}
+unsigned int gic_nr_irq_ids(void)
+{
+ return gic_hw_ops->info->nr_irq_ids;
+}
+
+bool_t gic_is_lpi(unsigned int irq)
+{
+ return (irq >= FIRST_GIC_LPI && irq < gic_nr_irq_ids());
+}
+
unsigned int gic_number_lines(void)
{
return gic_hw_ops->info->nr_lines;
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index d582f57..3a01f46 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -31,6 +31,18 @@
static unsigned int local_irqs_type[NR_LOCAL_IRQS];
static DEFINE_SPINLOCK(local_irqs_type_lock);
+/* Number of LPIs supported by Xen.
+ *
+ * The LPI identifier starts from 8192. Given that the hardware is
+ * providing the number of identifier bits, supporting LPIs requires at
+ * least 14 bits. This will represent 16384 interrupt ID of which 8192
+ * LPIs. So the minimum of LPIs supported when the hardware supports LPIs
+ * is 8192.
+ */
+#define DEFAULT_NR_LPIS 8192
+unsigned int nr_lpis = DEFAULT_NR_LPIS;
+integer_param("nr_lpis", nr_lpis);
+
/* Describe an IRQ assigned to a guest */
struct irq_guest
{
@@ -115,6 +127,9 @@ void __init init_IRQ(void)
{
int irq;
+ if ( nr_lpis < DEFAULT_NR_LPIS )
+ nr_lpis = DEFAULT_NR_LPIS;
+
spin_lock(&local_irqs_type_lock);
for ( irq = 0; irq < NR_LOCAL_IRQS; irq++ )
local_irqs_type[irq] = DT_IRQ_TYPE_INVALID;
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index e330fe3..ce2279e 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -20,6 +20,7 @@
#define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
#define NR_GIC_SGI 16
+#define FIRST_GIC_LPI 8192
#define MAX_RDIST_COUNT 4
#define GICD_CTLR (0x000)
@@ -299,6 +300,8 @@ struct gic_info {
unsigned int maintenance_irq;
/* Pointer to the device tree node representing the interrupt controller */
const struct dt_device_node *node;
+ /* Number of IRQ IDs supported */
+ uint32_t nr_irq_ids;
};
struct gic_hw_operations {
@@ -359,6 +362,8 @@ void register_gic_ops(const struct gic_hw_operations *ops);
int gic_make_hwdom_dt_node(const struct domain *d,
const struct dt_device_node *node,
void *fdt);
+unsigned int gic_nr_irq_ids(void);
+bool_t gic_is_lpi(unsigned int irq);
#endif /* __ASSEMBLY__ */
#endif
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index bafcd30..9e342d2 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -33,6 +33,9 @@ struct msi_desc {
#define nr_static_irqs NR_ITLINES
#define arch_hwdom_irqs(domid) NR_ITLINES
+/* Number of LPIs supported by Xen */
+extern unsigned int nr_lpis;
+
struct irq_desc;
struct irqaction;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 10/28] xen/arm: ITS: Implement hw_irq_controller for LPIs
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (8 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-21 14:35 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 11/28] xen/arm: ITS: Enable compilation of physical ITS driver vijay.kilari
` (18 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Implement hw_irq_controller callbacks required to
handle LPIs.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Split this patch into two. In this patch implement
only hw_irq_controller callbacks for LPIs.
v6: - Moved this patch #15 in v5 to patch #9
- Introduce inv command
- Moved msi_desc helper functions to separate
"xen/arm: ITS: Introduce msi_desc for LPIs"
- Exported LPI hw_irq_controller structure and removed
helper function to access.
v5: - Fixed review comments
- Exposed gicv3_[host|guest]_irq_end and hook to its
v4: - Implement separate hw_irq_controller for LPIs
- Drop setting LPI affinity
- virq and vid are moved under union
- Introduced inv command handling
- its_device is stored in irq_desc
---
xen/arch/arm/gic-v3-its.c | 119 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 2 +-
xen/include/asm-arm/gic_v3_defs.h | 2 +
3 files changed, 122 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index bc3b73c..0d5c61c 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -444,6 +444,125 @@ static void its_send_discard(struct its_device *dev, u32 event)
its_send_single_command(dev->its, &cmd, col);
}
+static void its_flush_and_invalidate_prop(struct irq_desc *desc, u8 *cfg)
+{
+ struct its_device *its_dev = irqdesc_get_its_device(desc);
+ u32 vid = irqdesc_get_lpi_event(desc);
+
+ ASSERT(vid < its_dev->event_map.nr_lpis);
+
+ /*
+ * Make the above write visible to the redistributors.
+ * And yes, we're flushing exactly: One. Single. Byte.
+ * Humpf...
+ */
+ if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
+ else
+ dsb(ishst);
+
+ its_send_inv(its_dev, vid);
+}
+
+static void its_set_lpi_state(struct irq_desc *desc, int enable)
+{
+ u8 *cfg;
+
+ ASSERT(spin_is_locked(&its_lock));
+
+ cfg = gic_rdists->prop_page + desc->irq - FIRST_GIC_LPI;
+ if ( enable )
+ *cfg |= LPI_PROP_ENABLED;
+ else
+ *cfg &= ~LPI_PROP_ENABLED;
+
+ its_flush_and_invalidate_prop(desc, cfg);
+}
+
+static void its_irq_enable(struct irq_desc *desc)
+{
+ unsigned long flags;
+
+ ASSERT(spin_is_locked(&desc->lock));
+
+ spin_lock_irqsave(&its_lock, flags);
+ clear_bit(_IRQ_DISABLED, &desc->status);
+ dsb(sy);
+ its_set_lpi_state(desc, 1);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
+static void its_irq_disable(struct irq_desc *desc)
+{
+ unsigned long flags;
+
+ ASSERT(spin_is_locked(&desc->lock));
+
+ spin_lock_irqsave(&its_lock, flags);
+ its_set_lpi_state(desc, 0);
+ set_bit(_IRQ_DISABLED, &desc->status);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
+static unsigned int its_irq_startup(struct irq_desc *desc)
+{
+ its_irq_enable(desc);
+
+ return 0;
+}
+
+static void its_irq_shutdown(struct irq_desc *desc)
+{
+ its_irq_disable(desc);
+}
+
+static void its_irq_ack(struct irq_desc *desc)
+{
+ /* No ACK -- reading IAR has done this for us */
+}
+
+static void its_host_irq_end(struct irq_desc *desc)
+{
+ /* Lower the priority */
+ gicv3_eoi_irq(desc);
+ /* LPIs does not have active state. Do not deactivate */
+}
+
+static void its_guest_irq_end(struct irq_desc *desc)
+{
+ gicv3_eoi_irq(desc);
+}
+
+static void its_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+{
+ /*TODO: Yet to support */
+ printk(XENLOG_G_WARNING "ITS: Setting Affinity of LPI is not supported\n");
+
+ return;
+}
+
+const hw_irq_controller its_host_lpi_type = {
+ .typename = "gic-its",
+ .startup = its_irq_startup,
+ .shutdown = its_irq_shutdown,
+ .enable = its_irq_enable,
+ .disable = its_irq_disable,
+ .ack = its_irq_ack,
+ .end = its_host_irq_end,
+ .set_affinity = its_irq_set_affinity,
+};
+
+const hw_irq_controller its_guest_lpi_type = {
+ .typename = "gic-its",
+ .startup = its_irq_startup,
+ .shutdown = its_irq_shutdown,
+ .enable = its_irq_enable,
+ .disable = its_irq_disable,
+ .ack = its_irq_ack,
+ .end = its_guest_irq_end,
+ .set_affinity = its_irq_set_affinity,
+};
+
/*
* How we allocate LPIs:
*
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 6680bd2..4f0b28b 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -440,7 +440,7 @@ static void gicv3_mask_irq(struct irq_desc *irqd)
gicv3_poke_irq(irqd, GICD_ICENABLER);
}
-static void gicv3_eoi_irq(struct irq_desc *irqd)
+void gicv3_eoi_irq(struct irq_desc *irqd)
{
/* Lower the priority */
WRITE_SYSREG32(irqd->irq, ICC_EOIR1_EL1);
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 2c322da..1bc88f6 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -199,6 +199,8 @@ struct rdist_prop {
DECLARE_PER_CPU(struct rdist, rdist);
+void gicv3_eoi_irq(struct irq_desc *irqd);
+
#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 11/28] xen/arm: ITS: Enable compilation of physical ITS driver
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (9 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 10/28] xen/arm: ITS: Implement hw_irq_controller for LPIs vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-18 13:08 ` [PATCH v7 12/28] xen/arm: ITS: Plumbing hw_irq_controller for LPIs vijay.kilari
` (17 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Enable compilation of pITS driver.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.c
---
xen/arch/arm/Makefile | 1 +
1 file changed, 1 insertion(+)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 1ef39f7..4708716 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -14,6 +14,7 @@ obj-y += domain_build.o
obj-y += gic.o gic-v2.o
obj-$(CONFIG_ARM_32) += gic-hip04.o
obj-$(HAS_GICV3) += gic-v3.o
+obj-$(HAS_GICV3) += gic-v3-its.o
obj-y += io.o
obj-y += irq.o
obj-y += kernel.o
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 12/28] xen/arm: ITS: Plumbing hw_irq_controller for LPIs
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (10 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 11/28] xen/arm: ITS: Enable compilation of physical ITS driver vijay.kilari
@ 2015-09-18 13:08 ` vijay.kilari
2015-09-21 14:44 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 13/28] xen/arm: Move vgic rank locking inside get_irq_priority vijay.kilari
` (16 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:08 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, Zoltan Kiss, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Change callbacks gic_host_irq_type and gic_guest_irq_type
to gic_get_host_irq_type and gic_get_guest_irq_type
in gic_hw_operations, which returns hw_irq_controller based
on irq type (SPI or LPI).
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
CC: Zoltan Kiss <zoltan.kiss@huawei.com>
---
xen/arch/arm/gic-hip04.c | 14 ++++++++++++--
xen/arch/arm/gic-v2.c | 14 ++++++++++++--
xen/arch/arm/gic-v3.c | 21 +++++++++++++++++++--
xen/arch/arm/gic.c | 14 ++++++++++++--
xen/include/asm-arm/gic-its.h | 2 ++
xen/include/asm-arm/gic.h | 4 ++--
6 files changed, 59 insertions(+), 10 deletions(-)
diff --git a/xen/arch/arm/gic-hip04.c b/xen/arch/arm/gic-hip04.c
index 398881b..f0a4c0b 100644
--- a/xen/arch/arm/gic-hip04.c
+++ b/xen/arch/arm/gic-hip04.c
@@ -630,6 +630,16 @@ static hw_irq_controller hip04gic_guest_irq_type = {
.set_affinity = hip04gic_irq_set_affinity,
};
+static hw_irq_controller *hip04gic_get_host_irq_type(unsigned int irq)
+{
+ return &hip04gic_host_irq_type;
+}
+
+static hw_irq_controller *hip04gic_get_guest_irq_type(unsigned int irq)
+{
+ return &hip04gic_guest_irq_type;
+}
+
static int __init hip04gic_init(void)
{
int res;
@@ -712,8 +722,8 @@ const static struct gic_hw_operations hip04gic_ops = {
.save_state = hip04gic_save_state,
.restore_state = hip04gic_restore_state,
.dump_state = hip04gic_dump_state,
- .gic_host_irq_type = &hip04gic_host_irq_type,
- .gic_guest_irq_type = &hip04gic_guest_irq_type,
+ .gic_get_host_irq_type = hip04gic_get_host_irq_type,
+ .gic_get_guest_irq_type = hip04gic_get_guest_irq_type,
.eoi_irq = hip04gic_eoi_irq,
.deactivate_irq = hip04gic_dir_irq,
.read_irq = hip04gic_read_irq,
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 6673f29..a214f3b 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -616,6 +616,16 @@ static hw_irq_controller gicv2_guest_irq_type = {
.set_affinity = gicv2_irq_set_affinity,
};
+static hw_irq_controller *gicv2_get_host_irq_type(unsigned int irq)
+{
+ return &gicv2_host_irq_type;
+}
+
+static hw_irq_controller *gicv2_get_guest_irq_type(unsigned int irq)
+{
+ return &gicv2_guest_irq_type;
+}
+
static int __init gicv2_init(void)
{
int res;
@@ -698,8 +708,8 @@ const static struct gic_hw_operations gicv2_ops = {
.save_state = gicv2_save_state,
.restore_state = gicv2_restore_state,
.dump_state = gicv2_dump_state,
- .gic_host_irq_type = &gicv2_host_irq_type,
- .gic_guest_irq_type = &gicv2_guest_irq_type,
+ .gic_get_host_irq_type = gicv2_get_host_irq_type,
+ .gic_get_guest_irq_type = gicv2_get_guest_irq_type,
.eoi_irq = gicv2_eoi_irq,
.deactivate_irq = gicv2_dir_irq,
.read_irq = gicv2_read_irq,
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 4f0b28b..c4c77a7 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -40,6 +40,7 @@
#include <asm/device.h>
#include <asm/gic.h>
#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
#include <asm/cpufeature.h>
/* Global state */
@@ -1156,6 +1157,22 @@ static const hw_irq_controller gicv3_guest_irq_type = {
.set_affinity = gicv3_irq_set_affinity,
};
+static hw_irq_controller *gicv3_get_host_irq_type(unsigned int irq)
+{
+ if ( gic_is_lpi(irq) )
+ return &its_host_lpi_type;
+
+ return &gicv3_host_irq_type;
+}
+
+static hw_irq_controller *gicv3_get_guest_irq_type(unsigned int irq)
+{
+ if ( gic_is_lpi(irq) )
+ return &its_guest_lpi_type;
+
+ return &gicv3_guest_irq_type;
+}
+
static int __init cmp_rdist(const void *a, const void *b)
{
const struct rdist_region *l = a, *r = a;
@@ -1309,8 +1326,8 @@ static const struct gic_hw_operations gicv3_ops = {
.save_state = gicv3_save_state,
.restore_state = gicv3_restore_state,
.dump_state = gicv3_dump_state,
- .gic_host_irq_type = &gicv3_host_irq_type,
- .gic_guest_irq_type = &gicv3_guest_irq_type,
+ .gic_get_host_irq_type = gicv3_get_host_irq_type,
+ .gic_get_guest_irq_type = gicv3_get_guest_irq_type,
.eoi_irq = gicv3_eoi_irq,
.deactivate_irq = gicv3_dir_irq,
.read_irq = gicv3_read_irq,
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index a0ed7df..cefd8fc 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -104,6 +104,16 @@ void gic_restore_state(struct vcpu *v)
gic_restore_pending_irqs(v);
}
+static inline hw_irq_controller *get_host_hw_irq_controller(unsigned int irq)
+{
+ return gic_hw_ops->gic_get_host_irq_type(irq);
+}
+
+static inline hw_irq_controller *get_guest_hw_irq_controller(unsigned int irq)
+{
+ return gic_hw_ops->gic_get_guest_irq_type(irq);
+}
+
/*
* needs to be called with a valid cpu_mask, ie each cpu in the mask has
* already called gic_cpu_init
@@ -128,7 +138,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
- desc->handler = gic_hw_ops->gic_host_irq_type;
+ desc->handler = get_host_hw_irq_controller(desc->irq);
gic_set_irq_properties(desc, cpu_mask, priority);
}
@@ -159,7 +169,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
goto out;
- desc->handler = gic_hw_ops->gic_guest_irq_type;
+ desc->handler = get_guest_hw_irq_controller(desc->irq);
set_bit(_IRQ_GUEST, &desc->status);
gic_set_irq_properties(desc, cpumask_of(v_target->processor), priority);
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index f5fba49..4455178 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -275,6 +275,8 @@ struct its_device {
struct rb_node node;
};
+extern const hw_irq_controller its_host_lpi_type;
+extern const hw_irq_controller its_guest_lpi_type;
void irqdesc_set_lpi_event(struct irq_desc *desc, unsigned id);
unsigned int irqdesc_get_lpi_event(struct irq_desc *desc);
struct its_device *irqdesc_get_its_device(struct irq_desc *desc);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index ce2279e..2073759 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -317,10 +317,10 @@ struct gic_hw_operations {
void (*dump_state)(const struct vcpu *);
/* hw_irq_controller to enable/disable/eoi host irq */
- hw_irq_controller *gic_host_irq_type;
+ hw_irq_controller *(*gic_get_host_irq_type)(unsigned int irq);
/* hw_irq_controller to enable/disable/eoi guest irq */
- hw_irq_controller *gic_guest_irq_type;
+ hw_irq_controller *(*gic_get_guest_irq_type)(unsigned int irq);
/* End of Interrupt */
void (*eoi_irq)(struct irq_desc *irqd);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 13/28] xen/arm: Move vgic rank locking inside get_irq_priority
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (11 preceding siblings ...)
2015-09-18 13:08 ` [PATCH v7 12/28] xen/arm: ITS: Plumbing hw_irq_controller for LPIs vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-21 14:49 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support vijay.kilari
` (15 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Move vgic rank locking inside get_irq_priority callback.
LPIs does not have vgic rank lock for reading LPI priority.
So make generic vgic code cleaner.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Updated commit message
---
xen/arch/arm/vgic-v2.c | 4 +++-
xen/arch/arm/vgic-v3.c | 4 +++-
xen/arch/arm/vgic.c | 3 ---
3 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index 524787b..600d27e 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -519,11 +519,13 @@ static struct vcpu *vgic_v2_get_target_vcpu(struct vcpu *v, unsigned int irq)
static int vgic_v2_get_irq_priority(struct vcpu *v, unsigned int irq)
{
int priority;
+ unsigned long flags;
struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
- ASSERT(spin_is_locked(&rank->lock));
+ vgic_lock_rank(v, rank, flags);
priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
irq, DABT_WORD)], 0, irq & 0x3);
+ vgic_unlock_rank(v, rank, flags);
return priority;
}
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 598f634..12c5d87 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1096,11 +1096,13 @@ static const struct mmio_handler_ops vgic_distr_mmio_handler = {
static int vgic_v3_get_irq_priority(struct vcpu *v, unsigned int irq)
{
int priority;
+ unsigned long flags;
struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
- ASSERT(spin_is_locked(&rank->lock));
+ vgic_lock_rank(v, rank, flags);
priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
irq, DABT_WORD)], 0, irq & 0x3);
+ vgic_unlock_rank(v, rank, flags);
return priority;
}
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 80a8f4e..fb2a205 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -407,14 +407,11 @@ void vgic_clear_pending_irqs(struct vcpu *v)
void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
{
uint8_t priority;
- struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
struct pending_irq *iter, *n = irq_to_pending(v, virq);
unsigned long flags;
bool_t running;
- vgic_lock_rank(v, rank, flags);
priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
- vgic_unlock_rank(v, rank, flags);
spin_lock_irqsave(&v->arch.vgic.lock, flags);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (12 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 13/28] xen/arm: Move vgic rank locking inside get_irq_priority vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-21 15:20 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 15/28] xen/arm: ITS: Add virtual ITS driver vijay.kilari
` (14 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Initialize physical ITS if HW supports LPIs.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Export lpi support information to vgic-v3 driver from gic-v3.
- Drop gic_lpi_supported() helper function
- Add boot param to enable or disable physical ITS
v6: - Updated lpi_supported gic_info member for GICv2 and GICv3
- Introduced helper gic_lpi_supported() and exported
v5: - Made check of its dt node availability before
setting lpi_supported flag
---
xen/arch/arm/gic-v3.c | 38 ++++++++++++++++++++++++++++++++++---
xen/arch/arm/vgic-v3.c | 5 ++++-
xen/include/asm-arm/gic_v3_defs.h | 4 +++-
xen/include/asm-arm/vgic.h | 2 +-
4 files changed, 43 insertions(+), 6 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index c4c77a7..ac8a0ea 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -55,6 +55,18 @@ static struct {
} gicv3;
static struct gic_info gicv3_info;
+/* Enable/Disable ITS support */
+static bool_t its_enable = 1;
+/* Availability of ITS support after successful ITS initialization */
+static bool_t its_enabled = 0;
+
+static void __init parse_its_param(char *s)
+{
+ if ( !parse_bool(s) )
+ its_enable = 0;
+}
+
+custom_param("its", parse_its_param);
/* per-cpu re-distributor base */
DEFINE_PER_CPU(struct rdist, rdist);
@@ -590,7 +602,7 @@ static void __init gicv3_dist_init(void)
* Here we override HW supported number of LPIs and
* limit to to LPIs specified in nr_lpis.
*/
- if ( gicv3_dist_supports_lpis() )
+ if ( its_enabled && gicv3_dist_supports_lpis() )
gicv3_info.nr_irq_ids = nr_lpis + FIRST_GIC_LPI;
else
{
@@ -714,6 +726,10 @@ static int __cpuinit gicv3_cpu_init(void)
if ( gicv3_enable_redist() )
return -ENODEV;
+ /* Give LPIs a spin */
+ if ( its_enabled && gicv3_dist_supports_lpis() )
+ its_cpu_init();
+
/* Set priority on PPI and SGI interrupts */
priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
GIC_PRI_IPI);
@@ -1303,14 +1319,30 @@ static int __init gicv3_init(void)
i, r->base, r->base + r->size);
}
- vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions,
- gicv3.rdist_stride);
+ reg = readl_relaxed(GICD + GICD_TYPER);
+
+ gicv3.rdist_data.id_bits = ((reg >> GICD_TYPE_ID_BITS_SHIFT) &
+ GICD_TYPE_ID_BITS_MASK) + 1;
+
gicv3_init_v2(node, dbase);
spin_lock_init(&gicv3.lock);
spin_lock(&gicv3.lock);
+ if ( its_enable && gicv3_dist_supports_lpis() )
+ {
+ /*
+ * LPI support is enabled only if HW supports it and
+ * ITS dt node is available
+ */
+ if ( !its_init(&gicv3.rdist_data) )
+ its_enabled = 1;
+ }
+
+ vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions,
+ gicv3.rdist_stride, its_enabled);
+
gicv3_dist_init();
res = gicv3_cpu_init();
gicv3_hyp_init();
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 12c5d87..52d4277 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -51,6 +51,8 @@
static struct {
bool_t enabled;
+ /* Check if its supported */
+ bool_t lpi_support;
/* Distributor interface address */
paddr_t dbase;
/* Re-distributor regions */
@@ -62,9 +64,10 @@ static struct {
void vgic_v3_setup_hw(paddr_t dbase,
unsigned int nr_rdist_regions,
const struct rdist_region *regions,
- uint32_t rdist_stride)
+ uint32_t rdist_stride, bool_t lpi_support)
{
vgic_v3_hw.enabled = 1;
+ vgic_v3_hw.lpi_support = lpi_support;
vgic_v3_hw.dbase = dbase;
vgic_v3_hw.nr_rdist_regions = nr_rdist_regions;
vgic_v3_hw.regions = regions;
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 1bc88f6..f819589 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -46,7 +46,9 @@
#define GICC_SRE_EL2_ENEL1 (1UL << 3)
/* Additional bits in GICD_TYPER defined by GICv3 */
-#define GICD_TYPE_ID_BITS_SHIFT 19
+#define GICD_TYPE_ID_BITS_SHIFT (19)
+#define GICD_TYPE_ID_BITS_MASK (0x1f)
+#define GICD_TYPE_LPIS (0x1UL << 17)
#define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
#define GICD_CTLR_RWP (1UL << 31)
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index cf5a790..2bf061f 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -362,7 +362,7 @@ struct rdist_region;
void vgic_v3_setup_hw(paddr_t dbase,
unsigned int nr_rdist_regions,
const struct rdist_region *regions,
- uint32_t rdist_stride);
+ uint32_t rdist_stride, bool_t lpi_support);
#endif
#endif /* __ASM_ARM_VGIC_H__ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 15/28] xen/arm: ITS: Add virtual ITS driver
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (13 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-21 15:34 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 16/28] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
` (13 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
This patch introduces virtual ITS driver with following
functionality
- Introduces helper functions to manage device table and
ITT table in guest memory
- Helper function to handle virtual ITS devices assigned
to domain
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Moved vits_access_guest_table() from vgic-v3-its.c to vgic.c
and renamed as vgic_access_guest_memory().
- Renamed entry paramter in vgic_access_guest_memory() to gipa
- Introduced new header file vits.h for vITS and moved vits structures
from gic-its.h to vits.h
v6: - Exported vits_access_guest_memory() api
v5: - Removed RB tree that manages vitual ITS devices
v4: - Rename functions {find,remove,insert}_vits_* to
vits_{find,remove,insert}.
- Add common helper function to map and read/write dt
or vitt table entry.
- Removed unused code
---
xen/arch/arm/vgic-v3-its.c | 123 ++++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/vgic.c | 39 ++++++++++++++
xen/include/asm-arm/domain.h | 2 +
xen/include/asm-arm/vits.h | 68 +++++++++++++++++++++++
4 files changed, 232 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
new file mode 100644
index 0000000..df54ce5
--- /dev/null
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -0,0 +1,123 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/bitops.h>
+#include <xen/config.h>
+#include <xen/init.h>
+#include <xen/irq.h>
+#include <xen/list.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+#include <xen/domain_page.h>
+#include <asm/device.h>
+#include <asm/mmio.h>
+#include <asm/io.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/vgic.h>
+#include <asm/gic-its.h>
+#include <asm/vits.h>
+#include <xen/log2.h>
+
+/* ITS device table helper functions */
+static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
+ struct vdevice_table *entry, bool_t set)
+{
+ uint64_t offset;
+ paddr_t dt_entry;
+ const struct vgic_its *vits = d->arch.vgic.vits;
+
+ BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
+
+ offset = dev_id * sizeof(struct vdevice_table);
+ if ( offset > vits->dt_size )
+ {
+ printk(XENLOG_G_ERR
+ "d%d: vITS: Out of range off 0x%"PRIx64" id 0x%"PRIx32"\n",
+ d->domain_id, offset, dev_id);
+ return -EINVAL;
+ }
+
+ dt_entry = vits->dt_ipa + offset;
+
+ return vgic_access_guest_memory(d, dt_entry, entry,
+ sizeof(struct vdevice_table), set);
+}
+
+static int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 1);
+}
+
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 0);
+}
+
+static int vits_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry, bool_t set)
+{
+ struct vdevice_table dt_entry;
+ paddr_t vitt_entry;
+ uint64_t offset;
+
+ BUILD_BUG_ON(sizeof(struct vitt) != 8);
+
+ if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
+ {
+ printk(XENLOG_G_ERR
+ "d%d: vITS: Fail to get vdevice for vdevid 0x%"PRIx32"\n",
+ d->domain_id, devid);
+ return -EINVAL;
+ }
+
+ /* dt_entry has been validated in vits_get_vdevice_entry */
+ offset = event * sizeof(struct vitt);
+ if ( offset > dt_entry.vitt_size )
+ {
+ printk(XENLOG_G_ERR "d%d: vITS: ITT out of range\n", d->domain_id);
+ return -EINVAL;
+ }
+
+ vitt_entry = dt_entry.vitt_ipa + offset;
+
+ return vgic_access_guest_memory(d, vitt_entry, entry,
+ sizeof(struct vitt), set);
+}
+
+static int vits_set_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 1);
+}
+
+int vits_get_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 0);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index fb2a205..baa74c7 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -25,6 +25,7 @@
#include <xen/irq.h>
#include <xen/sched.h>
#include <xen/perfc.h>
+#include <xen/domain_page.h>
#include <asm/current.h>
@@ -527,6 +528,44 @@ void vgic_free_virq(struct domain *d, unsigned int virq)
clear_bit(virq, d->arch.vgic.allocated_irqs);
}
+int vgic_access_guest_memory(struct domain *d, paddr_t gipa, void *addr,
+ uint32_t size, bool_t set)
+{
+ struct page_info *page;
+ uint64_t offset;
+ p2m_type_t p2mt;
+ void *p;
+
+ page = get_page_from_gfn(d, paddr_to_pfn(gipa), &p2mt, P2M_ALLOC);
+ if ( !page )
+ {
+ printk(XENLOG_G_ERR "d%d: vITS: Failed to get table entry\n",
+ d->domain_id);
+ return -EINVAL;
+ }
+
+ if ( !p2m_is_ram(p2mt) )
+ {
+ put_page(page);
+ printk(XENLOG_G_ERR "d%d: vITS: with wrong attributes\n", d->domain_id);
+ return -EINVAL;
+ }
+
+ p = __map_domain_page(page);
+ /* Offset within the mapped page */
+ offset = gipa & ~PAGE_MASK;
+
+ if ( set )
+ memcpy(p + offset, addr, size);
+ else
+ memcpy(addr, p + offset, size);
+
+ unmap_domain_page(p);
+ put_page(page);
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 7ddaeaa..0f2aa66 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -112,6 +112,8 @@ struct arch_domain
} rdist_regions[MAX_RDIST_COUNT];
int nr_regions; /* Number of rdist regions */
uint32_t rdist_stride; /* Re-Distributor stride */
+ /* Virtual ITS */
+ struct vgic_its *vits;
#endif
} vgic;
diff --git a/xen/include/asm-arm/vits.h b/xen/include/asm-arm/vits.h
new file mode 100644
index 0000000..6cb5d01
--- /dev/null
+++ b/xen/include/asm-arm/vits.h
@@ -0,0 +1,68 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_ARM_VITS_H__
+#define __ASM_ARM_VITS_H__
+
+/*
+ * Per domain virtual ITS structure.
+ */
+struct vgic_its
+{
+ /* vITT device table ipa */
+ paddr_t dt_ipa;
+ /* vITT device table size */
+ uint64_t dt_size;
+};
+
+/*
+ * struct vdevice_table and struct vitt are typically stored in memory
+ * which has been provided by the guest out of its own address space
+ * and which remains accessible to the guest.
+ *
+ * Therefore great care _must_ be taken when accessing an entry in
+ * either table to validate the sanity of any values which are used.
+ */
+struct vdevice_table {
+ uint64_t vitt_ipa;
+ uint32_t vitt_size;
+ uint32_t padding;
+};
+
+struct vitt {
+ uint16_t valid:1;
+ uint16_t pad:15;
+ uint16_t vcollection;
+ uint32_t vlpi;
+};
+
+int vgic_access_guest_memory(struct domain *d, paddr_t gipa, void *addr,
+ uint32_t size, bool_t set);
+int vits_get_vitt_entry(struct domain *d, uint32_t devid, uint32_t event,
+ struct vitt *entry);
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry);
+
+#endif /* __ASM_ARM_VITS_H__ */
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 16/28] xen/arm: ITS: Add virtual ITS commands support
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (14 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 15/28] xen/arm: ITS: Add virtual ITS driver vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-22 13:47 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation vijay.kilari
` (12 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add Virtual ITS command processing support to Virtual ITS driver
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Moved is_valid_collection() declaration to vits.h
- Added check for number of vcpus of a domain in
vits_domain_init()
- Moved changes in gic-its.h to vits.h
- vits_process_cmd() is made static
- Fixed coding styles
v6: - Updated printk to use correct PRI*
- Moved vits_get_max_collection and is_valid_collection
helper to this patch
- Added vits_domain_free()
- Few more review comments
v5: - Rename vgic_its_*() to vits_*()
v4: - Use helper function to read from command queue
- Add MOVALL
- Removed check for entry in device in domain RB-tree
---
xen/arch/arm/vgic-v3-its.c | 430 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 2 +
xen/include/asm-arm/vits.h | 14 ++
3 files changed, 446 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index df54ce5..dba0b9e 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -26,6 +26,7 @@
#include <asm/device.h>
#include <asm/mmio.h>
#include <asm/io.h>
+#include <asm/atomic.h>
#include <asm/gic_v3_defs.h>
#include <asm/gic.h>
#include <asm/vgic.h>
@@ -33,6 +34,47 @@
#include <asm/vits.h>
#include <xen/log2.h>
+//#define DEBUG_ITS
+
+#ifdef DEBUG_ITS
+# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#ifdef DEBUG_ITS
+static void dump_cmd(const its_cmd_block *cmd)
+{
+ printk("VITS:CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
+}
+#else
+static void dump_cmd(const its_cmd_block *cmd) { }
+#endif
+
+static inline uint16_t vits_get_max_collections(struct domain *d)
+{
+ /*
+ * ITS only supports up to 256 collections without
+ * provisioning external memory. As per vITS design, number of
+ * vCPUS should not exceed max number of collections.
+ */
+ ASSERT(d->max_vcpus < 256);
+
+ /* Each collection corresponds to one CPU(vCPU). collections are
+ * used to move interrupts from one CPU to another. The ITS
+ * mandated to implement N + 1 collections where N is the number of
+ * processor on the platform (i.e max number of VCPUs for a given
+ * guest).
+ */
+ return (d->max_vcpus + 1);
+}
+
+bool_t is_valid_collection(struct domain *d, uint16_t col)
+{
+ return (col <= vits_get_max_collections(d));
+}
+
/* ITS device table helper functions */
static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
struct vdevice_table *entry, bool_t set)
@@ -113,6 +155,394 @@ int vits_get_vitt_entry(struct domain *d, uint32_t devid,
return vits_vitt_entry(d, devid, event, entry, 0);
}
+static int vits_process_sync(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: SYNC: ta 0x%"PRIx32" \n", v, virt_cmd->sync.ta);
+
+ return 0;
+}
+
+static int vits_process_mapvi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint16_t vcol_id;
+ uint8_t cmd;
+ uint32_t vid, dev_id, event;
+
+ vcol_id = virt_cmd->mapvi.col;
+ vid = virt_cmd->mapvi.phy_id;
+ cmd = virt_cmd->mapvi.cmd;
+ dev_id = virt_cmd->mapvi.devid;
+
+ DPRINTK("%pv: vITS: MAPVI: dev 0x%"PRIx32" vcol %"PRIu16" vid %"PRIu32"\n",
+ v, dev_id, vcol_id, vid);
+
+ entry.valid = true;
+ entry.vcollection = vcol_id;
+ entry.vlpi = vid;
+
+ if ( cmd == GITS_CMD_MAPI )
+ vits_set_vitt_entry(d, dev_id, vid, &entry);
+ else
+ {
+ event = virt_cmd->mapvi.event;
+ vits_set_vitt_entry(d, dev_id, event, &entry);
+ }
+
+ return 0;
+}
+
+static int vits_process_movi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint32_t dev_id, event;
+ uint16_t vcol_id;
+
+ vcol_id = virt_cmd->movi.col;
+ event = virt_cmd->movi.event;
+ dev_id = virt_cmd->movi.devid;
+
+ DPRINTK("%pv vITS: MOVI: dev_id 0x%"PRIx32" vcol %"PRIu16" event %"PRIu32"\n",
+ v, dev_id, vcol_id, event);
+
+ if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ entry.vcollection = vcol_id;
+
+ if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vits_process_movall(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: MOVALL: ta1 0x%"PRIx32" ta2 0x%"PRIx32" \n",
+ v, virt_cmd->movall.ta1, virt_cmd->movall.ta2);
+
+ return 0;
+}
+
+static int vits_process_discard(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint32_t event, dev_id;
+
+ event = virt_cmd->discard.event;
+ dev_id = virt_cmd->discard.devid;
+
+ DPRINTK("%pv vITS: DISCARD: dev_id 0x%"PRIx32" id %"PRIu32"\n",
+ v, virt_cmd->discard.devid, event);
+
+ if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ entry.valid = false;
+
+ if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vits_process_inv(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv vITS: INV: dev_id 0x%"PRIx32" id %"PRIu32"\n",
+ v, virt_cmd->inv.devid, virt_cmd->inv.event);
+
+ return 0;
+}
+
+static int vits_process_clear(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: CLEAR: dev_id 0x%"PRIx32" id %"PRIu32"\n",
+ v, virt_cmd->clear.devid, virt_cmd->clear.event);
+
+ return 0;
+}
+
+static int vits_process_invall(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: INVALL: vCID %"PRIu16"\n", v, virt_cmd->invall.col);
+
+ return 0;
+}
+
+static int vits_process_int(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint32_t event, dev_id;
+
+ event = virt_cmd->int_cmd.cmd;
+ dev_id = virt_cmd->int_cmd.devid;
+
+ DPRINTK("%pv: vITS: INT: Device 0x%"PRIx32" id %"PRIu32"\n",
+ v, dev_id, event);
+
+ /* TODO: Inject LPI */
+
+ return 0;
+}
+
+static int vits_add_device(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct domain *d = v->domain;
+ struct vdevice_table dt_entry;
+ uint32_t dev_id = virt_cmd->mapd.devid;
+
+ DPRINTK("%pv: vITS:Add dev 0x%"PRIx32" ipa = 0x%"PRIx64" size %"PRIu32"\n",
+ v, dev_id, (u64)virt_cmd->mapd.itt << MAPC_ITT_IPA_SHIFT,
+ virt_cmd->mapd.size);
+
+ if ( virt_cmd->mapd.valid )
+ {
+ /* itt field is 40 bit. extract 48 bit address by shifting */
+ dt_entry.vitt_ipa = virt_cmd->mapd.itt << MAPC_ITT_IPA_SHIFT;
+ dt_entry.vitt_size = (1 << (virt_cmd->mapd.size + 1)) *
+ sizeof(struct vitt);
+ }
+ else
+ {
+ dt_entry.vitt_ipa = INVALID_PADDR;
+ dt_entry.vitt_size = 0;
+ }
+
+ if ( vits_set_vdevice_entry(d, dev_id, &dt_entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vits_process_mapc(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint16_t vcol_id = virt_cmd->mapc.col;
+ uint64_t vta = virt_cmd->mapc.ta;
+
+ DPRINTK("%pv: vITS: MAPC: vCID %"PRIu16" vTA 0x%"PRIx64" valid %"PRIu8"\n",
+ v, vcol_id, vta, virt_cmd->mapc.valid);
+
+ if ( !is_valid_collection(v->domain, vcol_id) )
+ return -EINVAL;
+
+ if ( virt_cmd->mapc.valid )
+ {
+ if ( vta > v->domain->max_vcpus )
+ return -EINVAL;
+ vits->collections[vcol_id].target_address = vta;
+ }
+ else
+ vits->collections[vcol_id].target_address = INVALID_PADDR;
+
+ return 0;
+}
+
+#ifdef DEBUG_ITS
+char *cmd_str[] = {
+ [GITS_CMD_MOVI] = "MOVI",
+ [GITS_CMD_INT] = "INT",
+ [GITS_CMD_CLEAR] = "CLEAR",
+ [GITS_CMD_SYNC] = "SYNC",
+ [GITS_CMD_MAPD] = "MAPD",
+ [GITS_CMD_MAPC] = "MAPC",
+ [GITS_CMD_MAPVI] = "MAPVI",
+ [GITS_CMD_MAPI] = "MAPI",
+ [GITS_CMD_INV] = "INV",
+ [GITS_CMD_INVALL] = "INVALL",
+ [GITS_CMD_MOVALL] = "MOVALL",
+ [GITS_CMD_DISCARD] = "DISCARD",
+ };
+#endif
+
+static int vits_parse_its_command(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint8_t cmd = virt_cmd->hdr.cmd;
+ int ret;
+
+ DPRINTK("%pv: vITS: Received cmd %s (0x%"PRIx8")\n", v, cmd_str[cmd], cmd);
+ dump_cmd(virt_cmd);
+
+ switch ( cmd )
+ {
+ case GITS_CMD_MAPD:
+ ret = vits_add_device(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPC:
+ ret = vits_process_mapc(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPI:
+ /* MAPI is same as MAPVI */
+ case GITS_CMD_MAPVI:
+ ret = vits_process_mapvi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MOVI:
+ ret = vits_process_movi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MOVALL:
+ ret = vits_process_movall(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_DISCARD:
+ ret = vits_process_discard(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INV:
+ ret = vits_process_inv(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INVALL:
+ ret = vits_process_invall(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INT:
+ ret = vits_process_int(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_CLEAR:
+ ret = vits_process_clear(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_SYNC:
+ ret = vits_process_sync(v, vits, virt_cmd);
+ break;
+ default:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Unhandled command cmd %"PRIu8"\n",
+ v, cmd);
+ return 1;
+ }
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to handle cmd %"PRIu8"\n",
+ v, cmd);
+ return 1;
+ }
+
+ return 0;
+}
+
+static int vits_read_virt_cmd(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ paddr_t maddr;
+ struct domain *d = v->domain;
+ int ret;
+
+ ASSERT(spin_is_locked(&vits->lock));
+
+ if ( !(vits->cmd_base & GITS_CBASER_VALID) )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Invalid CBASER\n", v);
+ return 0;
+ }
+
+ /* Get command queue offset */
+ maddr = (vits->cmd_base & GITS_CBASER_PA_MASK) +
+ atomic_read(&vits->cmd_read);
+
+ DPRINTK("%pv: vITS: Mapping CMD Q maddr 0x%"PRIx64" read 0x%"PRIx32"\n",
+ v, maddr, atomic_read(&vits->cmd_read));
+
+ ret = vgic_access_guest_memory(d, maddr, (void *)virt_cmd,
+ sizeof(its_cmd_block), 0);
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: Failed to get command page @page 0x%"PRIx32"\n",
+ v, atomic_read(&vits->cmd_read));
+ return -EINVAL;
+ }
+
+ atomic_add(sizeof(its_cmd_block), &vits->cmd_read);
+ if ( atomic_read(&vits->cmd_read) == vits->cmd_qsize )
+ {
+ DPRINTK("%pv: vITS: Reset read @ 0x%"PRIx32" qsize 0x%"PRIx64"\n",
+ v, atomic_read(&vits->cmd_read), vits->cmd_qsize);
+
+ atomic_set(&vits->cmd_read, 0);
+ }
+
+ return 0;
+}
+
+static int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
+{
+ its_cmd_block virt_cmd;
+
+ ASSERT(spin_is_locked(&vits->lock));
+
+ do {
+ if ( vits_read_virt_cmd(v, vits, &virt_cmd) )
+ goto err;
+ if ( vits_parse_its_command(v, vits, &virt_cmd) )
+ goto err;
+ } while ( vits->cmd_write != atomic_read(&vits->cmd_read) );
+
+ DPRINTK("%pv: vITS: read @ 0x%"PRIx32" write @ 0x%"PRIx64"\n",
+ v, atomic_read(&vits->cmd_read),
+ vits->cmd_write);
+
+ return 1;
+err:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to process guest cmd\n", v);
+ domain_crash_synchronous();
+
+ return 0;
+}
+
+int vits_domain_init(struct domain *d)
+{
+ struct vgic_its *vits;
+ int i;
+
+ if ( d->max_vcpus >= 256 )
+ {
+ printk(XENLOG_G_ERR
+ "vITS: Cannot support guest with >= 256 vCPUs for domaind %d\n",
+ d->domain_id);
+ return -ENXIO;
+ }
+
+ ASSERT(is_hardware_domain(d));
+
+ d->arch.vgic.vits = xzalloc(struct vgic_its);
+ if ( !d->arch.vgic.vits )
+ return -ENOMEM;
+
+ vits = d->arch.vgic.vits;
+
+ spin_lock_init(&vits->lock);
+
+ vits->collections = xzalloc_array(struct its_collection,
+ vits_get_max_collections(d));
+ if ( !vits->collections )
+ return -ENOMEM;
+
+ for ( i = 0; i < vits_get_max_collections(d); i++ )
+ vits->collections[i].target_address = INVALID_PADDR;
+
+ return 0;
+}
+
+void vits_domain_free(struct domain *d)
+{
+ xfree(d->arch.vgic.vits->collections);
+ xfree(d->arch.vgic.vits);
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 4455178..57a652c 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -59,6 +59,7 @@
#define GITS_CBASER_InnerShareable (1UL << 10)
#define GITS_CBASER_SHAREABILITY_MASK (3UL << 10)
#define GITS_CBASER_CACHEABILITY_MASK (7UL << 59)
+#define GITS_CBASER_PA_MASK (0xfffffffff000UL)
#define GITS_BASER_NR_REGS 8
@@ -137,6 +138,7 @@ typedef union {
u64 res5;
} mapd;
struct __packed {
+#define MAPC_ITT_IPA_SHIFT 8
u8 cmd;
u8 res1[7];
u64 res2;
diff --git a/xen/include/asm-arm/vits.h b/xen/include/asm-arm/vits.h
index 6cb5d01..295cce4 100644
--- a/xen/include/asm-arm/vits.h
+++ b/xen/include/asm-arm/vits.h
@@ -23,10 +23,21 @@
*/
struct vgic_its
{
+ spinlock_t lock;
+ /* Command queue base */
+ paddr_t cmd_base;
+ /* Command queue write pointer */
+ paddr_t cmd_write;
+ /* Command queue read pointer */
+ atomic_t cmd_read;
+ /* Command queue size */
+ unsigned long cmd_qsize;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
uint64_t dt_size;
+ /* collections mapped */
+ struct its_collection *collections;
};
/*
@@ -50,6 +61,9 @@ struct vitt {
uint32_t vlpi;
};
+bool_t is_valid_collection(struct domain *d, uint16_t col);
+int vits_domain_init(struct domain *d);
+void vits_domain_free(struct domain *d);
int vgic_access_guest_memory(struct domain *d, paddr_t gipa, void *addr,
uint32_t size, bool_t set);
int vits_get_vitt_entry(struct domain *d, uint32_t devid, uint32_t event,
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (15 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 16/28] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-22 14:30 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS vijay.kilari
` (11 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate GITS* registers
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Fixed wrong usage of vgic_regN* helpers
- coding styles and comments
- GITS_BASER0 is always overwritten with new value every time
it is updated.
v6: - Removed unrelated code of this patch
- Used vgic_regN_{read,write}
v4: - Removed GICR register emulation
---
xen/arch/arm/vgic-v3-its.c | 315 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 8 ++
xen/include/asm-arm/vits.h | 8 ++
3 files changed, 331 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index dba0b9e..0d97fcb 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -36,6 +36,15 @@
//#define DEBUG_ITS
+/* GITS_PIDRn register values for ARM implementations */
+#define GITS_PIDR0_VAL (0x94)
+#define GITS_PIDR1_VAL (0xb4)
+#define GITS_PIDR2_VAL (0x3b)
+#define GITS_PIDR3_VAL (0x00)
+#define GITS_PIDR4_VAL (0x04)
+#define GITS_BASER0_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
+ (sizeof(struct vitt) << \
+ GITS_BASER_ENTRY_SIZE_SHIFT))
#ifdef DEBUG_ITS
# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
#else
@@ -503,6 +512,301 @@ err:
return 0;
}
+static inline void vits_spin_lock(struct vgic_its *vits)
+{
+ spin_lock(&vits->lock);
+}
+
+static inline void vits_spin_unlock(struct vgic_its *vits)
+{
+ spin_unlock(&vits->lock);
+}
+
+static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ uint64_t val;
+ uint32_t gits_reg;
+
+ gits_reg = info->gpa - vits->gits_base;
+ DPRINTK("%pv: vITS: GITS_MMIO_READ offset 0x%"PRIx32"\n", v, gits_reg);
+
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vits_spin_lock(vits);
+ /*
+ * vITS is always quiescent, has no translations in progress and
+ * completed all operations. So set quiescent by default.
+ */
+ *r = vgic_reg32_read(vits->ctrl, info);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_IIDR:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_read(GICV3_GICD_IIDR_VAL, info);
+ return 1;
+ case GITS_TYPER:
+ case GITS_TYPER + 4:
+ /*
+ * GITS_TYPER.HCC = max_vcpus + 1 (max collection supported)
+ * GITS_TYPER.Devbits = HW supported Devbits size
+ * GITS_TYPER.IDbits = HW supported IDbits size
+ * GITS_TYPER.PTA = 0 (Target addresses are linear processor numbers)
+ * GITS_TYPER.ITTSize = Size of struct vitt
+ * GITS_TYPER.Physical = 1
+ */
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ val = ((vits_get_max_collections(v->domain) << GITS_TYPER_HCC_SHIFT ) |
+ ((vits_hw.devID_bits - 1) << GITS_TYPER_DEVBITS_SHIFT) |
+ ((vits_hw.eventID_bits - 1) << GITS_TYPER_IDBITS_SHIFT) |
+ ((sizeof(struct vitt) - 1) << GITS_TYPER_ITT_SIZE_SHIFT) |
+ GITS_TYPER_PHYSICAL_LPIS);
+ *r = vgic_reg64_read(val, info);
+ return 1;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- read ignored */
+ goto read_as_zero;
+ case GITS_CBASER:
+ case GITS_CBASER + 4:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ vits_spin_lock(vits);
+ *r = vgic_reg64_read(vits->cmd_base, info);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ vits_spin_lock(vits);
+ *r = vgic_reg64_read(vits->cmd_write, info);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER + 4:
+ /* Bits[63:20] are RES0 */
+ goto read_as_zero_32;
+ case GITS_CREADR:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ *r = vgic_reg64_read(atomic_read(&vits->cmd_read), info);
+ return 1;
+ case GITS_CREADR + 4:
+ /* Bits[63:20] are RES0 */
+ goto read_as_zero_32;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- read ignored */
+ goto read_as_zero;
+ case GITS_BASER0:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ vits_spin_lock(vits);
+ *r = vgic_reg64_read(vits->baser0, info);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_BASER1 ... GITS_BASERN:
+ goto read_as_zero_64;
+ case GITS_PIDR0:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_read(GITS_PIDR0_VAL, info);
+ return 1;
+ case GITS_PIDR1:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_read(GITS_PIDR1_VAL, info);
+ return 1;
+ case GITS_PIDR2:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_read(GITS_PIDR2_VAL, info);
+ return 1;
+ case GITS_PIDR3:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_read(GITS_PIDR3_VAL, info);
+ return 1;
+ case GITS_PIDR4:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = vgic_reg32_read(GITS_PIDR4_VAL, info);
+ return 1;
+ case GITS_PIDR5 ... GITS_PIDR7:
+ goto read_as_zero_32;
+ default:
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: unhandled read r%d offset 0x%#08"PRIx32"\n",
+ v, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: bad read width %d r%d offset 0x%#08"PRIx32"\n",
+ v, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+read_as_zero_64:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ *r = 0;
+ return 1;
+read_as_zero_32:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+read_as_zero:
+ *r = 0;
+ return 1;
+}
+
+/*
+ * GITS_BASER.Type[58:56], GITS_BASER.Entry_size[55:48]
+ * and GITS_BASER.Shareability[11:10] are read-only,
+ * Only flat table is supported. So GITS_BASER.Indirect[62]
+ * is RAZ/WI.
+ * Mask those fields while emulating GITS_BASER reg.
+ * Shareability is set to 0x0 (Reserved) which is fixed.
+ * TODO: Support shareability as fixed value is deprecated
+ */
+#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
+ (0x1UL << GITS_BASER_INDIRECT_SHIFT) | \
+ (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
+ (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
+
+static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ int ret;
+ uint32_t gits_reg, sz, psz;
+ uint64_t val;
+
+ gits_reg = info->gpa - vits->gits_base;
+
+ DPRINTK("%pv: vITS: GITS_MMIO_WRITE offset 0x%"PRIx32"\n", v, gits_reg);
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vits_spin_lock(vits);
+ vgic_reg32_write(&vits->ctrl,
+ (*r & GITS_CTLR_ENABLE) | GITS_CTLR_QUIESCENT, info);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_IIDR:
+ /* RO -- write ignored */
+ goto write_ignore;
+ case GITS_TYPER:
+ case GITS_TYPER + 4:
+ /* RO -- write ignored */
+ goto write_ignore;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- write ignored */
+ goto write_ignore;
+ case GITS_CBASER:
+ /* XXX: support 32-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ if ( vits->ctrl & GITS_CTLR_ENABLE )
+ {
+ /* RO -- write ignored */
+ vits_spin_unlock(vits);
+ goto write_ignore;
+ }
+ vgic_reg64_write(&vits->cmd_base, *r, info);
+ if ( vits->cmd_base & GITS_BASER_VALID )
+ {
+ val = ((vits->cmd_base & GITS_BASER_PAGES_MASK_VAL) + 1) * SZ_4K;
+ vits->cmd_qsize = val;
+ atomic_set(&vits->cmd_read, 0);
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ vits_spin_lock(vits);
+ /* Only Bits[19:5] are writable */
+ vgic_reg64_write(&vits->cmd_write, (*r & 0xfffe0), info);
+ ret = 1;
+ /* CWRITER should be within command queue range */
+ if ( (vits->ctrl & GITS_CTLR_ENABLE) &&
+ (vits->cmd_write < vits->cmd_qsize) )
+ ret = vits_process_cmd(v, vits);
+ vits_spin_unlock(vits);
+ return ret;
+ case GITS_CWRITER + 4:
+ /* Bits[63:20] are RES0 */
+ goto write_ignore_32;
+ case GITS_CREADR:
+ /* RO -- write ignored */
+ goto write_ignore_64;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- write ignored */
+ goto write_ignore;
+ case GITS_BASER0:
+ /* Support only 64-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ /* RO -- write ignored if GITS_CTLR.Enable = 1 */
+ if ( vits->ctrl & GITS_CTLR_ENABLE )
+ {
+ vits_spin_unlock(vits);
+ goto write_ignore;
+ }
+ val = GITS_BASER0_INIT_VAL | (*r & GITS_BASER_MASK);
+ vgic_reg64_write(&vits->baser0, val, info);
+ vits->dt_ipa = vits->baser0 & GITS_BASER_PA_MASK;
+ psz = (vits->baser0 >> GITS_BASER_PAGE_SIZE_SHIFT) &
+ GITS_BASER_PAGE_SIZE_MASK_VAL;
+ if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
+ sz = 4;
+ else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
+ sz = 16;
+ else
+ /* 0x11 and 0x10 are treated as 64K size */
+ sz = 64;
+
+ vits->dt_size = (vits->baser0 & GITS_BASER_PAGES_MASK_VAL) * sz * SZ_1K;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_BASER1 ... GITS_BASERN:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ goto write_ignore;
+ case GITS_PIDR7 ... GITS_PIDR0:
+ /* RO -- write ignored */
+ goto write_ignore_32;
+ default:
+ dprintk(XENLOG_G_ERR,
+ "%pv vITS: unhandled write r%d offset 0x%#08"PRIx32"\n",
+ v, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: bad write width %d r%d offset 0x%#08"PRIx32"\n",
+ v, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+write_ignore_64:
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ return 1;
+write_ignore_32:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ return 1;
+write_ignore:
+ return 1;
+}
+
+static const struct mmio_handler_ops vgic_gits_mmio_handler = {
+ .read_handler = vgic_v3_gits_mmio_read,
+ .write_handler = vgic_v3_gits_mmio_write,
+};
+
int vits_domain_init(struct domain *d)
{
struct vgic_its *vits;
@@ -534,6 +838,17 @@ int vits_domain_init(struct domain *d)
for ( i = 0; i < vits_get_max_collections(d); i++ )
vits->collections[i].target_address = INVALID_PADDR;
+ vits->baser0 = GITS_BASER0_INIT_VAL;
+
+ /*
+ * Only one virtual ITS is provided to domain.
+ * Assign first physical ITS address to Dom0 virtual ITS.
+ */
+ vits->gits_base = vits_hw.phys_base;
+ vits->gits_size = vits_hw.phys_size;
+
+ register_mmio_handler(d, &vgic_gits_mmio_handler, vits->gits_base, SZ_64K);
+
return 0;
}
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 57a652c..7936256 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -52,6 +52,8 @@
#define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
#define GITS_TYPER_PTA (1UL << 19)
#define GITS_TYPER_HCC_SHIFT (24)
+#define GITS_TYPER_PHYSICAL_LPIS (1UL)
+#define GITS_TYPER_ITT_SIZE_SHIFT (4)
#define GITS_CBASER_VALID (1UL << 63)
#define GITS_CBASER_nC (1UL << 59)
@@ -64,6 +66,7 @@
#define GITS_BASER_NR_REGS 8
#define GITS_BASER_VALID (1UL << 63)
+#define GITS_BASER_INDIRECT_SHIFT (62)
#define GITS_BASER_nC (1UL << 59)
#define GITS_BASER_WaWb (5UL << 59)
#define GITS_BASER_TYPE_SHIFT (56)
@@ -79,6 +82,10 @@
#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_4K_VAL (0)
+#define GITS_BASER_PAGE_SIZE_16K_VAL (1)
+#define GITS_BASER_PAGE_SIZE_MASK_VAL (0x3)
+#define GITS_BASER_PAGES_MASK_VAL (0xff)
#define GITS_BASER_TYPE_NONE 0
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_VCPU 2
@@ -87,6 +94,7 @@
#define GITS_BASER_TYPE_RESERVED5 5
#define GITS_BASER_TYPE_RESERVED6 6
#define GITS_BASER_TYPE_RESERVED7 7
+#define GITS_BASER_PA_MASK (0xfffffffff000UL)
/*
* ITS commands
diff --git a/xen/include/asm-arm/vits.h b/xen/include/asm-arm/vits.h
index 295cce4..dc02762 100644
--- a/xen/include/asm-arm/vits.h
+++ b/xen/include/asm-arm/vits.h
@@ -24,6 +24,10 @@
struct vgic_its
{
spinlock_t lock;
+ /* Emulation of BASER0 */
+ uint64_t baser0;
+ /* GICR ctrl register */
+ uint32_t ctrl;
/* Command queue base */
paddr_t cmd_base;
/* Command queue write pointer */
@@ -32,6 +36,10 @@ struct vgic_its
atomic_t cmd_read;
/* Command queue size */
unsigned long cmd_qsize;
+ /* ITS mmio physical base */
+ paddr_t gits_base;
+ /* ITS mmio physical size */
+ unsigned long gits_size;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (16 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-23 8:31 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 19/28] xen/arm: ITS: Store LPIs allocated per domain vijay.kilari
` (10 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Export physical ITS information to virtual ITS driver
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Moved gic_its_info from gic-its.h to gic-v3-its.c
- s/dev_bits/devID_bits
- s/eventid_bits/eventID_bits
- Dropped patch #20 and merged changes to this patch
- Enabled compilation of vITS driver
v6: - Passed only one physical ITS info
- Passed all the values as parameters
- Initialize vITS only if physical ITS is available
---
xen/arch/arm/Makefile | 1 +
xen/arch/arm/gic-v3-its.c | 21 +++++++++++++++++++++
xen/arch/arm/vgic-v3-its.c | 21 +++++++++++++++++++++
xen/include/asm-arm/vits.h | 2 ++
4 files changed, 45 insertions(+)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 4708716..cd10d42 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -33,6 +33,7 @@ obj-y += shutdown.o
obj-y += traps.o
obj-y += vgic.o vgic-v2.o
obj-$(CONFIG_ARM_64) += vgic-v3.o
+obj-$(CONFIG_ARM_64) += vgic-v3-its.o
obj-y += vtimer.o
obj-y += vuart.o
obj-y += hvm.o
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 0d5c61c..f3346d3 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -37,6 +37,7 @@
#include <asm/gic.h>
#include <asm/gic_v3_defs.h>
#include <asm/gic-its.h>
+#include <asm/vits.h>
#include <xen/log2.h>
#define its_print(lvl, fmt, ...) \
@@ -88,6 +89,11 @@ struct its_node {
struct dt_device_node *dt_node;
};
+static struct {
+ uint8_t eventID_bits;
+ uint8_t devID_bits;
+} its_data;
+
#define ITS_ITT_ALIGN SZ_256
static LIST_HEAD(its_nodes);
@@ -914,6 +920,8 @@ int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
for ( i = 0; i < pdev->event_map.nr_lpis; i++ )
{
+ ASSERT(i < (1 << its_data.eventID_bits));
+
plpi = its_get_plpi(pdev, i);
/* TODO: Route lpi */
}
@@ -1366,6 +1374,8 @@ static int its_probe(struct dt_device_node *node)
its->phys_size = its_size;
typer = readl_relaxed(its_base + GITS_TYPER);
its->ite_size = ((typer >> 4) & 0xf) + 1;
+ its_data.eventID_bits = GITS_TYPER_IDBITS(typer);
+ its_data.devID_bits = GITS_TYPER_DEVBITS(typer);
its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
if ( !its->cmd_base )
@@ -1457,6 +1467,7 @@ int its_cpu_init(void)
int __init its_init(struct rdist_prop *rdists)
{
+ struct its_node *its;
struct dt_device_node *np = NULL;
static const struct dt_device_match its_device_ids[] __initconst =
@@ -1479,6 +1490,16 @@ int __init its_init(struct rdist_prop *rdists)
its_alloc_lpi_tables();
its_lpi_init(rdists->id_bits);
+ its = list_first_entry(&its_nodes, struct its_node, entry);
+ /*
+ * As per vITS design spec, Xen exposes only one virtual ITS per domain.
+ * This simplifies vITS command model. i.e Simplifies processing global
+ * ITS commands which does not have device ID on platform having more
+ * than one physical ITS.
+ */
+ vits_setup_hw(its_data.devID_bits, its_data.eventID_bits,
+ its->phys_base, its->phys_size);
+
return 0;
}
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 0d97fcb..3679d11 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -61,6 +61,26 @@ static void dump_cmd(const its_cmd_block *cmd)
static void dump_cmd(const its_cmd_block *cmd) { }
#endif
+static struct {
+ bool_t enabled;
+ uint8_t devID_bits;
+ uint8_t eventID_bits;
+ /* GITS physical base */
+ paddr_t phys_base;
+ /* GITS physical size */
+ unsigned long phys_size;
+} vits_hw;
+
+void vits_setup_hw(uint8_t devID_bits, uint8_t eventID_bits,
+ paddr_t phys_base, unsigned long phys_size)
+{
+ vits_hw.enabled = 1;
+ vits_hw.devID_bits = devID_bits;
+ vits_hw.eventID_bits = eventID_bits;
+ vits_hw.phys_base = phys_base;
+ vits_hw.phys_size = phys_size;
+}
+
static inline uint16_t vits_get_max_collections(struct domain *d)
{
/*
@@ -821,6 +841,7 @@ int vits_domain_init(struct domain *d)
}
ASSERT(is_hardware_domain(d));
+ ASSERT(vits_hw.enabled);
d->arch.vgic.vits = xzalloc(struct vgic_its);
if ( !d->arch.vgic.vits )
diff --git a/xen/include/asm-arm/vits.h b/xen/include/asm-arm/vits.h
index dc02762..d864c1e 100644
--- a/xen/include/asm-arm/vits.h
+++ b/xen/include/asm-arm/vits.h
@@ -78,6 +78,8 @@ int vits_get_vitt_entry(struct domain *d, uint32_t devid, uint32_t event,
struct vitt *entry);
int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
struct vdevice_table *entry);
+void vits_setup_hw(uint8_t dev_bits, uint8_t eventid_bits,
+ paddr_t base, unsigned long size);
#endif /* __ASM_ARM_VITS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 19/28] xen/arm: ITS: Store LPIs allocated per domain
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (17 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-23 8:37 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper vijay.kilari
` (9 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Store number of lpis allocated per domain in vgic structure
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Change commit message.
- Store only nr_lpis per domain in vgic structure and drop
id_bits.
---
xen/arch/arm/vgic-v3-its.c | 2 ++
xen/arch/arm/vgic.c | 1 +
xen/include/asm-arm/domain.h | 1 +
3 files changed, 4 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 3679d11..4285af3 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -843,6 +843,8 @@ int vits_domain_init(struct domain *d)
ASSERT(is_hardware_domain(d));
ASSERT(vits_hw.enabled);
+ d->arch.vgic.nr_lpis = gic_nr_irq_ids() - FIRST_GIC_LPI;
+
d->arch.vgic.vits = xzalloc(struct vgic_its);
if ( !d->arch.vgic.vits )
return -ENOMEM;
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index baa74c7..a5ab99d9 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -75,6 +75,7 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
int ret;
d->arch.vgic.ctlr = 0;
+ d->arch.vgic.nr_lpis = 0;
/* Limit the number of virtual SPIs supported to (1020 - 32) = 988 */
if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 0f2aa66..011f85b 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -92,6 +92,7 @@ struct arch_domain
spinlock_t lock;
int ctlr;
int nr_spis; /* Number of SPIs */
+ int nr_lpis; /* Number of LPIs */
unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
struct vgic_irq_rank *shared_irqs;
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (18 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 19/28] xen/arm: ITS: Store LPIs allocated per domain vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-23 8:52 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation vijay.kilari
` (8 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Introduce vgic_is_lpi_supported() helper function
to know virtual ITS availability for a domain
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - its_enabled field is added to vgic structure
---
xen/arch/arm/vgic-v2.c | 2 ++
xen/arch/arm/vgic-v3.c | 3 +++
xen/arch/arm/vgic.c | 5 +++++
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/vgic.h | 1 +
5 files changed, 12 insertions(+)
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index 600d27e..de0cd4b 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -594,6 +594,8 @@ static int vgic_v2_domain_init(struct domain *d)
register_mmio_handler(d, &vgic_v2_distr_mmio_handler, d->arch.vgic.dbase,
PAGE_SIZE);
+ d->arch.vgic.its_enabled = 0;
+
return 0;
}
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 52d4277..d99dedb 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1250,6 +1250,9 @@ static int vgic_v3_domain_init(struct domain *d)
d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
+ if ( is_hardware_domain(d) && vgic_v3_hw.lpi_support )
+ d->arch.vgic.its_enabled = 1;
+
return 0;
}
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index a5ab99d9..3555833 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -62,6 +62,11 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
return vgic_get_rank(v, rank);
}
+bool_t vgic_is_lpi_supported(struct domain *d)
+{
+ return d->arch.vgic.its_enabled;
+}
+
static void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
{
INIT_LIST_HEAD(&p->inflight);
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 011f85b..0f6f9d8 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -93,6 +93,7 @@ struct arch_domain
int ctlr;
int nr_spis; /* Number of SPIs */
int nr_lpis; /* Number of LPIs */
+ bool_t its_enabled;
unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
struct vgic_irq_rank *shared_irqs;
/*
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 2bf061f..edeec9c 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -309,6 +309,7 @@ enum gic_sgi_mode;
#define vgic_num_irq_lines(d) ((d)->arch.vgic.nr_spis + 32)
+extern bool_t vgic_is_lpi_supported(struct domain *d);
extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
extern void domain_vgic_free(struct domain *d);
extern int vcpu_vgic_init(struct vcpu *v);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (19 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-23 10:22 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 22/28] xen/arm: ITS: Allocate irq descriptors for LPIs vijay.kilari
` (7 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate LPI related changes to GICR registers
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Merged patch#23 to this patch. patch#23 is just one line.
- Add 32-bit access to GICR_TYPER register
- Changed dprintk to printk
- LPI property table is handling is changed. Call {enable,disable}_lpi
only for nr_lpis.
- Changes to GICD_TYPER emulation.
v6: - Moved LPI handling code to vgic-v3.c
- parse LPI property table on GICR_PROPBASER update
- use vgic_is_lpi_supported()
v5: - Handled all sizes access to LPI configuration table
- Rename vits_unmap_lpi_prop as vits_map_lpi_prop
v4: - Added LPI configuration table emulation
- Rename function inline with vits
- Copied guest lpi configuration table to xen
---
xen/arch/arm/vgic-v3-its.c | 4 +
xen/arch/arm/vgic-v3.c | 348 +++++++++++++++++++++++++++++++++++--
xen/include/asm-arm/domain.h | 3 +
xen/include/asm-arm/gic-its.h | 1 +
xen/include/asm-arm/gic_v3_defs.h | 3 +
xen/include/asm-arm/vits.h | 8 +
6 files changed, 354 insertions(+), 13 deletions(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 4285af3..37c2105 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -29,6 +29,7 @@
#include <asm/atomic.h>
#include <asm/gic_v3_defs.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
#include <asm/gic-its.h>
#include <asm/vits.h>
@@ -852,6 +853,7 @@ int vits_domain_init(struct domain *d)
vits = d->arch.vgic.vits;
spin_lock_init(&vits->lock);
+ spin_lock_init(&vits->prop_lock);
vits->collections = xzalloc_array(struct its_collection,
vits_get_max_collections(d));
@@ -877,6 +879,8 @@ int vits_domain_init(struct domain *d)
void vits_domain_free(struct domain *d)
{
+ free_xenheap_pages(d->arch.vgic.vits->prop_page,
+ get_order_from_bytes(d->arch.vgic.vits->prop_size));
xfree(d->arch.vgic.vits->collections);
xfree(d->arch.vgic.vits);
}
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index d99dedb..c631c7b 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -29,7 +29,10 @@
#include <asm/current.h>
#include <asm/mmio.h>
#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
+#include <asm/vits.h>
/* GICD_PIDRn register values for ARM implementations */
#define GICV3_GICD_PIDR0 0x92
@@ -106,23 +109,269 @@ static struct vcpu *vgic_v3_get_target_vcpu(struct vcpu *v, unsigned int irq)
return v_target;
}
+static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
+{
+ struct pending_irq *p;
+ struct vcpu *v_target = v->domain->vcpu[0];
+
+ p = irq_to_pending(v_target, vlpi);
+ if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+ {
+ clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+ gic_remove_from_queues(v_target, vlpi);
+ }
+}
+
+static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
+{
+ struct pending_irq *p;
+ unsigned long flags;
+ struct vcpu *v_target = v->domain->vcpu[0];
+
+ /*
+ * We don't have vlpi to plpi mapping and hence we cannot
+ * have target on which corresponding vlpi is enabled.
+ * So for now we are always injecting vlpi on vcpu0.
+ * (See vgic_vcpu_inject_lpi() function) and so we get pending_irq
+ * structure on vcpu0.
+ * TODO: Get correct target vcpu
+ */
+ p = irq_to_pending(v_target, vlpi);
+
+ set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+
+ spin_lock_irqsave(&v_target->arch.vgic.lock, flags);
+
+ if ( !list_empty(&p->inflight) &&
+ !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
+ gic_raise_guest_irq(v_target, vlpi, p->priority);
+
+ spin_unlock_irqrestore(&v_target->arch.vgic.lock, flags);
+}
+
+static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset;
+ void *addr;
+ uint64_t val;
+ unsigned long flags;
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+
+ offset = info->gpa - (vits->propbase & GICR_PROPBASER_PA_MASK);
+ addr = (void *)((u8*)vits->prop_page + offset);
+
+ spin_lock_irqsave(&vits->prop_lock, flags);
+ switch (dabt.size)
+ {
+ case DABT_DOUBLE_WORD:
+ val = *((u64*)addr);
+ *r = val;
+ break;
+ case DABT_WORD:
+ val = *((u32*)addr);
+ *r = (u32)val;
+ break;
+ case DABT_HALF_WORD:
+ val = *((u16*)addr);
+ *r = (u16)val;
+ break;
+ default:
+ val = *((u8*)addr);
+ *r = (u8)val;
+ }
+ spin_unlock_irqrestore(&vits->prop_lock, flags);
+
+ return 1;
+}
+
+static void vgic_v3_gits_update_lpis_state(struct vcpu *v, uint32_t vid,
+ uint32_t size)
+{
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ uint32_t i;
+ uint8_t cfg, *p;
+ bool_t enable;
+
+ p = ((u8*)vits->prop_page + vid);
+
+ for ( i = 0 ; i < size; i++ )
+ {
+ cfg = *p;
+ enable = cfg & LPI_PROP_ENABLED;
+
+ /* LPIs start from 8192, So add 8192 to point to correct LPI number */
+ if ( !enable )
+ vits_enable_lpi(v, vid + FIRST_GIC_LPI, (cfg & LPI_PRIORITY_MASK));
+ else
+ vits_disable_lpi(v, vid + FIRST_GIC_LPI);
+
+ p++;
+ vid++;
+ }
+}
+
+static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset;
+ uint8_t cfg, *p, *val, i, iter;
+ bool_t enable;
+ unsigned long flags;
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+
+ offset = info->gpa - (vits->propbase & GICR_PROPBASER_PA_MASK);
+
+ switch (dabt.size)
+ {
+ case DABT_DOUBLE_WORD:
+ iter = 8;
+ break;
+ case DABT_WORD:
+ iter = 4;
+ break;
+ case DABT_HALF_WORD:
+ iter = 2;
+ break;
+ default:
+ iter = 1;
+ }
+ spin_lock_irqsave(&vits->prop_lock, flags);
+
+ p = ((u8*)vits->prop_page + offset);
+ val = (u8*)r;
+
+ for ( i = 0 ; i < iter; i++ )
+ {
+ cfg = *p;
+ enable = (cfg & *val) & LPI_PROP_ENABLED;
+
+ if ( !enable )
+ vits_enable_lpi(v, offset + FIRST_GIC_LPI, (*val & LPI_PRIORITY_MASK));
+ else
+ vits_disable_lpi(v, offset + FIRST_GIC_LPI);
+
+ /* Update virtual prop page */
+ *p = (*val & 0xff);
+ val++;
+ p++;
+ offset++;
+ }
+
+ spin_unlock_irqrestore(&vits->prop_lock, flags);
+
+ return 1;
+}
+
+static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
+ .read_handler = vgic_v3_gits_lpi_mmio_read,
+ .write_handler = vgic_v3_gits_lpi_mmio_write,
+};
+
+static int vits_map_lpi_prop(struct vcpu *v)
+{
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ paddr_t gaddr, addr;
+ unsigned long mfn, flags;
+ uint32_t id_bits, vgic_id_bits;
+ int i;
+
+ gaddr = vits->propbase & GICR_PROPBASER_PA_MASK;
+ id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
+
+ vgic_id_bits = get_count_order(v->domain->arch.vgic.nr_lpis +
+ FIRST_GIC_LPI);
+ /*
+ * Here we limit the size of LPI property table to the number of LPIs
+ * that domain supports.
+ */
+ if ( id_bits > vgic_id_bits )
+ id_bits = vgic_id_bits;
+
+ vits->prop_size = 1 << id_bits;
+
+ /*
+ * Allocate Virtual LPI Property table.
+ * TODO: To re-use guest property table
+ */
+ vits->prop_page = alloc_xenheap_pages(get_order_from_bytes(vits->prop_size),
+ 0);
+ if ( !vits->prop_page )
+ {
+ printk(XENLOG_G_ERR
+ "d%d: vITS: Fail to allocate LPI Prop page\n",
+ v->domain->domain_id);
+ return 0;
+ }
+
+ addr = gaddr;
+
+ spin_lock_irqsave(&vits->prop_lock, flags);
+ /* Copy only configuration of supported LPIs (vgic.nr_lpis) */
+ for ( i = 0; i < v->domain->arch.vgic.nr_lpis / PAGE_SIZE; i++ )
+ {
+ vgic_access_guest_memory(v->domain, addr,
+ (void *)(vits->prop_page + (i * PAGE_SIZE)), PAGE_SIZE, 0);
+
+ vgic_v3_gits_update_lpis_state(v, (i * PAGE_SIZE), PAGE_SIZE);
+ addr += PAGE_SIZE;
+ }
+ spin_unlock_irqrestore(&vits->prop_lock, flags);
+
+ /*
+ * Each re-distributor shares a common LPI configuration table
+ * So one set of mmio handlers to manage configuration table is enough
+ *
+ * Copy and unmap LPI property table.
+ */
+ addr = gaddr;
+ for ( i = 0; i < vits->prop_size / PAGE_SIZE; i++ )
+ {
+ mfn = gmfn_to_mfn(v->domain, paddr_to_pfn(addr));
+ if ( unlikely(!mfn_valid(mfn)) )
+ {
+ printk(XENLOG_G_ERR
+ "vITS: Invalid propbaser address for domain %d\n",
+ v->domain->domain_id);
+ return 0;
+ }
+ guest_physmap_remove_page(v->domain, paddr_to_pfn(addr), mfn, 0);
+ addr += PAGE_SIZE;
+ }
+
+ /* Register mmio handlers for this region */
+ register_mmio_handler(v->domain, &vgic_gits_lpi_mmio_handler,
+ gaddr, vits->prop_size);
+
+ return 1;
+}
+
static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
uint32_t gicr_reg)
{
struct hsr_dabt dabt = info->dabt;
struct cpu_user_regs *regs = guest_cpu_user_regs();
register_t *r = select_user_reg(regs, dabt.reg);
+ struct vgic_its *vits;
switch ( gicr_reg )
{
case GICR_CTLR:
- /* We have not implemented LPI's, read zero */
- goto read_as_zero_32;
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vgic_lock(v);
+ *r = vgic_reg32_read(v->domain->arch.vgic.gicr_ctlr, info);
+ vgic_unlock(v);
+ return 1;
case GICR_IIDR:
if ( dabt.size != DABT_WORD ) goto bad_width;
*r = vgic_reg32_read(GICV3_GICR_IIDR_VAL, info);
return 1;
case GICR_TYPER:
+ case GICR_TYPER + 4:
{
uint64_t typer, aff;
@@ -137,6 +386,12 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
typer |= GICR_TYPER_LAST;
+ /* Set Physical LPIs support */
+ if ( vgic_is_lpi_supported(v->domain) )
+ typer |= GICR_TYPER_PLPIS;
+ /* GITS_TYPER.PTA is 0. Provide vcpu number as target address */
+ typer |= (v->vcpu_id << GICR_TYPER_PROCESSOR_SHIFT);
+
*r = vgic_reg64_read(typer, info);
return 1;
@@ -154,10 +409,29 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
/* WO. Read as zero */
goto read_as_zero_64;
case GICR_PROPBASER:
- /* LPI's not implemented */
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ if ( vgic_is_lpi_supported(v->domain) )
+ {
+ vits = v->domain->arch.vgic.vits;
+ vgic_lock(v);
+ *r = vgic_reg64_read(vits->propbase, info);
+ vgic_unlock(v);
+ return 1;
+ }
goto read_as_zero_64;
case GICR_PENDBASER:
- /* LPI's not implemented */
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ if ( vgic_is_lpi_supported(v->domain) )
+ {
+ uint64_t val;
+
+ vgic_lock(v);
+ val = vgic_reg64_read(v->arch.vgic.pendbase, info);
+ /* PTZ field is WO */
+ *r = val & ~GICR_PENDBASER_PTZ_MASK;
+ vgic_unlock(v);
+ return 1;
+ }
goto read_as_zero_64;
case GICR_INVLPIR:
/* WO. Read as zero */
@@ -232,7 +506,19 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case GICR_CTLR:
- /* LPI's not implemented */
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ if ( vgic_is_lpi_supported(v->domain) )
+ {
+ /*
+ * Enable LPI's for ITS. Direct injection of LPI
+ * by writing to GICR_{SET,CLR}LPIR is not supported.
+ */
+ vgic_lock(v);
+ vgic_reg32_write(&v->domain->arch.vgic.gicr_ctlr,
+ (*r & GICR_CTLR_ENABLE_LPIS), info);
+ vgic_unlock(v);
+ return 1;
+ }
goto write_ignore_32;
case GICR_IIDR:
/* RO */
@@ -253,10 +539,42 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
/* LPI is not implemented */
goto write_ignore_64;
case GICR_PROPBASER:
- /* LPI is not implemented */
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ if ( vgic_is_lpi_supported(v->domain) )
+ {
+ vgic_lock(v);
+ /*
+ * LPI configuration tables are shared across cpus. Should be same.
+ *
+ * Allow updating on propbase only once with below check.
+ * TODO: Manage change in property table.
+ */
+ if ( v->domain->arch.vgic.vits->propbase != 0 )
+ {
+ printk(XENLOG_G_WARNING
+ "%pv: vGICR: Updating LPI propbase is not allowed\n", v);
+ vgic_unlock(v);
+ return 1;
+ }
+ /*
+ * TODO: As per spec, updating GICR_PROPBASER when GICR_CTLR.EnableLPIs = 1
+ * is unpredictable. Handle this scenario
+ */
+ vgic_reg64_write(&v->domain->arch.vgic.vits->propbase, *r, info);
+ vgic_unlock(v);
+ return vits_map_lpi_prop(v);
+ }
goto write_ignore_64;
case GICR_PENDBASER:
- /* LPI is not implemented */
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ if ( vgic_is_lpi_supported(v->domain) )
+ {
+ /* Just hold pendbaser value for guest read */
+ vgic_lock(v);
+ vgic_reg64_write(&v->arch.vgic.pendbase, *r, info);
+ vgic_unlock(v);
+ return 1;
+ }
goto write_ignore_64;
case GICR_INVLPIR:
/* LPI is not implemented */
@@ -720,11 +1038,7 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
return 1;
case GICD_TYPER:
{
- /*
- * Number of interrupt identifier bits supported by the GIC
- * Stream Protocol Interface
- */
- unsigned int irq_bits = get_count_order(vgic_num_irq_lines(v->domain));
+ unsigned int irqs;
/*
* Number of processors that may be used as interrupt targets when ARE
* bit is zero. The maximum is 8.
@@ -737,7 +1051,15 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
typer = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
- typer |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
+ if ( vgic_is_lpi_supported(v->domain) )
+ {
+ irqs = v->domain->arch.vgic.nr_lpis + FIRST_GIC_LPI;
+ typer |= GICD_TYPE_LPIS;
+ }
+ else
+ irqs = vgic_num_irq_lines(v->domain);
+
+ typer |= (get_count_order(irqs) - 1) << GICD_TYPE_ID_BITS_SHIFT;
*r = vgic_reg32_read(typer, info);
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 0f6f9d8..6a5c6a0 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -116,6 +116,7 @@ struct arch_domain
uint32_t rdist_stride; /* Re-Distributor stride */
/* Virtual ITS */
struct vgic_its *vits;
+ uint32_t gicr_ctlr;
#endif
} vgic;
@@ -249,6 +250,8 @@ struct arch_vcpu
/* GICv3: redistributor base and flags for this vCPU */
paddr_t rdist_base;
+ /* GICv3-ITS: LPI pending table for this vCPU */
+ paddr_t pendbase;
#define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
uint8_t flags;
} vgic;
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 7936256..8f3d0fe 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -114,6 +114,7 @@
#define LPI_PROP_ENABLED (1 << 0)
#define LPI_PROP_GROUP1 (1 << 1)
+#define LPI_PRIORITY_MASK (0xfc)
/*
* Collection structure - just an ID, and a redistributor address to
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index f819589..5d39033 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -126,14 +126,17 @@
#define GICR_PROPBASER_WaWb (5U << 7)
#define GICR_PROPBASER_CACHEABILITY_MASK (7U << 7)
#define GICR_PROPBASER_IDBITS_MASK (0x1f)
+#define GICR_PROPBASER_PA_MASK (0xfffffffff000UL)
#define GICR_TYPER_PLPIS (1U << 0)
#define GICR_TYPER_VLPIS (1U << 1)
#define GICR_TYPER_LAST (1U << 4)
+#define GICR_TYPER_PROCESSOR_SHIFT (8)
#define GICR_PENDBASER_InnerShareable (1U << 10)
#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
#define GICR_PENDBASER_nC (1U << 7)
#define GICR_PENDBASER_CACHEABILITY_MASK (7U << 7)
+#define GICR_PENDBASER_PTZ_MASK (1UL << 62)
#define DEFAULT_PMR_VALUE 0xff
diff --git a/xen/include/asm-arm/vits.h b/xen/include/asm-arm/vits.h
index d864c1e..e1188f7 100644
--- a/xen/include/asm-arm/vits.h
+++ b/xen/include/asm-arm/vits.h
@@ -36,6 +36,14 @@ struct vgic_its
atomic_t cmd_read;
/* Command queue size */
unsigned long cmd_qsize;
+ /* LPI propbase */
+ paddr_t propbase;
+ /* Virtual LPI property table */
+ void *prop_page;
+ /* Virtual LPI property size */
+ uint32_t prop_size;
+ /* spinlock to protect lpi property table */
+ spinlock_t prop_lock;
/* ITS mmio physical base */
paddr_t gits_base;
/* ITS mmio physical size */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 22/28] xen/arm: ITS: Allocate irq descriptors for LPIs
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (20 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-23 10:28 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 23/28] xen/arm: ITS: Allocate pending_lpi " vijay.kilari
` (6 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Allocate dynamically irq descriptors for LPIs
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v6: - Add separate patch for irq_pending structures
- renamed and moved is_domain_lpi to vgic
- Updated __irq_to_domain
---
xen/arch/arm/gic-v3-its.c | 4 ++++
xen/arch/arm/irq.c | 34 +++++++++++++++++++++++++++++++++-
xen/include/asm-arm/irq.h | 1 +
3 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index f3346d3..4875b0f 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -1490,6 +1490,10 @@ int __init its_init(struct rdist_prop *rdists)
its_alloc_lpi_tables();
its_lpi_init(rdists->id_bits);
+ /* Allocate irq descriptors for LPIs */
+ if ( init_lpi() )
+ return -ENOMEM;
+
its = list_first_entry(&its_nodes, struct its_node, entry);
/*
* As per vITS design spec, Xen exposes only one virtual ITS per domain.
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 3a01f46..32c3b53 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -31,6 +31,8 @@
static unsigned int local_irqs_type[NR_LOCAL_IRQS];
static DEFINE_SPINLOCK(local_irqs_type_lock);
+static irq_desc_t *irq_desc_lpi;
+
/* Number of LPIs supported by Xen.
*
* The LPI identifier starts from 8192. Given that the hardware is
@@ -73,7 +75,15 @@ static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
irq_desc_t *__irq_to_desc(int irq)
{
if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
- return &irq_desc[irq-NR_LOCAL_IRQS];
+ else if ( gic_is_lpi(irq) )
+ {
+ ASSERT(irq_desc_lpi != NULL);
+ return &irq_desc_lpi[irq - FIRST_GIC_LPI];
+ }
+ else
+ return &irq_desc[irq - NR_LOCAL_IRQS];
+
+ return NULL;
}
int __init arch_init_one_irq_desc(struct irq_desc *desc)
@@ -123,6 +133,28 @@ static int __cpuinit init_local_irq_data(void)
return 0;
}
+int init_lpi(void)
+{
+ struct irq_desc *desc;
+ unsigned int i;
+
+ /* Allocate LPI irq descriptors */
+ irq_desc_lpi = xzalloc_array(struct irq_desc, nr_lpis);
+ if ( !irq_desc_lpi )
+ return -ENOSPC;
+
+ for ( i = 0; i < nr_lpis; i++ )
+ {
+ desc = &irq_desc_lpi[i];
+ init_one_irq_desc(desc);
+ desc->irq = FIRST_GIC_LPI + i;
+ desc->arch.type = DT_IRQ_TYPE_EDGE_BOTH;
+ desc->action = NULL;
+ }
+
+ return 0;
+}
+
void __init init_IRQ(void)
{
int irq;
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 9e342d2..c009a5e 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -68,6 +68,7 @@ int platform_get_irq(const struct dt_device_node *device, int index);
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask);
void irq_set_msi_desc(struct irq_desc *desc, struct msi_desc *msi);
struct msi_desc *irq_get_msi_desc(struct irq_desc *desc);
+int init_lpi(void);
#endif /* _ASM_HW_IRQ_H */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 23/28] xen/arm: ITS: Allocate pending_lpi descriptors for LPIs
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (21 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 22/28] xen/arm: ITS: Allocate irq descriptors for LPIs vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-24 12:38 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 24/28] xen/arm: ITS: Route LPIs vijay.kilari
` (5 subsequent siblings)
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Allocate dynamically pending_lpi descriptors for LPIs
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/vgic-v3-its.c | 9 +++++++++
xen/arch/arm/vgic.c | 20 +++++++++++++++++---
xen/include/asm-arm/domain.h | 3 +++
xen/include/asm-arm/vgic.h | 4 +++-
4 files changed, 32 insertions(+), 4 deletions(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 37c2105..33b6e2e 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -852,6 +852,14 @@ int vits_domain_init(struct domain *d)
vits = d->arch.vgic.vits;
+ d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq,
+ d->arch.vgic.nr_lpis);
+ if ( d->arch.vgic.pending_lpis == NULL )
+ return -ENOMEM;
+
+ for ( i = 0; i < d->arch.vgic.nr_lpis; i++ )
+ vgic_init_pending_irq(&d->arch.vgic.pending_lpis[i], i + FIRST_GIC_LPI);
+
spin_lock_init(&vits->lock);
spin_lock_init(&vits->prop_lock);
@@ -881,6 +889,7 @@ void vits_domain_free(struct domain *d)
{
free_xenheap_pages(d->arch.vgic.vits->prop_page,
get_order_from_bytes(d->arch.vgic.vits->prop_size));
+ xfree(d->arch.vgic.pending_lpis);
xfree(d->arch.vgic.vits->collections);
xfree(d->arch.vgic.vits);
}
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 3555833..160a873 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -67,7 +67,13 @@ bool_t vgic_is_lpi_supported(struct domain *d)
return d->arch.vgic.its_enabled;
}
-static void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
+bool_t vgic_is_domain_lpi(struct domain *d, unsigned int lpi)
+{
+ return ((lpi >= FIRST_GIC_LPI) &&
+ (lpi < (d->arch.vgic.nr_lpis + FIRST_GIC_LPI)));
+}
+
+void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
{
INIT_LIST_HEAD(&p->inflight);
INIT_LIST_HEAD(&p->lr_queue);
@@ -382,13 +388,21 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
{
- struct pending_irq *n;
+ struct pending_irq *n = NULL;
/* Pending irqs allocation strategy: the first vgic.nr_spis irqs
- * are used for SPIs; the rests are used for per cpu irqs */
+ * are used for SPIs; the rests are used for per cpu irqs.
+ * For LPIs pending_irq structures are allocated separately */
if ( irq < 32 )
n = &v->arch.vgic.pending_irqs[irq];
+#ifdef HAS_GICV3
+ else if ( vgic_is_domain_lpi(v->domain, irq) )
+ n = &v->domain->arch.vgic.pending_lpis[irq - FIRST_GIC_LPI];
+#endif
else
n = &v->domain->arch.vgic.pending_irqs[irq - 32];
+
+ ASSERT(n != NULL);
+
return n;
}
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 6a5c6a0..49630a3 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -101,6 +101,9 @@ struct arch_domain
* struct arch_vcpu.
*/
struct pending_irq *pending_irqs;
+#ifdef HAS_GICV3
+ struct pending_irq *pending_lpis;
+#endif
/* Base address for guest GIC */
paddr_t dbase; /* Distributor base address */
paddr_t cbase; /* CPU base address */
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index edeec9c..5ed840c 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -333,6 +333,7 @@ extern int vgic_to_sgi(struct vcpu *v, register_t sgir,
enum gic_sgi_mode irqmode, int virq,
const struct sgi_target *target);
extern void vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq);
+extern void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq);
/* Reserve a specific guest vIRQ */
extern bool_t vgic_reserve_virq(struct domain *d, unsigned int virq);
@@ -365,7 +366,8 @@ void vgic_v3_setup_hw(paddr_t dbase,
const struct rdist_region *regions,
uint32_t rdist_stride, bool_t lpi_support);
#endif
-
+bool_t vgic_is_domain_lpi(struct domain *d, unsigned int lpi);
+
#endif /* __ASM_ARM_VGIC_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 24/28] xen/arm: ITS: Route LPIs
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (22 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 23/28] xen/arm: ITS: Allocate pending_lpi " vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-18 13:09 ` [PATCH v7 25/28] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
` (4 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Allocate and initialize irq descriptor for LPIs and
route LPIs to guest
For LPIs deactivation is not required. Hence
GICH_LR.HW is not required to set.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Subtract 8192 from irq number to index to lpi property
table to read priority
v6: - Moved ITS specific code from irq.c to vgic.c and
introduced vgic_vcpu_raise_lpi()
- Renamed gicv3_set_properties to gicv3_set_irq_properties
- Always Inject LPI to vcpu0
- Changed route_lpi_to_guest definition
---
xen/arch/arm/gic-v3-its.c | 17 ++++++-
xen/arch/arm/gic-v3.c | 16 ++++--
xen/arch/arm/gic.c | 32 +++++++++++-
xen/arch/arm/irq.c | 112 ++++++++++++++++++++++++++++++++++++++---
xen/arch/arm/vgic-v3-its.c | 2 +-
xen/arch/arm/vgic-v3.c | 40 +++++++++++++--
xen/arch/arm/vgic.c | 65 ++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 3 ++
xen/include/asm-arm/gic.h | 8 ++-
xen/include/asm-arm/irq.h | 2 +
xen/include/asm-arm/vgic.h | 3 ++
11 files changed, 278 insertions(+), 22 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 4875b0f..891a5e2 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -470,6 +470,21 @@ static void its_flush_and_invalidate_prop(struct irq_desc *desc, u8 *cfg)
its_send_inv(its_dev, vid);
}
+void its_set_lpi_properties(struct irq_desc *desc,
+ const cpumask_t *cpu_mask,
+ unsigned int priority)
+{
+ unsigned long flags;
+ u8 *cfg;
+
+ spin_lock_irqsave(&its_lock, flags);
+ cfg = gic_rdists->prop_page + desc->irq - FIRST_GIC_LPI;
+ *cfg = (*cfg & 3) | (priority & LPI_PRIORITY_MASK) ;
+
+ its_flush_and_invalidate_prop(desc, cfg);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
static void its_set_lpi_state(struct irq_desc *desc, int enable)
{
u8 *cfg;
@@ -923,7 +938,7 @@ int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
ASSERT(i < (1 << its_data.eventID_bits));
plpi = its_get_plpi(pdev, i);
- /* TODO: Route lpi */
+ route_lpi_to_guest(d, plpi, "LPI");
}
return 0;
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index ac8a0ea..2cdc065 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -481,9 +481,9 @@ static inline uint64_t gicv3_mpidr_to_affinity(int cpu)
MPIDR_AFFINITY_LEVEL(mpidr, 0));
}
-static void gicv3_set_irq_properties(struct irq_desc *desc,
- const cpumask_t *cpu_mask,
- unsigned int priority)
+static void gicv3_set_line_properties(struct irq_desc *desc,
+ const cpumask_t *cpu_mask,
+ unsigned int priority)
{
uint32_t cfg, actual, edgebit;
uint64_t affinity;
@@ -547,6 +547,16 @@ static int gicv3_dist_supports_lpis(void)
return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
}
+static void gicv3_set_irq_properties(struct irq_desc *desc,
+ const cpumask_t *cpu_mask,
+ unsigned int priority)
+{
+ if ( gic_is_lpi(desc->irq) )
+ its_set_lpi_properties(desc, cpu_mask, priority);
+ else
+ gicv3_set_line_properties(desc, cpu_mask, priority);
+}
+
static void __init gicv3_dist_init(void)
{
uint32_t type;
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index cefd8fc..ff24bb1 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -72,6 +72,13 @@ bool_t gic_is_lpi(unsigned int irq)
return (irq >= FIRST_GIC_LPI && irq < gic_nr_irq_ids());
}
+/* Validates PPIs/SGIs/SPIs/LPIs supported */
+bool_t gic_is_valid_irq(unsigned int irq)
+{
+ return (irq < gic_hw_ops->info->nr_lines || gic_is_lpi(irq));
+}
+
+/* Returns number of PPIs/SGIs/SPIs supported */
unsigned int gic_number_lines(void)
{
return gic_hw_ops->info->nr_lines;
@@ -134,7 +141,8 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
unsigned int priority)
{
ASSERT(priority <= 0xff); /* Only 8 bits of priority */
- ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
+ /* Can't route interrupts that don't exist */
+ ASSERT(gic_is_valid_irq(desc->irq));
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
@@ -183,6 +191,26 @@ out:
return res;
}
+int gic_route_lpi_to_guest(struct domain *d, struct irq_desc *desc,
+ unsigned int priority)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ desc->handler = get_guest_hw_irq_controller(desc->irq);
+ set_bit(_IRQ_GUEST, &desc->status);
+
+ /* Set cpumask to current processor */
+ gic_set_irq_properties(desc, cpumask_of(smp_processor_id()), priority);
+
+ /*
+ * Enable LPI by default. Each pLPI is enabled and routed
+ * when device is assigned.
+ */
+ desc->handler->enable(desc);
+
+ return 0;
+}
+
/* This function only works with SPIs for now */
int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
struct irq_desc *desc)
@@ -663,7 +691,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
/* Reading IRQ will ACK it */
irq = gic_hw_ops->read_irq();
- if ( likely(irq >= 16 && irq < 1020) )
+ if ( likely((irq >= 16 && irq < 1020) || gic_is_lpi(irq)) )
{
local_irq_enable();
do_IRQ(regs, irq, is_fiq);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 32c3b53..c258ab4 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -221,7 +221,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
* which interrupt is which (messes up the interrupt freeing
* logic etc).
*/
- if ( irq >= nr_irqs )
+ if ( !gic_is_valid_irq(irq) )
return -EINVAL;
if ( !handler )
return -EINVAL;
@@ -279,11 +279,16 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
set_bit(_IRQ_INPROGRESS, &desc->status);
- /*
- * The irq cannot be a PPI, we only support delivery of SPIs to
- * guests.
- */
- vgic_vcpu_inject_spi(info->d, info->virq);
+#ifdef HAS_GICV3
+ if ( gic_is_lpi(irq) )
+ vgic_vcpu_raise_lpi(info->d, desc);
+ else
+#endif
+ /*
+ * The irq cannot be a PPI, we only support delivery of SPIs to
+ * guests
+ */
+ vgic_vcpu_inject_spi(info->d, info->virq);
goto out_no_end;
}
@@ -368,6 +373,9 @@ void release_irq(unsigned int irq, const void *dev_id)
/* Wait to make sure it's not being used on another CPU */
do { smp_mb(); } while ( test_bit(_IRQ_INPROGRESS, &desc->status) );
+ if ( gic_is_lpi(irq) )
+ xfree(desc->msi_desc);
+
if ( action->free_on_release )
xfree(action);
}
@@ -449,9 +457,97 @@ err:
bool_t is_assignable_irq(unsigned int irq)
{
- /* For now, we can only route SPIs to the guest */
- return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
+ /* For now, we can only route SPI/LPIs to the guest */
+ return (((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines())) ||
+ gic_is_lpi(irq));
+}
+
+#ifdef HAS_GICV3
+/*
+ * Route an LPI to a specific guest.
+ */
+int route_lpi_to_guest(struct domain *d, unsigned int plpi, const char *devname)
+{
+ struct irqaction *action;
+ struct irq_guest *info;
+ struct irq_desc *desc;
+ unsigned long flags;
+ int retval = 0;
+
+ if ( !gic_is_lpi(plpi) )
+ {
+ printk(XENLOG_G_ERR "Only LPI can be routed \n");
+ return -EINVAL;
+ }
+
+ action = xmalloc(struct irqaction);
+ if ( !action )
+ return -ENOMEM;
+
+ info = xmalloc(struct irq_guest);
+ if ( !info )
+ {
+ xfree(action);
+ return -ENOMEM;
+ }
+ info->d = d;
+
+ action->dev_id = info;
+ action->name = devname;
+ action->free_on_release = 1;
+
+ desc = irq_to_desc(plpi);
+ spin_lock_irqsave(&desc->lock, flags);
+
+ ASSERT(desc->msi_desc != NULL);
+
+ if ( desc->arch.type == DT_IRQ_TYPE_INVALID )
+ {
+ printk(XENLOG_G_ERR "LPI %u has not been configured\n", plpi);
+ retval = -EIO;
+ goto out;
+ }
+
+ /* If the IRQ is already used by same domain, do not setup again.*/
+ if ( desc->action != NULL )
+ {
+ struct domain *ad = irq_get_domain(desc);
+
+ if ( test_bit(_IRQ_GUEST, &desc->status) && d == ad )
+ {
+ printk(XENLOG_G_ERR
+ "d%u: LPI %u is already assigned to domain %u\n",
+ d->domain_id, plpi, d->domain_id);
+ retval = -EBUSY;
+ goto out;
+ }
+ }
+
+ retval = __setup_irq(desc, 0, action);
+ if ( retval )
+ goto out;
+
+ retval = gic_route_lpi_to_guest(d, desc, GIC_PRI_IRQ);
+
+ spin_unlock_irqrestore(&desc->lock, flags);
+
+ if ( retval )
+ {
+ release_irq(desc->irq, info);
+ goto free_info;
+ }
+
+ return 0;
+
+out:
+ spin_unlock_irqrestore(&desc->lock, flags);
+ xfree(action);
+free_info:
+ xfree(info);
+
+ return retval;
}
+#endif
/*
* Route an IRQ to a specific guest.
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 33b6e2e..4f907d2 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -326,7 +326,7 @@ static int vits_process_int(struct vcpu *v, struct vgic_its *vits,
DPRINTK("%pv: vITS: INT: Device 0x%"PRIx32" id %"PRIu32"\n",
v, dev_id, event);
- /* TODO: Inject LPI */
+ vgic_vcpu_inject_lpi(v->domain, dev_id, event);
return 0;
}
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index c631c7b..4fd9c54 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1420,14 +1420,44 @@ static const struct mmio_handler_ops vgic_distr_mmio_handler = {
static int vgic_v3_get_irq_priority(struct vcpu *v, unsigned int irq)
{
- int priority;
+ int priority = 0;
unsigned long flags;
- struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
+ struct vgic_irq_rank *rank;
- vgic_lock_rank(v, rank, flags);
- priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
+ if ( !gic_is_lpi(irq) )
+ {
+ rank = vgic_rank_irq(v, irq);
+
+ vgic_lock_rank(v, rank, flags);
+ priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
irq, DABT_WORD)], 0, irq & 0x3);
- vgic_unlock_rank(v, rank, flags);
+ vgic_unlock_rank(v, rank, flags);
+ }
+ else if ( vgic_is_domain_lpi(v->domain, irq) )
+ {
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+
+ /*
+ * Guest can receive LPI before availability of LPI property table.
+ * Hence assert is wrong.
+ * TODO: Handle LPI which is valid and does not have LPI property
+ * table entry and remove below assert.
+ */
+ ASSERT(irq < vits->prop_size);
+
+ spin_lock_irqsave(&vits->prop_lock, flags);
+ /*
+ * LPI property table always starts from 0 (==8192 LPI).
+ * So, subtract 8192 from irq value.
+ */
+ priority = *((u8*)vits->prop_page + irq - FIRST_GIC_LPI);
+ /*
+ * Bits[7:2] specify priority with bits[1:0] of priority
+ * is set to zero. Hence only mask bits[7:2]
+ */
+ priority &= LPI_PRIORITY_MASK;
+ spin_unlock_irqrestore(&vits->prop_lock, flags);
+ }
return priority;
}
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 160a873..ca9ecf0 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -31,7 +31,9 @@
#include <asm/mmio.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
+#include <asm/vits.h>
static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
{
@@ -489,6 +491,69 @@ void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
vgic_vcpu_inject_irq(v, virq);
}
+#ifdef HAS_GICV3
+void vgic_vcpu_inject_lpi(struct domain *d, unsigned int vdevid,
+ unsigned int eventID)
+{
+ struct vdevice_table dt_entry;
+ struct vitt vitt_entry;
+ uint32_t col_id;
+
+ if ( vits_get_vdevice_entry(d, vdevid, &dt_entry) )
+ {
+ dprintk(XENLOG_WARNING,
+ "Failed to read dt entry for dev 0x%"PRIx32" ..dropping\n",
+ vdevid);
+ return;
+ }
+
+ if ( dt_entry.vitt_ipa == INVALID_PADDR )
+ {
+ dprintk(XENLOG_WARNING,
+ "Event %"PRId32" of dev 0x%"PRIx32" is invalid..dropping\n",
+ eventID, vdevid);
+ return;
+ }
+
+ if ( vits_get_vitt_entry(d, vdevid, eventID, &vitt_entry) )
+ {
+ dprintk(XENLOG_WARNING,
+ "Event %"PRId32" of dev 0x%"PRIx32" is invalid..dropping\n",
+ eventID, vdevid);
+ return;
+ }
+
+ col_id = vitt_entry.vcollection;
+
+ if ( !vitt_entry.valid || !is_valid_collection(d, col_id) ||
+ !vgic_is_domain_lpi(d, vitt_entry.vlpi) )
+ {
+ dprintk(XENLOG_WARNING,
+ "vlpi %"PRId32" for dev 0x%"PRIx32" is not valid..dropping\n",
+ vitt_entry.vlpi, vdevid);
+ return;
+ }
+
+ /*
+ * We don't have vlpi to plpi mapping and hence we cannot
+ * have target on which corresponding vlpi is enabled.
+ * So for now we are always injecting vlpi on vcpu0.
+ * (See vgic_vcpu_inject_lpi() function) and so we get pending_irq
+ * structure on vcpu0.
+ * TODO: Get correct target vcpu
+ */
+ vgic_vcpu_inject_irq(d->vcpu[0], vitt_entry.vlpi);
+}
+
+void vgic_vcpu_raise_lpi(struct domain *d, struct irq_desc *desc)
+{
+ struct its_device *dev = irqdesc_get_its_device(desc);
+ unsigned int eventID = irqdesc_get_lpi_event(desc);
+
+ vgic_vcpu_inject_lpi(d, dev->virt_device_id, eventID);
+}
+#endif
+
void arch_evtchn_inject(struct vcpu *v)
{
vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 8f3d0fe..52ced48 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -296,6 +296,9 @@ int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its);
int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid);
+void its_set_lpi_properties(struct irq_desc *desc,
+ const cpumask_t *cpu_mask,
+ unsigned int priority);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 2073759..b6b3a4f 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -225,6 +225,9 @@ extern void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mas
extern int gic_route_irq_to_guest(struct domain *, unsigned int virq,
struct irq_desc *desc,
unsigned int priority);
+extern int gic_route_lpi_to_guest(struct domain *d,
+ struct irq_desc *desc,
+ unsigned int priority);
/* Remove an IRQ passthrough to a guest */
int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
@@ -281,9 +284,10 @@ extern void send_SGI_allbutself(enum gic_sgi sgi);
/* print useful debug info */
extern void gic_dump_info(struct vcpu *v);
-/* Number of interrupt lines */
+/* Number of interrupt lines (PPIs + SGIs + SPIs)*/
extern unsigned int gic_number_lines(void);
-
+/* Check if irq is valid */
+bool_t gic_is_valid_irq(unsigned int irq);
/* IRQ translation function for the device tree */
int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
unsigned int *out_hwirq, unsigned int *out_type);
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index c009a5e..25ed272 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -52,6 +52,8 @@ bool_t is_assignable_irq(unsigned int irq);
void init_IRQ(void);
void init_secondary_IRQ(void);
+int route_lpi_to_guest(struct domain *d, unsigned int irq,
+ const char *devname);
int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned int irq, const char *devname);
int release_guest_irq(struct domain *d, unsigned int irq);
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 5ed840c..701556c 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -316,6 +316,9 @@ extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
+extern void vgic_vcpu_inject_lpi(struct domain *d, unsigned int devid,
+ unsigned int eventID);
+extern void vgic_vcpu_raise_lpi(struct domain *d, struct irq_desc *desc);
extern void vgic_clear_pending_irqs(struct vcpu *v);
extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
extern struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 25/28] xen/arm: ITS: Add domain specific ITS initialization
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (23 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 24/28] xen/arm: ITS: Route LPIs vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-18 13:09 ` [PATCH v7 26/28] xen/arm: ITS: Map ITS translation space vijay.kilari
` (3 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Call domain specific ITS initialization and introduce
callback in vgic for domain free
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Deprecate use of gic_lpi_supported and instead use
vgic_v3_hw.lpi_support
v6: - Moved vits_domain_free() out of this patch
---
xen/arch/arm/vgic-v3.c | 12 +++++++++++-
xen/arch/arm/vgic.c | 3 +++
xen/include/asm-arm/vgic.h | 2 ++
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 4fd9c54..b85b6ee 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1603,14 +1603,24 @@ static int vgic_v3_domain_init(struct domain *d)
d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
if ( is_hardware_domain(d) && vgic_v3_hw.lpi_support )
- d->arch.vgic.its_enabled = 1;
+ {
+ if ( !vits_domain_init(d) )
+ d->arch.vgic.its_enabled = 1;
+ }
return 0;
}
+void vgic_v3_domain_free(struct domain *d)
+{
+ if ( is_hardware_domain(d) && vgic_v3_hw.lpi_support )
+ vits_domain_free(d);
+}
+
static const struct vgic_ops v3_ops = {
.vcpu_init = vgic_v3_vcpu_init,
.domain_init = vgic_v3_domain_init,
+ .domain_free = vgic_v3_domain_free,
.get_irq_priority = vgic_v3_get_irq_priority,
.get_target_vcpu = vgic_v3_get_target_vcpu,
.emulate_sysreg = vgic_v3_emulate_sysreg,
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index ca9ecf0..f10c2cd 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -174,6 +174,9 @@ void domain_vgic_free(struct domain *d)
xfree(d->arch.vgic.shared_irqs);
xfree(d->arch.vgic.pending_irqs);
xfree(d->arch.vgic.allocated_irqs);
+
+ if ( d->arch.vgic.handler->domain_free )
+ d->arch.vgic.handler->domain_free(d);
}
int vcpu_vgic_init(struct vcpu *v)
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 701556c..9fc261c 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -115,6 +115,8 @@ struct vgic_ops {
int (*vcpu_init)(struct vcpu *v);
/* Domain specific initialization of vGIC */
int (*domain_init)(struct domain *d);
+ /* Free domain specific resources */
+ void (*domain_free)(struct domain *d);
/* Get priority for a given irq stored in vgic structure */
int (*get_irq_priority)(struct vcpu *v, unsigned int irq);
/* Get the target vcpu for a given virq. The rank lock is already taken
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 26/28] xen/arm: ITS: Map ITS translation space
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (24 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 25/28] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-18 13:09 ` [PATCH v7 27/28] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
` (2 subsequent siblings)
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
ITS translation space contains GITS_TRANSLATER register
which is written by device to raise LPI. This space needs
to mapped to every domain address space for all physical
ITS available,so that device can access GITS_TRANSLATER
register using SMMU.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v7: - Added comment on allowing CPU to access GITS_TRANSLATER
v6: - corrected typo in commit message
- Removed check for dom0
---
xen/arch/arm/vgic-v3-its.c | 49 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 48 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 4f907d2..042592f 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -828,6 +828,53 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
.write_handler = vgic_v3_gits_mmio_write,
};
+/*
+ * Map the 64K ITS translation space in guest.
+ * This is required purely for device smmu writes.
+*/
+
+static int vits_map_translation_space(struct domain *d)
+{
+ uint64_t addr, size;
+ int ret;
+
+ ASSERT(is_domain_direct_mapped(d));
+
+ addr = d->arch.vgic.vits->gits_base + SZ_64K;
+ size = SZ_64K;
+
+ /* For DomU translation space is mapped 1:1.
+ *
+ * As per spec IHI0069A, 8.1.3 there is undefined behavior when CPU
+ * writes to this register with wrong access size.
+ * Currently the page table are shared between the processor and the SMMU,
+ * So that means that a domain will be able to deadlock the processor
+ * and therefore the whole platform.
+ *
+ * TODO: A CPU should *never* be able to write to the GITS_TRANSLATER
+ * register. We have to make sure a guest cannot directly write to the HW.
+ * So we should never expose GITS_TRANSLATER into the processor page table.
+ * Which means we should not share page tables between the processor
+ * and the SMMU
+ *
+ * TODO: Also Handle DomU mapping
+ */
+ ret = map_mmio_regions(d,
+ paddr_to_pfn(addr & PAGE_MASK),
+ DIV_ROUND_UP(size, PAGE_SIZE),
+ paddr_to_pfn(addr & PAGE_MASK));
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "vITS: Unable to map to"
+ " 0x%"PRIx64" - 0x%"PRIx64" to dom%d\n",
+ addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1,
+ d->domain_id);
+ }
+
+ return ret;
+}
+
int vits_domain_init(struct domain *d)
{
struct vgic_its *vits;
@@ -882,7 +929,7 @@ int vits_domain_init(struct domain *d)
register_mmio_handler(d, &vgic_gits_mmio_handler, vits->gits_base, SZ_64K);
- return 0;
+ return vits_map_translation_space(d);
}
void vits_domain_free(struct domain *d)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 27/28] xen/arm: ITS: Generate ITS node for Dom0
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (25 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 26/28] xen/arm: ITS: Map ITS translation space vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-18 13:09 ` [PATCH v7 28/28] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
2015-09-18 13:51 ` [PATCH v7 00/28] Add ITS support Julien Grall
28 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Parse host dt and generate ITS node for Dom0.
ITS node resides inside GIC node so when GIC node
is encountered look for ITS node.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v7: - Check on return value
v6: - Introduced get_its_phandle in gic_hw_ops
- Removed make_hwdom_its_dt_node callback in gic_hw_ops
- Renamed gic_get_msi_handle() as gic_update_msi_phandle()
- Renamed its_get_phandle to its_update_phandle()
- ITS node generation is wrapped inside GICv3
- Update msi-parent phandle only if msi-parent is ITS
and find its phandle on demand
v5: - Moved ITS dt node generation to ITS driver
v4: - Generate only one ITS node for Dom0
- Replace msi-parent references to single its phandle
---
xen/arch/arm/domain_build.c | 14 +++++
xen/arch/arm/gic-v3-its.c | 120 +++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 17 +++++-
xen/arch/arm/gic.c | 8 +++
xen/include/asm-arm/gic-its.h | 2 +
xen/include/asm-arm/gic.h | 3 ++
6 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index a059de6..5119f47 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -469,6 +469,20 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
continue;
}
+ /*
+ * Replace all msi-parent phandle references to single ITS node
+ * generated for Dom0
+ */
+ if ( dt_property_name_is_equal(prop, "msi-parent") )
+ {
+ DPRINT(" Set msi-parent with ITS phandle (if ITS available)\n");
+ res = gic_update_msi_phandle(kinfo->fdt, prop);
+ if ( res )
+ return res;
+
+ continue;
+ }
+
res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
xfree(new_data);
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 891a5e2..2b3cbe1 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -27,6 +27,8 @@
#include <xen/sched.h>
#include <xen/errno.h>
#include <xen/delay.h>
+#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
#include <xen/list.h>
#include <xen/sizes.h>
#include <xen/vmap.h>
@@ -1298,6 +1300,124 @@ static void its_cpu_init_collection(void)
spin_unlock(&its_lock);
}
+int its_make_dt_node(const struct domain *d, void *fdt)
+{
+ struct its_node *its;
+ const struct dt_device_node *node;
+ const void *compatible = NULL;
+ u32 len;
+ __be32 *new_cells, *tmp;
+ int res = 0;
+
+ /* Will pass only first ITS node info */
+ its = list_first_entry(&its_nodes, struct its_node, entry);
+ if ( !its )
+ {
+ dprintk(XENLOG_ERR, "ITS node not found\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ node = its->dt_node;
+
+ compatible = dt_get_property(node, "compatible", &len);
+ if ( !compatible )
+ {
+ dprintk(XENLOG_ERR, "Can't find compatible property for the its node\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ res = fdt_begin_node(fdt, "gic-its");
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "compatible", compatible, len);
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "msi-controller", NULL, 0);
+ if ( res )
+ return res;
+
+ len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+
+ new_cells = xzalloc_bytes(len);
+ if ( new_cells == NULL )
+ return -FDT_ERR_XEN(ENOMEM);
+ tmp = new_cells;
+
+ dt_set_range(&tmp, node, its->phys_base, its->phys_size);
+
+ res = fdt_property(fdt, "reg", new_cells, len);
+ xfree(new_cells);
+ if ( res )
+ return res;
+
+ if ( node->phandle )
+ {
+ res = fdt_property_cell(fdt, "phandle", node->phandle);
+ if ( res )
+ return res;
+ }
+
+ res = fdt_end_node(fdt);
+
+ return res;
+}
+
+static int its_find_compatible_phandle(fdt32_t msi_parent_phandle)
+{
+ struct its_node *its;
+ const struct dt_device_node *node;
+ fdt32_t phandle;
+
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ node = its->dt_node;
+
+ if ( node->phandle )
+ {
+ phandle = cpu_to_fdt32(node->phandle);
+ if ( phandle == msi_parent_phandle )
+ return 0;
+ }
+ }
+
+ return -FDT_ERR_XEN(ENOENT);
+}
+
+int its_update_phandle(void *fdt, const struct dt_property *prop)
+{
+ struct its_node *its;
+ const struct dt_device_node *node;
+ fdt32_t phandle, msi_parent_phandle;
+
+ memcpy(&msi_parent_phandle, prop->value, sizeof(msi_parent_phandle));
+
+ /* chech if msi-parent phandle is ITS */
+ if ( its_find_compatible_phandle(msi_parent_phandle) )
+ return -FDT_ERR_XEN(ENOENT);
+
+ /* Only first ITS node phandle is considered */
+ its = list_first_entry(&its_nodes, struct its_node, entry);
+ if ( !its )
+ {
+ dprintk(XENLOG_ERR, "ITS node not found\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ node = its->dt_node;
+
+ if ( node->phandle )
+ {
+ phandle = cpu_to_fdt32(node->phandle);
+ fdt_property(fdt, prop->name, (void *)&phandle, sizeof(phandle));
+
+ return 0;
+ }
+
+ return -FDT_ERR_XEN(ENOENT);
+}
+
static int its_force_quiescent(void __iomem *base)
{
u32 count = 1000000; /* 1s */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 2cdc065..b1977b0 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1105,6 +1105,14 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
spin_unlock(&gicv3.lock);
}
+static int gicv3_update_its_phandle(void *fdt, const struct dt_property *prop)
+{
+ if ( its_enabled )
+ return its_update_phandle(fdt, prop);
+
+ return 0;
+}
+
static int gicv3_make_hwdom_dt_node(const struct domain *d,
const struct dt_device_node *node,
void *fdt)
@@ -1126,6 +1134,10 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
if ( res )
return res;
+ res = fdt_property(fdt, "ranges", NULL, 0);
+ if ( res )
+ return res;
+
res = fdt_property_cell(fdt, "redistributor-stride",
d->arch.vgic.rdist_stride);
if ( res )
@@ -1157,8 +1169,10 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
res = fdt_property(fdt, "reg", new_cells, len);
xfree(new_cells);
+ if ( res )
+ return res;
- return res;
+ return its_make_dt_node(d, fdt);
}
static const hw_irq_controller gicv3_host_irq_type = {
@@ -1384,6 +1398,7 @@ static const struct gic_hw_operations gicv3_ops = {
.read_vmcr_priority = gicv3_read_vmcr_priority,
.read_apr = gicv3_read_apr,
.secondary_init = gicv3_secondary_cpu_init,
+ .update_its_phandle = gicv3_update_its_phandle,
.make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
};
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index ff24bb1..072382e 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -749,6 +749,14 @@ void __cpuinit init_maintenance_interrupt(void)
"irq-maintenance", NULL);
}
+int gic_update_msi_phandle(void *fdt, const struct dt_property *prop)
+{
+ if ( gic_hw_ops->update_its_phandle != NULL )
+ return gic_hw_ops->update_its_phandle(fdt, prop);
+
+ return 0;
+}
+
int gic_make_hwdom_dt_node(const struct domain *d,
const struct dt_device_node *node,
void *fdt)
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 52ced48..d27b52c 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -299,6 +299,8 @@ int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid);
void its_set_lpi_properties(struct irq_desc *desc,
const cpumask_t *cpu_mask,
unsigned int priority);
+int its_update_phandle(void *fdt, const struct dt_property *prop);
+int its_make_dt_node(const struct domain *d, void *fdt);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index b6b3a4f..4ca00ca 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -151,6 +151,7 @@
#ifndef __ASSEMBLY__
#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
#include <xen/irq.h>
#include <asm-arm/vgic.h>
@@ -360,6 +361,7 @@ struct gic_hw_operations {
int (*secondary_init)(void);
int (*make_hwdom_dt_node)(const struct domain *d,
const struct dt_device_node *node, void *fdt);
+ int (*update_its_phandle)(void *fdt, const struct dt_property *prop);
};
void register_gic_ops(const struct gic_hw_operations *ops);
@@ -368,6 +370,7 @@ int gic_make_hwdom_dt_node(const struct domain *d,
void *fdt);
unsigned int gic_nr_irq_ids(void);
bool_t gic_is_lpi(unsigned int irq);
+int gic_update_msi_phandle(void *fdt, const struct dt_property *prop);
#endif /* __ASSEMBLY__ */
#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v7 28/28] xen/arm: ITS: Add pci devices in ThunderX
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (26 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 27/28] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
@ 2015-09-18 13:09 ` vijay.kilari
2015-09-22 15:09 ` Julien Grall
2015-09-18 13:51 ` [PATCH v7 00/28] Add ITS support Julien Grall
28 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-09-18 13:09 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
ITS initialization required for all PCI devices in
ThunderX platform are done by calling from specific
mapping function.
This patch can be reverted once XEN PCI passthrough
framework for arm64 is in available.
For now all the PCI devices are assigned to Dom0
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v7: Calculate size of bdf[] instead of hardcoding count
---
xen/arch/arm/platforms/Makefile | 1 +
xen/arch/arm/platforms/thunderx.c | 200 +++++++++++++++++++++++++++++++++++++
2 files changed, 201 insertions(+)
diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makefile
index e173fec..d9f98f9 100644
--- a/xen/arch/arm/platforms/Makefile
+++ b/xen/arch/arm/platforms/Makefile
@@ -7,3 +7,4 @@ obj-$(CONFIG_ARM_32) += sunxi.o
obj-$(CONFIG_ARM_32) += rcar2.o
obj-$(CONFIG_ARM_64) += seattle.o
obj-$(CONFIG_ARM_64) += xgene-storm.o
+obj-$(CONFIG_ARM_64) += thunderx.o
diff --git a/xen/arch/arm/platforms/thunderx.c b/xen/arch/arm/platforms/thunderx.c
new file mode 100644
index 0000000..1557d98
--- /dev/null
+++ b/xen/arch/arm/platforms/thunderx.c
@@ -0,0 +1,200 @@
+/*
+ * xen/arch/arm/platforms/thunderx.c
+ *
+ * Cavium Thunder specific settings
+ *
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ * Copyright (c) 2015 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/config.h>
+#include <asm/platform.h>
+#include <asm/gic-its.h>
+
+struct pci_dev_list
+{
+ uint32_t seg;
+ uint32_t bus;
+ uint32_t dev;
+ uint32_t func;
+};
+
+static struct pci_dev_list bdf[] =
+{
+ {0, 0, 1, 0},
+ {0, 0, 2, 0},
+ {0, 0, 6, 0},
+ {0, 0, 7, 0},
+ {0, 0, 10, 0},
+ {0, 0, 11, 0},
+ {0, 0, 14, 0},
+ {0, 0, 15, 0},
+ {0, 0, 16, 0},
+ {0, 1, 0, 0},
+ {0, 1, 0, 1},
+ {0, 1, 0, 5},
+ {0, 1, 1, 4},
+ {0, 1, 9, 0},
+ {0, 1, 9, 1},
+ {0, 1, 9, 2},
+ {0, 1, 9, 3},
+ {0, 1, 9, 4},
+ {0, 1, 9, 5},
+ {0, 1, 10, 0},
+ {0, 1, 10, 1},
+ {0, 1, 10, 2},
+ {0, 1, 10, 3},
+ {0, 1, 14, 0},
+ {0, 1, 14, 2},
+ {0, 1, 14, 4},
+ {0, 1, 16, 0},
+ {0, 1, 16, 1},
+ {0, 2, 0, 0},
+ {0, 3, 0, 0},
+ {0, 4, 0, 0},
+ {1, 0, 1, 0},
+ {1, 0, 4, 0},
+ {1, 0, 5, 0},
+ {1, 0, 6, 0},
+ {1, 0, 7, 0},
+ {1, 0, 8, 0},
+ {1, 0, 9, 0},
+ {1, 0, 10, 0},
+ {1, 0, 11, 0},
+ {2, 0, 1, 0},
+ {2, 0, 2, 0},
+ {2, 0, 3, 0},
+ {2, 1, 0, 0},
+ {2, 1, 0, 1},
+ {2, 1, 0, 2},
+ {2, 1, 0, 3},
+ {2, 1, 0, 4},
+ {2, 1, 0, 5},
+ {2, 1, 0, 6},
+ {2, 1, 0, 7},
+ {2, 1, 1, 0},
+ {2, 1, 1, 1},
+ {2, 1, 1, 2},
+ {2, 1, 1, 3},
+ {2, 1, 1, 4},
+ {2, 1, 1, 5},
+ {2, 1, 1, 6},
+ {2, 1, 1, 7},
+ {2, 1, 2, 0},
+ {2, 1, 2, 1},
+ {2, 1, 2, 2},
+ {2, 1, 2, 3},
+ {2, 1, 2, 4},
+ {2, 1, 2, 5},
+ {2, 1, 2, 6},
+ {2, 1, 2, 7},
+ {2, 1, 3, 0},
+ {2, 1, 3, 1},
+ {2, 1, 3, 2},
+ {2, 1, 3, 3},
+ {2, 1, 3, 4},
+ {2, 1, 3, 5},
+ {2, 1, 3, 6},
+ {2, 1, 3, 7},
+ {2, 1, 4, 0},
+ {2, 1, 4, 1},
+ {2, 1, 4, 2},
+ {2, 1, 4, 3},
+ {2, 1, 4, 4},
+ {2, 1, 4, 5},
+ {2, 1, 4, 6},
+ {2, 1, 4, 7},
+ {2, 1, 5, 0},
+ {2, 1, 5, 1},
+ {2, 1, 5, 2},
+ {2, 1, 5, 3},
+ {2, 1, 5, 4},
+ {2, 1, 5, 5},
+ {2, 1, 5, 6},
+ {2, 1, 5, 7},
+ {2, 1, 6, 0},
+ {2, 1, 6, 1},
+ {2, 1, 6, 2},
+ {2, 1, 6, 3},
+ {2, 1, 6, 4},
+ {2, 1, 6, 5},
+ {2, 1, 6, 6},
+ {2, 1, 6, 7},
+ {2, 1, 7, 0},
+ {2, 1, 7, 1},
+ {2, 1, 7, 2},
+ {2, 1, 7, 3},
+ {2, 1, 7, 4},
+ {3, 0, 1, 0},
+};
+
+#define BDF_TO_DEVID(seg, bus, dev, func) (seg << 16 | bus << 8 | dev << 3| func)
+
+/* TODO: add and assign devices using PCI framework */
+static int thunderx_specific_mapping(struct domain *d)
+{
+ struct dt_device_node *dt_its;
+ uint32_t devid, i;
+ int res;
+
+ static const struct dt_device_match its_device_ids[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+
+ for (dt_its = dt_find_matching_node(NULL, its_device_ids); dt_its;
+ dt_its = dt_find_matching_node(dt_its, its_device_ids))
+ {
+ break;
+ }
+
+ if ( dt_its == NULL )
+ {
+ dprintk(XENLOG_ERR, "ThunderX: ITS node not found to add device\n");
+ return 0;
+ }
+
+ for ( i = 0; i < ARRAY_SIZE(bdf); i++ )
+ {
+ devid = BDF_TO_DEVID(bdf[i].seg, bdf[i].bus,bdf[i].dev, bdf[i].func);
+ res = its_add_device(devid, 32, dt_its);
+ if ( res )
+ return res;
+ res = its_assign_device(d, devid, devid);
+ if ( res )
+ return res;
+ }
+
+ return 0;
+}
+
+static const char * const thunderx_dt_compat[] __initconst =
+{
+ "cavium,thunder-88xx",
+ NULL
+};
+
+PLATFORM_START(thunderx, "THUNDERX")
+ .compatible = thunderx_dt_compat,
+ .specific_mapping = thunderx_specific_mapping,
+PLATFORM_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread
* Re: [PATCH v7 00/28] Add ITS support
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
` (27 preceding siblings ...)
2015-09-18 13:09 ` [PATCH v7 28/28] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
@ 2015-09-18 13:51 ` Julien Grall
2015-09-21 6:52 ` Vijay Kilari
28 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-18 13:51 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, vijaya.kumar, manish.jaggi
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> This is based on DraftG version
> http://xenbits.xen.org/people/ianc/vits/draftG.pdf
>
> Following major features are supported
> - GICv3 ITS support for arm64 platform
> - Only Dom0 is supported. For DomU pci passthrough feature
> is required.
>
> Changes in v7:
>
> - Rebased to latest staging branch.
> - Compiled all the patches individually for both arm32 and arm64
> - Patch xen-arm-ITS-implement-hw_irq_controller-for-LPIs.patch
> split into two patches.
> - xen-arm-ITS-implement-hw_irq_controller-for-LPIs.patch and
> - xen-arm-ITS-Plumbing-hw_irq_controller-for-LPIs.patch.
> - Moved patch xen-arm-ITS-Add-GITS-registers-emulation.patch
> before vITS compilation.
> - Merged xen-arm-ITS-Enable-virtual-ITS-driver.patch with
> patch xen-arm-ITS-Export-ITS-info-to-Virtual-ITS.patch.
> - Dropped patch xen-arm-ITS-Add-32-bit-access-to-GICR_TYPER.patch.
> This is only minor change. Hence merged with
> xen-arm-ITS-Add-GICR-register-emulation.patch.
> - Dropped patch xen-arm-ITS-Introduce-helper-to-get-number-of-event-.patch
> - Dropped gic_lpi_supported helper function.
> - Changed LPI property table usage and handling.
> - Added support to pass nr_lpis from bootargs.
> - Added support to enable/disable its support from bootargs.
> - Fixed issues around freeing of its_device on error.
> - Introduced vits.h file
> - Dropped xen-dt-Handle-correctly-node-with-interrupt-map-in-d.patch
> from this series as it is already merged.
>
> Could not share code via Github. Network is very slow.
> Please, let me know if required.
One of the major reason to provide a branch is to know exactly your
baseline.
In this case, your baseline is not staging because I'm not able to apply
cleanly this branch when there was no {,v}gic change at all in the last
2 months.
I bet it's staging + my vgic series + some of your modification to this
series.
In general, you should mention what is the baseline, if you not staging,
so people can apply your series to try it and understand the changes.
So please provide a git branch with this series and required patch applied.
> Changes in v6:
>
> - Rebased to latest staging branch.
> - Compiled all the patches individually for both arm32 and arm64
> - Split the patch "xen/arm: ITS: Allocate irq descriptors for LPIs" into two.
> One for allocating LPI irq_desc and other patch for allocating pending_irq desc
> for LPIs
> - Following new patches are introduced
> 1) xen/arm: Rename NR_IRQs and vgic_num_irqs helper function
> 2) xen/arm: ITS: Introduce msi_desc for LPIs
> 3) xen/arm: Move vgic locking inside get_irq_priority callback
> 4) xen/arm: ITS: Store LPIs allocated and IRQ ID bits per domain
> 5) xen/arm: ITS: Introduce helper to get number of event IDs
> 6) xen/arm: ITS: Add virtual ITS availability check helper
> 7) xen/arm: ITS: Add 32-bit access to GICR_TYPER)
> 8) xen/arm: ITS: Allocate pending_lpi descriptors for LPIs
>
> - Based on below patch set
> http://lists.xen.org/archives/html/xen-devel/2015-08/msg00168.html
>
> Some Major TODOs:
> 1) Avoid making vits_process_cmd() static in later point of time
> 2) How to handle LPI that does not have LPI config table entry.
> 3) Enable/disable ITS to Dom0
The TODO in the middle of the changelog is a bit strange. Is the current
TODO list or the one of the previous version?
I was also expected to see in the TODO some item mentioned by Ian on
the previous version [1] such as:
C. The issue regarding what to do with physical LPIs which arrive
before the guest has mapped the corresponding event in the vits or which
is masked
Regards,
[1]
http://lists.xenproject.org/archives/html/xen-devel/2015-09/msg01134.html
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices
2015-09-18 13:08 ` [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
@ 2015-09-18 14:15 ` Julien Grall
2015-09-21 11:26 ` Julien Grall
1 sibling, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-18 14:15 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Helper functions to manage its devices using RB-tree
> are introduced in physical ITS driver.
>
> This is global list of all the devices.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
> Reviewed-by: Julien Grall <julien.grall@citrix.com>
> ---
> v7: - Introduce its_remove_device api to remove device
> from rb-tree
You should drop any Reviewed-by, Acked-by of a patch if you introduce
new code or fix bug.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 00/28] Add ITS support
2015-09-18 13:51 ` [PATCH v7 00/28] Add ITS support Julien Grall
@ 2015-09-21 6:52 ` Vijay Kilari
0 siblings, 0 replies; 81+ messages in thread
From: Vijay Kilari @ 2015-09-21 6:52 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Fri, Sep 18, 2015 at 7:21 PM, Julien Grall <julien.grall@citrix.com> wrote:
> On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> This is based on DraftG version
>> http://xenbits.xen.org/people/ianc/vits/draftG.pdf
>>
>> Following major features are supported
>> - GICv3 ITS support for arm64 platform
>> - Only Dom0 is supported. For DomU pci passthrough feature
>> is required.
>>
>> Changes in v7:
>>
>> - Rebased to latest staging branch.
>> - Compiled all the patches individually for both arm32 and arm64
>> - Patch xen-arm-ITS-implement-hw_irq_controller-for-LPIs.patch
>> split into two patches.
>> - xen-arm-ITS-implement-hw_irq_controller-for-LPIs.patch and
>> - xen-arm-ITS-Plumbing-hw_irq_controller-for-LPIs.patch.
>> - Moved patch xen-arm-ITS-Add-GITS-registers-emulation.patch
>> before vITS compilation.
>> - Merged xen-arm-ITS-Enable-virtual-ITS-driver.patch with
>> patch xen-arm-ITS-Export-ITS-info-to-Virtual-ITS.patch.
>> - Dropped patch xen-arm-ITS-Add-32-bit-access-to-GICR_TYPER.patch.
>> This is only minor change. Hence merged with
>> xen-arm-ITS-Add-GICR-register-emulation.patch.
>> - Dropped patch xen-arm-ITS-Introduce-helper-to-get-number-of-event-.patch
>> - Dropped gic_lpi_supported helper function.
>> - Changed LPI property table usage and handling.
>> - Added support to pass nr_lpis from bootargs.
>> - Added support to enable/disable its support from bootargs.
>> - Fixed issues around freeing of its_device on error.
>> - Introduced vits.h file
>> - Dropped xen-dt-Handle-correctly-node-with-interrupt-map-in-d.patch
>> from this series as it is already merged.
>>
>> Could not share code via Github. Network is very slow.
>> Please, let me know if required.
>
> One of the major reason to provide a branch is to know exactly your
> baseline.
>
> In this case, your baseline is not staging because I'm not able to apply
> cleanly this branch when there was no {,v}gic change at all in the last
> 2 months.
>
> I bet it's staging + my vgic series + some of your modification to this
> series.
Yes it depends on your vgic series. New branch 'staging_its_v7" is
pushed to github
https://github.com/vijaykilari/its_v6.git
>
> In general, you should mention what is the baseline, if you not staging,
> so people can apply your series to try it and understand the changes.
>
> So please provide a git branch with this series and required patch applied.
>
>> Changes in v6:
>>
>> - Rebased to latest staging branch.
>> - Compiled all the patches individually for both arm32 and arm64
>> - Split the patch "xen/arm: ITS: Allocate irq descriptors for LPIs" into two.
>> One for allocating LPI irq_desc and other patch for allocating pending_irq desc
>> for LPIs
>> - Following new patches are introduced
>> 1) xen/arm: Rename NR_IRQs and vgic_num_irqs helper function
>> 2) xen/arm: ITS: Introduce msi_desc for LPIs
>> 3) xen/arm: Move vgic locking inside get_irq_priority callback
>> 4) xen/arm: ITS: Store LPIs allocated and IRQ ID bits per domain
>> 5) xen/arm: ITS: Introduce helper to get number of event IDs
>> 6) xen/arm: ITS: Add virtual ITS availability check helper
>> 7) xen/arm: ITS: Add 32-bit access to GICR_TYPER)
>> 8) xen/arm: ITS: Allocate pending_lpi descriptors for LPIs
>>
>> - Based on below patch set
>> http://lists.xen.org/archives/html/xen-devel/2015-08/msg00168.html
>>
>> Some Major TODOs:
>> 1) Avoid making vits_process_cmd() static in later point of time
>> 2) How to handle LPI that does not have LPI config table entry.
>> 3) Enable/disable ITS to Dom0
>
> The TODO in the middle of the changelog is a bit strange. Is the current
> TODO list or the one of the previous version?
These are previous version TODOs that are under changes in v6.
>
> I was also expected to see in the TODO some item mentioned by Ian on
> the previous version [1] such as:
>
> C. The issue regarding what to do with physical LPIs which arrive
> before the guest has mapped the corresponding event in the vits or which
> is masked
yes, I will add this TODO to v7.
>
> Regards,
>
> [1]
> http://lists.xenproject.org/archives/html/xen-devel/2015-09/msg01134.html
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 04/28] xen/arm: Rename NR_IRQs and vgic_num_irqs helper function
2015-09-18 13:08 ` [PATCH v7 04/28] xen/arm: Rename NR_IRQs and vgic_num_irqs helper function vijay.kilari
@ 2015-09-21 10:40 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 10:40 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> NR_IRQS represents the number of lines (i.e SGIs, PPIs and SPIs).
> With the introduction of LPIs, NR_IRQs is renamed to NR_ITLINES.
> Similarly vgic_num_irqs() is renamed as vgic_num_irq_lines().
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen
2015-09-18 13:08 ` [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
@ 2015-09-21 11:23 ` Julien Grall
2015-09-22 9:55 ` Vijay Kilari
2015-09-22 10:34 ` Vijay Kilari
2015-09-21 13:08 ` Julien Grall
1 sibling, 2 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 11:23 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
The only things I haven't check on this patch was the ITS command structure.
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> +/* ITS command structure */
> +typedef union {
Can you please sort this union by command name in alphabetical order.
It's way easier to find a command in the list.
> + u64 bits[4];
> + struct __packed {
> + uint8_t cmd;
NIT: Please stay consistent with usage of the type. You are using u8
everywhere within this union except here.
> + uint8_t pad[7];
Why a padding of only 56 bits? Shouldn't it be 248 bits (i.e 31 * 8
bits) to fit all the command?
> + } hdr;
> + struct __packed {
> + u8 cmd;
> + u8 res1[3];
> + u32 devid;
> + u64 size:5;
> + u64 res2:59;
> + /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
> + u64 res3:8;
It's very confusing for the reviewer to see a mix of usage (u8 res1[n],
u64 res3:8) within the same structure.
The later (i.e u64 field:n) is the best to use because we can match
quickly the size with the spec.
> + u64 itt:40;
> + u64 res4:15;
> + u64 valid:1;
> + u64 res5;
> + } mapd;
[..]
> + struct __packed {
> + u8 cmd;
> + u8 res1[3];
> + u32 devid;
> + u32 event;
> + u32 res2;
> + u64 res3;
> + u64 res4;
> + } int_cmd;
Maybe you want to add the suffix _cmd to everyone to avoid having only
one because of C spec issue.
> + struct __packed {
> + u8 cmd;
> + u8 res1[3];
> + u32 devid;
> + u32 event;
> + u32 res2;
> + u64 res3;
> + u64 res4;
> + } clear;
> + struct __packed {
> + u8 cmd;
> + u8 res1[7];
> + u64 res2;
> + u16 res3;
> + u32 ta;
It would have been better to have a full name or a description rather
than only a 2 letter field "ta". I won't ask for a documentation of this
field right now, but it would be a nice follow-up of this series.
> + u16 res4;
> + u64 res5;
> + } sync;
> +} its_cmd_block;
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices
2015-09-18 13:08 ` [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
2015-09-18 14:15 ` Julien Grall
@ 2015-09-21 11:26 ` Julien Grall
1 sibling, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 11:26 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> +static void its_remove_device(struct its_device *dev)
> +{
> + if ( dev )
> + rb_erase(&dev->node, &rb_its_dev);
Either the caller or this function has to take the lock which protect
the RB-tree.
If it's the caller, please add an ASSERT to check the lock is taken. If
not, you know what to do ;).
Regards,
> +}
> +
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen
2015-09-18 13:08 ` [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
2015-09-21 11:23 ` Julien Grall
@ 2015-09-21 13:08 ` Julien Grall
1 sibling, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 13:08 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> +static void its_lpi_free(struct its_device *dev)
> +{
> + int lpi;
> +
> + spin_lock(&lpi_lock);
> +
> + for ( lpi = dev->event_map.lpi_base;
> + lpi < (dev->event_map.lpi_base + dev->event_map.nr_lpis);
> + lpi += IRQS_PER_CHUNK )
> + {
> + int chunk = its_lpi_to_chunk(lpi);
> +
> + if (chunk > lpi_chunks)
> + its_err("Bad LPI chunk %d\n", chunk);
> + if ( test_bit(chunk, lpi_bitmap) )
> + clear_bit(chunk, lpi_bitmap);
> + }
> +
> + spin_unlock(&lpi_lock);
> +
> + xfree(dev->event_map.lpi_map);
You didn't import correctly the patch which support collection for
Linux. It misses:
xfree(dev->event_map.col_map);
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device
2015-09-18 13:08 ` [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-09-21 13:47 ` Julien Grall
2015-09-22 7:33 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-21 13:47 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add APIs to add devices to RB-tree, assign and remove
> devices to domain.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v7: - Added check for domain in its_assign_device() to avoid
> assigning device to DomU
> - Added comments whereever requested
> - Called its_remove_device to remove from rb-tree
> when failed to add device
> - Changed dprintk to printk
> - Store domain pointer instead of domain id in its_device struct
> - Free msi_desc and reset irq_desc when lpis are discarded
> - Add function its_free_msi_descs() to free msi_desc
> v6: - Moved this patch #19 to patch #8
> - Used col_map to store collection id
> - Use helper functions to update msi_desc members
> v5: - Removed its_detach_device API
> - Pass nr_ites as parameter to its_add_device
> v4: - Introduced helper to populate its_device struct
> - Fixed freeing of its_device memory
> - its_device struct holds domain id
> ---
> xen/arch/arm/gic-v3-its.c | 261 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 6 +
> 2 files changed, 267 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 4b7d9ed..bc3b73c 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -145,6 +145,19 @@ static struct its_collection *dev_event_to_col(struct its_device *dev,
> return its->collections + dev->event_map.col_map[event];
> }
>
> +static struct its_node *its_get_phys_node(struct dt_device_node *dt)
> +{
> + struct its_node *its;
> +
> + list_for_each_entry(its, &its_nodes, entry)
> + {
> + if ( its->dt_node == dt )
> + return its;
> + }
> +
> + return NULL;
> +}
> +
> /* RB-tree helpers for its_device */
> static struct its_device *its_find_device(u32 devid)
> {
> @@ -537,8 +550,256 @@ static void its_lpi_free(struct its_device *dev)
> }
>
> spin_unlock(&lpi_lock);
> +}
> +
> +static void its_discard_lpis(struct its_device *dev, u32 ids)
> +{
> + int i;
u32 i;
> +
> + for ( i = 0; i < ids; i++ )
> + its_send_discard(dev, i);
> +}
> +
> +static void its_free_msi_descs(struct its_device *dev, int count)
its_free_msi_descs can be merge with its_discard_lpis. It would avoid
looping twice which can be expensive with device having a lot of MSI and
make clear the dependency.
For instance its_free_msi_descs should never be called before
its_discard_lpis.
> +{
> + int i, irq;
Why do you need them signed? Same for the parameter "count".
> + struct irq_desc *desc;
> +
> + for ( i = 0; i < count; i++ )
> + {
> + irq = dev->event_map.lpi_base + i;
You could have use its_get_plpi here.
> + desc = irq_to_desc(irq);
> +
> + spin_lock(&desc->lock);
> + irqdesc_set_lpi_event(desc, 0);
> + irqdesc_set_its_device(desc, NULL);
Why do you need these 2 calls?
> + xfree(irq_get_msi_desc(desc));
> + irq_set_msi_desc(desc, NULL);
After this, the IRQ desc is still left in an unknown state (give a look
to gic_remove_irq_from_guest).
Although, it may not need necessary for now. So I would be fine with a
TODO in the code.
> + spin_unlock(&desc->lock);
> + }
> +}
> +
> +static inline u32 its_get_plpi(struct its_device *dev, u32 event)
> +{
> + return dev->event_map.lpi_base + event;
> +}
> +
> +static int its_alloc_device_irq(struct its_device *dev, u32 *hwirq)
> +{
> + int idx;
> +
> + idx = find_first_zero_bit(dev->event_map.lpi_map, dev->event_map.nr_lpis);
> + if ( idx == dev->event_map.nr_lpis )
> + return -ENOSPC;
> +
> + *hwirq = its_get_plpi(dev, idx);
> + set_bit(idx, dev->event_map.lpi_map);
>
> + return 0;
> +}
> +
> +static void its_free_device(struct its_device *dev)
> +{
> + xfree(dev->itt);
> + its_lpi_free(dev);
> xfree(dev->event_map.lpi_map);
Why did you move this call from its_lpi_free to here?
> + xfree(dev->event_map.col_map);
This line should have been added in its_lpi_free in patch #5.
> + xfree(dev);
> +}
> +
> +static struct its_device *its_alloc_device(u32 devid, u32 nr_ites,
> + struct dt_device_node *dt_its)
Can you explain why you weren't able to re-use its_create_device from Linux?
AFAICT there is nothing different and you miss key code such as rounding
to a power of 2 the number of event IDs.
> +{
> + struct its_device *dev;
> + unsigned long *lpi_map;
> + int lpi_base, sz;
> + u16 *col_map = NULL;
> +
> + dev = xzalloc(struct its_device);
> + if ( dev == NULL )
> + return NULL;
> +
> + dev->its = its_get_phys_node(dt_its);
You can re-order the code to get the ITS before allocate the memory. And
then you can drop one goto.
> + if (dev->its == NULL)
> + {
> + printk(XENLOG_G_ERR
As already asked on v6, why XENLOG_G_ERR? Any call to this function will
be done via privileged guest.
> + "ITS: Failed to find ITS node for devid 0x%"PRIx32"\n", devid);
> + goto err;
> + }
> +
> + sz = nr_ites * dev->its->ite_size;
> + sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
> + dev->itt = xzalloc_bytes(sz);
> + if ( !dev->itt )
> + goto err;
> +
> + lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base);
> + if ( !lpi_map )
> + goto lpi_err;
> +
> + col_map = xzalloc_bytes(sizeof(*col_map) * nr_ites);
> + if ( !col_map )
> + goto col_err;
> + dev->event_map.lpi_map = lpi_map;
> + dev->event_map.lpi_base = lpi_base;
> + dev->event_map.col_map = col_map;
> + dev->event_map.nr_lpis = nr_ites;
> + dev->device_id = devid;
> +
> + return dev;
> +
> +col_err:
> + its_free_device(dev);
> + return NULL;
> +lpi_err:
> + xfree(dev->itt);
> +err:
> + xfree(dev);
> +
> + return NULL;
> +}
> +
> +/* Device assignment */
> +int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its)
> +{
> + struct its_device *dev;
> + u32 i, plpi = 0;
> + struct its_collection *col;
> + struct irq_desc *desc;
> + struct msi_desc *msi = NULL;
> + int res = 0;
> +
> + spin_lock(&rb_its_dev_lock);
> + dev = its_find_device(devid);
> + if ( dev )
> + {
> + printk(XENLOG_G_ERR "ITS: Device already exists 0x%"PRIx32"\n",
Same question as before for XENLOG_G_ERR.
> + dev->device_id);
> + res = -EEXIST;
> + goto err_unlock;
> + }
> +
> + dev = its_alloc_device(devid, nr_ites, dt_its);
> + if ( !dev )
> + {
> + res = -ENOMEM;
> + goto err_unlock;
> + }
> +
> + if ( its_insert_device(dev) )
The only way that this call can fail is because the device already
exists in the RB-tree. Although, this can never happen because you have
the lock taken and check beforehand if someone has inserted a device
with this devID.
So I would turn this if into an BUG_ON(its_insert_device(dev)) and save
6 lines.
> + {
> + its_free_device(dev);
> + printk(XENLOG_G_ERR "ITS: Failed to insert device 0x%"PRIx32"\n", devid);
> + res = -EINVAL;
> + goto err_unlock;
> + }
> + /* Assign device to dom0 by default */
This wasn't present on the previous version. Why do you need that? DOM0
is just a guest and you make the code in the assignation function (see
its_assign_device) for nothing.
IHMO, the code to add a device and assign to a guest should be
completely separate.
> + dev->domain = hardware_domain;
> + spin_unlock(&rb_its_dev_lock);
> +
> + DPRINTK("ITS:Add Device 0x%"PRIx32" lpis %"PRIu32" base 0x%"PRIx32"\n",
> + dev->device_id, dev->event_map.nr_lpis, dev->event_map.lpi_base);
> +
> + /* Map device to ITS ITT */
> + its_send_mapd(dev, 1);
> +
> + for ( i = 0; i < dev->event_map.nr_lpis; i++ )
> + {
> + msi = xzalloc(struct msi_desc);
> + if ( its_alloc_device_irq(dev, &plpi) || !msi )
> + {
> + /* Discard LPIs and free device on failure to allocate pLPI */
> + its_discard_lpis(dev, i);
> + its_free_msi_descs(dev, i);
> + its_send_mapd(dev, 0);
> +
> + spin_lock(&rb_its_dev_lock);
> + its_remove_device(dev);
> + spin_unlock(&rb_its_dev_lock);
> +
> + its_free_device(dev);
> +
> + printk(XENLOG_G_ERR "ITS: Cannot add device 0x%"PRIx32"\n", devid);
Same question as before for XENLOG_G_ERR.
> + res = -ENOSPC;
> + goto err;
> + }
> +
> + /*
> + * Each Collection is mapped to one physical CPU and
> + * each pLPI allocated to this device is mapped one collection
> + * in a roundrobin fashion. Hence all pLPIs are distributed
s/roundrobin/round robin/
> + * across all processors in the system.
> + */
> + col = &dev->its->collections[(i % nr_cpu_ids)];
Your round robin is done per device. So if we have 10 devices with a
single event ID, all the LPIs will be routed to CPU0.
I guess this is fine for now, but it worth mentioning it in the comment.
> + desc = irq_to_desc(plpi);
> +
> + spin_lock(&desc->lock);
> + dev->event_map.col_map[i] = col->col_id;
> + irq_set_msi_desc(desc, msi);
> + irqdesc_set_lpi_event(desc, i);
> + irqdesc_set_its_device(desc, dev);
While I gave my reviewed-by on the patch with add those helpers (see
#7), I'm continuing to think that they are not useful and make the usage
more difficult due the ordering requirement.
Something like below would have been better:
msi->eventID = i;
msi->dev = dev;
irq_set_msi_desc(msi);
This will also turn 5 functions call (2 for each irqdesc_* + 1 for
irq_set_msi_*) into a single one.
This code is still ok, but it would make a different in the LPI handling
code later.
> + spin_unlock(&desc->lock);
> +
> + /* For each pLPI send MAPVI command */
> + its_send_mapvi(dev, plpi, i);
> + }
> +
> + return 0;
> +
> +err_unlock:
> + spin_unlock(&rb_its_dev_lock);
> +err:
> + return res;
> +}
> +
> +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
> +{
> + struct its_device *pdev;
> + u32 plpi, i;
> +
> + DPRINTK("ITS: Assign request for dev 0x%"PRIx32" to domain %"PRIu16"\n",
> + vdevid, d->domain_id);
> +
> + spin_lock(&rb_its_dev_lock);
> + pdev = its_find_device(pdevid);
> + if ( !pdev )
> + {
> + spin_unlock(&rb_its_dev_lock);
> + return -ENODEV;
> + }
> + spin_unlock(&rb_its_dev_lock);
You can rework this code to avoid 2 spin_unlock:
spin_lock(&rb...)
pdev = its_find_device(pdevid);
spin_unlock(&rb...)
if ( !pdev )
return -ENODEV;
> +
> + /*
> + * TODO: For pass-through following has to be implemented
> + * 1) Allow device to be assigned to other domains (Dom0 -> DomU).
> + * 2) Allow device to be re-assigned to Dom0 (DomU -> Dom0).
> + * Implement separate function to handle this or rework this function.
> + * For now do not allow assigning devices other than Dom0.
> + *
> + * By default all devices are assigned to Dom0.
See my question above about the "why?".
> + * Device should be with Dom0 before assigning to other domain.
> + */
> + if ( !is_hardware_domain(d) || !is_hardware_domain(pdev->domain) )
> + {
> + printk(XENLOG_ERR
> + "ITS: PCI-Passthrough not supported!! to assign from d%d to d%d",
> + pdev->domain->domain_id, d->domain_id);
> + return -ENXIO;
> + }
> +
> + pdev->domain = d;
> + pdev->virt_device_id = vdevid;
> +
> + DPRINTK("ITS: Assign pdevid 0x%"PRIx32" lpis %"PRIu32" for dom %"PRIu16"\n",
> + pdevid, pdev->event_map.nr_lpis, d->domain_id);
> +
> + for ( i = 0; i < pdev->event_map.nr_lpis; i++ )
> + {
> + plpi = its_get_plpi(pdev, i);
> + /* TODO: Route lpi */
> + }
> +
> + return 0;
> }
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function
2015-09-18 13:08 ` [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function vijay.kilari
@ 2015-09-21 14:20 ` Julien Grall
2015-09-22 9:10 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-21 14:20 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, Zoltan Kiss, manish.jaggi
Hi Vijay,
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Helper function gic_is_lpi() is used to find
> if irq is lpi or not. For GICv2 platforms this function
> returns number of irq ids which represents only number of line irqs.
> For GICv3 platform irq ids are calculated from nr_lpis if
> HW supports LPIs
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> CC: Zoltan Kiss <zoltan.kiss@huawei.com>
> ---
> v7: - Rename gic_info.nr_id_bits as gic_info.nr_irq_ids
> - gic_is_lpi() is common for both ARM64/32
> - Introduce global variable nr_lpis from patch #17
> - Reset nr_lpis to 0 when HW does not support LPIs
> - pass nr_lpis as boot args to Xen. Set minimum to 8192
> v6: - Added gic_info.nr_id_bits to hold number of SPI/LPIs
> supported
> - Dropped is_lpi() callback
> ---
> xen/arch/arm/gic-hip04.c | 3 ++-
> xen/arch/arm/gic-v2.c | 3 ++-
> xen/arch/arm/gic-v3.c | 20 ++++++++++++++++++++
> xen/arch/arm/gic.c | 10 ++++++++++
> xen/arch/arm/irq.c | 15 +++++++++++++++
> xen/include/asm-arm/gic.h | 5 +++++
> xen/include/asm-arm/irq.h | 3 +++
> 7 files changed, 57 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/arm/gic-hip04.c b/xen/arch/arm/gic-hip04.c
> index c5ed545..398881b 100644
> --- a/xen/arch/arm/gic-hip04.c
> +++ b/xen/arch/arm/gic-hip04.c
> @@ -301,7 +301,8 @@ static void __init hip04gic_dist_init(void)
>
> /* Only 1020 interrupts are supported */
> gicv2_info.nr_lines = min(1020U, nr_lines);
> -
This empty line was useful to separate internal initialization from
enabling the distributor. Please retain the empty line.
> + /* Number of IRQ ids supported */
> + gicv2_info.nr_irq_ids = gicv2_info.nr_lines;
> /* Turn on the distributor */
> writel_gicd(GICD_CTL_ENABLE, GICD_CTLR);
> }
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 596126d..6673f29 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -291,7 +291,8 @@ static void __init gicv2_dist_init(void)
>
> /* Only 1020 interrupts are supported */
> gicv2_info.nr_lines = min(1020U, nr_lines);
> -
Ditto.
> + /* Number of IRQ ids supported */
> + gicv2_info.nr_irq_ids = nr_lines;
> /* Turn on the distributor */
> writel_gicd(GICD_CTL_ENABLE, GICD_CTLR);
> }
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 5076706..6680bd2 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -529,6 +529,11 @@ static void gicv3_set_irq_properties(struct irq_desc *desc,
> spin_unlock(&gicv3.lock);
> }
>
> +static int gicv3_dist_supports_lpis(void)
> +{
> + return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
> +}
> +
> static void __init gicv3_dist_init(void)
> {
> uint32_t type;
> @@ -578,6 +583,21 @@ static void __init gicv3_dist_init(void)
>
> /* Only 1020 interrupts are supported */
> gicv3_info.nr_lines = min(1020U, nr_lines);
> +
> + /*
> + * Number of IRQ ids supported.
> + * Here we override HW supported number of LPIs and
> + * limit to to LPIs specified in nr_lpis.
> + */
You still have to check that the number of LPIs requesting by the user
is supported by the hardware.
The user parameter can be seen as a restriction rather than a blind
overriding.
> + if ( gicv3_dist_supports_lpis() )
I'm sure we already speak about it in a previous series. The GICv3 may
support LPIs even without an ITS. Here you would claim that Xen is
supporting LPIs which is clearly not true.
When the ITS is not present, this value should be the number of
interrupt line supported.
I haven't checked if you fixed it later, but all the patch should be
bisectable. I.e I shouldn't see any unwanted modification on supported
platform with GICv3 (for instance the foundation model).
> + gicv3_info.nr_irq_ids = nr_lpis + FIRST_GIC_LPI;
> + else
> + {
> + gicv3_info.nr_irq_ids = gicv3_info.nr_lines;
> + /* LPIs are not supported by HW. Reset to 0 */
> + nr_lpis = 0;
That's really ugly and you still let GICv2 hardware with nr_lpis != 0.
As said on v6 I don't want to see any usage of nr_lpis in code except
for letting the user restricting the number of LPIs supported.
That means that all the code should use gic_nr_irq_ids to know the
number of LPIs supported. Mixing the 2 usage will lead to big trouble
latter.
So please move nr_lpis in gic-v3 as a parameter and don't export it.
> + }
> +
> }
>
> static int gicv3_enable_redist(void)
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 1d94e5e..a0ed7df 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -62,6 +62,16 @@ enum gic_version gic_hw_version(void)
> return gic_hw_ops->info->hw_version;
> }
>
> +unsigned int gic_nr_irq_ids(void)
> +{
> + return gic_hw_ops->info->nr_irq_ids;
> +}
> +
> +bool_t gic_is_lpi(unsigned int irq)
> +{
> + return (irq >= FIRST_GIC_LPI && irq < gic_nr_irq_ids());
> +}
> +
> unsigned int gic_number_lines(void)
> {
> return gic_hw_ops->info->nr_lines;
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index d582f57..3a01f46 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -31,6 +31,18 @@
> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
> static DEFINE_SPINLOCK(local_irqs_type_lock);
>
> +/* Number of LPIs supported by Xen.
> + *
> + * The LPI identifier starts from 8192. Given that the hardware is
> + * providing the number of identifier bits, supporting LPIs requires at
> + * least 14 bits. This will represent 16384 interrupt ID of which 8192
> + * LPIs. So the minimum of LPIs supported when the hardware supports LPIs
> + * is 8192.
> + */
> +#define DEFAULT_NR_LPIS 8192
> +unsigned int nr_lpis = DEFAULT_NR_LPIS;
> +integer_param("nr_lpis", nr_lpis);
> +
Any new parameter should be documented in
docs/misc/xen-command-line.markdown
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 10/28] xen/arm: ITS: Implement hw_irq_controller for LPIs
2015-09-18 13:08 ` [PATCH v7 10/28] xen/arm: ITS: Implement hw_irq_controller for LPIs vijay.kilari
@ 2015-09-21 14:35 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 14:35 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Implement hw_irq_controller callbacks required to
> handle LPIs.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
With the 3 changes requested below:
Reviewed-by: Julien Grall <julien.grall@citrix.com>
> ---
> xen/arch/arm/gic-v3-its.c | 119 +++++++++++++++++++++++++++++++++++++
> xen/arch/arm/gic-v3.c | 2 +-
> xen/include/asm-arm/gic_v3_defs.h | 2 +
> 3 files changed, 122 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index bc3b73c..0d5c61c 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -444,6 +444,125 @@ static void its_send_discard(struct its_device *dev, u32 event)
> its_send_single_command(dev->its, &cmd, col);
> }
>
> +static void its_flush_and_invalidate_prop(struct irq_desc *desc, u8 *cfg)
NIT: cfg could be const i.e
const u8 *cfg
> +{
> + struct its_device *its_dev = irqdesc_get_its_device(desc);
> + u32 vid = irqdesc_get_lpi_event(desc);
> +
> + ASSERT(vid < its_dev->event_map.nr_lpis);
> +
> + /*
> + * Make the above write visible to the redistributors.
> + * And yes, we're flushing exactly: One. Single. Byte.
> + * Humpf...
> + */
> + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
> + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
> + else
> + dsb(ishst);
> +
> + its_send_inv(its_dev, vid);
> +}
> +const hw_irq_controller its_host_lpi_type = {
As said on v6, the const is not necessary.
> + .typename = "gic-its",
> + .startup = its_irq_startup,
> + .shutdown = its_irq_shutdown,
> + .enable = its_irq_enable,
> + .disable = its_irq_disable,
> + .ack = its_irq_ack,
> + .end = its_host_irq_end,
> + .set_affinity = its_irq_set_affinity,
> +};
> +
> +const hw_irq_controller its_guest_lpi_type = {
Ditto.
> + .typename = "gic-its",
> + .startup = its_irq_startup,
> + .shutdown = its_irq_shutdown,
> + .enable = its_irq_enable,
> + .disable = its_irq_disable,
> + .ack = its_irq_ack,
> + .end = its_guest_irq_end,
> + .set_affinity = its_irq_set_affinity,
> +};
> +
> /*
> * How we allocate LPIs:
> *
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 12/28] xen/arm: ITS: Plumbing hw_irq_controller for LPIs
2015-09-18 13:08 ` [PATCH v7 12/28] xen/arm: ITS: Plumbing hw_irq_controller for LPIs vijay.kilari
@ 2015-09-21 14:44 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 14:44 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, Zoltan Kiss, manish.jaggi
Hi Vijay,
Title: I would replace "Plumbing" by "Plumb"
On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Change callbacks gic_host_irq_type and gic_guest_irq_type
> to gic_get_host_irq_type and gic_get_guest_irq_type
> in gic_hw_operations, which returns hw_irq_controller based
> on irq type (SPI or LPI).
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
With the title and 2 remarks below fixed:
Reviewed-by: Julien Grall <julien.grall@citrix.com>
[...]
> +static inline hw_irq_controller *get_host_hw_irq_controller(unsigned int irq)
> +{
> + return gic_hw_ops->gic_get_host_irq_type(irq);
> +}
> +
> +static inline hw_irq_controller *get_guest_hw_irq_controller(unsigned int irq)
> +{
> + return gic_hw_ops->gic_get_guest_irq_type(irq);
> +}
> +
Each of these helpers are used in a single call-site and doesn't bring
much improvement. I would prefer to see those two helpers dropped in
favor of open-coding the call to the callback.
> /*
> * needs to be called with a valid cpu_mask, ie each cpu in the mask has
> * already called gic_cpu_init
> @@ -128,7 +138,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
> ASSERT(spin_is_locked(&desc->lock));
>
> - desc->handler = gic_hw_ops->gic_host_irq_type;
> + desc->handler = get_host_hw_irq_controller(desc->irq);
> gic_set_irq_properties(desc, cpu_mask, priority);
> }
> @@ -159,7 +169,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> goto out;
>
> - desc->handler = gic_hw_ops->gic_guest_irq_type;
> + desc->handler = get_guest_hw_irq_controller(desc->irq);
> set_bit(_IRQ_GUEST, &desc->status);
>
> gic_set_irq_properties(desc, cpumask_of(v_target->processor), priority);
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index f5fba49..4455178 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -275,6 +275,8 @@ struct its_device {
> struct rb_node node;
> };
>
> +extern const hw_irq_controller its_host_lpi_type;
> +extern const hw_irq_controller its_guest_lpi_type;
Newline here.
> void irqdesc_set_lpi_event(struct irq_desc *desc, unsigned id);
> unsigned int irqdesc_get_lpi_event(struct irq_desc *desc);
> struct its_device *irqdesc_get_its_device(struct irq_desc *desc);
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 13/28] xen/arm: Move vgic rank locking inside get_irq_priority
2015-09-18 13:09 ` [PATCH v7 13/28] xen/arm: Move vgic rank locking inside get_irq_priority vijay.kilari
@ 2015-09-21 14:49 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 14:49 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Move vgic rank locking inside get_irq_priority callback.
> LPIs does not have vgic rank lock for reading LPI priority.
> So make generic vgic code cleaner.
The commit message is not clear. I would rewrite like that:
"The LPIs will require a different locking scheme to retrieve the priority.
So move the vGIC rank locking inside the callback get_irq_priority."
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Assuming that the suggested commit message is fine for you:
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-18 13:09 ` [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support vijay.kilari
@ 2015-09-21 15:20 ` Julien Grall
2015-09-22 9:17 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-21 15:20 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Initialize physical ITS if HW supports LPIs.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v7: - Export lpi support information to vgic-v3 driver from gic-v3.
> - Drop gic_lpi_supported() helper function
> - Add boot param to enable or disable physical ITS
> v6: - Updated lpi_supported gic_info member for GICv2 and GICv3
> - Introduced helper gic_lpi_supported() and exported
> v5: - Made check of its dt node availability before
> setting lpi_supported flag
> ---
> xen/arch/arm/gic-v3.c | 38 ++++++++++++++++++++++++++++++++++---
> xen/arch/arm/vgic-v3.c | 5 ++++-
> xen/include/asm-arm/gic_v3_defs.h | 4 +++-
> xen/include/asm-arm/vgic.h | 2 +-
> 4 files changed, 43 insertions(+), 6 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index c4c77a7..ac8a0ea 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -55,6 +55,18 @@ static struct {
> } gicv3;
>
> static struct gic_info gicv3_info;
> +/* Enable/Disable ITS support */
> +static bool_t its_enable = 1;
> +/* Availability of ITS support after successful ITS initialization */
> +static bool_t its_enabled = 0;
> +
> +static void __init parse_its_param(char *s)
> +{
> + if ( !parse_bool(s) )
> + its_enable = 0;
> +}
> +
> +custom_param("its", parse_its_param);
Why do you need a command line option to enable/disable the physical ITS?
>
> /* per-cpu re-distributor base */
> DEFINE_PER_CPU(struct rdist, rdist);
> @@ -590,7 +602,7 @@ static void __init gicv3_dist_init(void)
> * Here we override HW supported number of LPIs and
> * limit to to LPIs specified in nr_lpis.
> */
> - if ( gicv3_dist_supports_lpis() )
> + if ( its_enabled && gicv3_dist_supports_lpis() )
> gicv3_info.nr_irq_ids = nr_lpis + FIRST_GIC_LPI;
> else
> {
> @@ -714,6 +726,10 @@ static int __cpuinit gicv3_cpu_init(void)
> if ( gicv3_enable_redist() )
> return -ENODEV;
>
> + /* Give LPIs a spin */
> + if ( its_enabled && gicv3_dist_supports_lpis() )
If the ITS is not enabled, the list of ITS nodes will be empty and
therefore its_cpu_init will return directly.
So its_enabled is not necessary.
> + its_cpu_init();
> +
> /* Set priority on PPI and SGI interrupts */
> priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
> GIC_PRI_IPI);
> @@ -1303,14 +1319,30 @@ static int __init gicv3_init(void)
> i, r->base, r->base + r->size);
> }
>
> - vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions,
> - gicv3.rdist_stride);
> + reg = readl_relaxed(GICD + GICD_TYPER);
> +
> + gicv3.rdist_data.id_bits = ((reg >> GICD_TYPE_ID_BITS_SHIFT) &
> + GICD_TYPE_ID_BITS_MASK) + 1;
> +
> gicv3_init_v2(node, dbase);
>
> spin_lock_init(&gicv3.lock);
>
> spin_lock(&gicv3.lock);
>
> + if ( its_enable && gicv3_dist_supports_lpis() )
> + {
> + /*
> + * LPI support is enabled only if HW supports it and
> + * ITS dt node is available
> + */
> + if ( !its_init(&gicv3.rdist_data) )
> + its_enabled = 1;
> + }
> +
> + vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions,
> + gicv3.rdist_stride, its_enabled);
> +
Why do you need to execute all this new code with the gicv3.lock taken?
AFAICT there is no use of GICv3 registers inside the ITS initialization.
> gicv3_dist_init();
> res = gicv3_cpu_init();
> gicv3_hyp_init();
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 12c5d87..52d4277 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -51,6 +51,8 @@
>
> static struct {
> bool_t enabled;
> + /* Check if its supported */
> + bool_t lpi_support;
While ITS means LPIs is supported, the invert is not true.
Can you please rename this variable to its_enabled.
> /* Distributor interface address */
> paddr_t dbase;
> /* Re-distributor regions */
> @@ -62,9 +64,10 @@ static struct {
> void vgic_v3_setup_hw(paddr_t dbase,
> unsigned int nr_rdist_regions,
> const struct rdist_region *regions,
> - uint32_t rdist_stride)
> + uint32_t rdist_stride, bool_t lpi_support)
Ditto.
> {
> vgic_v3_hw.enabled = 1;
> + vgic_v3_hw.lpi_support = lpi_support;
Ditto.
> vgic_v3_hw.dbase = dbase;
> vgic_v3_hw.nr_rdist_regions = nr_rdist_regions;
> vgic_v3_hw.regions = regions;
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 1bc88f6..f819589 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -46,7 +46,9 @@
> #define GICC_SRE_EL2_ENEL1 (1UL << 3)
>
> /* Additional bits in GICD_TYPER defined by GICv3 */
> -#define GICD_TYPE_ID_BITS_SHIFT 19
> +#define GICD_TYPE_ID_BITS_SHIFT (19)
> +#define GICD_TYPE_ID_BITS_MASK (0x1f)
> +#define GICD_TYPE_LPIS (0x1UL << 17)
Please correct the typo in the name rather than keeping it everywhere. I.e
s/TYPE/TYPER/
Also, you've introduced GICD_TYPER_LPIS_SUPPORTED in patch #5 which does
exactly the same things as GICD_TYPE_LPIS. Please use it rather
introducing a new one.
>
> #define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
> #define GICD_CTLR_RWP (1UL << 31)
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index cf5a790..2bf061f 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -362,7 +362,7 @@ struct rdist_region;
> void vgic_v3_setup_hw(paddr_t dbase,
> unsigned int nr_rdist_regions,
> const struct rdist_region *regions,
> - uint32_t rdist_stride);
> + uint32_t rdist_stride, bool_t lpi_support);
Ditto.
> #endif
>
> #endif /* __ASM_ARM_VGIC_H__ */
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 15/28] xen/arm: ITS: Add virtual ITS driver
2015-09-18 13:09 ` [PATCH v7 15/28] xen/arm: ITS: Add virtual ITS driver vijay.kilari
@ 2015-09-21 15:34 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-21 15:34 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> This patch introduces virtual ITS driver with following
> functionality
> - Introduces helper functions to manage device table and
> ITT table in guest memory
> - Helper function to handle virtual ITS devices assigned
> to domain
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device
2015-09-21 13:47 ` Julien Grall
@ 2015-09-22 7:33 ` Vijay Kilari
2015-09-22 13:19 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-22 7:33 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Mon, Sep 21, 2015 at 7:17 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Add APIs to add devices to RB-tree, assign and remove
>> devices to domain.
>>
[...]
>> +
>> +static void its_free_device(struct its_device *dev)
>> +{
>> + xfree(dev->itt);
>> + its_lpi_free(dev);
>> xfree(dev->event_map.lpi_map);
>
> Why did you move this call from its_lpi_free to here?
its_lpi_free() is called along with its_free_device. So moved into
this code instead of calling it separately. This helps to avoid
chances of missing it out.
>
>> + xfree(dev->event_map.col_map);
>
> This line should have been added in its_lpi_free in patch #5.
>
>> + xfree(dev);
>> +}
>> +
>> +static struct its_device *its_alloc_device(u32 devid, u32 nr_ites,
>> + struct dt_device_node *dt_its)
>
> Can you explain why you weren't able to re-use its_create_device from Linux?
>
> AFAICT there is nothing different and you miss key code such as rounding
> to a power of 2 the number of event IDs.
>
It is almost re-used with following changes
1) Rounding of nr_ites, which is missing. However nr_ites is passed
from platform
file.
2) Freeing of memory on failure. Here I re-used its_free_device() which does
the same
>> +{
>> + struct its_device *dev;
>> + unsigned long *lpi_map;
>> + int lpi_base, sz;
>> + u16 *col_map = NULL;
>> +
>> + dev = xzalloc(struct its_device);
>> + if ( dev == NULL )
>> + return NULL;
>> +
>> + dev->its = its_get_phys_node(dt_its);
>
> You can re-order the code to get the ITS before allocate the memory. And
> then you can drop one goto.
>
>> + if (dev->its == NULL)
>> + {
>> + printk(XENLOG_G_ERR
>
> As already asked on v6, why XENLOG_G_ERR? Any call to this function will
> be done via privileged guest.
>
>> + "ITS: Failed to find ITS node for devid 0x%"PRIx32"\n", devid);
>> + goto err;
>> + }
>> +
>> + sz = nr_ites * dev->its->ite_size;
>> + sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
>> + dev->itt = xzalloc_bytes(sz);
>> + if ( !dev->itt )
>> + goto err;
>> +
>> + lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base);
>> + if ( !lpi_map )
>> + goto lpi_err;
>> +
>> + col_map = xzalloc_bytes(sizeof(*col_map) * nr_ites);
>> + if ( !col_map )
>> + goto col_err;
>> + dev->event_map.lpi_map = lpi_map;
>> + dev->event_map.lpi_base = lpi_base;
>> + dev->event_map.col_map = col_map;
>> + dev->event_map.nr_lpis = nr_ites;
>> + dev->device_id = devid;
>> +
>> + return dev;
>> +
>> +col_err:
>> + its_free_device(dev);
>> + return NULL;
>> +lpi_err:
>> + xfree(dev->itt);
>> +err:
>> + xfree(dev);
>> +
>> + return NULL;
>> +}
>> +
>> +/* Device assignment */
>> +int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its)
>> +{
>> + struct its_device *dev;
>> + u32 i, plpi = 0;
>> + struct its_collection *col;
>> + struct irq_desc *desc;
>> + struct msi_desc *msi = NULL;
>> + int res = 0;
>> +
>> + spin_lock(&rb_its_dev_lock);
>> + dev = its_find_device(devid);
>> + if ( dev )
>> + {
>> + printk(XENLOG_G_ERR "ITS: Device already exists 0x%"PRIx32"\n",
>
> Same question as before for XENLOG_G_ERR.
I should have used XENLOG_ERR?
>
>> + dev->device_id);
>> + res = -EEXIST;
>> + goto err_unlock;
>> + }
>> +
>> + dev = its_alloc_device(devid, nr_ites, dt_its);
>> + if ( !dev )
>> + {
>> + res = -ENOMEM;
>> + goto err_unlock;
>> + }
>> +
>> + if ( its_insert_device(dev) )
>
> The only way that this call can fail is because the device already
> exists in the RB-tree. Although, this can never happen because you have
> the lock taken and check beforehand if someone has inserted a device
> with this devID.
>
> So I would turn this if into an BUG_ON(its_insert_device(dev)) and save
> 6 lines.
With below code it prints message, free's up memory and return error.
The caller can handle as required.
>
>> + {
>> + its_free_device(dev);
>> + printk(XENLOG_G_ERR "ITS: Failed to insert device 0x%"PRIx32"\n", devid);
>> + res = -EINVAL;
>> + goto err_unlock;
>> + }
>> + /* Assign device to dom0 by default */
>
[...]
>
>> +
>> + /*
>> + * TODO: For pass-through following has to be implemented
>> + * 1) Allow device to be assigned to other domains (Dom0 -> DomU).
>> + * 2) Allow device to be re-assigned to Dom0 (DomU -> Dom0).
>> + * Implement separate function to handle this or rework this function.
>> + * For now do not allow assigning devices other than Dom0.
>> + *
>> + * By default all devices are assigned to Dom0.
>
> See my question above about the "why?".
AIUI, by default all devices are assigned to Dom0.
By updating dev->domain in add device to Dom0, we are making
sure that at the time of add device the device is with Dom0.
In assign_device the below check ensures that devices are with Dom0
before assigning to other Doms.
If we don't want to update dev->domain in add_device then
I think, we can explicitly initialize dev->domain = NULL and
check here (assign_device) that dev->domain should always either NULL or
should be hardware_domain.
>
>> + * Device should be with Dom0 before assigning to other domain.
>> + */
>> + if ( !is_hardware_domain(d) || !is_hardware_domain(pdev->domain) )
>> + {
>> + printk(XENLOG_ERR
>> + "ITS: PCI-Passthrough not supported!! to assign from d%d to d%d",
>> + pdev->domain->domain_id, d->domain_id);
>> + return -ENXIO;
>> + }
>> +
>> + pdev->domain = d;
>> + pdev->virt_device_id = vdevid;
>> +
>> + DPRINTK("ITS: Assign pdevid 0x%"PRIx32" lpis %"PRIu32" for dom %"PRIu16"\n",
>> + pdevid, pdev->event_map.nr_lpis, d->domain_id);
>> +
>> + for ( i = 0; i < pdev->event_map.nr_lpis; i++ )
>> + {
>> + plpi = its_get_plpi(pdev, i);
>> + /* TODO: Route lpi */
>> + }
>> +
>> + return 0;
>> }
>
> Regards,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function
2015-09-21 14:20 ` Julien Grall
@ 2015-09-22 9:10 ` Vijay Kilari
2015-09-22 13:24 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-22 9:10 UTC (permalink / raw)
To: Julien Grall
Cc: Zoltan Kiss, Ian Campbell, Stefano Stabellini, Prasun Kapoor,
Vijaya Kumar K, Tim Deegan, xen-devel, Stefano Stabellini,
manish.jaggi
On Mon, Sep 21, 2015 at 7:50 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Helper function gic_is_lpi() is used to find
>> if irq is lpi or not. For GICv2 platforms this function
>> returns number of irq ids which represents only number of line irqs.
>> For GICv3 platform irq ids are calculated from nr_lpis if
>> HW supports LPIs
>>
[...]
>> static void __init gicv3_dist_init(void)
>> {
>> uint32_t type;
>> @@ -578,6 +583,21 @@ static void __init gicv3_dist_init(void)
>>
>> /* Only 1020 interrupts are supported */
>> gicv3_info.nr_lines = min(1020U, nr_lines);
>> +
>> + /*
>> + * Number of IRQ ids supported.
>> + * Here we override HW supported number of LPIs and
>> + * limit to to LPIs specified in nr_lpis.
>> + */
>
> You still have to check that the number of LPIs requesting by the user
> is supported by the hardware.
>
> The user parameter can be seen as a restriction rather than a blind
> overriding.
>
>> + if ( gicv3_dist_supports_lpis() )
>
> I'm sure we already speak about it in a previous series. The GICv3 may
> support LPIs even without an ITS. Here you would claim that Xen is
> supporting LPIs which is clearly not true.
>
> When the ITS is not present, this value should be the number of
> interrupt line supported.
>
> I haven't checked if you fixed it later, but all the patch should be
> bisectable. I.e I shouldn't see any unwanted modification on supported
> platform with GICv3 (for instance the foundation model).
>
>> + gicv3_info.nr_irq_ids = nr_lpis + FIRST_GIC_LPI;
>> + else
>> + {
>> + gicv3_info.nr_irq_ids = gicv3_info.nr_lines;
>> + /* LPIs are not supported by HW. Reset to 0 */
>> + nr_lpis = 0;
>
> That's really ugly and you still let GICv2 hardware with nr_lpis != 0.
> As said on v6 I don't want to see any usage of nr_lpis in code except
> for letting the user restricting the number of LPIs supported.
>
> That means that all the code should use gic_nr_irq_ids to know the
> number of LPIs supported. Mixing the 2 usage will lead to big trouble
> latter.
Here nr_lpis is used to update nr_irq_ids which is used by gic_nr_irq_ids().
>
> So please move nr_lpis in gic-v3 as a parameter and don't export it.
nr_lpis is user defined/initialized in irq.c . How can we pass this to gic-v3 as
parameter?. You mean to have helper function to read/update nr_lpis?
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-21 15:20 ` Julien Grall
@ 2015-09-22 9:17 ` Vijay Kilari
2015-09-22 9:45 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-22 9:17 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Mon, Sep 21, 2015 at 8:50 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Initialize physical ITS if HW supports LPIs.
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> v7: - Export lpi support information to vgic-v3 driver from gic-v3.
>> - Drop gic_lpi_supported() helper function
>> - Add boot param to enable or disable physical ITS
>> v6: - Updated lpi_supported gic_info member for GICv2 and GICv3
>> - Introduced helper gic_lpi_supported() and exported
>> v5: - Made check of its dt node availability before
>> setting lpi_supported flag
>> ---
>> xen/arch/arm/gic-v3.c | 38 ++++++++++++++++++++++++++++++++++---
>> xen/arch/arm/vgic-v3.c | 5 ++++-
>> xen/include/asm-arm/gic_v3_defs.h | 4 +++-
>> xen/include/asm-arm/vgic.h | 2 +-
>> 4 files changed, 43 insertions(+), 6 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index c4c77a7..ac8a0ea 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -55,6 +55,18 @@ static struct {
>> } gicv3;
>>
>> static struct gic_info gicv3_info;
>> +/* Enable/Disable ITS support */
>> +static bool_t its_enable = 1;
>> +/* Availability of ITS support after successful ITS initialization */
>> +static bool_t its_enabled = 0;
>> +
>> +static void __init parse_its_param(char *s)
>> +{
>> + if ( !parse_bool(s) )
>> + its_enable = 0;
>> +}
>> +
>> +custom_param("its", parse_its_param);
>
> Why do you need a command line option to enable/disable the physical ITS?
I have added this to remove ITS support?.
Did I misunderstood it. May be I have to avoid generating its dt node
to disable its for dom0?
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-22 9:17 ` Vijay Kilari
@ 2015-09-22 9:45 ` Julien Grall
2015-09-22 10:05 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-22 9:45 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 22/09/2015 10:17, Vijay Kilari wrote:
>> Why do you need a command line option to enable/disable the physical ITS?
>
> I have added this to remove ITS support?.
> Did I misunderstood it. May be I have to avoid generating its dt node
> to disable its for dom0?
My question is why do you want to let the user disabling the ITS using
the command line? What's the use case?
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen
2015-09-21 11:23 ` Julien Grall
@ 2015-09-22 9:55 ` Vijay Kilari
2015-09-22 10:01 ` Julien Grall
2015-09-22 10:34 ` Vijay Kilari
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-22 9:55 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Mon, Sep 21, 2015 at 4:53 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> The only things I haven't check on this patch was the ITS command structure.
[...]
> Why a padding of only 56 bits? Shouldn't it be 248 bits (i.e 31 * 8
> bits) to fit all the command?
>
>> + } hdr;
>> + struct __packed {
>> + u8 cmd;
>> + u8 res1[3];
>> + u32 devid;
>> + u64 size:5;
>> + u64 res2:59;
>> + /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
>> + u64 res3:8;
>
> It's very confusing for the reviewer to see a mix of usage (u8 res1[n],
> u64 res3:8) within the same structure.
>
> The later (i.e u64 field:n) is the best to use because we can match
> quickly the size with the spec.
>
Do you mean to convert all field type as 64 as below?
struct __packed {
u64 cmd:8;
u64 res:24;
u64 devid:32;
u64 size:5;
u64 res2:59;
/* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
u64 res3:8;
u64 itt:40;
u64 res4:15;
u64 valid:1;
u64 res5;
} mapd_cmd;
>> + u64 itt:40;
>> + u64 res4:15;
>> + u64 valid:1;
>> + u64 res5;
>> + } mapd;
>
> [..]
l
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen
2015-09-22 9:55 ` Vijay Kilari
@ 2015-09-22 10:01 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-22 10:01 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Vijay,
On 22/09/2015 10:55, Vijay Kilari wrote:
> On Mon, Sep 21, 2015 at 4:53 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> The only things I haven't check on this patch was the ITS command structure.
> [...]
>> Why a padding of only 56 bits? Shouldn't it be 248 bits (i.e 31 * 8
>> bits) to fit all the command?
>>
>>> + } hdr;
>>> + struct __packed {
>>> + u8 cmd;
>>> + u8 res1[3];
>>> + u32 devid;
>>> + u64 size:5;
>>> + u64 res2:59;
>>> + /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
>>> + u64 res3:8;
>>
>> It's very confusing for the reviewer to see a mix of usage (u8 res1[n],
>> u64 res3:8) within the same structure.
>>
>> The later (i.e u64 field:n) is the best to use because we can match
>> quickly the size with the spec.
>>
>
> Do you mean to convert all field type as 64 as below?
Yes.
>
> struct __packed {
> u64 cmd:8;
> u64 res:24;
> u64 devid:32;
> u64 size:5;
> u64 res2:59;
> /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
BTW, this comment is not clear. Please explain in the comment why you
want to use 48 bit. I.e
"XXX: The ITT holds the upper bits of the address [47-8]. Include res3
to avoid shifting?"
> u64 res3:8;
> u64 itt:40;
> u64 res4:15;
> u64 valid:1;
> u64 res5;
> } mapd_cmd;
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-22 9:45 ` Julien Grall
@ 2015-09-22 10:05 ` Ian Campbell
2015-09-22 10:29 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-09-22 10:05 UTC (permalink / raw)
To: Julien Grall, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On Tue, 2015-09-22 at 10:45 +0100, Julien Grall wrote:
>
> On 22/09/2015 10:17, Vijay Kilari wrote:
> > > Why do you need a command line option to enable/disable the physical
> > > ITS?
> >
> > I have added this to remove ITS support?.
> > Did I misunderstood it. May be I have to avoid generating its dt node
> > to disable its for dom0?
>
> My question is why do you want to let the user disabling the ITS using
> the command line? What's the use case?
It's always useful to be able to nobble this sort of thing, either for
debugging/development purposes or to allow users to workaround issues.
Assuming the switch is simple an reasonably self contained (and it really
should be, since it should just be checked at start of day and then arrange
to behave like no ITS is present) then I can't see why not to have it.
Ian.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-22 10:05 ` Ian Campbell
@ 2015-09-22 10:29 ` Julien Grall
2015-09-22 10:44 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-22 10:29 UTC (permalink / raw)
To: Ian Campbell, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 22/09/2015 11:05, Ian Campbell wrote:
> On Tue, 2015-09-22 at 10:45 +0100, Julien Grall wrote:
>>
>> On 22/09/2015 10:17, Vijay Kilari wrote:
>>>> Why do you need a command line option to enable/disable the physical
>>>> ITS?
>>>
>>> I have added this to remove ITS support?.
>>> Did I misunderstood it. May be I have to avoid generating its dt node
>>> to disable its for dom0?
>>
>> My question is why do you want to let the user disabling the ITS using
>> the command line? What's the use case?
>
> It's always useful to be able to nobble this sort of thing, either for
> debugging/development purposes or to allow users to workaround issues.
This is can be done via the firmware table (see the property "status" in
the DT). I see no advantage to use the command line for a such purpose,
mainly for the debugging/development point.
Also what kind of workaround do you expect to be fixed by disabling ITS?
I know that it will disable all the PCI MSI in general. But should not
it be done with a PCI related parameter?
> Assuming the switch is simple an reasonably self contained (and it really
> should be, since it should just be checked at start of day and then arrange
> to behave like no ITS is present) then I can't see why not to have it.
I think that disabling the ITS is more complex than not using in Xen.
This will be tied with PCI passthrough very soon and we will have to
spread this decision everywhere.
We would also have to remove everything related to ITS (i.e
msi-parent...) to the DT to avoid to pass an half-valid DT to DOM0.
Otherwise we may just not be able to boot or using PCI (even without MSI).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen
2015-09-21 11:23 ` Julien Grall
2015-09-22 9:55 ` Vijay Kilari
@ 2015-09-22 10:34 ` Vijay Kilari
2015-09-22 12:34 ` Julien Grall
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-22 10:34 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Mon, Sep 21, 2015 at 4:53 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> The only things I haven't check on this patch was the ITS command structure.
>
> On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
>> +/* ITS command structure */
>> +typedef union {
>
> Can you please sort this union by command name in alphabetical order.
> It's way easier to find a command in the list.
>
>> + u64 bits[4];
>> + struct __packed {
>> + uint8_t cmd;
>
> NIT: Please stay consistent with usage of the type. You are using u8
> everywhere within this union except here.
>
>> + uint8_t pad[7];
>
> Why a padding of only 56 bits? Shouldn't it be 248 bits (i.e 31 * 8
> bits) to fit all the command?
>
>> + } hdr;
>> + struct __packed {
>> + u8 cmd;
>> + u8 res1[3];
>> + u32 devid;
>> + u64 size:5;
>> + u64 res2:59;
>> + /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
>> + u64 res3:8;
>
> It's very confusing for the reviewer to see a mix of usage (u8 res1[n],
> u64 res3:8) within the same structure.
>
> The later (i.e u64 field:n) is the best to use because we can match
> quickly the size with the spec.
>
>> + u64 itt:40;
>> + u64 res4:15;
>> + u64 valid:1;
>> + u64 res5;
>> + } mapd;
>
> [..]
>
>> + struct __packed {
>> + u8 cmd;
>> + u8 res1[3];
>> + u32 devid;
>> + u32 event;
>> + u32 res2;
>> + u64 res3;
>> + u64 res4;
>> + } int_cmd;
>
> Maybe you want to add the suffix _cmd to everyone to avoid having only
> one because of C spec issue.
suffixing _cmd will look ugly ( as below ex) where "cmd" is repeated twice.
cmd.mapd_cmd.cmd = GITS_CMD_MAPD;
cmd.mapd_cmd.devid = dev->device_id;
cmd.mapd_cmd.size = size - 1;
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-22 10:29 ` Julien Grall
@ 2015-09-22 10:44 ` Ian Campbell
2015-09-22 12:31 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-09-22 10:44 UTC (permalink / raw)
To: Julien Grall, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On Tue, 2015-09-22 at 11:29 +0100, Julien Grall wrote:
>
> On 22/09/2015 11:05, Ian Campbell wrote:
> > On Tue, 2015-09-22 at 10:45 +0100, Julien Grall wrote:
> > >
> > > On 22/09/2015 10:17, Vijay Kilari wrote:
> > > > > Why do you need a command line option to enable/disable the
> > > > > physical
> > > > > ITS?
> > > >
> > > > I have added this to remove ITS support?.
> > > > Did I misunderstood it. May be I have to avoid generating its dt
> > > > node
> > > > to disable its for dom0?
> > >
> > > My question is why do you want to let the user disabling the ITS
> > > using
> > > the command line? What's the use case?
> >
> > It's always useful to be able to nobble this sort of thing, either for
> > debugging/development purposes or to allow users to workaround issues.
>
> This is can be done via the firmware table (see the property "status" in
> the DT).
This is not always trivial to override (think DTB provided by the UEFI
firmware).
> I see no advantage to use the command line for a such purpose,
> mainly for the debugging/development point.
>
> Also what kind of workaround do you expect to be fixed by disabling ITS?
Any bug relating to running on an ITS which a user trips over in the field.
This isn't really any different from the myriad of options which already
exist to disable particular hardware features lists in
http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html (many of
which are x86 specific, including I notice now msi=<boolean>)
> I know that it will disable all the PCI MSI in general. But should not
> it be done with a PCI related parameter?
Maybe there should be a PCI option as well, but disabling PCI MSI should
inherit from the disabling of the ITS. Just as it should be disabled if the
ITS isn't present at all.
> > Assuming the switch is simple an reasonably self contained (and it really
> > should be, since it should just be checked at start of day and then arrange
> > to behave like no ITS is present) then I can't see why not to have it.
>
> I think that disabling the ITS is more complex than not using in Xen.
> This will be tied with PCI passthrough very soon and we will have to
> spread this decision everywhere.
>
> We would also have to remove everything related to ITS (i.e
> msi-parent...) to the DT to avoid to pass an half-valid DT to DOM0.
> Otherwise we may just not be able to boot or using PCI (even without
> MSI).
So this might fall into the "assuming the switch is simple" caveat which I
mentioned, although I'm not convinced we need to go this far. We could
equally well just mark the ITS node disabled and leave everything else
alone.
Ian.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support
2015-09-22 10:44 ` Ian Campbell
@ 2015-09-22 12:31 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-22 12:31 UTC (permalink / raw)
To: Ian Campbell, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 22/09/15 11:44, Ian Campbell wrote:
> On Tue, 2015-09-22 at 11:29 +0100, Julien Grall wrote:
>>
>> On 22/09/2015 11:05, Ian Campbell wrote:
>>> On Tue, 2015-09-22 at 10:45 +0100, Julien Grall wrote:
>>>>
>>>> On 22/09/2015 10:17, Vijay Kilari wrote:
>>>>>> Why do you need a command line option to enable/disable the
>>>>>> physical
>>>>>> ITS?
>>>>>
>>>>> I have added this to remove ITS support?.
>>>>> Did I misunderstood it. May be I have to avoid generating its dt
>>>>> node
>>>>> to disable its for dom0?
>>>>
>>>> My question is why do you want to let the user disabling the ITS
>>>> using
>>>> the command line? What's the use case?
>>>
>>> It's always useful to be able to nobble this sort of thing, either for
>>> debugging/development purposes or to allow users to workaround issues.
>>
>> This is can be done via the firmware table (see the property "status" in
>> the DT).
>
> This is not always trivial to override (think DTB provided by the UEFI
> firmware).
Hmmm, right. I always have in mind the U-boot case and forgot the UEFI one.
>> I see no advantage to use the command line for a such purpose,
>> mainly for the debugging/development point.
>>
>> Also what kind of workaround do you expect to be fixed by disabling ITS?
>
> Any bug relating to running on an ITS which a user trips over in the field.
TBH, I asked this question because it has been introduced in this
version without any explanation. Maybe I'm too picky but I think it's
important to know what the usage of a new option.
Aside that, just reading the code, I can tell that this parameter can't
even work on Thunder-X.
With its_enable=false, the platform code (see patch #28) will fail to
add the PCI and therefore return an error which will make Xen to fail
building DOM0.
> This isn't really any different from the myriad of options which already
> exist to disable particular hardware features lists in
> http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html (many of
> which are x86 specific, including I notice now msi=<boolean>)
>> I know that it will disable all the PCI MSI in general. But should not
>> it be done with a PCI related parameter?
>
> Maybe there should be a PCI option as well, but disabling PCI MSI should
> inherit from the disabling of the ITS. Just as it should be disabled if the
> ITS isn't present at all.
While today we support MSI only with ITS, we will have at some point
GICv2m and maybe new interrupt controllers.
I would much prefer having a common parameter (such a "msi=...") to
disable MSI on the platform rather adding a new parameter for each new
interrupt controller using MSI.
It much easier to tell the user "please disable msi" rather than "please
check what interrupt controller you are using" and after another round
of mail "please use this option".
>
>>> Assuming the switch is simple an reasonably self contained (and it really
>>> should be, since it should just be checked at start of day and then arrange
>>> to behave like no ITS is present) then I can't see why not to have it.
>>
>> I think that disabling the ITS is more complex than not using in Xen.
>> This will be tied with PCI passthrough very soon and we will have to
>> spread this decision everywhere.
>>
>> We would also have to remove everything related to ITS (i.e
>> msi-parent...) to the DT to avoid to pass an half-valid DT to DOM0.
>> Otherwise we may just not be able to boot or using PCI (even without
>> MSI).
>
> So this might fall into the "assuming the switch is simple" caveat which I
> mentioned, although I'm not convinced we need to go this far. We could
> equally well just mark the ITS node disabled and leave everything else
> alone.
Today the ITS driver in Linux doesn't check if the ITS node is disabled
or not.
If we don't take into account this fact, the ITS node are creating from
scratch for DOM0. If the vITS is not present there will no region to
create it. So adding fake node maybe more complex than just having a DT
partially valid and hoping the guest doing the right thing.
The behavior chosen should be documented in the
docs/misc/xen-command-line.markdown to make clear what is expected with
this option turned on.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen
2015-09-22 10:34 ` Vijay Kilari
@ 2015-09-22 12:34 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-22 12:34 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 22/09/15 11:34, Vijay Kilari wrote:
> On Mon, Sep 21, 2015 at 4:53 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Maybe you want to add the suffix _cmd to everyone to avoid having only
>> one because of C spec issue.
>
> suffixing _cmd will look ugly ( as below ex) where "cmd" is repeated twice.
Well, you already have this problem with "int_cmd", the cmd is repeated
twice ;).
Anyway that was only a suggestion, if it doesn't work for you add a
comment to explain, even if it's obvious why the "_cmd" for int.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device
2015-09-22 7:33 ` Vijay Kilari
@ 2015-09-22 13:19 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-22 13:19 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 22/09/15 08:33, Vijay Kilari wrote:
> On Mon, Sep 21, 2015 at 7:17 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> On 18/09/15 14:08, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Add APIs to add devices to RB-tree, assign and remove
>>> devices to domain.
>>>
> [...]
>>> +
>>> +static void its_free_device(struct its_device *dev)
>>> +{
>>> + xfree(dev->itt);
>>> + its_lpi_free(dev);
>>> xfree(dev->event_map.lpi_map);
>>
>> Why did you move this call from its_lpi_free to here?
>
> its_lpi_free() is called along with its_free_device. So moved into
> this code instead of calling it separately. This helps to avoid
> chances of missing it out.
I wasn't asking about the call to its_lpi_free... but the line:
xfree(dev->event_map.lpi_map).
It belonged to its_lpi_free before but now it's part of its_free_device.
I don't see any reason to move it here.
>>
>>> + xfree(dev->event_map.col_map);
>>
>> This line should have been added in its_lpi_free in patch #5.
>>
>>> + xfree(dev);
>>> +}
>>> +
>>> +static struct its_device *its_alloc_device(u32 devid, u32 nr_ites,
>>> + struct dt_device_node *dt_its)
>>
>> Can you explain why you weren't able to re-use its_create_device from Linux?
>>
>> AFAICT there is nothing different and you miss key code such as rounding
>> to a power of 2 the number of event IDs.
>>
>
> It is almost re-used with following changes
Well if you compare the code it looks very different... it was only for
the purpose to keep the code as close as possible as the Linux ITS driver
I guess we already diverge a lot so having one functions more...
> 1) Rounding of nr_ites, which is missing. However nr_ites is passed
> from platform
> file.
While today it's only called in the Thunder-X platform specific code and
the value are hardcoded. With the addition of PCI passthrough, this
value will be directly retrieved from the PCI configuration space.
Furthermore, AFAICT, the round up to a power of 2 is an ITS restriction
and not mandated by the PCI spec. So you can't really on the caller
giving nicely the correct value.
e caller giving you the correct
> 2) Freeing of memory on failure. Here I re-used its_free_device() which does
> the same
[...]
>>> +/* Device assignment */
>>> +int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its)
>>> +{
>>> + struct its_device *dev;
>>> + u32 i, plpi = 0;
>>> + struct its_collection *col;
>>> + struct irq_desc *desc;
>>> + struct msi_desc *msi = NULL;
>>> + int res = 0;
>>> +
>>> + spin_lock(&rb_its_dev_lock);
>>> + dev = its_find_device(devid);
>>> + if ( dev )
>>> + {
>>> + printk(XENLOG_G_ERR "ITS: Device already exists 0x%"PRIx32"\n",
>>
>> Same question as before for XENLOG_G_ERR.
>
> I should have used XENLOG_ERR?
Yes. I don't see the point to use XENLOG_G_* here.
>
>>
>>> + dev->device_id);
>>> + res = -EEXIST;
>>> + goto err_unlock;
>>> + }
>>> +
>>> + dev = its_alloc_device(devid, nr_ites, dt_its);
>>> + if ( !dev )
>>> + {
>>> + res = -ENOMEM;
>>> + goto err_unlock;
>>> + }
>>> +
>>> + if ( its_insert_device(dev) )
>>
>> The only way that this call can fail is because the device already
>> exists in the RB-tree. Although, this can never happen because you have
>> the lock taken and check beforehand if someone has inserted a device
>> with this devID.
>>
>> So I would turn this if into an BUG_ON(its_insert_device(dev)) and save
>> 6 lines.
>
> With below code it prints message, free's up memory and return error.
> The caller can handle as required.
This call can only fail if you are implementation is buggy. It's true
you may get in trouble elsewhere later.
As you know that this call should never fail, you can use a BUG_ON to
check your assertion. This is very common within Xen and Linux. It
allows you to catch the root cause rather than waiting to have unrelated
error making harder to debug.
>>
>>> + {
>>> + its_free_device(dev);
>>> + printk(XENLOG_G_ERR "ITS: Failed to insert device 0x%"PRIx32"\n", devid);
>>> + res = -EINVAL;
>>> + goto err_unlock;
>>> + }
>>> + /* Assign device to dom0 by default */
>>
> [...]
>>
>>> +
>>> + /*
>>> + * TODO: For pass-through following has to be implemented
>>> + * 1) Allow device to be assigned to other domains (Dom0 -> DomU).
>>> + * 2) Allow device to be re-assigned to Dom0 (DomU -> Dom0).
>>> + * Implement separate function to handle this or rework this function.
>>> + * For now do not allow assigning devices other than Dom0.
>>> + *
>>> + * By default all devices are assigned to Dom0.
>>
>> See my question above about the "why?".
>
> AIUI, by default all devices are assigned to Dom0.
> By updating dev->domain in add device to Dom0, we are making
> sure that at the time of add device the device is with Dom0.
No, you are not assigning the device to DOM0 in the function add_device.
You are just setting a value and left the ITS incorrectly setup to allow
DOM0 using this device.
You still have to call assign_device to setup the ITS for this device.
Furthermore, that make the requirement to call its_add_device always
after DOM0 has been created. We may want to add device in the ITS before
ITS.
> In assign_device the below check ensures that devices are with Dom0
> before assigning to other Doms.
>
> If we don't want to update dev->domain in add_device then
> I think, we can explicitly initialize dev->domain = NULL and
> check here (assign_device) that dev->domain should always either NULL or
> should be hardware_domain.
For now, you just have to check that d is the hardware domain and
pdev->domain is not NULL. This is fine because the caller is only the
platform code.
We will need to revisit this code when PCI passthrough will be added to
add some missing lock on the device and adding more check on the used
domain.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function
2015-09-22 9:10 ` Vijay Kilari
@ 2015-09-22 13:24 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-22 13:24 UTC (permalink / raw)
To: Vijay Kilari
Cc: Zoltan Kiss, Ian Campbell, Stefano Stabellini, Prasun Kapoor,
Vijaya Kumar K, Tim Deegan, xen-devel, Stefano Stabellini,
manish.jaggi
On 22/09/15 10:10, Vijay Kilari wrote:
> On Mon, Sep 21, 2015 at 7:50 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> That's really ugly and you still let GICv2 hardware with nr_lpis != 0.
>> As said on v6 I don't want to see any usage of nr_lpis in code except
>> for letting the user restricting the number of LPIs supported.
>>
>> That means that all the code should use gic_nr_irq_ids to know the
>> number of LPIs supported. Mixing the 2 usage will lead to big trouble
>> latter.
>
> Here nr_lpis is used to update nr_irq_ids which is used by gic_nr_irq_ids().
That not the only usage of nr_lpis within this series. You are using it
to create the IRQ desc for LPIs.
We should have only one way to get the number of LPIs (i.e
gic_nr_irq_ids()). Any attempt to have 2 different to retrieve the
number of LPIs will be a problem later.
>>
>> So please move nr_lpis in gic-v3 as a parameter and don't export it.
>
> nr_lpis is user defined/initialized in irq.c . How can we pass this to gic-v3 as
> parameter?. You mean to have helper function to read/update nr_lpis?
I meant moving the parameter in gic-v3 (i.e moving integer_param(...)).
We can speak about moving the parameter in another place when we will
have the need to.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 16/28] xen/arm: ITS: Add virtual ITS commands support
2015-09-18 13:09 ` [PATCH v7 16/28] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-09-22 13:47 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-22 13:47 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add Virtual ITS command processing support to Virtual ITS driver
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
I've got minor comments in vits_get_max_collections. With that addressed:
Reviewed-by: Julien Grall <julien.grall@citrix.com>
[...]
> +static inline uint16_t vits_get_max_collections(struct domain *d)
> +{
> + /*
> + * ITS only supports up to 256 collections without
> + * provisioning external memory. As per vITS design, number of
> + * vCPUS should not exceed max number of collections.
> + */
> + ASSERT(d->max_vcpus < 256);
> +
> + /* Each collection corresponds to one CPU(vCPU). collections are
NIT: The coding style for comment is
/*
* My comments
* ...
*/
s/collections are/Collections/
> + * used to move interrupts from one CPU to another. The ITS
> + * mandated to implement N + 1 collections where N is the number of
s/mandated/mandates/
> + * processor on the platform (i.e max number of VCPUs for a given
> + * guest).
I was about to suggestion mention the section in the spec. Although I'm
not able to find a related section in the Public doc (IHI0069A).
Might still be worth to mention the PRD03-GENC-010745 24 section 4.9.15?
> + */
> + return (d->max_vcpus + 1);
> +}
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation
2015-09-18 13:09 ` [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-09-22 14:30 ` Julien Grall
2015-09-24 5:07 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-22 14:30 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Emulate GITS* registers
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v7: - Fixed wrong usage of vgic_regN* helpers
> - coding styles and comments
> - GITS_BASER0 is always overwritten with new value every time
> it is updated.
> v6: - Removed unrelated code of this patch
> - Used vgic_regN_{read,write}
> v4: - Removed GICR register emulation
> ---
> xen/arch/arm/vgic-v3-its.c | 315 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 8 ++
> xen/include/asm-arm/vits.h | 8 ++
> 3 files changed, 331 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index dba0b9e..0d97fcb 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -36,6 +36,15 @@
>
> //#define DEBUG_ITS
>
> +/* GITS_PIDRn register values for ARM implementations */
> +#define GITS_PIDR0_VAL (0x94)
> +#define GITS_PIDR1_VAL (0xb4)
> +#define GITS_PIDR2_VAL (0x3b)
> +#define GITS_PIDR3_VAL (0x00)
> +#define GITS_PIDR4_VAL (0x04)
> +#define GITS_BASER0_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
> + (sizeof(struct vitt) << \
> + GITS_BASER_ENTRY_SIZE_SHIFT))
As asked on v6, I'd like to see a comment explaining the different
values and the usage of BASER0. I.e something like "BASER0 is used to
ask the guest to allocate memory for the Device table".
> #ifdef DEBUG_ITS
> # define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
> #else
> @@ -503,6 +512,301 @@ err:
> return 0;
> }
>
> +static inline void vits_spin_lock(struct vgic_its *vits)
> +{
> + spin_lock(&vits->lock);
> +}
> +
> +static inline void vits_spin_unlock(struct vgic_its *vits)
> +{
> + spin_unlock(&vits->lock);
> +}
> +
> +static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + uint64_t val;
> + uint32_t gits_reg;
> +
> + gits_reg = info->gpa - vits->gits_base;
> + DPRINTK("%pv: vITS: GITS_MMIO_READ offset 0x%"PRIx32"\n", v, gits_reg);
> +
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + vits_spin_lock(vits);
> + /*
> + * vITS is always quiescent, has no translations in progress and
> + * completed all operations. So set quiescent by default.
> + */
Well, this comment should have been moved at the same place as you moved
the setting of QUIESCENT bit (i.e write).
> + *r = vgic_reg32_read(vits->ctrl, info);
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_IIDR:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + *r = vgic_reg32_read(GICV3_GICD_IIDR_VAL, info);
> + return 1;
> + case GITS_TYPER:
> + case GITS_TYPER + 4:
> + /*
> + * GITS_TYPER.HCC = max_vcpus + 1 (max collection supported)
> + * GITS_TYPER.Devbits = HW supported Devbits size
> + * GITS_TYPER.IDbits = HW supported IDbits size
> + * GITS_TYPER.PTA = 0 (Target addresses are linear processor numbers)
> + * GITS_TYPER.ITTSize = Size of struct vitt
> + * GITS_TYPER.Physical = 1
> + */
> + if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> + val = ((vits_get_max_collections(v->domain) << GITS_TYPER_HCC_SHIFT ) |
> + ((vits_hw.devID_bits - 1) << GITS_TYPER_DEVBITS_SHIFT) |
> + ((vits_hw.eventID_bits - 1) << GITS_TYPER_IDBITS_SHIFT) |
> + ((sizeof(struct vitt) - 1) << GITS_TYPER_ITT_SIZE_SHIFT) |
> + GITS_TYPER_PHYSICAL_LPIS);
> + *r = vgic_reg64_read(val, info);
> + return 1;
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- read ignored */
> + goto read_as_zero;
> + case GITS_CBASER:
> + case GITS_CBASER + 4:
> + if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> + vits_spin_lock(vits);
> + *r = vgic_reg64_read(vits->cmd_base, info);
I mentioned it in v6 for cmd_qsize but fail to notice it for cmd_base.
You are using cmd_base as a 64-bit register, although you've defined
cmd_base a paddr_t. Even though the two are 64-bit this is conceptually
completely different, you have to define cmd_base using uint64_t.
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CWRITER:
> + if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> + vits_spin_lock(vits);
> + *r = vgic_reg64_read(vits->cmd_write, info);
Ditto for cmd_write.
> + vits_spin_unlock(vits);
> + return 1;
[...]
> +static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
[...]
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- write ignored */
> + goto write_ignore;
> + case GITS_CBASER:
> + /* XXX: support 32-bit access */
You haven't asked my question on v6... What is missing to support 32-bt
support? AFAICT the register helpers should the job for you. Correct?
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + if ( vits->ctrl & GITS_CTLR_ENABLE )
> + {
> + /* RO -- write ignored */
> + vits_spin_unlock(vits);
> + goto write_ignore;
> + }
> + vgic_reg64_write(&vits->cmd_base, *r, info);
> + if ( vits->cmd_base & GITS_BASER_VALID )
> + {
> + val = ((vits->cmd_base & GITS_BASER_PAGES_MASK_VAL) + 1) * SZ_4K;
> + vits->cmd_qsize = val;
> + atomic_set(&vits->cmd_read, 0);
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CWRITER:
> + if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> + vits_spin_lock(vits);
> + /* Only Bits[19:5] are writable */
> + vgic_reg64_write(&vits->cmd_write, (*r & 0xfffe0), info);
> + ret = 1;
> + /* CWRITER should be within command queue range */
> + if ( (vits->ctrl & GITS_CTLR_ENABLE) &&
> + (vits->cmd_write < vits->cmd_qsize) )
> + ret = vits_process_cmd(v, vits);
> + vits_spin_unlock(vits);
> + return ret;
> + case GITS_CWRITER + 4:
> + /* Bits[63:20] are RES0 */
> + goto write_ignore_32;
> + case GITS_CREADR:
> + /* RO -- write ignored */
> + goto write_ignore_64;
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- write ignored */
> + goto write_ignore;
> + case GITS_BASER0:
> + /* Support only 64-bit access */
You didn't answer my question on v6. What's the problem to support
32-bit access?
If it's not fixed right now, this have to be fixed before Xen 4.7 is
released because some OS may decide to use 32 bit which are entirely valid.
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + /* RO -- write ignored if GITS_CTLR.Enable = 1 */
> + if ( vits->ctrl & GITS_CTLR_ENABLE )
> + {
> + vits_spin_unlock(vits);
> + goto write_ignore;
> + }
> + val = GITS_BASER0_INIT_VAL | (*r & GITS_BASER_MASK);
> + vgic_reg64_write(&vits->baser0, val, info);
> + vits->dt_ipa = vits->baser0 & GITS_BASER_PA_MASK;
> + psz = (vits->baser0 >> GITS_BASER_PAGE_SIZE_SHIFT) &
> + GITS_BASER_PAGE_SIZE_MASK_VAL;
> + if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
> + sz = 4;
> + else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
> + sz = 16;
> + else
> + /* 0x11 and 0x10 are treated as 64K size */
> + sz = 64;
> +
> + vits->dt_size = (vits->baser0 & GITS_BASER_PAGES_MASK_VAL) * sz * SZ_1K;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_BASER1 ... GITS_BASERN:
> + if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> + goto write_ignore;
NIT: These 2 lines can be replaced by
goto write_ignore_64;
[...]
> diff --git a/xen/include/asm-arm/vits.h b/xen/include/asm-arm/vits.h
> index 295cce4..dc02762 100644
> --- a/xen/include/asm-arm/vits.h
> +++ b/xen/include/asm-arm/vits.h
> @@ -24,6 +24,10 @@
> struct vgic_its
> {
> spinlock_t lock;
> + /* Emulation of BASER0 */
As asked on v6, please add a comment to explain the usage of baser0.
> + uint64_t baser0;
> + /* GICR ctrl register */
> + uint32_t ctrl;
> /* Command queue base */
> paddr_t cmd_base;
> /* Command queue write pointer */
> @@ -32,6 +36,10 @@ struct vgic_its
> atomic_t cmd_read;
> /* Command queue size */
> unsigned long cmd_qsize;
> + /* ITS mmio physical base */
> + paddr_t gits_base;
> + /* ITS mmio physical size */
> + unsigned long gits_size;
> /* vITT device table ipa */
> paddr_t dt_ipa;
> /* vITT device table size */
>
Regards
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 28/28] xen/arm: ITS: Add pci devices in ThunderX
2015-09-18 13:09 ` [PATCH v7 28/28] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
@ 2015-09-22 15:09 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-22 15:09 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> ITS initialization required for all PCI devices in
> ThunderX platform are done by calling from specific
> mapping function.
>
> This patch can be reverted once XEN PCI passthrough
> framework for arm64 is in available.
When a new feature is added in Xen (such as ITS and basic PCI support
with Thunder-X), we are trying to avoid any breakage no matter the
version of Linux.
With this series, any Linux with Thunder-X support is able to boot on
Xen and use basic PCI without Manish's series. If we decide to revert
this patch, a such Linux won't be able to boot any more on Xen.
IIUC, what matters is having Xen booting on Thunder-X with PCI supported
when 4.7 released. If we can get PCI passthrough fully working by then,
this patch will become unnecessary.
Xen 4.6 hasn't yet been released, so it's entirely feasible to get the
PCI passthrough for the next release if we start to review it very soon.
If the feature doesn't land in staging before the freeze, then we can
talk about having this patch in 4.7 and come up with a plan to revert it
when PCI passthrough will be fully supported.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS
2015-09-18 13:09 ` [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS vijay.kilari
@ 2015-09-23 8:31 ` Julien Grall
2015-09-24 5:26 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-23 8:31 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 0d5c61c..f3346d3 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -37,6 +37,7 @@
> #include <asm/gic.h>
> #include <asm/gic_v3_defs.h>
> #include <asm/gic-its.h>
> +#include <asm/vits.h>
> #include <xen/log2.h>
>
> #define its_print(lvl, fmt, ...) \
> @@ -88,6 +89,11 @@ struct its_node {
> struct dt_device_node *dt_node;
> };
>
> +static struct {
> + uint8_t eventID_bits;
> + uint8_t devID_bits;
> +} its_data;
> +
> #define ITS_ITT_ALIGN SZ_256
>
> static LIST_HEAD(its_nodes);
> @@ -914,6 +920,8 @@ int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
>
> for ( i = 0; i < pdev->event_map.nr_lpis; i++ )
> {
> + ASSERT(i < (1 << its_data.eventID_bits));
I'd like to understand why you choose to use an ASSERT here.
ASSERT means that we expect this condition never happen, although the
nr_lpis is calculated using the number of MSI handled by a device.
Given that we don't control the number of MSI, a proper check in
its_add_device seem better than this ASSERT.
> +
> plpi = its_get_plpi(pdev, i);
> /* TODO: Route lpi */
> }
> @@ -1366,6 +1374,8 @@ static int its_probe(struct dt_device_node *node)
> its->phys_size = its_size;
> typer = readl_relaxed(its_base + GITS_TYPER);
> its->ite_size = ((typer >> 4) & 0xf) + 1;
> + its_data.eventID_bits = GITS_TYPER_IDBITS(typer);
> + its_data.devID_bits = GITS_TYPER_DEVBITS(typer);
The 2 fields will be override every time a new ITS node will be initialized.
What ensure that all the ITS have the same number of Event ID and Device ID?
>
> its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
> if ( !its->cmd_base )
> @@ -1457,6 +1467,7 @@ int its_cpu_init(void)
>
> int __init its_init(struct rdist_prop *rdists)
> {
> + struct its_node *its;
> struct dt_device_node *np = NULL;
>
> static const struct dt_device_match its_device_ids[] __initconst =
> @@ -1479,6 +1490,16 @@ int __init its_init(struct rdist_prop *rdists)
> its_alloc_lpi_tables();
> its_lpi_init(rdists->id_bits);
>
> + its = list_first_entry(&its_nodes, struct its_node, entry);
> + /*
> + * As per vITS design spec, Xen exposes only one virtual ITS per domain.
> + * This simplifies vITS command model. i.e Simplifies processing global
s/i.e Simplifies/I.e simplifies/
> + * ITS commands which does not have device ID on platform having more
> + * than one physical ITS.
> + */
> + vits_setup_hw(its_data.devID_bits, its_data.eventID_bits,
> + its->phys_base, its->phys_size);
> +
> return 0;
> }
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 19/28] xen/arm: ITS: Store LPIs allocated per domain
2015-09-18 13:09 ` [PATCH v7 19/28] xen/arm: ITS: Store LPIs allocated per domain vijay.kilari
@ 2015-09-23 8:37 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-23 8:37 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
Commit title: Store the number of LPIs allocated per domain
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Store number of lpis allocated per domain in vgic structure
NIT: It's not neccessary to repeat the commit title in the commit
message. It would just let it empty.
If you want to keep it: s/Store number of lpis/Store the number of LPIs/
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v7: - Change commit message.
> - Store only nr_lpis per domain in vgic structure and drop
> id_bits.
> ---
> xen/arch/arm/vgic-v3-its.c | 2 ++
> xen/arch/arm/vgic.c | 1 +
> xen/include/asm-arm/domain.h | 1 +
> 3 files changed, 4 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 3679d11..4285af3 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -843,6 +843,8 @@ int vits_domain_init(struct domain *d)
> ASSERT(is_hardware_domain(d));
> ASSERT(vits_hw.enabled);
>
> + d->arch.vgic.nr_lpis = gic_nr_irq_ids() - FIRST_GIC_LPI;
Please add a comment to explain that we expose the same number of LPIs
than the HW
> +
> d->arch.vgic.vits = xzalloc(struct vgic_its);
> if ( !d->arch.vgic.vits )
> return -ENOMEM;
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index baa74c7..a5ab99d9 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -75,6 +75,7 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> int ret;
>
> d->arch.vgic.ctlr = 0;
> + d->arch.vgic.nr_lpis = 0;
The struct domain is always initialized to 0, so it's not necessary to
set nr_lpis.
>
> /* Limit the number of virtual SPIs supported to (1020 - 32) = 988 */
> if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 0f2aa66..011f85b 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -92,6 +92,7 @@ struct arch_domain
> spinlock_t lock;
> int ctlr;
> int nr_spis; /* Number of SPIs */
> + int nr_lpis; /* Number of LPIs */
> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
> struct vgic_irq_rank *shared_irqs;
> /*
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper
2015-09-18 13:09 ` [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper vijay.kilari
@ 2015-09-23 8:52 ` Julien Grall
2015-09-24 6:44 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-23 8:52 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Introduce vgic_is_lpi_supported() helper function
> to know virtual ITS availability for a domain
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v7: - its_enabled field is added to vgic structure
> ---
> xen/arch/arm/vgic-v2.c | 2 ++
> xen/arch/arm/vgic-v3.c | 3 +++
> xen/arch/arm/vgic.c | 5 +++++
> xen/include/asm-arm/domain.h | 1 +
> xen/include/asm-arm/vgic.h | 1 +
> 5 files changed, 12 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index 600d27e..de0cd4b 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -594,6 +594,8 @@ static int vgic_v2_domain_init(struct domain *d)
> register_mmio_handler(d, &vgic_v2_distr_mmio_handler, d->arch.vgic.dbase,
> PAGE_SIZE);
>
> + d->arch.vgic.its_enabled = 0;
> +
Not necessary and very confusing.
> return 0;
> }
>
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 52d4277..d99dedb 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1250,6 +1250,9 @@ static int vgic_v3_domain_init(struct domain *d)
>
> d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
>
> + if ( is_hardware_domain(d) && vgic_v3_hw.lpi_support )
> + d->arch.vgic.its_enabled = 1;
We don't want platform support ITS to use the vITS until everything is
supported in Xen. Otherwise you would break bisection.
This vITS inialization in the vGICv3 driver should be done in one of the
latest patch.
> +
> return 0;
> }
>
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index a5ab99d9..3555833 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -62,6 +62,11 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
> return vgic_get_rank(v, rank);
> }
>
> +bool_t vgic_is_lpi_supported(struct domain *d)
> +{
> + return d->arch.vgic.its_enabled;
> +}
Please move this helper as a static inline to allow the compiler
inlining the code.
> +
> static void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
> {
> INIT_LIST_HEAD(&p->inflight);
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 011f85b..0f6f9d8 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -93,6 +93,7 @@ struct arch_domain
> int ctlr;
> int nr_spis; /* Number of SPIs */
> int nr_lpis; /* Number of LPIs */
> + bool_t its_enabled;
The field is GICv3 specific and should go within the HAS_GICV3 section
below.
Although, I don't think having this boolean is necessary. You can know
whether we support LPIs or not by checking either the nr_lpis > 0 or
vits != NULL.
The former would be the best if you plan to keep an helper.
> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
> struct vgic_irq_rank *shared_irqs;
> /*
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 2bf061f..edeec9c 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -309,6 +309,7 @@ enum gic_sgi_mode;
>
> #define vgic_num_irq_lines(d) ((d)->arch.vgic.nr_spis + 32)
>
> +extern bool_t vgic_is_lpi_supported(struct domain *d);
> extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
> extern void domain_vgic_free(struct domain *d);
> extern int vcpu_vgic_init(struct vcpu *v);
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation
2015-09-18 13:09 ` [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation vijay.kilari
@ 2015-09-23 10:22 ` Julien Grall
2015-09-26 16:08 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-23 10:22 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 4285af3..37c2105 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -29,6 +29,7 @@
> #include <asm/atomic.h>
> #include <asm/gic_v3_defs.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <asm/vgic.h>
> #include <asm/gic-its.h>
> #include <asm/vits.h>
> @@ -852,6 +853,7 @@ int vits_domain_init(struct domain *d)
> vits = d->arch.vgic.vits;
>
> spin_lock_init(&vits->lock);
> + spin_lock_init(&vits->prop_lock);
Anything related to the LPI property table should have been moved in the
gic-v3.c and protected with proper check.
>
> vits->collections = xzalloc_array(struct its_collection,
> vits_get_max_collections(d));
> @@ -877,6 +879,8 @@ int vits_domain_init(struct domain *d)
>
> void vits_domain_free(struct domain *d)
> {
> + free_xenheap_pages(d->arch.vgic.vits->prop_page,
> + get_order_from_bytes(d->arch.vgic.vits->prop_size));
Ditto.
> xfree(d->arch.vgic.vits->collections);
> xfree(d->arch.vgic.vits);
> }
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index d99dedb..c631c7b 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -29,7 +29,10 @@
> #include <asm/current.h>
> #include <asm/mmio.h>
> #include <asm/gic_v3_defs.h>
> +#include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <asm/vgic.h>
> +#include <asm/vits.h>
>
> /* GICD_PIDRn register values for ARM implementations */
> #define GICV3_GICD_PIDR0 0x92
> @@ -106,23 +109,269 @@ static struct vcpu *vgic_v3_get_target_vcpu(struct vcpu *v, unsigned int irq)
> return v_target;
> }
>
> +static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
There is nothing vITS specific in this code. I was expecting you to
rename properly all the vits function you moved in the GICv3 code.
> +{
> + struct pending_irq *p;
> + struct vcpu *v_target = v->domain->vcpu[0];
> +
> + p = irq_to_pending(v_target, vlpi);
> + if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> + {
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + gic_remove_from_queues(v_target, vlpi);
> + }
> +}
> +
> +static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
The parameter priority is never used. Please drop it.
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> + struct vcpu *v_target = v->domain->vcpu[0];
> +
> + /*
> + * We don't have vlpi to plpi mapping and hence we cannot
> + * have target on which corresponding vlpi is enabled.
> + * So for now we are always injecting vlpi on vcpu0.
> + * (See vgic_vcpu_inject_lpi() function) and so we get pending_irq
> + * structure on vcpu0.
> + * TODO: Get correct target vcpu
> + */
I would much prefer to see get_vcpu_target implemented for LPIs which
will always return VCPU0 rather than open-coding it everywhere.
It will help later when we will properly support LPIs on other VCPU than
VCPU0.
> + p = irq_to_pending(v_target, vlpi);
> +
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + spin_lock_irqsave(&v_target->arch.vgic.lock, flags);
> +
> + if ( !list_empty(&p->inflight) &&
> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> + gic_raise_guest_irq(v_target, vlpi, p->priority);
> +
> + spin_unlock_irqrestore(&v_target->arch.vgic.lock, flags);
> +}
> +
> +static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + void *addr;
> + uint64_t val;
> + unsigned long flags;
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa - (vits->propbase & GICR_PROPBASER_PA_MASK);
> + addr = (void *)((u8*)vits->prop_page + offset);
Even if the current emulation doesn't allow to modify the LPI property
table, you still have to protect access to prop_page and propbase.
I would proptect those fields with the prop_lock rather than the
vits->lock. It would avoid having to take 2 locks everytime.
> +
> + spin_lock_irqsave(&vits->prop_lock, flags);
> + switch (dabt.size)
> + {
> + case DABT_DOUBLE_WORD:
> + val = *((u64*)addr);
I'm pretty sure you need a space between the type and the *. I.e
u64 *
> + *r = val;
There is no point to have a temporary variable. You can directly use
assign the value to *r. I.e
*r = *((u64 *)addr);
The remark is valid for all the other case in this switch.
> + break;
[...]
> +static void vgic_v3_gits_update_lpis_state(struct vcpu *v, uint32_t vid,
> + uint32_t size)
> +{
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + uint32_t i;
> + uint8_t cfg, *p;
> + bool_t enable;
> +
I would add a ASSERT(spin_is_locked(&vits->prop_lock));
> + p = ((u8*)vits->prop_page + vid);
> +
> + for ( i = 0 ; i < size; i++ )
> + {
> + cfg = *p;
> + enable = cfg & LPI_PROP_ENABLED;
> +
> + /* LPIs start from 8192, So add 8192 to point to correct LPI number */
> + if ( !enable )
> + vits_enable_lpi(v, vid + FIRST_GIC_LPI, (cfg & LPI_PRIORITY_MASK));
> + else
> + vits_disable_lpi(v, vid + FIRST_GIC_LPI);
> +
> + p++;
> + vid++;
> + }
> +}
> +
> +static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + uint8_t cfg, *p, *val, i, iter;
> + bool_t enable;
> + unsigned long flags;
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa - (vits->propbase & GICR_PROPBASER_PA_MASK);
Same remark as previously for propbase. It has to be protected by a lock.
> +
> + switch (dabt.size)
> + {
> + case DABT_DOUBLE_WORD:
> + iter = 8;
> + break;
> + case DABT_WORD:
> + iter = 4;
> + break;
> + case DABT_HALF_WORD:
> + iter = 2;
> + break;
> + default:
datb_size is an enum, you should enumerate all the case rather than
using the default label for the last item.
It would help the compiler to catch any missing item and make easier to
understand what is the last case.
Although, all this switch could be replace by a single line:
iter = 1 << dabt.size;
> + iter = 1;
> + }
> + spin_lock_irqsave(&vits->prop_lock, flags);
> +
> + p = ((u8*)vits->prop_page + offset);
> + val = (u8*)r;
> +
> + for ( i = 0 ; i < iter; i++ )
> + {
> + cfg = *p;
> + enable = (cfg & *val) & LPI_PROP_ENABLED;
You could have directly use *p rather than introducing cfg.
> +
> + if ( !enable )
> + vits_enable_lpi(v, offset + FIRST_GIC_LPI, (*val & LPI_PRIORITY_MASK));
> + else
> + vits_disable_lpi(v, offset + FIRST_GIC_LPI);
> +
> + /* Update virtual prop page */
> + *p = (*val & 0xff);
The mask is not necessary.
> + val++;
> + p++;
> + offset++;
> + }
> +
> + spin_unlock_irqrestore(&vits->prop_lock, flags);
> +
> + return 1;
> +}
> +
> +static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
> + .read_handler = vgic_v3_gits_lpi_mmio_read,
> + .write_handler = vgic_v3_gits_lpi_mmio_write,
> +};
> +
> +static int vits_map_lpi_prop(struct vcpu *v)
> +{
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + paddr_t gaddr, addr;
> + unsigned long mfn, flags;
> + uint32_t id_bits, vgic_id_bits;
> + int i;
> +
> + gaddr = vits->propbase & GICR_PROPBASER_PA_MASK;
> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
> +
> + vgic_id_bits = get_count_order(v->domain->arch.vgic.nr_lpis +
> + FIRST_GIC_LPI);
> + /*
> + * Here we limit the size of LPI property table to the number of LPIs
> + * that domain supports.
> + */
> + if ( id_bits > vgic_id_bits )
> + id_bits = vgic_id_bits;
As said on v4/v5, you are allowing the possibility to have a smaller
property table than the effective number of LPIs.
An ASSERT in vgic_v3_get_irq_priority (patch #25) doesn't ensure the
validity of the size of the property table provided by the guest. This
will surely crash Xen in debug mode, and who knows what will happen in
production mode.
We had a discussion about what to do on v5 and I was expecting from you
at least to put a TODO/comment in the code explaining what needs to be
done in order to fix the problem.
You could also have workaround the problem by always allocating the
prop_page using vgic_id_bits not matter the size provide by the caller.
> +
> + vits->prop_size = 1 << id_bits;
The field prop_size should only be updated if we succeed to allocat the
prop_page.
> +
> + /*
> + * Allocate Virtual LPI Property table.
> + * TODO: To re-use guest property table
> + */
> + vits->prop_page = alloc_xenheap_pages(get_order_from_bytes(vits->prop_size),
> + 0);
> + if ( !vits->prop_page )
> + {
> + printk(XENLOG_G_ERR
> + "d%d: vITS: Fail to allocate LPI Prop page\n",
> + v->domain->domain_id);
> + return 0;
> + }
> +
> + addr = gaddr;
> +
> + spin_lock_irqsave(&vits->prop_lock, flags);
On a previous version I clearly explain the consequence of syncing the
LPI property table. It's now getting worse with this lock because you
disable the interrupt.
Syncing the LPI property table may mean enabling an unknown amount
amount of LPIs. This could be very slow and you block the physical CPU
to receive any interrupt.
While this may be acceptable (?) for DOM0 is this unacceptable for a
guest. A valid guest could potentially block a CPU for a really long
time due to the number of LPIs.
> + /* Copy only configuration of supported LPIs (vgic.nr_lpis) */
> + for ( i = 0; i < v->domain->arch.vgic.nr_lpis / PAGE_SIZE; i++ )
As said above, the size of the property table may be smaller than the
number of LPIs supported. So you will copy garbagge.
Also please add a comment to explain why it's fine to assume that
nr_lpis is page aligned.
> + {
> + vgic_access_guest_memory(v->domain, addr,
> + (void *)(vits->prop_page + (i * PAGE_SIZE)), PAGE_SIZE, 0);
> +
> + vgic_v3_gits_update_lpis_state(v, (i * PAGE_SIZE), PAGE_SIZE);
> + addr += PAGE_SIZE;
> + }
> + spin_unlock_irqrestore(&vits->prop_lock, flags);
> +
> + /*
> + * Each re-distributor shares a common LPI configuration table
> + * So one set of mmio handlers to manage configuration table is enough
> + *
> + * Copy and unmap LPI property table.
> + */
> + addr = gaddr;
> + for ( i = 0; i < vits->prop_size / PAGE_SIZE; i++ )
Same remark as nr_lpis here.
> + {
> + mfn = gmfn_to_mfn(v->domain, paddr_to_pfn(addr));
> + if ( unlikely(!mfn_valid(mfn)) )
> + {
> + printk(XENLOG_G_ERR
> + "vITS: Invalid propbaser address for domain %d\n",
> + v->domain->domain_id);
It would have been nice to have the problematic address in the commit
message.
> + return 0;
> + }
> + guest_physmap_remove_page(v->domain, paddr_to_pfn(addr), mfn, 0);
> + addr += PAGE_SIZE;
> + }
> +
> + /* Register mmio handlers for this region */
> + register_mmio_handler(v->domain, &vgic_gits_lpi_mmio_handler,
> + gaddr, vits->prop_size);
> +
> + return 1;
> +}
> +
> static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> uint32_t gicr_reg)
> {
> struct hsr_dabt dabt = info->dabt;
> struct cpu_user_regs *regs = guest_cpu_user_regs();
> register_t *r = select_user_reg(regs, dabt.reg);
> + struct vgic_its *vits;
>
> switch ( gicr_reg )
> {
> case GICR_CTLR:
> - /* We have not implemented LPI's, read zero */
> - goto read_as_zero_32;
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + vgic_lock(v);
> + *r = vgic_reg32_read(v->domain->arch.vgic.gicr_ctlr, info);
The re-distributor is per-VCPU. Therefore, gicr_ctlr should be in arch_vcpu.
> + vgic_unlock(v);
> + return 1;
> case GICR_IIDR:
> if ( dabt.size != DABT_WORD ) goto bad_width;
> *r = vgic_reg32_read(GICV3_GICR_IIDR_VAL, info);
> return 1;
> case GICR_TYPER:
> + case GICR_TYPER + 4:
> {
> uint64_t typer, aff;
>
> @@ -137,6 +386,12 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
> typer |= GICR_TYPER_LAST;
>
> + /* Set Physical LPIs support */
> + if ( vgic_is_lpi_supported(v->domain) )
> + typer |= GICR_TYPER_PLPIS;
> + /* GITS_TYPER.PTA is 0. Provide vcpu number as target address */
Setting the processor in GICR_TYPER is not related to GITS_TYPER.PTA. It
could also be used for GICv2 compatibility
So please drop "GITS_TYPER.PTA is 0.".
> + typer |= (v->vcpu_id << GICR_TYPER_PROCESSOR_SHIFT);
> +
> *r = vgic_reg64_read(typer, info);
>
> return 1;
> @@ -154,10 +409,29 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> /* WO. Read as zero */
> goto read_as_zero_64;
> case GICR_PROPBASER:
> - /* LPI's not implemented */
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
AFAICT, this field be accessed using 32 bit access.
> + if ( vgic_is_lpi_supported(v->domain) )
> + {
> + vits = v->domain->arch.vgic.vits;
> + vgic_lock(v);
Based on my sugesstion above, I would lock probase with prop_lock to
avoid having to take multiple lock.
> + *r = vgic_reg64_read(vits->propbase, info);
> + vgic_unlock(v);
> + return 1;
> + }
Can you invert the condition to drop one layer of indentation (same for
all similar code within this patch).
I.e
if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
if ( !vgic_is_lpi_supported(v->domain) )
goto read_as_zero;
/* Emulation of the register */
> goto read_as_zero_64;
> case GICR_PENDBASER:
> - /* LPI's not implemented */
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
AFAICT, this field be accessed using 32 bit access.
> + if ( vgic_is_lpi_supported(v->domain) )
> + {
> + uint64_t val;
> +
> + vgic_lock(v);
> + val = vgic_reg64_read(v->arch.vgic.pendbase, info);
> + /* PTZ field is WO */
The more important reason is PTZ is RAZ not WO.
> + *r = val & ~GICR_PENDBASER_PTZ_MASK;
> + vgic_unlock(v);
> + return 1;
> + }
> goto read_as_zero_64;
> case GICR_INVLPIR:
> /* WO. Read as zero */
> @@ -232,7 +506,19 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> switch ( gicr_reg )
> {
> case GICR_CTLR:
> - /* LPI's not implemented */
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + if ( vgic_is_lpi_supported(v->domain) )
> + {
> + /*
> + * Enable LPI's for ITS. Direct injection of LPI
> + * by writing to GICR_{SET,CLR}LPIR is not supported.
> + */
> + vgic_lock(v);
> + vgic_reg32_write(&v->domain->arch.vgic.gicr_ctlr,
> + (*r & GICR_CTLR_ENABLE_LPIS), info);
> + vgic_unlock(v);
> + return 1;
> + }
> goto write_ignore_32;
> case GICR_IIDR:
> /* RO */
> @@ -253,10 +539,42 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> /* LPI is not implemented */
> goto write_ignore_64;
> case GICR_PROPBASER:
> - /* LPI is not implemented */
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
Any reason to not support 32-bit access?
> + if ( vgic_is_lpi_supported(v->domain) )
> + {
> + vgic_lock(v);
See my remark above about locking
> + /*
> + * LPI configuration tables are shared across cpus. Should be same.
> + *
> + * Allow updating on propbase only once with below check.
> + * TODO: Manage change in property table.
> + */
> + if ( v->domain->arch.vgic.vits->propbase != 0 )
> + {
> + printk(XENLOG_G_WARNING
> + "%pv: vGICR: Updating LPI propbase is not allowed\n", v);
> + vgic_unlock(v);
As said on v5, AFAICT this warning will be printed even in valid case.
i.e every VCPU except VCPU0.
This warning should only be printed when it's necessary to avoid
question from user about false positive.
If you disagree please explain why.
> + return 1;
> + }
> + /*
> + * TODO: As per spec, updating GICR_PROPBASER when GICR_CTLR.EnableLPIs = 1
This line is too long. If you invert the condition, it would drop one
layer of indentation.
> + * is unpredictable. Handle this scenario
> + */
> + vgic_reg64_write(&v->domain->arch.vgic.vits->propbase, *r, info);
> + vgic_unlock(v);
> + return vits_map_lpi_prop(v);
> + }
> goto write_ignore_64;
> case GICR_PENDBASER:
> - /* LPI is not implemented */
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
Any reason to not support 32-bit access?
> + if ( vgic_is_lpi_supported(v->domain) )
> + {
> + /* Just hold pendbaser value for guest read */
> + vgic_lock(v);
> + vgic_reg64_write(&v->arch.vgic.pendbase, *r, info);
> + vgic_unlock(v);
> + return 1;
> + }
> goto write_ignore_64;
> case GICR_INVLPIR:
> /* LPI is not implemented */
[...]
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 0f6f9d8..6a5c6a0 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -116,6 +116,7 @@ struct arch_domain
> uint32_t rdist_stride; /* Re-Distributor stride */
> /* Virtual ITS */
> struct vgic_its *vits;
> + uint32_t gicr_ctlr;
GICR_CTLR is per-redistributor i.e per-VCPU.
> #endif
> } vgic;
>
> @@ -249,6 +250,8 @@ struct arch_vcpu
>
> /* GICv3: redistributor base and flags for this vCPU */
> paddr_t rdist_base;
> + /* GICv3-ITS: LPI pending table for this vCPU */
> + paddr_t pendbase;
pendbase store a 64-bit register so it should be uint64_t.
> #define VGIC_V3_RDIST_LAST (1 << 0) /* last vCPU of the rdist */
> uint8_t flags;
> } vgic;
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 7936256..8f3d0fe 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -114,6 +114,7 @@
>
> #define LPI_PROP_ENABLED (1 << 0)
> #define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PRIORITY_MASK (0xfc)
>
> /*
> * Collection structure - just an ID, and a redistributor address to
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index f819589..5d39033 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -126,14 +126,17 @@
> #define GICR_PROPBASER_WaWb (5U << 7)
> #define GICR_PROPBASER_CACHEABILITY_MASK (7U << 7)
> #define GICR_PROPBASER_IDBITS_MASK (0x1f)
> +#define GICR_PROPBASER_PA_MASK (0xfffffffff000UL)
> #define GICR_TYPER_PLPIS (1U << 0)
> #define GICR_TYPER_VLPIS (1U << 1)
> #define GICR_TYPER_LAST (1U << 4)
> +#define GICR_TYPER_PROCESSOR_SHIFT (8)
>
> #define GICR_PENDBASER_InnerShareable (1U << 10)
> #define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
> #define GICR_PENDBASER_nC (1U << 7)
> #define GICR_PENDBASER_CACHEABILITY_MASK (7U << 7)
> +#define GICR_PENDBASER_PTZ_MASK (1UL << 62)
>
> #define DEFAULT_PMR_VALUE 0xff
>
> diff --git a/xen/include/asm-arm/vits.h b/xen/include/asm-arm/vits.h
> index d864c1e..e1188f7 100644
> --- a/xen/include/asm-arm/vits.h
> +++ b/xen/include/asm-arm/vits.h
> @@ -36,6 +36,14 @@ struct vgic_its
> atomic_t cmd_read;
> /* Command queue size */
> unsigned long cmd_qsize;
> + /* LPI propbase */
> + paddr_t propbase;
GICR_PROPBASER is a register so uint64_t
> + /* Virtual LPI property table */
> + void *prop_page;
> + /* Virtual LPI property size */
> + uint32_t prop_size;
> + /* spinlock to protect lpi property table */
> + spinlock_t prop_lock;
All those fields are not vITS specific but GICv3 specific. This should
go with the other GICv3 fields.
> /* ITS mmio physical base */
> paddr_t gits_base;
> /* ITS mmio physical size */
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 22/28] xen/arm: ITS: Allocate irq descriptors for LPIs
2015-09-18 13:09 ` [PATCH v7 22/28] xen/arm: ITS: Allocate irq descriptors for LPIs vijay.kilari
@ 2015-09-23 10:28 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-23 10:28 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Allocate dynamically irq descriptors for LPIs
This patch is the best example why we should only use nr_lpis to setup
the number of ID bits in the GICv3.
Here you are using gic_is_lpi to know if the IRQ is an LPIs. This
function is using the number of IRQ (gic_nr_irq_ids()).
Although, you allocate the LPIs using nr_lpis...
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v6: - Add separate patch for irq_pending structures
> - renamed and moved is_domain_lpi to vgic
> - Updated __irq_to_domain
> ---
> xen/arch/arm/gic-v3-its.c | 4 ++++
> xen/arch/arm/irq.c | 34 +++++++++++++++++++++++++++++++++-
> xen/include/asm-arm/irq.h | 1 +
> 3 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index f3346d3..4875b0f 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -1490,6 +1490,10 @@ int __init its_init(struct rdist_prop *rdists)
> its_alloc_lpi_tables();
> its_lpi_init(rdists->id_bits);
>
> + /* Allocate irq descriptors for LPIs */
> + if ( init_lpi() )
> + return -ENOMEM;
> +
> its = list_first_entry(&its_nodes, struct its_node, entry);
> /*
> * As per vITS design spec, Xen exposes only one virtual ITS per domain.
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 3a01f46..32c3b53 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -31,6 +31,8 @@
> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
> static DEFINE_SPINLOCK(local_irqs_type_lock);
>
> +static irq_desc_t *irq_desc_lpi;
> +
> /* Number of LPIs supported by Xen.
> *
> * The LPI identifier starts from 8192. Given that the hardware is
> @@ -73,7 +75,15 @@ static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
> irq_desc_t *__irq_to_desc(int irq)
> {
> if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
> - return &irq_desc[irq-NR_LOCAL_IRQS];
> + else if ( gic_is_lpi(irq) )
> + {
> + ASSERT(irq_desc_lpi != NULL);
> + return &irq_desc_lpi[irq - FIRST_GIC_LPI];
> + }
> + else
> + return &irq_desc[irq - NR_LOCAL_IRQS];
> +
> + return NULL;
> }
>
> int __init arch_init_one_irq_desc(struct irq_desc *desc)
> @@ -123,6 +133,28 @@ static int __cpuinit init_local_irq_data(void)
> return 0;
> }
>
> +int init_lpi(void)
> +{
> + struct irq_desc *desc;
> + unsigned int i;
> +
> + /* Allocate LPI irq descriptors */
> + irq_desc_lpi = xzalloc_array(struct irq_desc, nr_lpis);
Please use gic_nr_irq_ids()
> + if ( !irq_desc_lpi )
> + return -ENOSPC;
We can't allocate memory so the proper errno is ENOMEM.
> +
> + for ( i = 0; i < nr_lpis; i++ )
Same here.
> + {
> + desc = &irq_desc_lpi[i];
> + init_one_irq_desc(desc);
> + desc->irq = FIRST_GIC_LPI + i;
> + desc->arch.type = DT_IRQ_TYPE_EDGE_BOTH;
> + desc->action = NULL;
> + }
> +
> + return 0;
> +}
> +
> void __init init_IRQ(void)
> {
> int irq;
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation
2015-09-22 14:30 ` Julien Grall
@ 2015-09-24 5:07 ` Vijay Kilari
2015-09-24 11:05 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-24 5:07 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Tue, Sep 22, 2015 at 8:00 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
[...]
>> + case 0x0010 ... 0x007c:
>> + case 0xc000 ... 0xffcc:
>> + /* Implementation defined -- write ignored */
>> + goto write_ignore;
>> + case GITS_CBASER:
>> + /* XXX: support 32-bit access */
>
> You haven't asked my question on v6... What is missing to support 32-bt
> support? AFAICT the register helpers should the job for you. Correct?
GITS_CBASER.Physical Address holds bits[47:12], Being at
physical address at bits [47:12] cannot be updated with single 32-bit access.
I remember we discussed this on chat.
Regards
Vijay
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS
2015-09-23 8:31 ` Julien Grall
@ 2015-09-24 5:26 ` Vijay Kilari
2015-09-24 8:27 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-24 5:26 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Wed, Sep 23, 2015 at 2:01 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> index 0d5c61c..f3346d3 100644
>> --- a/xen/arch/arm/gic-v3-its.c
>> +++ b/xen/arch/arm/gic-v3-its.c
>> @@ -37,6 +37,7 @@
>> #include <asm/gic.h>
>> #include <asm/gic_v3_defs.h>
>> #include <asm/gic-its.h>
>> +#include <asm/vits.h>
>> #include <xen/log2.h>
>>
>> #define its_print(lvl, fmt, ...) \
>> @@ -88,6 +89,11 @@ struct its_node {
>> struct dt_device_node *dt_node;
>> };
>>
>> +static struct {
>> + uint8_t eventID_bits;
>> + uint8_t devID_bits;
>> +} its_data;
>> +
>> #define ITS_ITT_ALIGN SZ_256
>>
>> static LIST_HEAD(its_nodes);
>> @@ -914,6 +920,8 @@ int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
>>
>> for ( i = 0; i < pdev->event_map.nr_lpis; i++ )
>> {
>> + ASSERT(i < (1 << its_data.eventID_bits));
>
> I'd like to understand why you choose to use an ASSERT here.
>
> ASSERT means that we expect this condition never happen, although the
> nr_lpis is calculated using the number of MSI handled by a device.
>
> Given that we don't control the number of MSI, a proper check in
> its_add_device seem better than this ASSERT.
>
>> +
>> plpi = its_get_plpi(pdev, i);
>> /* TODO: Route lpi */
>> }
>> @@ -1366,6 +1374,8 @@ static int its_probe(struct dt_device_node *node)
>> its->phys_size = its_size;
>> typer = readl_relaxed(its_base + GITS_TYPER);
>> its->ite_size = ((typer >> 4) & 0xf) + 1;
>> + its_data.eventID_bits = GITS_TYPER_IDBITS(typer);
>> + its_data.devID_bits = GITS_TYPER_DEVBITS(typer);
>
> The 2 fields will be override every time a new ITS node will be initialized.
>
> What ensure that all the ITS have the same number of Event ID and Device ID?
Nothing of I thing of ensures this at HW level. Either we should
assume that all the ITS
have number of Event and Device ID or we choose lower of all the ITS?.
Regards
Vijay
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper
2015-09-23 8:52 ` Julien Grall
@ 2015-09-24 6:44 ` Vijay Kilari
2015-09-24 11:47 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-24 6:44 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Wed, Sep 23, 2015 at 2:22 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Introduce vgic_is_lpi_supported() helper function
>> to know virtual ITS availability for a domain
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> v7: - its_enabled field is added to vgic structure
>> ---
[...]
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index a5ab99d9..3555833 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -62,6 +62,11 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
>> return vgic_get_rank(v, rank);
>> }
>>
>> +bool_t vgic_is_lpi_supported(struct domain *d)
>> +{
>> + return d->arch.vgic.its_enabled;
>> +}
>
> Please move this helper as a static inline to allow the compiler
> inlining the code.
>
This function is used in vgic-v3.c later. Hence this function is exported.
>> +
>> static void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
>> {
>> INIT_LIST_HEAD(&p->inflight);
>> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
>> index 011f85b..0f6f9d8 100644
>> --- a/xen/include/asm-arm/domain.h
>> +++ b/xen/include/asm-arm/domain.h
>> @@ -93,6 +93,7 @@ struct arch_domain
>> int ctlr;
>> int nr_spis; /* Number of SPIs */
>> int nr_lpis; /* Number of LPIs */
>> + bool_t its_enabled;
>
> The field is GICv3 specific and should go within the HAS_GICV3 section
> below.
>
> Although, I don't think having this boolean is necessary. You can know
> whether we support LPIs or not by checking either the nr_lpis > 0 or
> vits != NULL.
>
> The former would be the best if you plan to keep an helper.
I will drop its_enabled field and helper will be updated as below.
bool_t vgic_is_lpi_supported(struct domain *d)
{
return (d->arch.vgic.nr_lpis != 0);
}
>
>> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>> struct vgic_irq_rank *shared_irqs;
>> /*
>> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
>> index 2bf061f..edeec9c 100644
>> --- a/xen/include/asm-arm/vgic.h
>> +++ b/xen/include/asm-arm/vgic.h
>> @@ -309,6 +309,7 @@ enum gic_sgi_mode;
>>
>> #define vgic_num_irq_lines(d) ((d)->arch.vgic.nr_spis + 32)
>>
>> +extern bool_t vgic_is_lpi_supported(struct domain *d);
>> extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
>> extern void domain_vgic_free(struct domain *d);
>> extern int vcpu_vgic_init(struct vcpu *v);
>>
>
> Regards,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS
2015-09-24 5:26 ` Vijay Kilari
@ 2015-09-24 8:27 ` Ian Campbell
2015-09-24 11:31 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-09-24 8:27 UTC (permalink / raw)
To: Vijay Kilari, Julien Grall
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On Thu, 2015-09-24 at 10:56 +0530, Vijay Kilari wrote:
> > > +
> > > plpi = its_get_plpi(pdev, i);
> > > /* TODO: Route lpi */
> > > }
> > > @@ -1366,6 +1374,8 @@ static int its_probe(struct dt_device_node
> > > *node)
> > > its->phys_size = its_size;
> > > typer = readl_relaxed(its_base + GITS_TYPER);
> > > its->ite_size = ((typer >> 4) & 0xf) + 1;
> > > + its_data.eventID_bits = GITS_TYPER_IDBITS(typer);
> > > + its_data.devID_bits = GITS_TYPER_DEVBITS(typer);
> >
> > The 2 fields will be override every time a new ITS node will be
> > initialized.
> >
> > What ensure that all the ITS have the same number of Event ID and
> > Device ID?
>
> Nothing of I thing of ensures this at HW level.
The question here is about the behaviour of the software emulation.
> Either we should
> assume that all the ITS
> have number of Event and Device ID or we choose lower of all the ITS?.
Don't we need, if anything, the highest?
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation
2015-09-24 5:07 ` Vijay Kilari
@ 2015-09-24 11:05 ` Julien Grall
2015-09-24 11:29 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-24 11:05 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 24/09/15 06:07, Vijay Kilari wrote:
> On Tue, Sep 22, 2015 at 8:00 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> [...]
>>> + case 0x0010 ... 0x007c:
>>> + case 0xc000 ... 0xffcc:
>>> + /* Implementation defined -- write ignored */
>>> + goto write_ignore;
>>> + case GITS_CBASER:
>>> + /* XXX: support 32-bit access */
>>
>> You haven't asked my question on v6... What is missing to support 32-bt
>> support? AFAICT the register helpers should the job for you. Correct?
>
> GITS_CBASER.Physical Address holds bits[47:12], Being at
> physical address at bits [47:12] cannot be updated with single 32-bit access.
I'd like you to explain why GITS_CBASER can't be updated with single
32-bit access. Because, based on the spec it's possible (see 8.19.2 ARM
IHI 0069A):
"Bits [63:32] and bits [31:0] are accessible separately."
> I remember we discussed this on chat.
If you are mentioning the chat we had back in June, this was based on
the hardware spec (PRD03-GENC-009432 23.0) where the support of 32-bit
access wasn't not clearly mandatory.
Although, we had multiple discussion after by the mail with you saying
that 32-bit access should be supported based on the developer spec (IHI
0069A 8.1.3).
After this discussion, you've implemented the 32-bit access on some
register, but left some unintended. A month ago, I provided to you a
patch series which introduce helpers to handle any size access on
register...
While Linux is mostly using 64-bit access on 64-bit register (expect for
GITS_TYPER where there is one 32-bit access), other OS may decide to use
32-bit access because they may want to support GICv3 on aarch32.
We did the mistake to left some access size not implemented on GICv3
which lead to FreeBSD crashing when booting as a Xen guest with GICv3.
We should avoid to reproduce knowingly the same mistake for the ITS.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation
2015-09-24 11:05 ` Julien Grall
@ 2015-09-24 11:29 ` Ian Campbell
2015-09-24 11:43 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-09-24 11:29 UTC (permalink / raw)
To: Julien Grall, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On Thu, 2015-09-24 at 12:05 +0100, Julien Grall wrote:
> On 24/09/15 06:07, Vijay Kilari wrote:
> > On Tue, Sep 22, 2015 at 8:00 PM, Julien Grall <julien.grall@citrix.com>
> > wrote:
> > > Hi Vijay,
> > >
> > > On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> > [...]
> > > > + case 0x0010 ... 0x007c:
> > > > + case 0xc000 ... 0xffcc:
> > > > + /* Implementation defined -- write ignored */
> > > > + goto write_ignore;
> > > > + case GITS_CBASER:
> > > > + /* XXX: support 32-bit access */
> > >
> > > You haven't asked my question on v6... What is missing to support 32
> > > -bt
> > > support? AFAICT the register helpers should the job for you. Correct?
> >
> > GITS_CBASER.Physical Address holds bits[47:12], Being at
> > physical address at bits [47:12] cannot be updated with single 32-bit
> > access.
>
> I'd like you to explain why GITS_CBASER can't be updated with single
> 32-bit access. Because, based on the spec it's possible (see 8.19.2 ARM
> IHI 0069A):
>
> "Bits [63:32] and bits [31:0] are accessible separately."
You clearly can't update the entire 47:12 range atomically with a 32-bit
access, which I suppose is what Vijay means.
However, I don't see why it wouldn't be possible to update either bits
[47:32] or [31:12] independently, by modifying just those bits. If for some
reason an OS wanted to do so.
More plausibly an OS might want to change some of the flag bits outside of
the [47:12] range, on either end with a single 32-bit write to just the
half they are trying to change. Writing just [63:32] to flip the valid bit
seems pretty plausible OS behaviour.
> Ian
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS
2015-09-24 8:27 ` Ian Campbell
@ 2015-09-24 11:31 ` Julien Grall
2015-09-24 11:41 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-24 11:31 UTC (permalink / raw)
To: Ian Campbell, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
Hi Ian,
On 24/09/15 09:27, Ian Campbell wrote:
> On Thu, 2015-09-24 at 10:56 +0530, Vijay Kilari wrote:
>
>>>> +
>>>> plpi = its_get_plpi(pdev, i);
>>>> /* TODO: Route lpi */
>>>> }
>>>> @@ -1366,6 +1374,8 @@ static int its_probe(struct dt_device_node
>>>> *node)
>>>> its->phys_size = its_size;
>>>> typer = readl_relaxed(its_base + GITS_TYPER);
>>>> its->ite_size = ((typer >> 4) & 0xf) + 1;
>>>> + its_data.eventID_bits = GITS_TYPER_IDBITS(typer);
>>>> + its_data.devID_bits = GITS_TYPER_DEVBITS(typer);
>>>
>>> The 2 fields will be override every time a new ITS node will be
>>> initialized.
>>>
>>> What ensure that all the ITS have the same number of Event ID and
>>> Device ID?
>>
>> Nothing of I thing of ensures this at HW level.
>
> The question here is about the behaviour of the software emulation.
>
>> Either we should
>> assume that all the ITS
>> have number of Event and Device ID or we choose lower of all the ITS?.
>
> Don't we need, if anything, the highest?
It's worth mentioning that there is no generic mechanism to notify if a
command is invalid. So passing an out of range DevID and EventID in a
command may potentially result to crashing Xen because we receive an SError.
So we can use the highest value only if we have all the safety in Xen to
prevent to send any command invalid out of range value.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS
2015-09-24 11:31 ` Julien Grall
@ 2015-09-24 11:41 ` Ian Campbell
0 siblings, 0 replies; 81+ messages in thread
From: Ian Campbell @ 2015-09-24 11:41 UTC (permalink / raw)
To: Julien Grall, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On Thu, 2015-09-24 at 12:31 +0100, Julien Grall wrote:
> Hi Ian,
>
> On 24/09/15 09:27, Ian Campbell wrote:
> > On Thu, 2015-09-24 at 10:56 +0530, Vijay Kilari wrote:
> >
> > > > > +
> > > > > plpi = its_get_plpi(pdev, i);
> > > > > /* TODO: Route lpi */
> > > > > }
> > > > > @@ -1366,6 +1374,8 @@ static int its_probe(struct dt_device_node
> > > > > *node)
> > > > > its->phys_size = its_size;
> > > > > typer = readl_relaxed(its_base + GITS_TYPER);
> > > > > its->ite_size = ((typer >> 4) & 0xf) + 1;
> > > > > + its_data.eventID_bits = GITS_TYPER_IDBITS(typer);
> > > > > + its_data.devID_bits = GITS_TYPER_DEVBITS(typer);
> > > >
> > > > The 2 fields will be override every time a new ITS node will be
> > > > initialized.
> > > >
> > > > What ensure that all the ITS have the same number of Event ID and
> > > > Device ID?
> > >
> > > Nothing of I thing of ensures this at HW level.
> >
> > The question here is about the behaviour of the software emulation.
> >
> > > Either we should
> > > assume that all the ITS
> > > have number of Event and Device ID or we choose lower of all the
> > > ITS?.
> >
> > Don't we need, if anything, the highest?
>
> It's worth mentioning that there is no generic mechanism to notify if a
> command is invalid. So passing an out of range DevID and EventID in a
> command may potentially result to crashing Xen because we receive an
> SError.
>
> So we can use the highest value only if we have all the safety in Xen to
> prevent to send any command invalid out of range value.
Oh, I thought this was the vits (so we could make whatever we wanted work).
If this is the physical ITS thing and these values can differ between
different ITSs in a system then surely these limits need to be recorded in
a per pits location.
Ian.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation
2015-09-24 11:29 ` Ian Campbell
@ 2015-09-24 11:43 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-24 11:43 UTC (permalink / raw)
To: Ian Campbell, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 24/09/15 12:29, Ian Campbell wrote:
> On Thu, 2015-09-24 at 12:05 +0100, Julien Grall wrote:
>> On 24/09/15 06:07, Vijay Kilari wrote:
>>> On Tue, Sep 22, 2015 at 8:00 PM, Julien Grall <julien.grall@citrix.com>
>>> wrote:
>>>> Hi Vijay,
>>>>
>>>> On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
>>> [...]
>>>>> + case 0x0010 ... 0x007c:
>>>>> + case 0xc000 ... 0xffcc:
>>>>> + /* Implementation defined -- write ignored */
>>>>> + goto write_ignore;
>>>>> + case GITS_CBASER:
>>>>> + /* XXX: support 32-bit access */
>>>>
>>>> You haven't asked my question on v6... What is missing to support 32
>>>> -bt
>>>> support? AFAICT the register helpers should the job for you. Correct?
>>>
>>> GITS_CBASER.Physical Address holds bits[47:12], Being at
>>> physical address at bits [47:12] cannot be updated with single 32-bit
>>> access.
>>
>> I'd like you to explain why GITS_CBASER can't be updated with single
>> 32-bit access. Because, based on the spec it's possible (see 8.19.2 ARM
>> IHI 0069A):
>>
>> "Bits [63:32] and bits [31:0] are accessible separately."
>
> You clearly can't update the entire 47:12 range atomically with a 32-bit
> access, which I suppose is what Vijay means.
The GITS_CBASER holds when address of the command queue which can only
be used when the ITS is enabled (i.e GITS_CTLR.Enable = 1).
As soon as GITS_CTLR.enable is set, it's not possible GITS_CBASER
becomes read-only. So we don't care about atomic access.
Although, the main problem is how the emulation for this register has
been written. As soon as the register is written, we are validating the
fields in the register and store in a convenient way. This give us
little possibility to correctly support 32-bit access.
Given that the register value is never used until GITS_CTLR.enable is
set, we could have validating GITS_CBASER and update our internal state
directly in GITS_CTLR.
Note that this remark is valid for any register relyiong on
GITS_CTLR.Enable and GICR_CTLR.Enable.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper
2015-09-24 6:44 ` Vijay Kilari
@ 2015-09-24 11:47 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-24 11:47 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 24/09/15 07:44, Vijay Kilari wrote:
> On Wed, Sep 23, 2015 at 2:22 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Introduce vgic_is_lpi_supported() helper function
>>> to know virtual ITS availability for a domain
>>>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>> ---
>>> v7: - its_enabled field is added to vgic structure
>>> ---
> [...]
>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>> index a5ab99d9..3555833 100644
>>> --- a/xen/arch/arm/vgic.c
>>> +++ b/xen/arch/arm/vgic.c
>>> @@ -62,6 +62,11 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
>>> return vgic_get_rank(v, rank);
>>> }
>>>
>>> +bool_t vgic_is_lpi_supported(struct domain *d)
>>> +{
>>> + return d->arch.vgic.its_enabled;
>>> +}
>>
>> Please move this helper as a static inline to allow the compiler
>> inlining the code.
>>
>
> This function is used in vgic-v3.c later. Hence this function is exported.
I though you knew that a static inline function can live in the header...
>
>>> +
>>> static void vgic_init_pending_irq(struct pending_irq *p, unsigned int virq)
>>> {
>>> INIT_LIST_HEAD(&p->inflight);
>>> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
>>> index 011f85b..0f6f9d8 100644
>>> --- a/xen/include/asm-arm/domain.h
>>> +++ b/xen/include/asm-arm/domain.h
>>> @@ -93,6 +93,7 @@ struct arch_domain
>>> int ctlr;
>>> int nr_spis; /* Number of SPIs */
>>> int nr_lpis; /* Number of LPIs */
>>> + bool_t its_enabled;
>>
>> The field is GICv3 specific and should go within the HAS_GICV3 section
>> below.
>>
>> Although, I don't think having this boolean is necessary. You can know
>> whether we support LPIs or not by checking either the nr_lpis > 0 or
>> vits != NULL.
>>
>> The former would be the best if you plan to keep an helper.
>
> I will drop its_enabled field and helper will be updated as below.
>
> bool_t vgic_is_lpi_supported(struct domain *d)
> {
> return (d->arch.vgic.nr_lpis != 0);
> }
+ static inline and move in the header.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 23/28] xen/arm: ITS: Allocate pending_lpi descriptors for LPIs
2015-09-18 13:09 ` [PATCH v7 23/28] xen/arm: ITS: Allocate pending_lpi " vijay.kilari
@ 2015-09-24 12:38 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-09-24 12:38 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
On 18/09/15 14:09, vijay.kilari@gmail.com wrote:
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 6a5c6a0..49630a3 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -101,6 +101,9 @@ struct arch_domain
> * struct arch_vcpu.
> */
> struct pending_irq *pending_irqs;
> +#ifdef HAS_GICV3
> + struct pending_irq *pending_lpis;
> +#endif
It's rather strange to expose nr_lpis, vgic_is_domain_lpi to anyone but
not pending_lpis.
Please either expose to anyone anything related to LPIs or nothing. But
not only some of them for convenience in the code.
As for physical LPIs I would much prefer to see this code compiled for
everyone.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation
2015-09-23 10:22 ` Julien Grall
@ 2015-09-26 16:08 ` Vijay Kilari
2015-09-28 9:53 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-26 16:08 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Julien,
On Wed, Sep 23, 2015 at 3:52 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
[...]
>> +static int vits_map_lpi_prop(struct vcpu *v)
>> +{
>> + struct vgic_its *vits = v->domain->arch.vgic.vits;
>> + paddr_t gaddr, addr;
>> + unsigned long mfn, flags;
>> + uint32_t id_bits, vgic_id_bits;
>> + int i;
>> +
>> + gaddr = vits->propbase & GICR_PROPBASER_PA_MASK;
>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>> +
>> + vgic_id_bits = get_count_order(v->domain->arch.vgic.nr_lpis +
>> + FIRST_GIC_LPI);
>> + /*
>> + * Here we limit the size of LPI property table to the number of LPIs
>> + * that domain supports.
>> + */
>> + if ( id_bits > vgic_id_bits )
>> + id_bits = vgic_id_bits;
>
> As said on v4/v5, you are allowing the possibility to have a smaller
> property table than the effective number of LPIs.
>
> An ASSERT in vgic_v3_get_irq_priority (patch #25) doesn't ensure the
> validity of the size of the property table provided by the guest. This
> will surely crash Xen in debug mode, and who knows what will happen in
> production mode.
>
> We had a discussion about what to do on v5 and I was expecting from you
> at least to put a TODO/comment in the code explaining what needs to be
> done in order to fix the problem.
>
> You could also have workaround the problem by always allocating the
> prop_page using vgic_id_bits not matter the size provide by the caller.
>
>> +
>> + vits->prop_size = 1 << id_bits;
>
> The field prop_size should only be updated if we succeed to allocat the
> prop_page.
>
>> +
>> + /*
>> + * Allocate Virtual LPI Property table.
>> + * TODO: To re-use guest property table
>> + */
>> + vits->prop_page = alloc_xenheap_pages(get_order_from_bytes(vits->prop_size),
>> + 0);
>> + if ( !vits->prop_page )
>> + {
>> + printk(XENLOG_G_ERR
>> + "d%d: vITS: Fail to allocate LPI Prop page\n",
>> + v->domain->domain_id);
>> + return 0;
>> + }
>> +
>> + addr = gaddr;
>> +
>> + spin_lock_irqsave(&vits->prop_lock, flags);
>
> On a previous version I clearly explain the consequence of syncing the
> LPI property table. It's now getting worse with this lock because you
> disable the interrupt.
LPI property table is accessed in interrupt context. So interrupts
are disabled.
In anycase GICR_PROPBASER can be updated only when
GICR_CTLR.Enable_LPIS = 0 ( I have missed this check ).
So we can enable/disable LPIs in GICR_CTLR instead of disabling interrupts.
But again question is (vgic.gicr_ctlr) is per vcpu. We have to disable
LPIs for this
vcpu and domain only. Check has to be made against gicr_ctlr value before
routing LPI for every domain?.
>
> Syncing the LPI property table may mean enabling an unknown amount
> amount of LPIs. This could be very slow and you block the physical CPU
> to receive any interrupt.
>
> While this may be acceptable (?) for DOM0 is this unacceptable for a
> guest. A valid guest could potentially block a CPU for a really long
> time due to the number of LPIs.
>
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation
2015-09-26 16:08 ` Vijay Kilari
@ 2015-09-28 9:53 ` Ian Campbell
2015-09-28 10:37 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-09-28 9:53 UTC (permalink / raw)
To: Vijay Kilari, Julien Grall
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On Sat, 2015-09-26 at 21:38 +0530, Vijay Kilari wrote:
> LPI property table is accessed in interrupt context. So interrupts
> are disabled.
Interrupts a reenabled while handling an interrupt, so a higher priority
interrupt can still interrupt things.
See the use of local_irq_enable in gic_interrupt.
Ian.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation
2015-09-28 9:53 ` Ian Campbell
@ 2015-09-28 10:37 ` Vijay Kilari
2015-09-28 11:03 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-09-28 10:37 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Sep 28, 2015 at 3:23 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Sat, 2015-09-26 at 21:38 +0530, Vijay Kilari wrote:
>
>> LPI property table is accessed in interrupt context. So interrupts
>> are disabled.
>
> Interrupts a reenabled while handling an interrupt, so a higher priority
> interrupt can still interrupt things.
>
> See the use of local_irq_enable in gic_interrupt.
true, for this reason, I have taken spin lock with disabled
interrrupts when reading guest LPI table.
Question is to avoid disabling interrupts for a long time for about 8K/16K lpis
state update.
The LPI state update from guest LPI property table is done when guest
updates GICR_CTLR.EnableLPIs to 1.
(I am doing this in next revision).
So we can make a check for guest's GICR_CTLR.EnableLPIs field before
injecting LPIs to the domain. As per design
GICR_CTLR.EnableLPIs should be 0 when updating guest property table.
i.e updating GICR_PROPBASER.
So that we don't need to disable interrupts(LPIs are already disabled)
while reading guest LPI table.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation
2015-09-28 10:37 ` Vijay Kilari
@ 2015-09-28 11:03 ` Julien Grall
2015-09-29 9:35 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-09-28 11:03 UTC (permalink / raw)
To: Vijay Kilari, Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 28/09/15 11:37, Vijay Kilari wrote:
> On Mon, Sep 28, 2015 at 3:23 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>> On Sat, 2015-09-26 at 21:38 +0530, Vijay Kilari wrote:
>>
>>> LPI property table is accessed in interrupt context. So interrupts
>>> are disabled.
>>
>> Interrupts a reenabled while handling an interrupt, so a higher priority
>> interrupt can still interrupt things.
>>
>> See the use of local_irq_enable in gic_interrupt.
>
> true, for this reason, I have taken spin lock with disabled
> interrrupts when reading guest LPI table.
>
> Question is to avoid disabling interrupts for a long time for about 8K/16K lpis
> state update.
Why only 8K/16K LPIs? From the ITS spec, the number of LPIs is encoded
on 32bit so it would be possible to have 500K LPIs...
> The LPI state update from guest LPI property table is done when guest
> updates GICR_CTLR.EnableLPIs to 1.
> (I am doing this in next revision).
>
> So we can make a check for guest's GICR_CTLR.EnableLPIs field before
> injecting LPIs to the domain. As per design
> GICR_CTLR.EnableLPIs should be 0 when updating guest property table.
> i.e updating GICR_PROPBASER.
GICR_CTLR is per vCPU and the property table is per domain. How would
you know when to parse the property table?
> So that we don't need to disable interrupts(LPIs are already disabled)
> while reading guest LPI table.
That won't help. As already said on a previous mail, it was for command
emulation, the vCPU will monopolize the physical CPU for a very long
time. That means that no other vCPU can run this physical CPU.
I think the best solution would be tasklet which will divide the parsing
in small chunk. This would avoid to get Xen stuck on the physical CPU
for a long time.
A tasklet would be created when GICR_PROPBASER is written. Any vCPU
sending an INV command would get block until the tasklet is finished.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation
2015-09-28 11:03 ` Julien Grall
@ 2015-09-29 9:35 ` Vijay Kilari
0 siblings, 0 replies; 81+ messages in thread
From: Vijay Kilari @ 2015-09-29 9:35 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Mon, Sep 28, 2015 at 4:33 PM, Julien Grall <julien.grall@citrix.com> wrote:
> On 28/09/15 11:37, Vijay Kilari wrote:
>> On Mon, Sep 28, 2015 at 3:23 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>>> On Sat, 2015-09-26 at 21:38 +0530, Vijay Kilari wrote:
>>>
>>>> LPI property table is accessed in interrupt context. So interrupts
>>>> are disabled.
>>>
>>> Interrupts a reenabled while handling an interrupt, so a higher priority
>>> interrupt can still interrupt things.
>>>
>>> See the use of local_irq_enable in gic_interrupt.
>>
>> true, for this reason, I have taken spin lock with disabled
>> interrrupts when reading guest LPI table.
>>
>> Question is to avoid disabling interrupts for a long time for about 8K/16K lpis
>> state update.
>
> Why only 8K/16K LPIs? From the ITS spec, the number of LPIs is encoded
> on 32bit so it would be possible to have 500K LPIs...
>
>> The LPI state update from guest LPI property table is done when guest
>> updates GICR_CTLR.EnableLPIs to 1.
>> (I am doing this in next revision).
>>
>> So we can make a check for guest's GICR_CTLR.EnableLPIs field before
>> injecting LPIs to the domain. As per design
>> GICR_CTLR.EnableLPIs should be 0 when updating guest property table.
>> i.e updating GICR_PROPBASER.
>
> GICR_CTLR is per vCPU and the property table is per domain. How would
> you know when to parse the property table?
property table is shared across all vCPUs. So we can parse the property table
on update of GICR_CTLR for vCPU0.
>
>> So that we don't need to disable interrupts(LPIs are already disabled)
>> while reading guest LPI table.
>
> That won't help. As already said on a previous mail, it was for command
> emulation, the vCPU will monopolize the physical CPU for a very long
> time. That means that no other vCPU can run this physical CPU.
>
> I think the best solution would be tasklet which will divide the parsing
> in small chunk. This would avoid to get Xen stuck on the physical CPU
> for a long time.
>
> A tasklet would be created when GICR_PROPBASER is written. Any vCPU
> sending an INV command would get block until the tasklet is finished.
Below is the psuedo code for handling LPI property table. Please
let me know if the locking mechanism is ok or not.
On GICR_CTLR update:
----------------------------------
case GICR_CTLR:
...
if ( v->vcpu_id == 0 && v->arch.vgic.gicr_ctlr && propbase == NULL )
{
spin_lock_irqsave(&vgic.lpi_enable_lock, flags);
vgic.enableLPIs = 0;
spin_unlock_irqrestore(&vgic.lpi_enable_lock, flags);
spin_lock_irqsave(&vgic.prop_lock, flags);
// Allocate memory for property table
// Copy guest LPI prop table to Xen table.
// Unmap guest table.
// Register mmio handlers.
spin_unlock_irqrestore(&vgic.prop_lock, flags);
schedule_tasklet();
return 1;
}
....
In Tasklet function
--------------------------
spin_lock(&vgic.prop_lock);
for ( i = 0 ; i < nr_lpis; i++ )
{
enable = prop_table[i] & LPI_PROP_ENABLED;
...
if ( !enable )
vgic_v3_enable_lpi(v, i + 8192);
else
vgic_v3_disable_lpi(v, i + 8192);
...
}
spin_unlock(&vgic.prop_lock);
spin_lock_irqsave(&vgic.lpi_enable_lock, flags);
vgic.enableLPIs = 1;
spin_unlock_irqrestore(&vgic.lpi_enable_lock, flags);
In vits_process_inv()
------------------------------
done = 0;
do {
spin_lock_irqsave(&vgic.lpi_enable_lock, flags);
if ( vgic.enableLPIs )
done = 1;
spin_unlock_irqrestore(&vgic.lpi_enable_lock, flags);
cpu_relax();
} while ( !done );
In vgic_vcpu_inject_irq()
----------------------------------
...
spin_lock_irqsave(&vgic.lpi_enable_lock, flags);
lpi_state = vgic.enableLPIs;
spin_unlock_irqrestore(&vgic.lpi_enable_lock, flags);
/*
* This function is called from interrupt context. For lpis,
get_irq_priority() callback
* takes prop_lock to read priority from lpi property table.
* So we take default priority if property table is under update by
tasklet (vgic.enableLPIs == 0).
*/
if ( vgic_is_domain_lpi(d, lpi) && !lpi_state )
{
/*
* LPI is received for this domain, while LPIs property table
* is not updated, set irq with default priority of LPI. Priority
is needed to enqueue this irq.
*/
priority = DEFAULT_LPI_PRIORITY; //0xa2?
}
else
priority = vgic_ops->get_irq_priority();
if ( vgic_is_domain_lpi(d, lpi) )
{
if ( vgic.enableLPIs && test_bit(GIC_IRQ_GUEST_ENABLED, &n->status) )
gic_raise_guest_irq(v, virq, priority);
}
else
{
if ( test_bit(GIC_IRQ_GUEST_ENABLED, &n->status) )
gic_raise_guest_irq(v, virq, priority);
}
...
^ permalink raw reply [flat|nested] 81+ messages in thread
end of thread, other threads:[~2015-09-29 9:35 UTC | newest]
Thread overview: 81+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-09-18 13:08 [PATCH v7 00/28] Add ITS support vijay.kilari
2015-09-18 13:08 ` [PATCH v7 01/28] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
2015-09-18 13:08 ` [PATCH v7 02/28] xen: Add log2 functionality vijay.kilari
2015-09-18 13:08 ` [PATCH v7 03/28] xen/arm: Set nr_cpu_ids to available number of cpus vijay.kilari
2015-09-18 13:08 ` [PATCH v7 04/28] xen/arm: Rename NR_IRQs and vgic_num_irqs helper function vijay.kilari
2015-09-21 10:40 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 05/28] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
2015-09-21 11:23 ` Julien Grall
2015-09-22 9:55 ` Vijay Kilari
2015-09-22 10:01 ` Julien Grall
2015-09-22 10:34 ` Vijay Kilari
2015-09-22 12:34 ` Julien Grall
2015-09-21 13:08 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 06/28] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
2015-09-18 14:15 ` Julien Grall
2015-09-21 11:26 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 07/28] xen/arm: ITS: Introduce msi_desc for LPIs vijay.kilari
2015-09-18 13:08 ` [PATCH v7 08/28] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
2015-09-21 13:47 ` Julien Grall
2015-09-22 7:33 ` Vijay Kilari
2015-09-22 13:19 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 09/28] xen/arm: ITS: Introduce gic_is_lpi helper function vijay.kilari
2015-09-21 14:20 ` Julien Grall
2015-09-22 9:10 ` Vijay Kilari
2015-09-22 13:24 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 10/28] xen/arm: ITS: Implement hw_irq_controller for LPIs vijay.kilari
2015-09-21 14:35 ` Julien Grall
2015-09-18 13:08 ` [PATCH v7 11/28] xen/arm: ITS: Enable compilation of physical ITS driver vijay.kilari
2015-09-18 13:08 ` [PATCH v7 12/28] xen/arm: ITS: Plumbing hw_irq_controller for LPIs vijay.kilari
2015-09-21 14:44 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 13/28] xen/arm: Move vgic rank locking inside get_irq_priority vijay.kilari
2015-09-21 14:49 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 14/28] xen/arm: ITS: Initialize physical ITS and export lpi support vijay.kilari
2015-09-21 15:20 ` Julien Grall
2015-09-22 9:17 ` Vijay Kilari
2015-09-22 9:45 ` Julien Grall
2015-09-22 10:05 ` Ian Campbell
2015-09-22 10:29 ` Julien Grall
2015-09-22 10:44 ` Ian Campbell
2015-09-22 12:31 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 15/28] xen/arm: ITS: Add virtual ITS driver vijay.kilari
2015-09-21 15:34 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 16/28] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
2015-09-22 13:47 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 17/28] xen/arm: ITS: Add GITS registers emulation vijay.kilari
2015-09-22 14:30 ` Julien Grall
2015-09-24 5:07 ` Vijay Kilari
2015-09-24 11:05 ` Julien Grall
2015-09-24 11:29 ` Ian Campbell
2015-09-24 11:43 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 18/28] xen/arm: ITS: Export ITS info to Virtual ITS vijay.kilari
2015-09-23 8:31 ` Julien Grall
2015-09-24 5:26 ` Vijay Kilari
2015-09-24 8:27 ` Ian Campbell
2015-09-24 11:31 ` Julien Grall
2015-09-24 11:41 ` Ian Campbell
2015-09-18 13:09 ` [PATCH v7 19/28] xen/arm: ITS: Store LPIs allocated per domain vijay.kilari
2015-09-23 8:37 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 20/28] xen/arm: ITS: Add virtual ITS availability check helper vijay.kilari
2015-09-23 8:52 ` Julien Grall
2015-09-24 6:44 ` Vijay Kilari
2015-09-24 11:47 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 21/28] xen/arm: ITS: Add GICR register emulation vijay.kilari
2015-09-23 10:22 ` Julien Grall
2015-09-26 16:08 ` Vijay Kilari
2015-09-28 9:53 ` Ian Campbell
2015-09-28 10:37 ` Vijay Kilari
2015-09-28 11:03 ` Julien Grall
2015-09-29 9:35 ` Vijay Kilari
2015-09-18 13:09 ` [PATCH v7 22/28] xen/arm: ITS: Allocate irq descriptors for LPIs vijay.kilari
2015-09-23 10:28 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 23/28] xen/arm: ITS: Allocate pending_lpi " vijay.kilari
2015-09-24 12:38 ` Julien Grall
2015-09-18 13:09 ` [PATCH v7 24/28] xen/arm: ITS: Route LPIs vijay.kilari
2015-09-18 13:09 ` [PATCH v7 25/28] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
2015-09-18 13:09 ` [PATCH v7 26/28] xen/arm: ITS: Map ITS translation space vijay.kilari
2015-09-18 13:09 ` [PATCH v7 27/28] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
2015-09-18 13:09 ` [PATCH v7 28/28] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
2015-09-22 15:09 ` Julien Grall
2015-09-18 13:51 ` [PATCH v7 00/28] Add ITS support Julien Grall
2015-09-21 6:52 ` Vijay Kilari
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).