* [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 9:01 ` Jan Beulich
2015-07-10 7:42 ` [PATCH v4 02/17] xen: Add log2 functionality vijay.kilari
` (15 subsequent siblings)
16 siblings, 1 reply; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Keir Fraser, Ian Campbell, vijay.kilari, Prasun.Kapoor,
Vijaya Kumar K, Ian Jackson, Jan Beulich, manish.jaggi
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
bitmap_find_next_zero_area helper function will be used
by physical ITS driver. This is imported from linux
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Keir Fraser <keir@xen.org>
CC: Tim Deegan <tim@xen.org>
---
v4: Removed spaces and added tabs
Moved ALIGN macro to lib.h
v3: Moved changes to xen/common/bitmap.c and
xen/include/xen/bitmap.h
---
xen/common/bitmap.c | 37 +++++++++++++++++++++++++++++++++++++
xen/include/xen/bitmap.h | 5 +++++
xen/include/xen/lib.h | 2 ++
3 files changed, 44 insertions(+)
diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index 61d1ea4..77c68b0 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -489,6 +489,43 @@ int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
}
EXPORT_SYMBOL(bitmap_allocate_region);
+/*
+ * bitmap_find_next_zero_area - find a contiguous aligned zero area
+ * @map: The address to base the search on
+ * @size: The bitmap size in bits
+ * @start: The bitnumber to start searching at
+ * @nr: The number of zeroed bits we're looking for
+ * @align_mask: Alignment mask for zero area
+ *
+ * The @align_mask should be one less than a power of 2; the effect is that
+ * the bit offset of all zero areas this function finds is multiples of that
+ * power of 2. A @align_mask of 0 means no alignment is required.
+ */
+unsigned long bitmap_find_next_zero_area(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask)
+{
+ unsigned long index, end, i;
+again:
+ index = find_next_zero_bit(map, size, start);
+
+ /* Align allocation */
+ index = ALIGN_MASK(index, align_mask);
+
+ end = index + nr;
+ if (end > size)
+ return end;
+ i = find_next_bit(map, end, index);
+ if (i < end) {
+ start = i + 1;
+ goto again;
+ }
+ return index;
+}
+EXPORT_SYMBOL(bitmap_find_next_zero_area);
+
#ifdef __BIG_ENDIAN
void bitmap_long_to_byte(uint8_t *bp, const unsigned long *lp, int nbits)
diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
index e2a3686..238b976 100644
--- a/xen/include/xen/bitmap.h
+++ b/xen/include/xen/bitmap.h
@@ -101,6 +101,11 @@ extern int bitmap_scnlistprintf(char *buf, unsigned int len,
extern int bitmap_find_free_region(unsigned long *bitmap, int bits, int order);
extern void bitmap_release_region(unsigned long *bitmap, int pos, int order);
extern int bitmap_allocate_region(unsigned long *bitmap, int pos, int order);
+extern unsigned long bitmap_find_next_zero_area(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask);
#define BITMAP_LAST_WORD_MASK(nbits) \
( \
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 4258912..e7d9d95 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -55,6 +55,8 @@
#define ROUNDUP(x, a) (((x) + (a) - 1) & ~((a) - 1))
+#define ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
+
#define reserve_bootmem(_p,_l) ((void)0)
struct domain;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-10 7:42 ` [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
@ 2015-07-10 9:01 ` Jan Beulich
2015-07-10 9:28 ` Vijay Kilari
2015-07-10 9:45 ` Vijay Kilari
0 siblings, 2 replies; 113+ messages in thread
From: Jan Beulich @ 2015-07-10 9:01 UTC (permalink / raw)
To: vijay.kilari
Cc: Keir Fraser, Ian Campbell, stefano.stabellini, Prasun.Kapoor,
tim, manish.jaggi, Ian Jackson, xen-devel, julien.grall,
stefano.stabellini, Vijaya Kumar K
>>> On 10.07.15 at 09:42, <vijay.kilari@gmail.com> wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> bitmap_find_next_zero_area helper function will be used
> by physical ITS driver. This is imported from linux
version? Certainly not any 4.x, i.e. nothing reasonably recent.
Jan
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-10 9:01 ` Jan Beulich
@ 2015-07-10 9:28 ` Vijay Kilari
2015-07-10 9:30 ` Vijay Kilari
2015-07-10 9:45 ` Vijay Kilari
1 sibling, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-10 9:28 UTC (permalink / raw)
To: Jan Beulich
Cc: Keir Fraser, Ian Campbell, Stefano Stabellini, Prasun Kapoor,
Tim Deegan, manish.jaggi, Ian Jackson, xen-devel, Julien Grall,
Stefano Stabellini, Vijaya Kumar K
On Fri, Jul 10, 2015 at 2:31 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 10.07.15 at 09:42, <vijay.kilari@gmail.com> wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> bitmap_find_next_zero_area helper function will be used
>> by physical ITS driver. This is imported from linux
>
> version? Certainly not any 4.x, i.e. nothing reasonably recent.
There is no recent changes to this file in linux.
Below is the last commit that I have taken
linux/log2.h: Fix rounddown_pow_of_two(1)
(13c07b0286d340275f2d97adf085cecda37ede37)
>
> Jan
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-10 9:28 ` Vijay Kilari
@ 2015-07-10 9:30 ` Vijay Kilari
0 siblings, 0 replies; 113+ messages in thread
From: Vijay Kilari @ 2015-07-10 9:30 UTC (permalink / raw)
To: Jan Beulich
Cc: Keir Fraser, Ian Campbell, Stefano Stabellini, Prasun Kapoor,
Tim Deegan, manish.jaggi, Ian Jackson, xen-devel, Julien Grall,
Stefano Stabellini, Vijaya Kumar K
On Fri, Jul 10, 2015 at 2:58 PM, Vijay Kilari <vijay.kilari@gmail.com> wrote:
> On Fri, Jul 10, 2015 at 2:31 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 10.07.15 at 09:42, <vijay.kilari@gmail.com> wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> bitmap_find_next_zero_area helper function will be used
>>> by physical ITS driver. This is imported from linux
>>
>> version? Certainly not any 4.x, i.e. nothing reasonably recent.
>
> There is no recent changes to this file in linux.
> Below is the last commit that I have taken
>
> linux/log2.h: Fix rounddown_pow_of_two(1)
> (13c07b0286d340275f2d97adf085cecda37ede37)
Sorry. this is for log2.h not for this patch
>
>
>>
>> Jan
>>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-10 9:01 ` Jan Beulich
2015-07-10 9:28 ` Vijay Kilari
@ 2015-07-10 9:45 ` Vijay Kilari
2015-07-10 10:07 ` Jan Beulich
1 sibling, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-10 9:45 UTC (permalink / raw)
To: Jan Beulich
Cc: Keir Fraser, Ian Campbell, Stefano Stabellini, Prasun Kapoor,
Tim Deegan, manish.jaggi, Ian Jackson, xen-devel, Julien Grall,
Stefano Stabellini, Vijaya Kumar K
On Fri, Jul 10, 2015 at 2:31 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 10.07.15 at 09:42, <vijay.kilari@gmail.com> wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> bitmap_find_next_zero_area helper function will be used
>> by physical ITS driver. This is imported from linux
>
> version? Certainly not any 4.x, i.e. nothing reasonably recent.
This is based on 3.18. However this function in 4.x is calling by
bitmap_find_next_zero_area_off() with align_offset set to 0.
So functionality wise this is same.
Regards
Vijay
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-10 9:45 ` Vijay Kilari
@ 2015-07-10 10:07 ` Jan Beulich
0 siblings, 0 replies; 113+ messages in thread
From: Jan Beulich @ 2015-07-10 10:07 UTC (permalink / raw)
To: Vijay Kilari
Cc: Keir Fraser, Ian Campbell, Stefano Stabellini, Prasun Kapoor,
Tim Deegan, manish.jaggi, Ian Jackson, xen-devel, Julien Grall,
Stefano Stabellini, Vijaya Kumar K
>>> On 10.07.15 at 11:45, <vijay.kilari@gmail.com> wrote:
> On Fri, Jul 10, 2015 at 2:31 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 10.07.15 at 09:42, <vijay.kilari@gmail.com> wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> bitmap_find_next_zero_area helper function will be used
>>> by physical ITS driver. This is imported from linux
>>
>> version? Certainly not any 4.x, i.e. nothing reasonably recent.
>
> This is based on 3.18. However this function in 4.x is calling by
> bitmap_find_next_zero_area_off() with align_offset set to 0.
> So functionality wise this is same.
Yeah, but why would we not take the most up-to-date variant?
Jan
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 02/17] xen: Add log2 functionality
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
2015-07-10 7:42 ` [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 7:42 ` [PATCH v4 03/17] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
` (14 subsequent siblings)
16 siblings, 0 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Keir Fraser, Ian Campbell, vijay.kilari, Prasun.Kapoor,
Vijaya Kumar K, Ian Jackson, Jan Beulich, manish.jaggi
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
log2 helper apis are ported from linux from
commit 13c07b0286d340275f2d97adf085cecda37ede37
(linux/log2.h: Fix rounddown_pow_of_two(1))
Changes made for xen are:
- Only required functionality is retained
- Replace fls_long with flsl
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Keir Fraser <keir@xen.org>
CC: Tim Deegan <tim@xen.org>
---
v4: - Only retained required functionality
- Replaced fls_long with flsl
- Removed fls_long implementation in bitops.h in v3 version
---
xen/include/xen/log2.h | 167 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 167 insertions(+)
diff --git a/xen/include/xen/log2.h b/xen/include/xen/log2.h
new file mode 100644
index 0000000..86bd861
--- /dev/null
+++ b/xen/include/xen/log2.h
@@ -0,0 +1,167 @@
+/*
+ * Copyright (C) 2006 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _XEN_LOG2_H
+#define _XEN_LOG2_H
+
+#include <xen/types.h>
+#include <xen/bitops.h>
+
+/*
+ * deal with unrepresentable constant logarithms
+ */
+extern __attribute__((const))
+int ____ilog2_NaN(void);
+
+/*
+ * non-constant log of base 2 calculators
+ * - the arch may override these in asm/bitops.h if they can be implemented
+ * more efficiently than using fls() and fls64()
+ * - the arch is not required to handle n==0 if implementing the fallback
+ */
+static inline __attribute__((const))
+int __ilog2_u32(u32 n)
+{
+ return fls(n) - 1;
+}
+
+static inline __attribute__((const))
+int __ilog2_u64(u64 n)
+{
+ return flsl(n) - 1;
+}
+
+/*
+ * round up to nearest power of two
+ */
+static inline __attribute__((const))
+unsigned long __roundup_pow_of_two(unsigned long n)
+{
+ return 1UL << flsl(n - 1);
+}
+
+/**
+ * ilog2 - log of base 2 of 32-bit or a 64-bit unsigned value
+ * @n - parameter
+ *
+ * constant-capable log of base 2 calculation
+ * - this can be used to initialise global variables from constant data, hence
+ * the massive ternary operator construction
+ *
+ * selects the appropriately-sized optimised version depending on sizeof(n)
+ */
+#define ilog2(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n) < 1 ? ____ilog2_NaN() : \
+ (n) & (1ULL << 63) ? 63 : \
+ (n) & (1ULL << 62) ? 62 : \
+ (n) & (1ULL << 61) ? 61 : \
+ (n) & (1ULL << 60) ? 60 : \
+ (n) & (1ULL << 59) ? 59 : \
+ (n) & (1ULL << 58) ? 58 : \
+ (n) & (1ULL << 57) ? 57 : \
+ (n) & (1ULL << 56) ? 56 : \
+ (n) & (1ULL << 55) ? 55 : \
+ (n) & (1ULL << 54) ? 54 : \
+ (n) & (1ULL << 53) ? 53 : \
+ (n) & (1ULL << 52) ? 52 : \
+ (n) & (1ULL << 51) ? 51 : \
+ (n) & (1ULL << 50) ? 50 : \
+ (n) & (1ULL << 49) ? 49 : \
+ (n) & (1ULL << 48) ? 48 : \
+ (n) & (1ULL << 47) ? 47 : \
+ (n) & (1ULL << 46) ? 46 : \
+ (n) & (1ULL << 45) ? 45 : \
+ (n) & (1ULL << 44) ? 44 : \
+ (n) & (1ULL << 43) ? 43 : \
+ (n) & (1ULL << 42) ? 42 : \
+ (n) & (1ULL << 41) ? 41 : \
+ (n) & (1ULL << 40) ? 40 : \
+ (n) & (1ULL << 39) ? 39 : \
+ (n) & (1ULL << 38) ? 38 : \
+ (n) & (1ULL << 37) ? 37 : \
+ (n) & (1ULL << 36) ? 36 : \
+ (n) & (1ULL << 35) ? 35 : \
+ (n) & (1ULL << 34) ? 34 : \
+ (n) & (1ULL << 33) ? 33 : \
+ (n) & (1ULL << 32) ? 32 : \
+ (n) & (1ULL << 31) ? 31 : \
+ (n) & (1ULL << 30) ? 30 : \
+ (n) & (1ULL << 29) ? 29 : \
+ (n) & (1ULL << 28) ? 28 : \
+ (n) & (1ULL << 27) ? 27 : \
+ (n) & (1ULL << 26) ? 26 : \
+ (n) & (1ULL << 25) ? 25 : \
+ (n) & (1ULL << 24) ? 24 : \
+ (n) & (1ULL << 23) ? 23 : \
+ (n) & (1ULL << 22) ? 22 : \
+ (n) & (1ULL << 21) ? 21 : \
+ (n) & (1ULL << 20) ? 20 : \
+ (n) & (1ULL << 19) ? 19 : \
+ (n) & (1ULL << 18) ? 18 : \
+ (n) & (1ULL << 17) ? 17 : \
+ (n) & (1ULL << 16) ? 16 : \
+ (n) & (1ULL << 15) ? 15 : \
+ (n) & (1ULL << 14) ? 14 : \
+ (n) & (1ULL << 13) ? 13 : \
+ (n) & (1ULL << 12) ? 12 : \
+ (n) & (1ULL << 11) ? 11 : \
+ (n) & (1ULL << 10) ? 10 : \
+ (n) & (1ULL << 9) ? 9 : \
+ (n) & (1ULL << 8) ? 8 : \
+ (n) & (1ULL << 7) ? 7 : \
+ (n) & (1ULL << 6) ? 6 : \
+ (n) & (1ULL << 5) ? 5 : \
+ (n) & (1ULL << 4) ? 4 : \
+ (n) & (1ULL << 3) ? 3 : \
+ (n) & (1ULL << 2) ? 2 : \
+ (n) & (1ULL << 1) ? 1 : \
+ (n) & (1ULL << 0) ? 0 : \
+ ____ilog2_NaN() \
+ ) : \
+ (sizeof(n) <= 4) ? \
+ __ilog2_u32(n) : \
+ __ilog2_u64(n) \
+ )
+
+/**
+ * roundup_pow_of_two - round the given value up to nearest power of two
+ * @n - parameter
+ *
+ * round the given value up to the nearest power of two
+ * - the result is undefined when n == 0
+ * - this can be used to initialise global variables from constant data
+ */
+#define roundup_pow_of_two(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n == 1) ? 1 : \
+ (1UL << (ilog2((n) - 1) + 1)) \
+ ) : \
+ __roundup_pow_of_two(n) \
+ )
+
+/**
+ * order_base_2 - calculate the (rounded up) base 2 order of the argument
+ * @n: parameter
+ *
+ * The first few values calculated by this routine:
+ * ob2(0) = 0
+ * ob2(1) = 0
+ * ob2(2) = 1
+ * ob2(3) = 2
+ * ob2(4) = 2
+ * ob2(5) = 3
+ * ... and so on.
+ */
+
+#define order_base_2(n) ilog2(roundup_pow_of_two(n))
+#endif /* _XEN_LOG2_H */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH v4 03/17] xen/arm: ITS: Port ITS driver to Xen
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
2015-07-10 7:42 ` [PATCH v4 01/17] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
2015-07-10 7:42 ` [PATCH v4 02/17] xen: Add log2 functionality vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 13:01 ` Ian Campbell
2015-07-15 10:23 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
` (13 subsequent siblings)
16 siblings, 2 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
The linux driver is based on 4.1 with below commit id
3ad2a5f57656a14d964b673a5a0e4ab0e583c870
Only following code from Linux ITS driver is ported
and compiled
- LPI initialization
- ITS configuration code
- Physical command queue management
- ITS command building
Also redistributor information is split into rdist and
rdist_prop structures.
The rdist_prop struct holds the redistributor common
information for all re-distributor and rdist struct
holds the per-cpu specific information.
This per-cpu rdist is defined as global and shared with
physical ITS driver.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: Major changes
- Redistributor refactoring patch is merged
- Fixed comments from v3 related to coding style and
removing duplicate code.
- Target address is stored from bits[48:16] to avoid
shifting of target address while building ITS commands
- Removed non-static functions
- Removed usage of command builder functions
- Changed its_cmd_block union to include mix of bit and unsigned
variable types to define ITS command structure
v3:
- Only required changes from Linux ITS driver is ported
- Xen coding style is followed.
---
xen/arch/arm/gic-v3-its.c | 882 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 15 +-
xen/include/asm-arm/gic-its.h | 171 +++++++
xen/include/asm-arm/gic.h | 1 +
xen/include/asm-arm/gic_v3_defs.h | 117 ++++-
5 files changed, 1179 insertions(+), 7 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
new file mode 100644
index 0000000..60ab646
--- /dev/null
+++ b/xen/arch/arm/gic-v3-its.c
@@ -0,0 +1,882 @@
+/*
+ * Copyright (C) 2013, 2014 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Xen changes:
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ * Copyright (C) 2014, 2015 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/config.h>
+#include <xen/bitops.h>
+#include <xen/init.h>
+#include <xen/mm.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/errno.h>
+#include <xen/delay.h>
+#include <xen/list.h>
+#include <xen/sizes.h>
+#include <xen/vmap.h>
+#include <asm/p2m.h>
+#include <asm/domain.h>
+#include <asm/io.h>
+#include <asm/device.h>
+#include <asm/gic.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
+#include <xen/log2.h>
+
+#define its_print(lvl, fmt, ...) \
+ printk("GIC-ITS:" fmt, ## __VA_ARGS__)
+
+#define its_err(fmt, ...) its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
+
+#define its_dbg(fmt, ...) \
+ its_print(XENLOG_DEBUG, fmt, ## __VA_ARGS__)
+
+#define its_info(fmt, ...) \
+ its_print(XENLOG_INFO, fmt, ## __VA_ARGS__)
+
+#define its_warn(fmt, ...) \
+ its_print(XENLOG_WARNING, fmt, ## __VA_ARGS__)
+
+//#define DEBUG_GIC_ITS
+
+#ifdef DEBUG_GIC_ITS
+# define DPRINTK(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1 << 0)
+#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
+
+/*
+ * The ITS structure - contains most of the infrastructure, with the
+ * msi_controller, the command queue, the collections, and the list of
+ * devices writing to it.
+ */
+struct its_node {
+ spinlock_t lock;
+ struct list_head entry;
+ void __iomem *base;
+ unsigned long phys_base;
+ unsigned long phys_size;
+ its_cmd_block *cmd_base;
+ its_cmd_block *cmd_write;
+ void *tables[GITS_BASER_NR_REGS];
+ u32 order[GITS_BASER_NR_REGS];
+ struct its_collection *collections;
+ u64 flags;
+ u32 ite_size;
+ struct dt_device_node *dt_node;
+};
+
+#define ITS_ITT_ALIGN SZ_256
+
+static LIST_HEAD(its_nodes);
+static DEFINE_SPINLOCK(its_lock);
+static struct rdist_prop *gic_rdists;
+
+#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
+
+#ifdef DEBUG_GIC_ITS
+void dump_cmd(its_cmd_block *cmd)
+{
+ printk("ITS: Phys_cmd CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
+}
+#endif
+
+#define ITS_CMD_QUEUE_SZ SZ_64K
+#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
+
+static u64 its_cmd_ptr_to_offset(struct its_node *its, its_cmd_block *ptr)
+{
+ return (ptr - its->cmd_base) * sizeof(*ptr);
+}
+
+static int its_queue_full(struct its_node *its)
+{
+ int widx;
+ int ridx;
+
+ widx = its->cmd_write - its->cmd_base;
+ ridx = readl_relaxed(its->base + GITS_CREADR) / sizeof(its_cmd_block);
+
+ /* This is incredibly unlikely to happen, unless the ITS locks up. */
+ if ( ((widx + 1) % ITS_CMD_QUEUE_NR_ENTRIES) == ridx )
+ return 1;
+
+ return 0;
+}
+
+static its_cmd_block *its_allocate_entry(struct its_node *its)
+{
+ its_cmd_block *cmd;
+ u32 count = 1000000; /* 1s! */
+
+ while ( its_queue_full(its) )
+ {
+ count--;
+ if ( !count )
+ {
+ its_err("ITS queue not draining\n");
+ return NULL;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+
+ cmd = its->cmd_write++;
+
+ /* Handle queue wrapping */
+ if (its->cmd_write == (its->cmd_base + ITS_CMD_QUEUE_NR_ENTRIES))
+ its->cmd_write = its->cmd_base;
+
+ return cmd;
+}
+
+static its_cmd_block *its_post_commands(struct its_node *its)
+{
+ u64 wr = its_cmd_ptr_to_offset(its, its->cmd_write);
+
+ writel_relaxed(wr, its->base + GITS_CWRITER);
+
+ return its->cmd_write;
+}
+
+static void its_flush_cmd(struct its_node *its, its_cmd_block *cmd)
+{
+ /*
+ * Make sure the commands written to memory are observable by
+ * the ITS.
+ */
+ if ( its->flags & ITS_FLAGS_CMDQ_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cmd, sizeof(*cmd));
+ else
+ dsb(ishst);
+}
+
+static void its_wait_for_range_completion(struct its_node *its,
+ its_cmd_block *from,
+ its_cmd_block *to)
+{
+ u64 rd_idx, from_idx, to_idx;
+ u32 count = 1000000; /* 1s! */
+
+ from_idx = its_cmd_ptr_to_offset(its, from);
+ to_idx = its_cmd_ptr_to_offset(its, to);
+
+ while ( 1 )
+ {
+ rd_idx = readl_relaxed(its->base + GITS_CREADR);
+ if ( rd_idx >= to_idx || rd_idx < from_idx )
+ break;
+
+ count--;
+ if ( !count )
+ {
+ its_err("ITS queue timeout\n");
+ return;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static void its_send_single_command(struct its_node *its,
+ its_cmd_block *src,
+ struct its_collection *sync_col)
+{
+ its_cmd_block *cmd, *sync_cmd, *next_cmd;
+ unsigned long flags;
+
+ BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
+
+ spin_lock_irqsave(&its->lock, flags);
+
+ cmd = its_allocate_entry(its);
+ if ( !cmd )
+ {
+ its_err("ITS can't allocate, dropping command\n");
+ spin_unlock_irqrestore(&its->lock, flags);
+ return;
+ }
+
+ memcpy(cmd, src, sizeof(its_cmd_block));
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(cmd);
+#endif
+ its_flush_cmd(its, cmd);
+
+ if ( sync_col )
+ {
+ sync_cmd = its_allocate_entry(its);
+ if ( !sync_cmd )
+ {
+ its_err("ITS can't SYNC, skipping\n");
+ goto post;
+ }
+ sync_cmd->sync.cmd = GITS_CMD_SYNC;
+ sync_cmd->sync.ta = sync_col->target_address;
+
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(sync_cmd);
+#endif
+ its_flush_cmd(its, sync_cmd);
+ }
+
+post:
+ next_cmd = its_post_commands(its);
+ spin_unlock_irqrestore(&its->lock, flags);
+
+ its_wait_for_range_completion(its, cmd, next_cmd);
+}
+
+static void its_send_mapc(struct its_node *its, struct its_collection *col,
+ int valid)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapc.cmd = GITS_CMD_MAPC;
+ cmd.mapc.col = col->col_id;
+ cmd.mapc.ta = col->target_address;
+ cmd.mapc.valid = !!valid;
+
+ its_send_single_command(its, &cmd, col);
+}
+
+static void its_send_invall(struct its_node *its, struct its_collection *col)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.invall.cmd = GITS_CMD_INVALL;
+ cmd.invall.col = col->col_id;
+
+ its_send_single_command(its, &cmd, NULL);
+}
+
+/*
+ * How we allocate LPIs:
+ *
+ * The GIC has id_bits bits for interrupt identifiers. From there, we
+ * must subtract 8192 which are reserved for SGIs/PPIs/SPIs. Then, as
+ * we allocate LPIs by chunks of 32, we can shift the whole thing by 5
+ * bits to the right.
+ *
+ * This gives us (((1UL << id_bits) - 8192) >> 5) possible allocations.
+ */
+#define IRQS_PER_CHUNK_SHIFT 5
+#define IRQS_PER_CHUNK (1 << IRQS_PER_CHUNK_SHIFT)
+
+static unsigned long *lpi_bitmap;
+static u32 lpi_chunks;
+
+static int its_lpi_to_chunk(int lpi)
+{
+ return (lpi - 8192) >> IRQS_PER_CHUNK_SHIFT;
+}
+
+int its_lpi_init(u32 id_bits)
+{
+ lpi_chunks = its_lpi_to_chunk(1UL << id_bits);
+
+ lpi_bitmap = xzalloc_bytes(BITS_TO_LONGS(lpi_chunks) * sizeof(long));
+ if ( !lpi_bitmap )
+ {
+ lpi_chunks = 0;
+ return -ENOMEM;
+ }
+
+ its_info("ITS: Allocated %d chunks for LPIs\n", (int)lpi_chunks);
+
+ return 0;
+}
+
+/*
+ * We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
+ * deal with (one configuration byte per interrupt). PENDBASE has to
+ * be 64kB aligned (one bit per LPI, plus 8192 bits for SPI/PPI/SGI).
+ */
+#define LPI_PROPBASE_SZ SZ_64K
+#define LPI_PENDBASE_SZ (LPI_PROPBASE_SZ / 8 + SZ_1K)
+
+/*
+ * This is how many bits of ID we need, including the useless ones.
+ */
+#define LPI_NRBITS ilog2(LPI_PROPBASE_SZ + SZ_8K)
+
+static int __init its_alloc_lpi_tables(void)
+{
+ paddr_t paddr;
+
+ gic_rdists->prop_page =
+ alloc_xenheap_pages(get_order_from_bytes(LPI_PROPBASE_SZ), 0);
+
+ if ( !gic_rdists->prop_page )
+ {
+ its_err("Failed to allocate PROPBASE\n");
+ return -ENOMEM;
+ }
+
+ paddr = __pa(gic_rdists->prop_page);
+ its_info("GIC: using LPI property table @%pa\n", &paddr);
+
+ /* Set LPI priority and Group-1, but disabled */
+ memset(gic_rdists->prop_page,
+ GIC_PRI_IRQ | LPI_PROP_GROUP1,
+ LPI_PROPBASE_SZ);
+
+ /* Make sure the GIC will observe the written configuration */
+ clean_and_invalidate_dcache_va_range(gic_rdists->prop_page,
+ LPI_PROPBASE_SZ);
+
+ return 0;
+}
+
+static const char *its_base_type_string[] = {
+ [GITS_BASER_TYPE_DEVICE] = "Devices",
+ [GITS_BASER_TYPE_VCPU] = "Virtual CPUs",
+ [GITS_BASER_TYPE_CPU] = "Physical CPUs",
+ [GITS_BASER_TYPE_COLLECTION] = "Interrupt Collections",
+ [GITS_BASER_TYPE_RESERVED5] = "Reserved (5)",
+ [GITS_BASER_TYPE_RESERVED6] = "Reserved (6)",
+ [GITS_BASER_TYPE_RESERVED7] = "Reserved (7)",
+};
+
+static void its_free_tables(struct its_node *its)
+{
+ int i;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ if ( its->tables[i] )
+ {
+ free_xenheap_pages(its->tables[i], its->order[i]);
+ its->tables[i] = NULL;
+ its->order[i] = 0;
+ }
+ }
+}
+
+static int its_alloc_tables(struct its_node *its)
+{
+ int err;
+ int i;
+ int psz = SZ_64K;
+ u64 shr = GITS_BASER_InnerShareable;
+ u64 cache = GITS_BASER_WaWb;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
+ u64 type = GITS_BASER_TYPE(val);
+ u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
+ unsigned int order = get_order_from_bytes(psz);
+ int alloc_size;
+ u64 tmp;
+ void *base;
+
+ if ( type == GITS_BASER_TYPE_NONE )
+ continue;
+
+ /*
+ * Allocate as many entries as required to fit the
+ * range of device IDs that the ITS can grok... The ID
+ * space being incredibly sparse, this results in a
+ * massive waste of memory.
+ *
+ * For other tables, only allocate a single page.
+ */
+ if ( type == GITS_BASER_TYPE_DEVICE )
+ {
+ u64 typer = readq_relaxed(its->base + GITS_TYPER);
+ u32 ids = GITS_TYPER_DEVBITS(typer);
+
+ order = max(get_order_from_bytes((1UL << ids) * entry_size), order);
+ if (order >= MAX_ORDER)
+ {
+ order = MAX_ORDER - 1;
+ its_warn("Device Table too large,reduce its page order to %u\n",
+ order);
+ }
+ }
+
+ alloc_size = (1 << order) * PAGE_SIZE;
+ base = alloc_xenheap_pages(order, 0);
+ if ( !base )
+ {
+ err = -ENOMEM;
+ goto out_free;
+ }
+ memset(base, 0, alloc_size);
+ its->tables[i] = base;
+ its->order[i] = order;
+
+retry_baser:
+ val = (__pa(base) |
+ (type << GITS_BASER_TYPE_SHIFT) |
+ ((entry_size - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
+ cache |
+ shr |
+ GITS_BASER_VALID);
+
+ switch (psz) {
+ case SZ_4K:
+ val |= GITS_BASER_PAGE_SIZE_4K;
+ break;
+ case SZ_16K:
+ val |= GITS_BASER_PAGE_SIZE_16K;
+ break;
+ case SZ_64K:
+ val |= GITS_BASER_PAGE_SIZE_64K;
+ break;
+ }
+
+ val |= (alloc_size / psz) - 1;
+
+ writeq_relaxed(val, its->base + GITS_BASER + i * 8);
+ tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
+
+ if ( (val ^ tmp) & GITS_BASER_SHAREABILITY_MASK )
+ {
+ /*
+ * Shareability didn't stick. Just use
+ * whatever the read reported, which is likely
+ * to be the only thing this redistributor
+ * supports.
+ */
+ shr = tmp & GITS_BASER_SHAREABILITY_MASK;
+ if ( !shr )
+ cache = GITS_BASER_nC;
+ goto retry_baser;
+ }
+
+ if ( (val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK )
+ {
+ /*
+ * Page size didn't stick. Let's try a smaller
+ * size and retry. If we reach 4K, then
+ * something is horribly wrong...
+ */
+ switch (psz) {
+ case SZ_16K:
+ psz = SZ_4K;
+ goto retry_baser;
+ case SZ_64K:
+ psz = SZ_16K;
+ goto retry_baser;
+ }
+ }
+
+ if ( val != tmp )
+ {
+ its_err("ITS: GITS_BASER%d doesn't stick: %lx %lx\n",
+ i, (unsigned long) val, (unsigned long) tmp);
+ err = -ENXIO;
+ goto out_free;
+ }
+
+ its_info("ITS: allocated %d %s @%lx (psz %dK, shr %d)\n",
+ (int)(alloc_size / entry_size),
+ its_base_type_string[type],
+ (unsigned long)__pa(base),
+ psz / SZ_1K, (int)shr >> GITS_BASER_SHAREABILITY_SHIFT);
+ }
+
+ return 0;
+
+out_free:
+ its_free_tables(its);
+
+ return err;
+}
+
+static int its_alloc_collections(struct its_node *its)
+{
+ its->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
+ if ( !its->collections )
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void its_cpu_init_lpis(void)
+{
+ void __iomem *rbase = gic_data_rdist().rbase;
+ void *pend_page;
+ u64 val, tmp;
+
+ /* If we didn't allocate the pending table yet, do it now */
+ pend_page = gic_data_rdist().pend_page;
+ if ( !pend_page )
+ {
+ paddr_t paddr;
+ u32 order;
+ /*
+ * The pending pages have to be at least 64kB aligned,
+ * hence the 'max(LPI_PENDBASE_SZ, SZ_64K)' below.
+ */
+ order = get_order_from_bytes(max(LPI_PENDBASE_SZ, SZ_64K));
+ pend_page = alloc_xenheap_pages(order, 0);
+ if ( !pend_page )
+ {
+ its_err("Failed to allocate PENDBASE for CPU%d with order %d\n",
+ smp_processor_id(), order);
+ return;
+ }
+
+ memset(pend_page, 0, max(LPI_PENDBASE_SZ, SZ_64K));
+ /* Make sure the GIC will observe the zero-ed page */
+ clean_and_invalidate_dcache_va_range(pend_page, LPI_PENDBASE_SZ);
+
+ paddr = __pa(pend_page);
+
+ its_info("CPU%d: using LPI pending table @%pa\n",
+ smp_processor_id(), &paddr);
+
+ gic_data_rdist().pend_page = pend_page;
+ }
+
+ /* Disable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val &= ~GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /*
+ * Make sure any change to the table is observable by the GIC.
+ */
+ dsb(sy);
+
+ /* set PROPBASE */
+ val = (__pa(gic_rdists->prop_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb |
+ ((LPI_NRBITS - 1) & GICR_PROPBASER_IDBITS_MASK));
+
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ tmp = readq_relaxed(rbase + GICR_PROPBASER);
+
+ if ( (tmp ^ val) & GICR_PROPBASER_SHAREABILITY_MASK )
+ {
+ if ( !(tmp & GICR_PROPBASER_SHAREABILITY_MASK) )
+ {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as well.
+ */
+ val &= ~(GICR_PROPBASER_SHAREABILITY_MASK |
+ GICR_PROPBASER_CACHEABILITY_MASK);
+ val |= GICR_PROPBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ }
+
+ its_info("GIC: using cache flushing for LPI property table\n");
+ gic_rdists->flags |= RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING;
+ }
+
+ /* set PENDBASE */
+ val = (__pa(pend_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb);
+
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+ tmp = readq_relaxed(rbase + GICR_PENDBASER);
+
+ if ( !(tmp & GICR_PENDBASER_SHAREABILITY_MASK) )
+ {
+ /*
+ * The HW reports non-shareable, we must remove the
+ * cacheability attributes as well.
+ */
+ val &= ~(GICR_PENDBASER_SHAREABILITY_MASK |
+ GICR_PENDBASER_CACHEABILITY_MASK);
+ val |= GICR_PENDBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+ }
+
+ /* Enable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val |= GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /* Make sure the GIC has seen the above */
+ dsb(sy);
+}
+
+static void its_cpu_init_collection(void)
+{
+ struct its_node *its;
+ int cpu;
+
+ spin_lock(&its_lock);
+ cpu = smp_processor_id();
+
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ u64 target;
+ /*
+ * We now have to bind each collection to its target
+ * redistributor.
+ */
+ if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
+ {
+ /*
+ * This ITS wants the physical address of the
+ * redistributor.
+ */
+ target = gic_data_rdist().phys_base;
+ /*
+ * If Target address is GICR adddress, then it is aligned to 64K
+ * and hence ITS command field is only consider 32 bit skipping
+ * lower 16 bits.So take bit[48:16]
+ */
+ its->collections[cpu].target_address = target >> 16;
+ }
+ else
+ {
+ /*
+ * This ITS wants a linear CPU number.
+ */
+ target = readq_relaxed(gic_data_rdist().rbase + GICR_TYPER);
+ target = GICR_TYPER_CPU_NUMBER(target);
+ its->collections[cpu].target_address = target;
+ }
+
+ /* Perform collection mapping */
+ its->collections[cpu].col_id = cpu;
+
+ its_send_mapc(its, &its->collections[cpu], 1);
+ its_send_invall(its, &its->collections[cpu]);
+ }
+
+ spin_unlock(&its_lock);
+}
+
+static int its_force_quiescent(void __iomem *base)
+{
+ u32 count = 1000000; /* 1s */
+ u32 val;
+
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ /* Disable the generation of all interrupts to this ITS */
+ val &= ~GITS_CTLR_ENABLE;
+ writel_relaxed(val, base + GITS_CTLR);
+
+ /* Poll GITS_CTLR and wait until ITS becomes quiescent */
+ while ( 1 )
+ {
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ count--;
+ if ( !count )
+ return -EBUSY;
+
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static int its_probe(struct dt_device_node *node)
+{
+ paddr_t its_addr, its_size;
+ struct its_node *its;
+ void __iomem *its_base;
+ u32 val, typer;
+ u64 baser, tmp;
+ int err;
+
+ if ( !dt_get_property(node, "msi-controller", NULL) )
+ {
+ its_warn("%s: not a msi-controller\n", node->full_name);
+ return -ENXIO;
+ }
+
+ err = dt_device_get_address(node, 0, &its_addr, &its_size);
+ if ( err )
+ {
+ its_warn("%s: no regs?\n", node->full_name);
+ return -ENXIO;
+ }
+
+ its_base = ioremap_nocache(its_addr, its_size);
+ if ( !its_base )
+ {
+ its_warn("%s: unable to map registers\n", node->full_name);
+ return -ENOMEM;
+ }
+
+ val = readl_relaxed(its_base + GITS_PIDR2) & GIC_PIDR2_ARCH_REV_MASK;
+ if ( val != 0x30 && val != 0x40 )
+ {
+ its_warn("%s: no ITS detected, giving up\n", node->full_name);
+ err = -ENODEV;
+ goto out_unmap;
+ }
+
+ err = its_force_quiescent(its_base);
+ if ( err )
+ {
+ its_warn("%s: failed to quiesce, giving up\n",
+ node->full_name);
+ goto out_unmap;
+ }
+
+ its_info("ITS: %s\n", node->full_name);
+
+ its = xzalloc(struct its_node);
+ if ( !its )
+ {
+ err = -ENOMEM;
+ goto out_unmap;
+ }
+
+ spin_lock_init(&its->lock);
+ INIT_LIST_HEAD(&its->entry);
+ its->dt_node = node;
+ its->base = its_base;
+ its->phys_base = its_addr;
+ its->phys_size = its_size;
+ typer = readl_relaxed(its_base + GITS_TYPER);
+ its->ite_size = ((typer >> 4) & 0xf) + 1;
+
+ its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
+ if ( !its->cmd_base )
+ {
+ err = -ENOMEM;
+ goto out_free_its;
+ }
+ its->cmd_write = its->cmd_base;
+
+ err = its_alloc_tables(its);
+ if ( err )
+ goto out_free_cmd;
+
+ err = its_alloc_collections(its);
+ if ( err )
+ goto out_free_tables;
+
+ baser = (__pa(its->cmd_base) |
+ GITS_CBASER_WaWb |
+ GITS_CBASER_InnerShareable |
+ (ITS_CMD_QUEUE_SZ / SZ_4K - 1) |
+ GITS_CBASER_VALID);
+
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ tmp = readq_relaxed(its->base + GITS_CBASER);
+ if ( (tmp ^ baser) & GITS_CBASER_SHAREABILITY_MASK )
+ {
+ if (!(tmp & GITS_CBASER_SHAREABILITY_MASK))
+ {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as
+ * well.
+ */
+ baser &= ~(GITS_CBASER_SHAREABILITY_MASK |
+ GITS_CBASER_CACHEABILITY_MASK);
+ baser |= GITS_CBASER_nC;
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ }
+
+ its_info("ITS: using cache flushing for cmd queue\n");
+ its->flags |= ITS_FLAGS_CMDQ_NEEDS_FLUSHING;
+ }
+
+ writeq_relaxed(0, its->base + GITS_CWRITER);
+ writel_relaxed(GITS_CTLR_ENABLE, its->base + GITS_CTLR);
+
+ spin_lock(&its_lock);
+ list_add(&its->entry, &its_nodes);
+ spin_unlock(&its_lock);
+
+ return 0;
+
+out_free_tables:
+ its_free_tables(its);
+out_free_cmd:
+ xfree(its->cmd_base);
+out_free_its:
+ xfree(its);
+out_unmap:
+ iounmap(its_base);
+ its_err("ITS: failed probing %s (%d)\n", node->full_name, err);
+ return err;
+}
+
+static bool gic_rdists_supports_plpis(void)
+{
+ return !!(readl_relaxed(gic_data_rdist().rbase + GICR_TYPER) & GICR_TYPER_PLPIS);
+}
+
+int its_cpu_init(void)
+{
+ if ( !list_empty(&its_nodes) )
+ {
+ if ( !gic_rdists_supports_plpis() )
+ {
+ its_info("CPU%d: LPIs not supported\n", smp_processor_id());
+ return -ENXIO;
+ }
+ its_cpu_init_lpis();
+ its_cpu_init_collection();
+ }
+
+ return 0;
+}
+
+int __init its_init(struct rdist_prop *rdists)
+{
+ struct dt_device_node *np = NULL;
+
+ static const struct dt_device_match its_device_ids[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+
+ for (np = dt_find_matching_node(NULL, its_device_ids); np;
+ np = dt_find_matching_node(np, its_device_ids))
+ its_probe(np);
+
+ if ( list_empty(&its_nodes) )
+ {
+ its_warn("ITS: No ITS available, not enabling LPIs\n");
+ return -ENXIO;
+ }
+
+ gic_rdists = rdists;
+ its_alloc_lpi_tables();
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 30682cf..b5c59f6 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -53,6 +53,7 @@ static struct {
paddr_t dbase; /* Address of distributor registers */
paddr_t dbase_size;
void __iomem *map_dbase; /* Mapped address of distributor registers */
+ struct rdist_prop rdist_data;
struct rdist_region *rdist_regions;
uint32_t rdist_stride;
unsigned int rdist_count; /* Number of rdist regions count */
@@ -63,10 +64,10 @@ static struct {
static struct gic_info gicv3_info;
/* per-cpu re-distributor base */
-static DEFINE_PER_CPU(void __iomem*, rbase);
+DEFINE_PER_CPU(struct rdist, rdist);
#define GICD (gicv3.map_dbase)
-#define GICD_RDIST_BASE (this_cpu(rbase))
+#define GICD_RDIST_BASE (per_cpu(rdist, smp_processor_id()).rbase)
#define GICD_RDIST_SGI_BASE (GICD_RDIST_BASE + SZ_64K)
/*
@@ -613,6 +614,7 @@ static int __init gicv3_populate_rdist(void)
uint32_t aff;
uint32_t reg;
uint64_t typer;
+ uint64_t offset;
uint64_t mpidr = cpu_logical_map(smp_processor_id());
/*
@@ -648,9 +650,12 @@ static int __init gicv3_populate_rdist(void)
if ( (typer >> 32) == aff )
{
- this_cpu(rbase) = ptr;
- printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
- smp_processor_id(), i, ptr);
+ offset = ptr - gicv3.rdist_regions[i].map_base;
+ per_cpu(rdist, smp_processor_id()).rbase = ptr;
+ per_cpu(rdist, smp_processor_id()).phys_base = gicv3.rdist_regions[i].base + offset;
+ printk("GICv3: CPU%d: Found redistributor in region %d @%"PRIpaddr"\n",
+ smp_processor_id(), i,
+ per_cpu(rdist, smp_processor_id()).phys_base);
return 0;
}
if ( gicv3.rdist_stride )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
new file mode 100644
index 0000000..d24b039
--- /dev/null
+++ b/xen/include/asm-arm/gic-its.h
@@ -0,0 +1,171 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_ARM_GIC_ITS_H__
+#define __ASM_ARM_GIC_ITS_H__
+
+#include <asm/gic_v3_defs.h>
+
+/*
+ * Collection structure - just an ID, and a redistributor address to
+ * ping. We use one per CPU as collection of interrupts assigned to this
+ * CPU.
+ */
+struct its_collection {
+ u64 target_address;
+ u16 col_id;
+};
+
+/* ITS command structure */
+typedef union {
+ u64 bits[4];
+ struct __packed {
+ uint8_t cmd;
+ uint8_t pad[7];
+ } hdr;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u64 size:5;
+ u64 res2:59;
+ /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
+ u64 res3:8;
+ u64 itt:40;
+ u64 res4:15;
+ u64 valid:1;
+ u64 res5;
+ } mapd;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 col;
+ u32 ta;
+ u16 res3:15;
+ u16 valid:1;
+ u64 res4;
+ } mapc;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } mapi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 phy_id;
+ u16 col;
+ u8 res2[6];
+ u64 res3;
+ } mapvi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } movi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } discard;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } inv;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 res3;
+ u32 ta1;
+ u16 res4;
+ u16 res5;
+ u32 ta2;
+ u16 res6;
+ } movall;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } invall;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } int_cmd;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } clear;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 res3;
+ u32 ta;
+ u16 res4;
+ u64 res5;
+ } sync;
+} its_cmd_block;
+
+int its_lpi_init(u32 id_bits);
+int its_init(struct rdist_prop *rdists);
+int its_cpu_init(void);
+
+#endif /* __ASM_ARM_GIC_ITS_H__ */
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 9e2acb7..e9d5f36 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -161,6 +161,7 @@
DT_MATCH_COMPATIBLE("arm,gic-400")
#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
+#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
/*
* GICv3 registers that needs to be saved/restored
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 556f114..051a95e 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -48,6 +48,7 @@
/* Additional bits in GICD_TYPER defined by GICv3 */
#define GICD_TYPE_ID_BITS_SHIFT 19
+#define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
#define GICD_CTLR_RWP (1UL << 31)
#define GICD_CTLR_ARE_NS (1U << 4)
#define GICD_CTLR_ENABLE_G1A (1U << 1)
@@ -59,11 +60,12 @@
#define GICR_WAKER_ProcessorSleep (1U << 1)
#define GICR_WAKER_ChildrenAsleep (1U << 2)
-#define GICD_PIDR2_ARCH_REV_MASK (0xf0)
+#define GIC_PIDR2_ARCH_REV_MASK (0xf0)
+#define GICD_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICD_PIDR2_ARCH_REV_SHIFT (0x4)
#define GICD_PIDR2_ARCH_GICV3 (0x3)
-#define GICR_PIDR2_ARCH_REV_MASK GICD_PIDR2_ARCH_REV_MASK
+#define GICR_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICR_PIDR2_ARCH_REV_SHIFT GICD_PIDR2_ARCH_REV_SHIFT
#define GICR_PIDR2_ARCH_GICV3 GICD_PIDR2_ARCH_GICV3
@@ -113,10 +115,24 @@
#define GICR_ICFGR1 (0x0C04)
#define GICR_NSACR (0x0E00)
+#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
+#define GICR_TYPER_CPU_NUMBER(r) (((r) >> 8) & 0xffff)
+
+#define GICR_PROPBASER_InnerShareable (1U << 10)
+#define GICR_PROPBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PROPBASER_nC (1U << 7)
+#define GICR_PROPBASER_WaWb (5U << 7)
+#define GICR_PROPBASER_CACHEABILITY_MASK (7U << 7)
+#define GICR_PROPBASER_IDBITS_MASK (0x1f)
#define GICR_TYPER_PLPIS (1U << 0)
#define GICR_TYPER_VLPIS (1U << 1)
#define GICR_TYPER_LAST (1U << 4)
+#define GICR_PENDBASER_InnerShareable (1U << 10)
+#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PENDBASER_nC (1U << 7)
+#define GICR_PENDBASER_CACHEABILITY_MASK (7U << 7)
+
#define DEFAULT_PMR_VALUE 0xff
#define GICH_VMCR_EOI (1 << 9)
@@ -152,6 +168,103 @@
#define ICH_SGI_IRQ_SHIFT 24
#define ICH_SGI_IRQ_MASK 0xf
#define ICH_SGI_TARGETLIST_MASK 0xffff
+#define LPI_PROP_GROUP1 (1 << 1)
+
+/*
+ * ITS registers, offsets from ITS_base
+ */
+#define GITS_CTLR 0x0000
+#define GITS_IIDR 0x0004
+#define GITS_TYPER 0x0008
+#define GITS_CBASER 0x0080
+#define GITS_CWRITER 0x0088
+#define GITS_CREADR 0x0090
+#define GITS_BASER0 0x0100
+#define GITS_BASER1 0x0108
+#define GITS_BASER 0x0100
+#define GITS_BASERN 0x013c
+#define GITS_PIDR0 GICR_PIDR0
+#define GITS_PIDR1 GICR_PIDR1
+#define GITS_PIDR2 GICR_PIDR2
+#define GITS_PIDR3 GICR_PIDR3
+#define GITS_PIDR4 GICR_PIDR4
+#define GITS_PIDR5 GICR_PIDR5
+#define GITS_PIDR7 GICR_PIDR7
+
+#define GITS_TRANSLATER 0x10040
+#define GITS_CTLR_QUIESCENT (1U << 31)
+#define GITS_CTLR_ENABLE (1U << 0)
+
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_DEVBITS(r) ((((r) >> GITS_TYPER_DEVBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_IDBITS_SHIFT 8
+#define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_PTA (1UL << 19)
+#define GITS_TYPER_HCC_SHIFT (24)
+
+#define GITS_CBASER_VALID (1UL << 63)
+#define GITS_CBASER_nC (1UL << 59)
+#define GITS_CBASER_WaWb (5UL << 59)
+#define GITS_CBASER_InnerShareable (1UL << 10)
+#define GITS_CBASER_SHAREABILITY_MASK (3UL << 10)
+#define GITS_CBASER_CACHEABILITY_MASK (7UL << 59)
+
+#define GITS_BASER_NR_REGS 8
+
+#define GITS_BASER_VALID (1UL << 63)
+#define GITS_BASER_nC (1UL << 59)
+#define GITS_BASER_WaWb (5UL << 59)
+#define GITS_BASER_TYPE_SHIFT (56)
+#define GITS_BASER_TYPE_MASK (0x7)
+#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
+#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0xff) + 1)
+#define GITS_BASER_InnerShareable (1UL << 10)
+#define GITS_BASER_SHAREABILITY_SHIFT (10)
+#define GITS_BASER_SHAREABILITY_MASK (3UL << GITS_BASER_SHAREABILITY_SHIFT)
+#define GITS_BASER_PAGE_SIZE_SHIFT (8)
+#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_TYPE_NONE 0
+#define GITS_BASER_TYPE_DEVICE 1
+#define GITS_BASER_TYPE_VCPU 2
+#define GITS_BASER_TYPE_CPU 3
+#define GITS_BASER_TYPE_COLLECTION 4
+#define GITS_BASER_TYPE_RESERVED5 5
+#define GITS_BASER_TYPE_RESERVED6 6
+#define GITS_BASER_TYPE_RESERVED7 7
+
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD 0x08
+#define GITS_CMD_MAPC 0x09
+#define GITS_CMD_MAPVI 0x0a
+#define GITS_CMD_MAPI 0x0b
+#define GITS_CMD_MOVI 0x01
+#define GITS_CMD_DISCARD 0x0f
+#define GITS_CMD_INV 0x0c
+#define GITS_CMD_MOVALL 0x0e
+#define GITS_CMD_INVALL 0x0d
+#define GITS_CMD_INT 0x03
+#define GITS_CMD_CLEAR 0x04
+#define GITS_CMD_SYNC 0x05
+
+struct rdist {
+ void __iomem *rbase;
+ void *pend_page;
+ paddr_t phys_base;
+};
+
+struct rdist_prop {
+ void *prop_page;
+ uint64_t flags;
+};
+
+DECLARE_PER_CPU(struct rdist, rdist);
+
#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 03/17] xen/arm: ITS: Port ITS driver to Xen
2015-07-10 7:42 ` [PATCH v4 03/17] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
@ 2015-07-10 13:01 ` Ian Campbell
2015-07-15 10:23 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 13:01 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> +/*
> + * The ITS structure - contains most of the infrastructure, with the
> + * msi_controller, the command queue, the collections, and the list of
> + * devices writing to it.
> + */
> +struct its_node {
> + spinlock_t lock;
> + struct list_head entry;
> + void __iomem *base;
> + unsigned long phys_base;
> + unsigned long phys_size;
These two could be paddr_t I think.
> +#ifdef DEBUG_GIC_ITS
> +void dump_cmd(its_cmd_block *cmd)
> +{
> + printk("ITS: Phys_cmd CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
> + cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
> +}
Please can you include a #else with a dummy version of this function and
therefore avoid the #ifdef's at the callsite.
I think this also wants a XENLOG_DEBUG prefix, or better to use
its_debug.
> +
> +static void its_send_single_command(struct its_node *its,
> + its_cmd_block *src,
> + struct its_collection *sync_col)
> +{
> + its_cmd_block *cmd, *sync_cmd, *next_cmd;
> + unsigned long flags;
> +
> + BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
> +
> + spin_lock_irqsave(&its->lock, flags);
> +
> + cmd = its_allocate_entry(its);
> + if ( !cmd )
> + {
> + its_err("ITS can't allocate, dropping command\n");
> + spin_unlock_irqrestore(&its->lock, flags);
> + return;
> + }
> +
> + memcpy(cmd, src, sizeof(its_cmd_block));
> +#ifdef DEBUG_GIC_ITS
> + dump_cmd(cmd);
> +#endif
> + its_flush_cmd(its, cmd);
You could probably put the dump in its_flush_cmd, which might be a bit
less prone to forgetting it sometimes.
> +/* ITS command structure */
> +typedef union {
> + u64 bits[4];
> + struct __packed {
> + uint8_t cmd;
> + uint8_t pad[7];
> + } hdr;
> + struct __packed {
> + u8 cmd;
> + u8 res1[3];
> + u32 devid;
> + u64 size:5;
> + u64 res2:59;
> + /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
Is this intended as a TODO?
> [...]
> +} its_cmd_block;
Much preferred, thanks.
If you do at least the #else case for dump_cmd then I would ack this
patch.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 03/17] xen/arm: ITS: Port ITS driver to Xen
2015-07-10 7:42 ` [PATCH v4 03/17] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
2015-07-10 13:01 ` Ian Campbell
@ 2015-07-15 10:23 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 10:23 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> ---
> v4: Major changes
> - Redistributor refactoring patch is merged
> - Fixed comments from v3 related to coding style and
> removing duplicate code.
> - Target address is stored from bits[48:16] to avoid
> shifting of target address while building ITS commands
> - Removed non-static functions
> - Removed usage of command builder functions
> - Changed its_cmd_block union to include mix of bit and unsigned
> variable types to define ITS command structure
You also replaced all ratelimited printk by normal printk. On Xen, error
and warning are not ratelimited by default for Xen (only guest printk is
ratelimited).
The ratelimited is used in the ITS for the command queue. Which will be
partially exposed to the guest via the LPI configuration table
virtualisation.
I'd like to see at least the macro its_err_ratelimited kept, even though
it uses XENLOG_ERR today. This is at least to remind us that it needs to
be fixed in 4.7 in order to avoid possible security issue.
[..]
> +static void its_send_single_command(struct its_node *its,
> + its_cmd_block *src,
> + struct its_collection *sync_col)
> +{
> + its_cmd_block *cmd, *sync_cmd, *next_cmd;
> + unsigned long flags;
> +
> + BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
> +
> + spin_lock_irqsave(&its->lock, flags);
> +
> + cmd = its_allocate_entry(its);
> + if ( !cmd )
> + {
Can you keep /* We are soooo screwed .... */? I'm not sure why you
removed it since v3, but it was useful shows that it's can unlikely happen.
> + its_err("ITS can't allocate, dropping command\n");
> + spin_unlock_irqrestore(&its->lock, flags);
> + return;
> + }
[..]
> +int its_lpi_init(u32 id_bits)
You are calling its_lpi_init from the vgic driver. I'm struggling to
understand why... The vGIC driver is only here to virtualize the GIC for
a domain not initialize the physical GIC.
Why can't you call its_lpi_init from its_init as Linux does? This would
avoid to export this function.
[..]
> +static void its_cpu_init_collection(void)
> +{
> + struct its_node *its;
> + int cpu;
> +
> + spin_lock(&its_lock);
> + cpu = smp_processor_id();
> +
> + list_for_each_entry(its, &its_nodes, entry)
> + {
> + u64 target;
Newline here please.
> + /*
> + * We now have to bind each collection to its target
> + * redistributor.
> + */
> + if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
> + {
> + /*
> + * This ITS wants the physical address of the
> + * redistributor.
> + */
> + target = gic_data_rdist().phys_base;
> + /*
> + * If Target address is GICR adddress, then it is aligned to 64K
I think the "If Target address is GICR address..." is not necessary. We
are already in the if for the Target address will be the re-dist address.
> + * and hence ITS command field is only consider 32 bit skipping
> + * lower 16 bits.So take bit[48:16]
I would say : "The ITS command is only considering [48:16] of the GICR
address".
> + */
> + its->collections[cpu].target_address = target >> 16;
You could have done target >>= 16; To avoid duplicating
its->collections[cpu].target_address in both if.
> + }
> + else
> + {
> + /*
> + * This ITS wants a linear CPU number.
> + */
> + target = readq_relaxed(gic_data_rdist().rbase + GICR_TYPER);
> + target = GICR_TYPER_CPU_NUMBER(target);
> + its->collections[cpu].target_address = target;
> + }
> +
> + /* Perform collection mapping */
And then
its->collections[cpu].target_address = target;
> + its->collections[cpu].col_id = cpu;
> +
> + its_send_mapc(its, &its->collections[cpu], 1);
> + its_send_invall(its, &its->collections[cpu]);
> + }
> +
> + spin_unlock(&its_lock);
> +}
[..]
> +int __init its_init(struct rdist_prop *rdists)
> +{
> + struct dt_device_node *np = NULL;
> +
> + static const struct dt_device_match its_device_ids[] __initconst =
> + {
> + DT_MATCH_GIC_ITS,
> + { /* sentinel */ },
> + };
> +
> + for (np = dt_find_matching_node(NULL, its_device_ids); np;
> + np = dt_find_matching_node(np, its_device_ids))
> + its_probe(np);
> +
> + if ( list_empty(&its_nodes) )
> + {
> + its_warn("ITS: No ITS available, not enabling LPIs\n");
> + return -ENXIO;
> + }
> +
> + gic_rdists = rdists;
> + its_alloc_lpi_tables();
FIY, I think that its_lpi_init should be called here as it's done on Linux.
> +
> + return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 30682cf..b5c59f6 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -53,6 +53,7 @@ static struct {
> paddr_t dbase; /* Address of distributor registers */
> paddr_t dbase_size;
> void __iomem *map_dbase; /* Mapped address of distributor registers */
> + struct rdist_prop rdist_data;
> struct rdist_region *rdist_regions;
> uint32_t rdist_stride;
> unsigned int rdist_count; /* Number of rdist regions count */
> @@ -63,10 +64,10 @@ static struct {
> static struct gic_info gicv3_info;
>
> /* per-cpu re-distributor base */
> -static DEFINE_PER_CPU(void __iomem*, rbase);
> +DEFINE_PER_CPU(struct rdist, rdist);
>
> #define GICD (gicv3.map_dbase)
> -#define GICD_RDIST_BASE (this_cpu(rbase))
> +#define GICD_RDIST_BASE (per_cpu(rdist, smp_processor_id()).rbase)
You could use this_cpu here:
this_cpu(rdist).rbase
> #define GICD_RDIST_SGI_BASE (GICD_RDIST_BASE + SZ_64K)
>
> /*
> @@ -613,6 +614,7 @@ static int __init gicv3_populate_rdist(void)
> uint32_t aff;
> uint32_t reg;
> uint64_t typer;
> + uint64_t offset;
> uint64_t mpidr = cpu_logical_map(smp_processor_id());
>
> /*
> @@ -648,9 +650,12 @@ static int __init gicv3_populate_rdist(void)
>
> if ( (typer >> 32) == aff )
> {
> - this_cpu(rbase) = ptr;
> - printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
> - smp_processor_id(), i, ptr);
> + offset = ptr - gicv3.rdist_regions[i].map_base;
> + per_cpu(rdist, smp_processor_id()).rbase = ptr;
Please use: this_cpu(rdist).rbase = ptr;
> + per_cpu(rdist, smp_processor_id()).phys_base = gicv3.rdist_regions[i].base + offset;
this_cpu(rdist).phys_base;
> + printk("GICv3: CPU%d: Found redistributor in region %d @%"PRIpaddr"\n",
> + smp_processor_id(), i,
> + per_cpu(rdist, smp_processor_id()).phys_base);
this_cpu(rdist).phys_base;
> return 0;
> }
> if ( gicv3.rdist_stride )
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 9e2acb7..e9d5f36 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -161,6 +161,7 @@
> DT_MATCH_COMPATIBLE("arm,gic-400")
>
> #define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
> +#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
I don't think you need to export this.
>
> /*
> * GICv3 registers that needs to be saved/restored
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 556f114..051a95e 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -48,6 +48,7 @@
> /* Additional bits in GICD_TYPER defined by GICv3 */
> #define GICD_TYPE_ID_BITS_SHIFT 19
>
> +#define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
> #define GICD_CTLR_RWP (1UL << 31)
> #define GICD_CTLR_ARE_NS (1U << 4)
> #define GICD_CTLR_ENABLE_G1A (1U << 1)
> @@ -59,11 +60,12 @@
> #define GICR_WAKER_ProcessorSleep (1U << 1)
> #define GICR_WAKER_ChildrenAsleep (1U << 2)
>
> -#define GICD_PIDR2_ARCH_REV_MASK (0xf0)
> +#define GIC_PIDR2_ARCH_REV_MASK (0xf0)
> +#define GICD_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
> #define GICD_PIDR2_ARCH_REV_SHIFT (0x4)
> #define GICD_PIDR2_ARCH_GICV3 (0x3)
>
> -#define GICR_PIDR2_ARCH_REV_MASK GICD_PIDR2_ARCH_REV_MASK
> +#define GICR_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
> #define GICR_PIDR2_ARCH_REV_SHIFT GICD_PIDR2_ARCH_REV_SHIFT
> #define GICR_PIDR2_ARCH_GICV3 GICD_PIDR2_ARCH_GICV3
>
> @@ -113,10 +115,24 @@
> #define GICR_ICFGR1 (0x0C04)
> #define GICR_NSACR (0x0E00)
>
> +#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
> +#define GICR_TYPER_CPU_NUMBER(r) (((r) >> 8) & 0xffff)
> +
> +#define GICR_PROPBASER_InnerShareable (1U << 10)
> +#define GICR_PROPBASER_SHAREABILITY_MASK (3UL << 10)
> +#define GICR_PROPBASER_nC (1U << 7)
> +#define GICR_PROPBASER_WaWb (5U << 7)
> +#define GICR_PROPBASER_CACHEABILITY_MASK (7U << 7)
> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
> #define GICR_TYPER_PLPIS (1U << 0)
> #define GICR_TYPER_VLPIS (1U << 1)
> #define GICR_TYPER_LAST (1U << 4)
>
> +#define GICR_PENDBASER_InnerShareable (1U << 10)
> +#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
> +#define GICR_PENDBASER_nC (1U << 7)
> +#define GICR_PENDBASER_CACHEABILITY_MASK (7U << 7)
> +
> #define DEFAULT_PMR_VALUE 0xff
>
> #define GICH_VMCR_EOI (1 << 9)
> @@ -152,6 +168,103 @@
> #define ICH_SGI_IRQ_SHIFT 24
> #define ICH_SGI_IRQ_MASK 0xf
> #define ICH_SGI_TARGETLIST_MASK 0xffff
> +#define LPI_PROP_GROUP1 (1 << 1)
>
> +/*
> + * ITS registers, offsets from ITS_base
> + */
It would make sense to move all the GITS_* defines in the gic-its.h
which you've introduced within this patch.
> +#define GITS_CTLR 0x0000
> +#define GITS_IIDR 0x0004
> +#define GITS_TYPER 0x0008
> +#define GITS_CBASER 0x0080
> +#define GITS_CWRITER 0x0088
> +#define GITS_CREADR 0x0090
> +#define GITS_BASER0 0x0100
> +#define GITS_BASER1 0x0108
> +#define GITS_BASER 0x0100
> +#define GITS_BASERN 0x013c
> +#define GITS_PIDR0 GICR_PIDR0
> +#define GITS_PIDR1 GICR_PIDR1
> +#define GITS_PIDR2 GICR_PIDR2
> +#define GITS_PIDR3 GICR_PIDR3
> +#define GITS_PIDR4 GICR_PIDR4
> +#define GITS_PIDR5 GICR_PIDR5
> +#define GITS_PIDR7 GICR_PIDR7
[..]
> +struct rdist {
> + void __iomem *rbase;
> + void *pend_page;
> + paddr_t phys_base;
> +};
> +
> +struct rdist_prop {
> + void *prop_page;
> + uint64_t flags;
> +};
On v3, I asked the meaning of "prop" in both the name of the structure
and the field. Can you explain what does it mean?
It would also be nice to document the 2 structures.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (2 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 03/17] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 13:05 ` Ian Campbell
2015-07-15 10:37 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
` (12 subsequent siblings)
16 siblings, 2 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Helper functions to manage its devices using RB-tree
are introduced in physical ITS driver.
This is global list of all the devices.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Remove passing of root node as parameter
- Declare prototype in header file
- Rename find_its_device to its_find_device
---
xen/arch/arm/gic-v3-its.c | 49 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 13 +++++++++++
2 files changed, 62 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 60ab646..b421a6f 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -90,6 +90,7 @@ struct its_node {
static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
+static struct rb_root rb_its_dev;
#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
@@ -101,6 +102,53 @@ void dump_cmd(its_cmd_block *cmd)
}
#endif
+/* RB-tree helpers for its_device */
+struct its_device *its_find_device(u32 devid)
+{
+ struct rb_node *node = rb_its_dev.rb_node;
+
+ while ( node )
+ {
+ struct its_device *dev;
+
+ dev = container_of(node, struct its_device, node);
+ if ( devid < dev->device_id )
+ node = node->rb_left;
+ else if ( devid > dev->device_id )
+ node = node->rb_right;
+ else
+ return dev;
+ }
+
+ return NULL;
+}
+
+int its_insert_device(struct its_device *dev)
+{
+ struct rb_node **new, *parent;
+
+ new = &rb_its_dev.rb_node;
+ parent = NULL;
+ while ( *new )
+ {
+ struct its_device *this;
+
+ this = container_of(*new, struct its_device, node);
+ parent = *new;
+ if ( dev->device_id < this->device_id )
+ new = &((*new)->rb_left);
+ else if ( dev->device_id > this->device_id )
+ new = &((*new)->rb_right);
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&dev->node, parent, new);
+ rb_insert_color(&dev->node, &rb_its_dev);
+
+ return 0;
+}
+
#define ITS_CMD_QUEUE_SZ SZ_64K
#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
@@ -811,6 +859,7 @@ static int its_probe(struct dt_device_node *node)
list_add(&its->entry, &its_nodes);
spin_unlock(&its_lock);
+ rb_its_dev = RB_ROOT;
return 0;
out_free_tables:
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index d24b039..b5e09bd 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -19,6 +19,7 @@
#define __ASM_ARM_GIC_ITS_H__
#include <asm/gic_v3_defs.h>
+#include <xen/rbtree.h>
/*
* Collection structure - just an ID, and a redistributor address to
@@ -156,9 +157,21 @@ typedef union {
} sync;
} its_cmd_block;
+/*
+ * The ITS view of a device.
+ */
+struct its_device {
+ /* Physical Device id */
+ u32 device_id;
+ /* RB-tree entry */
+ struct rb_node node;
+};
+
int its_lpi_init(u32 id_bits);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
+struct its_device *its_find_device(u32 devid);
+int its_insert_device(struct its_device *dev);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices
2015-07-10 7:42 ` [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
@ 2015-07-10 13:05 ` Ian Campbell
2015-07-15 10:37 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 13:05 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Helper functions to manage its devices using RB-tree
> are introduced in physical ITS driver.
>
> This is global list of all the devices.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices
2015-07-10 7:42 ` [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
2015-07-10 13:05 ` Ian Campbell
@ 2015-07-15 10:37 ` Julien Grall
2015-07-15 14:21 ` Vijay Kilari
1 sibling, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 10:37 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> ---
> xen/arch/arm/gic-v3-its.c | 49 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 13 +++++++++++
> 2 files changed, 62 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 60ab646..b421a6f 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -90,6 +90,7 @@ struct its_node {
> static LIST_HEAD(its_nodes);
> static DEFINE_SPINLOCK(its_lock);
> static struct rdist_prop *gic_rdists;
> +static struct rb_root rb_its_dev;
>
> #define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
>
> @@ -101,6 +102,53 @@ void dump_cmd(its_cmd_block *cmd)
> }
> #endif
>
> +/* RB-tree helpers for its_device */
> +struct its_device *its_find_device(u32 devid)
> +{
> + struct rb_node *node = rb_its_dev.rb_node;
> +
> + while ( node )
> + {
> + struct its_device *dev;
> +
> + dev = container_of(node, struct its_device, node);
> + if ( devid < dev->device_id )
> + node = node->rb_left;
> + else if ( devid > dev->device_id )
> + node = node->rb_right;
> + else
> + return dev;
> + }
> +
> + return NULL;
> +}
> +
> +int its_insert_device(struct its_device *dev)
> +{
> + struct rb_node **new, *parent;
> +
> + new = &rb_its_dev.rb_node;
> + parent = NULL;
> + while ( *new )
> + {
> + struct its_device *this;
> +
> + this = container_of(*new, struct its_device, node);
> + parent = *new;
> + if ( dev->device_id < this->device_id )
> + new = &((*new)->rb_left);
> + else if ( dev->device_id > this->device_id )
> + new = &((*new)->rb_right);
> + else
> + return -EEXIST;
> + }
> +
> + rb_link_node(&dev->node, parent, new);
> + rb_insert_color(&dev->node, &rb_its_dev);
> +
> + return 0;
> +}
> +
> #define ITS_CMD_QUEUE_SZ SZ_64K
> #define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
>
> @@ -811,6 +859,7 @@ static int its_probe(struct dt_device_node *node)
> list_add(&its->entry, &its_nodes);
> spin_unlock(&its_lock);
>
> + rb_its_dev = RB_ROOT;
NIT: missing newline here.
A general question, how do you ensure any concurrent insert to the
RB-tree? Is it taken care by the ITS lock and the caller?
I would be ok to defer the locking for when the PCI passthrough is
supported as all the ITS device are added when Xen is booted.
A TODO would be worth to add. With the TODO and the answer to my questions:
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices
2015-07-15 10:37 ` Julien Grall
@ 2015-07-15 14:21 ` Vijay Kilari
2015-07-15 14:28 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-15 14:21 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Wed, Jul 15, 2015 at 4:07 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
>
> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>
>> ---
>> xen/arch/arm/gic-v3-its.c | 49
>> +++++++++++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/gic-its.h | 13 +++++++++++
>> 2 files changed, 62 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> index 60ab646..b421a6f 100644
>> --- a/xen/arch/arm/gic-v3-its.c
>> +++ b/xen/arch/arm/gic-v3-its.c
>> @@ -90,6 +90,7 @@ struct its_node {
>> static LIST_HEAD(its_nodes);
>> static DEFINE_SPINLOCK(its_lock);
>> static struct rdist_prop *gic_rdists;
>> +static struct rb_root rb_its_dev;
>>
>> #define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
>>
>> @@ -101,6 +102,53 @@ void dump_cmd(its_cmd_block *cmd)
>> }
>> #endif
>>
>> +/* RB-tree helpers for its_device */
>> +struct its_device *its_find_device(u32 devid)
>> +{
>> + struct rb_node *node = rb_its_dev.rb_node;
>> +
>> + while ( node )
>> + {
>> + struct its_device *dev;
>> +
>> + dev = container_of(node, struct its_device, node);
>> + if ( devid < dev->device_id )
>> + node = node->rb_left;
>> + else if ( devid > dev->device_id )
>> + node = node->rb_right;
>> + else
>> + return dev;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +int its_insert_device(struct its_device *dev)
>> +{
>> + struct rb_node **new, *parent;
>> +
>> + new = &rb_its_dev.rb_node;
>> + parent = NULL;
>> + while ( *new )
>> + {
>> + struct its_device *this;
>> +
>> + this = container_of(*new, struct its_device, node);
>> + parent = *new;
>> + if ( dev->device_id < this->device_id )
>> + new = &((*new)->rb_left);
>> + else if ( dev->device_id > this->device_id )
>> + new = &((*new)->rb_right);
>> + else
>> + return -EEXIST;
>> + }
>> +
>> + rb_link_node(&dev->node, parent, new);
>> + rb_insert_color(&dev->node, &rb_its_dev);
>> +
>> + return 0;
>> +}
>> +
>> #define ITS_CMD_QUEUE_SZ SZ_64K
>> #define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ /
>> sizeof(its_cmd_block))
>>
>> @@ -811,6 +859,7 @@ static int its_probe(struct dt_device_node *node)
>> list_add(&its->entry, &its_nodes);
>> spin_unlock(&its_lock);
>>
>> + rb_its_dev = RB_ROOT;
>
>
> NIT: missing newline here.
>
> A general question, how do you ensure any concurrent insert to the RB-tree?
> Is it taken care by the ITS lock and the caller?
>
> I would be ok to defer the locking for when the PCI passthrough is supported
> as all the ITS device are added when Xen is booted.
>
> A TODO would be worth to add. With the TODO and the answer to my questions:
I have added new lock "rb_its_dev_lock" in patch #8 where caller is managing.
>
> Reviewed-by: Julien Grall <julien.grall@citrix.com>
>
> Regards,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices
2015-07-15 14:21 ` Vijay Kilari
@ 2015-07-15 14:28 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 14:28 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 15/07/2015 16:21, Vijay Kilari wrote:
>> A general question, how do you ensure any concurrent insert to the RB-tree?
>> Is it taken care by the ITS lock and the caller?
>>
>> I would be ok to defer the locking for when the PCI passthrough is supported
>> as all the ITS device are added when Xen is booted.
>>
>> A TODO would be worth to add. With the TODO and the answer to my questions:
>
> I have added new lock "rb_its_dev_lock" in patch #8 where caller is managing.
It's rather strange to introduce the lock later given that anyone can
call these functions (they are exported).
You need to ensure that the caller does take the lock by adding an
ASSERT(spin_is_locked(...)) in each function.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (3 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 04/17] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 13:46 ` Ian Campbell
` (2 more replies)
2015-07-10 7:42 ` [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver vijay.kilari
` (11 subsequent siblings)
16 siblings, 3 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Implements hw_irq_controller api's required
to handle LPI's
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Implement separate hw_irq_controller for LPIs
- Drop setting LPI affinity
- virq and vid are moved under union
- Introduced inv command handling
- its_device is stored in irq_desc
---
xen/arch/arm/gic-v3-its.c | 132 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 5 +-
xen/arch/arm/gic.c | 32 +++++++--
xen/arch/arm/irq.c | 40 ++++++++++-
xen/include/asm-arm/gic-its.h | 4 ++
xen/include/asm-arm/gic.h | 13 ++++
xen/include/asm-arm/gic_v3_defs.h | 1 +
xen/include/asm-arm/irq.h | 8 ++-
8 files changed, 227 insertions(+), 8 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index b421a6f..b98d396 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -295,6 +295,19 @@ post:
its_wait_for_range_completion(its, cmd, next_cmd);
}
+static void its_send_inv(struct its_device *dev, struct its_collection *col,
+ u32 event_id)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.inv.cmd = GITS_CMD_INV;
+ cmd.inv.devid = dev->device_id;
+ cmd.inv.event = event_id;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
static void its_send_mapc(struct its_node *its, struct its_collection *col,
int valid)
{
@@ -320,6 +333,125 @@ static void its_send_invall(struct its_node *its, struct its_collection *col)
its_send_single_command(its, &cmd, NULL);
}
+static void lpi_set_config(struct irq_desc *desc, int enable)
+{
+ u8 *cfg;
+ struct its_collection *col;
+ struct its_device *its_dev = get_irq_device(desc);
+ u16 col_id;
+ u32 vid = irq_to_vid(desc);
+
+ ASSERT(vid < its_dev->nr_lpis);
+
+ cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
+ if ( enable )
+ *cfg |= LPI_PROP_ENABLED;
+ else
+ *cfg &= ~LPI_PROP_ENABLED;
+
+ /*
+ * Make the above write visible to the redistributors.
+ * And yes, we're flushing exactly: One. Single. Byte.
+ * Humpf...
+ */
+ if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
+ else
+ dsb(ishst);
+
+ /* Get collection id for this event id */
+ col_id = gic_get_irq_collection(desc->irq);
+ col = &its_dev->its->collections[col_id];
+ its_send_inv(its_dev, col, vid);
+}
+
+static void its_irq_enable(struct irq_desc *desc)
+{
+ unsigned long flags;
+
+ ASSERT(spin_is_locked(&desc->lock));
+
+ spin_lock_irqsave(&its_lock, flags);
+ clear_bit(_IRQ_DISABLED, &desc->status);
+ dsb(sy);
+ lpi_set_config(desc, 1);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
+static void its_irq_disable(struct irq_desc *desc)
+{
+ unsigned long flags;
+
+ ASSERT(spin_is_locked(&desc->lock));
+
+ spin_lock_irqsave(&its_lock, flags);
+ lpi_set_config(desc, 0);
+ set_bit(_IRQ_DISABLED, &desc->status);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
+static unsigned int its_irq_startup(struct irq_desc *desc)
+{
+ its_irq_enable(desc);
+
+ return 0;
+}
+
+static void its_irq_shutdown(struct irq_desc *desc)
+{
+ its_irq_disable(desc);
+}
+
+static void its_irq_ack(struct irq_desc *desc)
+{
+ /* No ACK -- reading IAR has done this for us */
+}
+
+static void its_host_irq_end(struct irq_desc *desc)
+{
+ /* Lower the priority */
+ gicv3_eoi_irq(desc);
+ /* Deactivate */
+ gicv3_dir_irq(desc);
+}
+
+static void its_guest_irq_end(struct irq_desc *desc)
+{
+ gicv3_eoi_irq(desc);
+}
+
+static void its_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+{
+ return;
+}
+
+static const hw_irq_controller its_host_lpi_type = {
+ .typename = "gic-its",
+ .startup = its_irq_startup,
+ .shutdown = its_irq_shutdown,
+ .enable = its_irq_enable,
+ .disable = its_irq_disable,
+ .ack = its_irq_ack,
+ .end = its_host_irq_end,
+ .set_affinity = its_irq_set_affinity,
+};
+
+static const hw_irq_controller its_guest_lpi_type = {
+ .typename = "gic-its",
+ .startup = its_irq_startup,
+ .shutdown = its_irq_shutdown,
+ .enable = its_irq_enable,
+ .disable = its_irq_disable,
+ .ack = its_irq_ack,
+ .end = its_guest_irq_end,
+ .set_affinity = its_irq_set_affinity,
+};
+
+static const struct its_hw_operations its_ops = {
+ .lpi_host_irq_type = &its_host_lpi_type,
+ .lpi_guest_irq_type = &its_guest_lpi_type,
+};
+
/*
* How we allocate LPIs:
*
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index b5c59f6..904fe57 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -40,6 +40,7 @@
#include <asm/device.h>
#include <asm/gic.h>
#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
#include <asm/cpufeature.h>
struct rdist_region {
@@ -435,14 +436,14 @@ static void gicv3_mask_irq(struct irq_desc *irqd)
gicv3_poke_irq(irqd, GICD_ICENABLER);
}
-static void gicv3_eoi_irq(struct irq_desc *irqd)
+void gicv3_eoi_irq(struct irq_desc *irqd)
{
/* Lower the priority */
WRITE_SYSREG32(irqd->irq, ICC_EOIR1_EL1);
isb();
}
-static void gicv3_dir_irq(struct irq_desc *irqd)
+void gicv3_dir_irq(struct irq_desc *irqd)
{
/* Deactivate */
WRITE_SYSREG32(irqd->irq, ICC_DIR_EL1);
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index c41e82e..4f3801b 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -46,12 +46,18 @@ static DEFINE_PER_CPU(uint64_t, lr_mask);
static void gic_update_one_lr(struct vcpu *v, int i);
static const struct gic_hw_operations *gic_hw_ops;
+static const struct its_hw_operations *its_hw_ops;
void register_gic_ops(const struct gic_hw_operations *ops)
{
gic_hw_ops = ops;
}
+void register_its_ops(const struct its_hw_operations *ops)
+{
+ its_hw_ops = ops;
+}
+
static void clear_cpu_lr_mask(void)
{
this_cpu(lr_mask) = 0ULL;
@@ -94,6 +100,22 @@ void gic_restore_state(struct vcpu *v)
gic_restore_pending_irqs(v);
}
+static inline hw_irq_controller *get_host_hw_irq_controller(unsigned int irq)
+{
+ if ( is_lpi(irq) )
+ return its_hw_ops->lpi_host_irq_type;
+ else
+ return gic_hw_ops->gic_host_irq_type;
+}
+
+static inline hw_irq_controller *get_guest_hw_irq_controller(unsigned int irq)
+{
+ if ( is_lpi(irq) )
+ return its_hw_ops->lpi_guest_irq_type;
+ else
+ return gic_hw_ops->gic_guest_irq_type;
+}
+
/*
* needs to be called with a valid cpu_mask, ie each cpu in the mask has
* already called gic_cpu_init
@@ -104,7 +126,8 @@ static void gic_set_irq_properties(struct irq_desc *desc,
const cpumask_t *cpu_mask,
unsigned int priority)
{
- gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
+ if ( desc->irq < gic_number_lines() )
+ gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
}
/* Program the GIC to route an interrupt to the host (i.e. Xen)
@@ -114,11 +137,12 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
unsigned int priority)
{
ASSERT(priority <= 0xff); /* Only 8 bits of priority */
- ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
+ /* Can't route interrupts that don't exist */
+ ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
- desc->handler = gic_hw_ops->gic_host_irq_type;
+ desc->handler = get_host_hw_irq_controller(desc->irq);
gic_set_irq_properties(desc, cpu_mask, priority);
}
@@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
goto out;
- desc->handler = gic_hw_ops->gic_guest_irq_type;
+ desc->handler = get_guest_hw_irq_controller(desc->irq);
set_bit(_IRQ_GUEST, &desc->status);
gic_set_irq_properties(desc, cpumask_of(v_target->processor), priority);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 2dd43ee..ba8528a 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
struct irq_guest
{
struct domain *d;
- unsigned int virq;
+ union
+ {
+ /* virq refer to virtual irq in case of spi */
+ unsigned int virq;
+ /* virq refer to event ID in case of lpi */
+ unsigned int vid;
+ };
};
static void ack_none(struct irq_desc *irq)
@@ -143,6 +149,38 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
return irq_get_guest_info(desc)->d;
}
+unsigned int irq_to_vid(struct irq_desc *desc)
+{
+ return irq_get_guest_info(desc)->vid;
+}
+
+unsigned int irq_to_virq(struct irq_desc *desc)
+{
+ return irq_get_guest_info(desc)->virq;
+}
+
+struct its_device *get_irq_device(struct irq_desc *desc)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ return desc->arch.dev;
+}
+
+void set_irq_device(struct irq_desc *desc, struct its_device *dev)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+ desc->arch.dev = dev;
+}
+
+u16 gic_get_irq_collection(unsigned int irq)
+{
+ struct irq_desc *desc = irq_to_desc(irq);
+
+ ASSERT(spin_is_locked(&desc->lock));
+
+ return desc->arch.col_id;
+}
+
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
{
if ( desc != NULL )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index b5e09bd..e8d244f 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -161,6 +161,10 @@ typedef union {
* The ITS view of a device.
*/
struct its_device {
+ /* Physical ITS */
+ struct its_node *its;
+ /* Number of Physical LPIs assigned */
+ int nr_lpis;
/* Physical Device id */
u32 device_id;
/* RB-tree entry */
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index e9d5f36..44c2317 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -20,6 +20,9 @@
#define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
#define NR_GIC_SGI 16
+#define FIRST_GIC_LPI 8192
+#define NR_GIC_LPI 4096
+#define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
#define MAX_RDIST_COUNT 4
#define GICD_CTLR (0x000)
@@ -163,6 +166,7 @@
#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
+#define is_lpi(lpi) (lpi >= FIRST_GIC_LPI && lpi < MAX_LPI)
/*
* GICv3 registers that needs to be saved/restored
*/
@@ -279,6 +283,8 @@ extern void gic_dump_info(struct vcpu *v);
/* Number of interrupt lines */
extern unsigned int gic_number_lines(void);
+void gicv3_eoi_irq(struct irq_desc *irqd);
+void gicv3_dir_irq(struct irq_desc *irqd);
/* IRQ translation function for the device tree */
int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
unsigned int *out_hwirq, unsigned int *out_type);
@@ -353,7 +359,14 @@ struct gic_hw_operations {
const struct dt_device_node *node, void *fdt);
};
+struct its_hw_operations {
+ /* hw_irq_controller to enable/disable/eoi host lpi */
+ hw_irq_controller *lpi_host_irq_type;
+ /* hw_irq_controller to enable/disable/eoi guest lpi */
+ hw_irq_controller *lpi_guest_irq_type;
+};
void register_gic_ops(const struct gic_hw_operations *ops);
+void register_its_ops(const struct its_hw_operations *ops);
int gic_make_node(const struct domain *d,const struct dt_device_node *node,
void *fdt);
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 051a95e..0443ae7 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -169,6 +169,7 @@
#define ICH_SGI_IRQ_MASK 0xf
#define ICH_SGI_TARGETLIST_MASK 0xffff
#define LPI_PROP_GROUP1 (1 << 1)
+#define LPI_PROP_ENABLED (1 << 0)
/*
* ITS registers, offsets from ITS_base
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 34b492b..55e219f 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -17,6 +17,8 @@ struct arch_pirq
struct arch_irq_desc {
int eoi_cpu;
unsigned int type;
+ struct its_device *dev;
+ u16 col_id;
};
#define NR_LOCAL_IRQS 32
@@ -50,7 +52,11 @@ void arch_move_irqs(struct vcpu *v);
/* Set IRQ type for an SPI */
int irq_set_spi_type(unsigned int spi, unsigned int type);
-
+unsigned int irq_to_virq(struct irq_desc *desc);
+unsigned int irq_to_vid(struct irq_desc *desc);
+struct its_device *get_irq_device(struct irq_desc *desc);
+void set_irq_device(struct irq_desc *desc, struct its_device *dev);
+u16 gic_get_irq_collection(unsigned int irq);
int platform_get_irq(const struct dt_device_node *device, int index);
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-10 7:42 ` [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
@ 2015-07-10 13:46 ` Ian Campbell
2015-07-11 14:40 ` Vijay Kilari
2015-07-13 21:18 ` Julien Grall
2015-07-15 18:19 ` Julien Grall
2 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 13:46 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Implements hw_irq_controller api's required
> to handle LPI's
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Implement separate hw_irq_controller for LPIs
> - Drop setting LPI affinity
> - virq and vid are moved under union
> - Introduced inv command handling
> - its_device is stored in irq_desc
> ---
> xen/arch/arm/gic-v3-its.c | 132 +++++++++++++++++++++++++++++++++++++
> xen/arch/arm/gic-v3.c | 5 +-
> xen/arch/arm/gic.c | 32 +++++++--
> xen/arch/arm/irq.c | 40 ++++++++++-
> xen/include/asm-arm/gic-its.h | 4 ++
> xen/include/asm-arm/gic.h | 13 ++++
> xen/include/asm-arm/gic_v3_defs.h | 1 +
> xen/include/asm-arm/irq.h | 8 ++-
> 8 files changed, 227 insertions(+), 8 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index b421a6f..b98d396 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -295,6 +295,19 @@ post:
> its_wait_for_range_completion(its, cmd, next_cmd);
> }
>
> +static void its_send_inv(struct its_device *dev, struct its_collection *col,
> + u32 event_id)
> +{
> + its_cmd_block cmd;
> +
> + memset(&cmd, 0x0, sizeof(its_cmd_block));
> + cmd.inv.cmd = GITS_CMD_INV;
> + cmd.inv.devid = dev->device_id;
> + cmd.inv.event = event_id;
> +
> + its_send_single_command(dev->its, &cmd, col);
> +}
This ought to be in the prior patch doing such things I think.
Oh I see, you didn't have struct its_device defined back then. I think
you can just reorder patches #3 and #4 to solve that.
> +static void its_host_irq_end(struct irq_desc *desc)
> +{
> + /* Lower the priority */
> + gicv3_eoi_irq(desc);
> + /* Deactivate */
> + gicv3_dir_irq(desc);
> +}
> +
> +static void its_guest_irq_end(struct irq_desc *desc)
> +{
> + gicv3_eoi_irq(desc);
> +}
Exposing those two gicv3 functions is a bit unfortunate, but I think it
will do for now.
Exposing gicv3_[host|guest]_irq_end might have been nicer, since you
could just insert them into your its_[host|guest]_lpi_type instead of
duplicating them.
Eventually we may want to refactor such that register_its_ops gives back
the ack/eoi hooks to use.
> +static void its_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
> +{
Please add
/* Not yet supported */
> @@ -104,7 +126,8 @@ static void gic_set_irq_properties(struct irq_desc *desc,
> const cpumask_t *cpu_mask,
> unsigned int priority)
> {
> - gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
> + if ( desc->irq < gic_number_lines() )
Should this be is_lpi as in other similar places?
> + gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
> }
>
> /* Program the GIC to route an interrupt to the host (i.e. Xen)
> @@ -114,11 +137,12 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> unsigned int priority)
> {
> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
> + /* Can't route interrupts that don't exist */
> + ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
As discussed in <1436284206.25646.258.camel@citrix.com> please make some
sort of is_valid_irq(irq) helper to encapsulate this logic.
> ocessor), priority);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 2dd43ee..ba8528a 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
> struct irq_guest
> {
> struct domain *d;
> - unsigned int virq;
> + union
> + {
> + /* virq refer to virtual irq in case of spi */
> + unsigned int virq;
> + /* virq refer to event ID in case of lpi */
"refers" in both cases
And I'd say "to the ..." not just "to ..." and "in the case of..." too.
> +unsigned int irq_to_vid(struct irq_desc *desc)
> +{
> + return irq_get_guest_info(desc)->vid;
> +}
> +
> +unsigned int irq_to_virq(struct irq_desc *desc)
> +{
> + return irq_get_guest_info(desc)->virq;
> +}
Please assert that irq_desc->arch.its_device is (non-)NULL as
appropriate in these two cases.
BTW, while checking the field name I spotted "struct msi_desc
*msi_desc" in the main struct irq_desc.
Since MSIs are effectively the same as LPIs as a future cleanup I think
we should s/its_device/msi_desc/g and use this field instead of adding a
second redundant type and pointer to it. THis is not a blocker for 4.6
though.
> +struct its_device *get_irq_device(struct irq_desc *desc)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> +
> + return desc->arch.dev;
> +}
> +
> +void set_irq_device(struct irq_desc *desc, struct its_device *dev)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> + desc->arch.dev = dev;
> +}
Please add _its to the names of both of these functions, ie..g
set_irq_its_device.
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index b5e09bd..e8d244f 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -161,6 +161,10 @@ typedef union {
> * The ITS view of a device.
> */
> struct its_device {
> + /* Physical ITS */
> + struct its_node *its;
> + /* Number of Physical LPIs assigned */
> + int nr_lpis;
> /* Physical Device id */
> u32 device_id;
> /* RB-tree entry */
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index e9d5f36..44c2317 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -20,6 +20,9 @@
>
> #define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
> #define NR_GIC_SGI 16
> +#define FIRST_GIC_LPI 8192
> +#define NR_GIC_LPI 4096
> +#define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
MAX_LPI and NR_GIC_LPI should be obtained from the hardware at init time
and put somewhere, like a global nr_lpis perhaps, to be used throughout.
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 051a95e..0443ae7 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -169,6 +169,7 @@
> #define ICH_SGI_IRQ_MASK 0xf
> #define ICH_SGI_TARGETLIST_MASK 0xffff
> #define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PROP_ENABLED (1 << 0)
Please order (1 << X) by the X.
> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
> index 34b492b..55e219f 100644
> --- a/xen/include/asm-arm/irq.h
> +++ b/xen/include/asm-arm/irq.h
> @@ -17,6 +17,8 @@ struct arch_pirq
> struct arch_irq_desc {
> int eoi_cpu;
> unsigned int type;
> + struct its_device *dev;
> + u16 col_id;
There are probably opportunities to make eoi_cpu or type smaller so as
to allow col_id to be added without increasing the size of the struct.
Not a blocker for 4.6 though.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-10 13:46 ` Ian Campbell
@ 2015-07-11 14:40 ` Vijay Kilari
2015-07-11 18:08 ` Julien Grall
2015-07-13 9:17 ` Ian Campbell
0 siblings, 2 replies; 113+ messages in thread
From: Vijay Kilari @ 2015-07-11 14:40 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Fri, Jul 10, 2015 at 7:16 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Implements hw_irq_controller api's required
>> to handle LPI's
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> v4: - Implement separate hw_irq_controller for LPIs
>> - Drop setting LPI affinity
>> - virq and vid are moved under union
>> - Introduced inv command handling
>> - its_device is stored in irq_desc
>> ---
>> xen/arch/arm/gic-v3-its.c | 132 +++++++++++++++++++++++++++++++++++++
>> xen/arch/arm/gic-v3.c | 5 +-
>> xen/arch/arm/gic.c | 32 +++++++--
>> xen/arch/arm/irq.c | 40 ++++++++++-
>> xen/include/asm-arm/gic-its.h | 4 ++
>> xen/include/asm-arm/gic.h | 13 ++++
>> xen/include/asm-arm/gic_v3_defs.h | 1 +
>> xen/include/asm-arm/irq.h | 8 ++-
>> 8 files changed, 227 insertions(+), 8 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> index b421a6f..b98d396 100644
>> --- a/xen/arch/arm/gic-v3-its.c
>> +++ b/xen/arch/arm/gic-v3-its.c
>> @@ -295,6 +295,19 @@ post:
>> its_wait_for_range_completion(its, cmd, next_cmd);
>> }
>>
>> +static void its_send_inv(struct its_device *dev, struct its_collection *col,
>> + u32 event_id)
>> +{
>> + its_cmd_block cmd;
>> +
>> + memset(&cmd, 0x0, sizeof(its_cmd_block));
>> + cmd.inv.cmd = GITS_CMD_INV;
>> + cmd.inv.devid = dev->device_id;
>> + cmd.inv.event = event_id;
>> +
>> + its_send_single_command(dev->its, &cmd, col);
>> +}
>
> This ought to be in the prior patch doing such things I think.
>
> Oh I see, you didn't have struct its_device defined back then. I think
> you can just reorder patches #3 and #4 to solve that.
INV is used only in this patch in lpi_set_config().
So introduced in this patch
>
>> +static void its_host_irq_end(struct irq_desc *desc)
>> +{
>> + /* Lower the priority */
>> + gicv3_eoi_irq(desc);
>> + /* Deactivate */
>> + gicv3_dir_irq(desc);
>> +}
>> +
>> +static void its_guest_irq_end(struct irq_desc *desc)
>> +{
>> + gicv3_eoi_irq(desc);
>> +}
>
> Exposing those two gicv3 functions is a bit unfortunate, but I think it
> will do for now.
>
> Exposing gicv3_[host|guest]_irq_end might have been nicer, since you
> could just insert them into your its_[host|guest]_lpi_type instead of
> duplicating them.
>
> Eventually we may want to refactor such that register_its_ops gives back
> the ack/eoi hooks to use.
>
>> +static void its_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
>> +{
>
> Please add
>
> /* Not yet supported */
>
>> @@ -104,7 +126,8 @@ static void gic_set_irq_properties(struct irq_desc *desc,
>> const cpumask_t *cpu_mask,
>> unsigned int priority)
>> {
>> - gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
>> + if ( desc->irq < gic_number_lines() )
>
> Should this be is_lpi as in other similar places?
>
>> + gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
>> }
>>
>> /* Program the GIC to route an interrupt to the host (i.e. Xen)
>> @@ -114,11 +137,12 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>> unsigned int priority)
>> {
>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
>> + /* Can't route interrupts that don't exist */
>> + ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
>
> As discussed in <1436284206.25646.258.camel@citrix.com> please make some
> sort of is_valid_irq(irq) helper to encapsulate this logic.
I have added it patch#12. I remove this change from this patch
>
>> ocessor), priority);
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 2dd43ee..ba8528a 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
>> struct irq_guest
>> {
>> struct domain *d;
>> - unsigned int virq;
>> + union
>> + {
>> + /* virq refer to virtual irq in case of spi */
>> + unsigned int virq;
>> + /* virq refer to event ID in case of lpi */
>
> "refers" in both cases
>
> And I'd say "to the ..." not just "to ..." and "in the case of..." too.
>
>> +unsigned int irq_to_vid(struct irq_desc *desc)
>> +{
>> + return irq_get_guest_info(desc)->vid;
>> +}
>> +
>> +unsigned int irq_to_virq(struct irq_desc *desc)
>> +{
>> + return irq_get_guest_info(desc)->virq;
>> +}
>
> Please assert that irq_desc->arch.its_device is (non-)NULL as
> appropriate in these two cases.
These two functions are accessing irq_guest structure not arch.its_device
>
> BTW, while checking the field name I spotted "struct msi_desc
> *msi_desc" in the main struct irq_desc.
>
> Since MSIs are effectively the same as LPIs as a future cleanup I think
> we should s/its_device/msi_desc/g and use this field instead of adding a
> second redundant type and pointer to it. THis is not a blocker for 4.6
> though.
>
>> +struct its_device *get_irq_device(struct irq_desc *desc)
>> +{
>> + ASSERT(spin_is_locked(&desc->lock));
>> +
>> + return desc->arch.dev;
>> +}
>> +
>> +void set_irq_device(struct irq_desc *desc, struct its_device *dev)
>> +{
>> + ASSERT(spin_is_locked(&desc->lock));
>> + desc->arch.dev = dev;
>> +}
>
> Please add _its to the names of both of these functions, ie..g
> set_irq_its_device.
>
>> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
>> index b5e09bd..e8d244f 100644
>> --- a/xen/include/asm-arm/gic-its.h
>> +++ b/xen/include/asm-arm/gic-its.h
>> @@ -161,6 +161,10 @@ typedef union {
>> * The ITS view of a device.
>> */
>> struct its_device {
>> + /* Physical ITS */
>> + struct its_node *its;
>> + /* Number of Physical LPIs assigned */
>> + int nr_lpis;
>> /* Physical Device id */
>> u32 device_id;
>> /* RB-tree entry */
>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>> index e9d5f36..44c2317 100644
>> --- a/xen/include/asm-arm/gic.h
>> +++ b/xen/include/asm-arm/gic.h
>> @@ -20,6 +20,9 @@
>>
>> #define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
>> #define NR_GIC_SGI 16
>> +#define FIRST_GIC_LPI 8192
>> +#define NR_GIC_LPI 4096
>> +#define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
>
> MAX_LPI and NR_GIC_LPI should be obtained from the hardware at init time
> and put somewhere, like a global nr_lpis perhaps, to be used throughout.
This MAX_LPI and NR_GIC_LPI is Xen limitation where in we
are allocating irq_descriptors statically upto NR_GIC_LPI.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-11 14:40 ` Vijay Kilari
@ 2015-07-11 18:08 ` Julien Grall
2015-07-13 9:17 ` Ian Campbell
1 sibling, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-11 18:08 UTC (permalink / raw)
To: Vijay Kilari, Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
Hi Vijay,
On 11/07/2015 16:40, Vijay Kilari wrote:
>>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>>> index e9d5f36..44c2317 100644
>>> --- a/xen/include/asm-arm/gic.h
>>> +++ b/xen/include/asm-arm/gic.h
>>> @@ -20,6 +20,9 @@
>>>
>>> #define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
>>> #define NR_GIC_SGI 16
>>> +#define FIRST_GIC_LPI 8192
>>> +#define NR_GIC_LPI 4096
>>> +#define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
>>
>> MAX_LPI and NR_GIC_LPI should be obtained from the hardware at init time
>> and put somewhere, like a global nr_lpis perhaps, to be used throughout.
>
> This MAX_LPI and NR_GIC_LPI is Xen limitation where in we
> are allocating irq_descriptors statically upto NR_GIC_LPI.
Why? The design doc [1] suggested to allocate dynamically the irq_desc
array for LPI at boot.
Using a static array will grow up the Xen binary by close to 4096 LPIs *
60 bytes (rough estimate) = 240 KB.
And this is for any AArch64 platform, no matter that ITS is in used or not.
Regards,
[1] http://xenbits.xen.org/people/ianc/vits/draftG.html#irq-descriptors
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-11 14:40 ` Vijay Kilari
2015-07-11 18:08 ` Julien Grall
@ 2015-07-13 9:17 ` Ian Campbell
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-13 9:17 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Sat, 2015-07-11 at 20:10 +0530, Vijay Kilari wrote:
> On Fri, Jul 10, 2015 at 7:16 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> > On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >>
> >> Implements hw_irq_controller api's required
> >> to handle LPI's
> >>
> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >> ---
> >> v4: - Implement separate hw_irq_controller for LPIs
> >> - Drop setting LPI affinity
> >> - virq and vid are moved under union
> >> - Introduced inv command handling
> >> - its_device is stored in irq_desc
> >> ---
> >> xen/arch/arm/gic-v3-its.c | 132 +++++++++++++++++++++++++++++++++++++
> >> xen/arch/arm/gic-v3.c | 5 +-
> >> xen/arch/arm/gic.c | 32 +++++++--
> >> xen/arch/arm/irq.c | 40 ++++++++++-
> >> xen/include/asm-arm/gic-its.h | 4 ++
> >> xen/include/asm-arm/gic.h | 13 ++++
> >> xen/include/asm-arm/gic_v3_defs.h | 1 +
> >> xen/include/asm-arm/irq.h | 8 ++-
> >> 8 files changed, 227 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> >> index b421a6f..b98d396 100644
> >> --- a/xen/arch/arm/gic-v3-its.c
> >> +++ b/xen/arch/arm/gic-v3-its.c
> >> @@ -295,6 +295,19 @@ post:
> >> its_wait_for_range_completion(its, cmd, next_cmd);
> >> }
> >>
> >> +static void its_send_inv(struct its_device *dev, struct its_collection *col,
> >> + u32 event_id)
> >> +{
> >> + its_cmd_block cmd;
> >> +
> >> + memset(&cmd, 0x0, sizeof(its_cmd_block));
> >> + cmd.inv.cmd = GITS_CMD_INV;
> >> + cmd.inv.devid = dev->device_id;
> >> + cmd.inv.event = event_id;
> >> +
> >> + its_send_single_command(dev->its, &cmd, col);
> >> +}
> >
> > This ought to be in the prior patch doing such things I think.
> >
> > Oh I see, you didn't have struct its_device defined back then. I think
> > you can just reorder patches #3 and #4 to solve that.
>
> INV is used only in this patch in lpi_set_config().
> So introduced in this patch
And the other patch introduces every (almost) every other cmd handler.
Having one patch do a bulk add of most commands and then other commands
dribbled in later as they are used just makes the series harder to
follow.
> >> @@ -114,11 +137,12 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> >> unsigned int priority)
> >> {
> >> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> >> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
> >> + /* Can't route interrupts that don't exist */
> >> + ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
> >
> > As discussed in <1436284206.25646.258.camel@citrix.com> please make some
> > sort of is_valid_irq(irq) helper to encapsulate this logic.
>
> I have added it patch#12. I remove this change from this patch
Please fix the ordering of the series so that you don't need to do
things like this. It just wastes review bandwidth since people reading
patch #5 have no idea what is going to happen in #12 and in any case bad
or redundant code shouldn't be added only to be removed later unless
there is really no option (a rare occurrence)
> >> +unsigned int irq_to_vid(struct irq_desc *desc)
> >> +{
> >> + return irq_get_guest_info(desc)->vid;
> >> +}
> >> +
> >> +unsigned int irq_to_virq(struct irq_desc *desc)
> >> +{
> >> + return irq_get_guest_info(desc)->virq;
> >> +}
> >
> > Please assert that irq_desc->arch.its_device is (non-)NULL as
> > appropriate in these two cases.
>
> These two functions are accessing irq_guest structure not arch.its_device
->vid and ->virq are members of a union. The distinguishing feature
which tells us which one is valid is whether or not
irq_desc->arch.its_device is NULL or not.
Therefore an assertion in each function should be added to catch cases
where people try to get the vid of an SPI or the virq of an LPI.
> >> #define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
> >> #define NR_GIC_SGI 16
> >> +#define FIRST_GIC_LPI 8192
> >> +#define NR_GIC_LPI 4096
> >> +#define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
> >
> > MAX_LPI and NR_GIC_LPI should be obtained from the hardware at init time
> > and put somewhere, like a global nr_lpis perhaps, to be used throughout.
>
> This MAX_LPI and NR_GIC_LPI is Xen limitation where in we
> are allocating irq_descriptors statically upto NR_GIC_LPI.
As I said later on, please make this allocation dynamic as described in
the design doc. The static LPI descriptor array used in this series is
not acceptable.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-10 7:42 ` [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
2015-07-10 13:46 ` Ian Campbell
@ 2015-07-13 21:18 ` Julien Grall
2015-07-15 7:16 ` Vijay Kilari
2015-07-15 18:19 ` Julien Grall
2 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-13 21:18 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, vijaya.kumar, manish.jaggi
Hi,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index c41e82e..4f3801b 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> +static inline hw_irq_controller *get_host_hw_irq_controller(unsigned int irq)
> +{
> + if ( is_lpi(irq) )
> + return its_hw_ops->lpi_host_irq_type;
> + else
> + return gic_hw_ops->gic_host_irq_type;
> +}
This is not what I asked on v3 [1]. The ITS hardware controller
shouldn't be exposed to the common GIC. We have to keep a clean and
comprehensible interface.
What I asked is to replace the gic_host_irq_type variable by a new
callback which will return the correct hw_irq_controller.
For GICv2, it will return the same hw_irq_controller as today. For
GICv3, it will check is the IRQ is an LPI and return the correct controller.
FWIW, it was "ack" by Ian [2].
> +
> +static inline hw_irq_controller *get_guest_hw_irq_controller(unsigned int irq)
> +{
> + if ( is_lpi(irq) )
> + return its_hw_ops->lpi_guest_irq_type;
> + else
> + return gic_hw_ops->gic_guest_irq_type;
> +}
> +
> /*
> * needs to be called with a valid cpu_mask, ie each cpu in the mask has
> * already called gic_cpu_init
> @@ -104,7 +126,8 @@ static void gic_set_irq_properties(struct irq_desc *desc,
> const cpumask_t *cpu_mask,
> unsigned int priority)
> {
> - gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
> + if ( desc->irq < gic_number_lines() )
> + gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
> }
Why this function can't be called for LPI? The configuration should
likely be the same...
> @@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> goto out;
>
> - desc->handler = gic_hw_ops->gic_guest_irq_type;
> + desc->handler = get_guest_hw_irq_controller(desc->irq);
> set_bit(_IRQ_GUEST, &desc->status);
>
> gic_set_irq_properties(desc, cpumask_of(v_target->processor), priority);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 2dd43ee..ba8528a 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
> struct irq_guest
> {
> struct domain *d;
> - unsigned int virq;
> + union
> + {
> + /* virq refer to virtual irq in case of spi */
> + unsigned int virq;
> + /* virq refer to event ID in case of lpi */
> + unsigned int vid;
Why can't we store the event ID in the irq_guest? As said on v3, this is
not domain specific [3]. Furthermore, you add support to route LPI in
Xen (see gic_route_irq_to_xen) where you will clearly need the event ID.
> void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
> {
> if ( desc != NULL )
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index b5e09bd..e8d244f 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -161,6 +161,10 @@ typedef union {
> * The ITS view of a device.
> */
> struct its_device {
> + /* Physical ITS */
> + struct its_node *its;
> + /* Number of Physical LPIs assigned */
> + int nr_lpis;
Why didn't you add this field directly in the patch #4? It would be more
logical.
> /*
> * ITS registers, offsets from ITS_base
> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
> index 34b492b..55e219f 100644
> --- a/xen/include/asm-arm/irq.h
> +++ b/xen/include/asm-arm/irq.h
> @@ -17,6 +17,8 @@ struct arch_pirq
> struct arch_irq_desc {
> int eoi_cpu;
> unsigned int type;
> + struct its_device *dev;
> + u16 col_id;
It has been suggested by Ian to move col_id in the its_device in the
previous version [4]. Any reason to not doing it?
Regards,
[1] http://www.gossamer-threads.com/lists/xen/devel/386493#386493
[2] http://www.gossamer-threads.com/lists/xen/devel/386771#386771
[3] http://www.gossamer-threads.com/lists/xen/devel/385521#385521
[4] http://www.gossamer-threads.com/lists/xen/devel/387586#387586
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-13 21:18 ` Julien Grall
@ 2015-07-15 7:16 ` Vijay Kilari
2015-07-15 8:26 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-15 7:16 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Tue, Jul 14, 2015 at 2:48 AM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi,
>
> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index c41e82e..4f3801b 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> +static inline hw_irq_controller *get_host_hw_irq_controller(unsigned int
>> irq)
>> +{
>> + if ( is_lpi(irq) )
>> + return its_hw_ops->lpi_host_irq_type;
>> + else
>> + return gic_hw_ops->gic_host_irq_type;
>> +}
>
>
> This is not what I asked on v3 [1]. The ITS hardware controller shouldn't be
> exposed to the common GIC. We have to keep a clean and comprehensible
> interface.
>
> What I asked is to replace the gic_host_irq_type variable by a new callback
> which will return the correct hw_irq_controller.
>
> For GICv2, it will return the same hw_irq_controller as today. For GICv3, it
> will check is the IRQ is an LPI and return the correct controller.
>
> FWIW, it was "ack" by Ian [2].
If we don't want to expose any ITS interfaces to common gic code, then we
have to register callbacks to GICv3 driver.
>
>> +
>> +static inline hw_irq_controller *get_guest_hw_irq_controller(unsigned int
>> irq)
>> +{
>> + if ( is_lpi(irq) )
>> + return its_hw_ops->lpi_guest_irq_type;
>> + else
>> + return gic_hw_ops->gic_guest_irq_type;
>> +}
>> +
>> /*
>> * needs to be called with a valid cpu_mask, ie each cpu in the mask has
>> * already called gic_cpu_init
>> @@ -104,7 +126,8 @@ static void gic_set_irq_properties(struct irq_desc
>> *desc,
>> const cpumask_t *cpu_mask,
>> unsigned int priority)
>> {
>> - gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
>> + if ( desc->irq < gic_number_lines() )
>> + gic_hw_ops->set_irq_properties(desc, cpu_mask, priority);
>> }
>
>
> Why this function can't be called for LPI? The configuration should likely
> be the same...
>
>> @@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned
>> int virq,
>> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>> goto out;
>>
>> - desc->handler = gic_hw_ops->gic_guest_irq_type;
>> + desc->handler = get_guest_hw_irq_controller(desc->irq);
>> set_bit(_IRQ_GUEST, &desc->status);
>>
>> gic_set_irq_properties(desc, cpumask_of(v_target->processor),
>> priority);
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 2dd43ee..ba8528a 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
>> struct irq_guest
>> {
>> struct domain *d;
>> - unsigned int virq;
>> + union
>> + {
>> + /* virq refer to virtual irq in case of spi */
>> + unsigned int virq;
>> + /* virq refer to event ID in case of lpi */
>> + unsigned int vid;
>
>
> Why can't we store the event ID in the irq_guest? As said on v3, this is not
Are you referring to irq_desc in above statement?
> domain specific [3]. Furthermore, you add support to route LPI in Xen (see
> gic_route_irq_to_xen) where you will clearly need the event ID.
>
>> void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
>> {
>> if ( desc != NULL )
>> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
>> index b5e09bd..e8d244f 100644
>> --- a/xen/include/asm-arm/gic-its.h
>> +++ b/xen/include/asm-arm/gic-its.h
>> @@ -161,6 +161,10 @@ typedef union {
>> * The ITS view of a device.
>> */
>> struct its_device {
>> + /* Physical ITS */
>> + struct its_node *its;
>> + /* Number of Physical LPIs assigned */
>> + int nr_lpis;
>
>
> Why didn't you add this field directly in the patch #4? It would be more
> logical.
>
>> /*
>> * ITS registers, offsets from ITS_base
>> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
>> index 34b492b..55e219f 100644
>> --- a/xen/include/asm-arm/irq.h
>> +++ b/xen/include/asm-arm/irq.h
>> @@ -17,6 +17,8 @@ struct arch_pirq
>> struct arch_irq_desc {
>> int eoi_cpu;
>> unsigned int type;
>> + struct its_device *dev;
>> + u16 col_id;
>
>
> It has been suggested by Ian to move col_id in the its_device in the
> previous version [4]. Any reason to not doing it?
In round robin fashion each plpi is attached to col_id. So storing
in its_device is not possible. In linux latest col_id is stored in its_device
structure for which set_affinity is called.
>
> Regards,
>
> [1] http://www.gossamer-threads.com/lists/xen/devel/386493#386493
> [2] http://www.gossamer-threads.com/lists/xen/devel/386771#386771
> [3] http://www.gossamer-threads.com/lists/xen/devel/385521#385521
> [4] http://www.gossamer-threads.com/lists/xen/devel/387586#387586
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 7:16 ` Vijay Kilari
@ 2015-07-15 8:26 ` Julien Grall
2015-07-15 9:32 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 8:26 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Vijay,
On 15/07/2015 09:16, Vijay Kilari wrote:
> On Tue, Jul 14, 2015 at 2:48 AM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi,
>>
>> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>>
>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>> index c41e82e..4f3801b 100644
>>> --- a/xen/arch/arm/gic.c
>>> +++ b/xen/arch/arm/gic.c
>>> +static inline hw_irq_controller *get_host_hw_irq_controller(unsigned int
>>> irq)
>>> +{
>>> + if ( is_lpi(irq) )
>>> + return its_hw_ops->lpi_host_irq_type;
>>> + else
>>> + return gic_hw_ops->gic_host_irq_type;
>>> +}
>>
>>
>> This is not what I asked on v3 [1]. The ITS hardware controller shouldn't be
>> exposed to the common GIC. We have to keep a clean and comprehensible
>> interface.
>>
>> What I asked is to replace the gic_host_irq_type variable by a new callback
>> which will return the correct hw_irq_controller.
>>
>> For GICv2, it will return the same hw_irq_controller as today. For GICv3, it
>> will check is the IRQ is an LPI and return the correct controller.
>>
>> FWIW, it was "ack" by Ian [2].
>
> If we don't want to expose any ITS interfaces to common gic code, then we
> have to register callbacks to GICv3 driver.
Why? In fine, the ITS is an integral part of the GICv3, so you could
directly call the ITS code within the GICv3 without any callback.
Actually, you already do that in some place. So I don't see why you
can't do it there...
>>> @@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned
>>> int virq,
>>> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>>> goto out;
>>>
>>> - desc->handler = gic_hw_ops->gic_guest_irq_type;
>>> + desc->handler = get_guest_hw_irq_controller(desc->irq);
>>> set_bit(_IRQ_GUEST, &desc->status);
>>>
>>> gic_set_irq_properties(desc, cpumask_of(v_target->processor),
>>> priority);
>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>> index 2dd43ee..ba8528a 100644
>>> --- a/xen/arch/arm/irq.c
>>> +++ b/xen/arch/arm/irq.c
>>> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
>>> struct irq_guest
>>> {
>>> struct domain *d;
>>> - unsigned int virq;
>>> + union
>>> + {
>>> + /* virq refer to virtual irq in case of spi */
>>> + unsigned int virq;
>>> + /* virq refer to event ID in case of lpi */
>>> + unsigned int vid;
>>
>>
>> Why can't we store the event ID in the irq_guest? As said on v3, this is not
>
> Are you referring to irq_desc in above statement?
Yes sorry.
>
>> domain specific [3]. Furthermore, you add support to route LPI in Xen (see
>> gic_route_irq_to_xen) where you will clearly need the event ID.
>>
>>> void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
>>> {
>>> if ( desc != NULL )
>>> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
>>> index b5e09bd..e8d244f 100644
>>> --- a/xen/include/asm-arm/gic-its.h
>>> +++ b/xen/include/asm-arm/gic-its.h
>>> @@ -161,6 +161,10 @@ typedef union {
>>> * The ITS view of a device.
>>> */
>>> struct its_device {
>>> + /* Physical ITS */
>>> + struct its_node *its;
>>> + /* Number of Physical LPIs assigned */
>>> + int nr_lpis;
>>
>>
>> Why didn't you add this field directly in the patch #4? It would be more
>> logical.
>>
>>> /*
>>> * ITS registers, offsets from ITS_base
>>> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
>>> index 34b492b..55e219f 100644
>>> --- a/xen/include/asm-arm/irq.h
>>> +++ b/xen/include/asm-arm/irq.h
>>> @@ -17,6 +17,8 @@ struct arch_pirq
>>> struct arch_irq_desc {
>>> int eoi_cpu;
>>> unsigned int type;
>>> + struct its_device *dev;
>>> + u16 col_id;
>>
>>
>> It has been suggested by Ian to move col_id in the its_device in the
>> previous version [4]. Any reason to not doing it?
>
> In round robin fashion each plpi is attached to col_id. So storing
> in its_device is not possible. In linux latest col_id is stored in its_device
> structure for which set_affinity is called.
You could do round robin on its_device... It would be exactly the same
and save 2 byte if not more with the alignment per irq_desc.
Don't forget that 1 byte in the irq_desc means 1KB added in Xen binary.
These bytes saved could be used to store the event ID.
That remind me, these 2 new fields should only be defined when GICv3 is
used (#ifdef HAS_GICV3).
I'm would be fine if you skip the former for 4.6, but the latter is
mandatory. ITS code shouldn't be compiled on arm32.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 8:26 ` Julien Grall
@ 2015-07-15 9:32 ` Ian Campbell
2015-07-15 9:49 ` Julien Grall
` (2 more replies)
0 siblings, 3 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-15 9:32 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Wed, 2015-07-15 at 10:26 +0200, Julien Grall wrote:
> >>> @@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned
> >>> int virq,
> >>> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> >>> goto out;
> >>>
> >>> - desc->handler = gic_hw_ops->gic_guest_irq_type;
> >>> + desc->handler = get_guest_hw_irq_controller(desc->irq);
> >>> set_bit(_IRQ_GUEST, &desc->status);
> >>>
> >>> gic_set_irq_properties(desc, cpumask_of(v_target->processor),
> >>> priority);
> >>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> >>> index 2dd43ee..ba8528a 100644
> >>> --- a/xen/arch/arm/irq.c
> >>> +++ b/xen/arch/arm/irq.c
> >>> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
> >>> struct irq_guest
> >>> {
> >>> struct domain *d;
> >>> - unsigned int virq;
> >>> + union
> >>> + {
> >>> + /* virq refer to virtual irq in case of spi */
> >>> + unsigned int virq;
> >>> + /* virq refer to event ID in case of lpi */
> >>> + unsigned int vid;
> >>
> >>
> >> Why can't we store the event ID in the irq_guest? As said on v3, this is not
> >
> > Are you referring to irq_desc in above statement?
>
> Yes sorry.
I'm afraid I don't follow your suggestion here, are you suggesting that
the vid field added above should be moved to irq_desc?
But the vid _is_ domain specific, it is the virtual event ID which is
per-domain (it's the thing looked up in the ITT to get a vLPI to be
injected). I think it is a pretty direct analogue of the virq field used
for non-LPI irq_guest structs.
If we had need for the physical event id then that would like belong in
the irq_desc.
Your proposal on v3 looks to be around moving the its_device pointer to
the irq_desc, which appears to have been done here, along with turning
the virq+vid into a union as requested there too.
> >> It has been suggested by Ian to move col_id in the its_device in the
> >> previous version [4]. Any reason to not doing it?
> >
> > In round robin fashion each plpi is attached to col_id. So storing
> > in its_device is not possible. In linux latest col_id is stored in its_device
> > structure for which set_affinity is called.
Are you saying that in Linux all Events/LPIs associated with a given ITS
device are routed to the same collection?
> You could do round robin on its_device... It would be exactly the same
Routing all LPIs associated with a given its_device to the same
collection is not exactly the same as round robin-ing all LPIs from the
device over the collections.
> and save 2 byte if not more with the alignment per irq_desc.
If this is a concern then I would say we would either want a separate
array of per-pLPI information which we do not want in irq_desc because
it is irq specific, or do add a pointer to its_desc which points to an
array of per-event information.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 9:32 ` Ian Campbell
@ 2015-07-15 9:49 ` Julien Grall
2015-07-15 10:01 ` Ian Campbell
2015-07-15 14:15 ` Vijay Kilari
2015-07-18 10:13 ` Julien Grall
2 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 9:49 UTC (permalink / raw)
To: Ian Campbell
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Ian,
On 15/07/2015 11:32, Ian Campbell wrote:
>>>> Why can't we store the event ID in the irq_guest? As said on v3, this is not
>>>
>>> Are you referring to irq_desc in above statement?
>>
>> Yes sorry.
>
> I'm afraid I don't follow your suggestion here, are you suggesting that
> the vid field added above should be moved to irq_desc?
Yes,
> But the vid _is_ domain specific, it is the virtual event ID which is
> per-domain (it's the thing looked up in the ITT to get a vLPI to be
> injected). I think it is a pretty direct analogue of the virq field used
> for non-LPI irq_guest structs.
No, vid is not specific to a domain but a device. The virtual event ID
is always the same as the physical event ID (See your design document
[1]). Furthermore, all the usage of the irq_to_vid in this series are
for physical command (see lpi_set_config within this patch).
>
> Your proposal on v3 looks to be around moving the its_device pointer to
> the irq_desc, which appears to have been done here, along with turning
> the virq+vid into a union as requested there too.
On v3 I said: "The event ID and
the its_device assigned are known when the device is added to Xen and
hence can be set in irq_desc (with a small memory impact, but we have
plenty of memory on ARM64)."
Sorry if it was confusing.
>
>>>> It has been suggested by Ian to move col_id in the its_device in the
>>>> previous version [4]. Any reason to not doing it?
>>>
>>> In round robin fashion each plpi is attached to col_id. So storing
>>> in its_device is not possible. In linux latest col_id is stored in its_device
>>> structure for which set_affinity is called.
>
> Are you saying that in Linux all Events/LPIs associated with a given ITS
> device are routed to the same collection?
>
>> You could do round robin on its_device... It would be exactly the same
>
> Routing all LPIs associated with a given its_device to the same
> collection is not exactly the same as round robin-ing all LPIs from the
> device over the collections.
Yes, sorry I was a bit lax on the writing. I wanted to meant that there
is not much difference to do it.
>> and save 2 byte if not more with the alignment per irq_desc.
>
> If this is a concern then I would say we would either want a separate
> array of per-pLPI information which we do not want in irq_desc because
> it is irq specific, or do add a pointer to its_desc which points to an
> array of per-event information.
That would be a good solution. Although, as I said, I don't really care
for Xen 4.6. It's more an optimization for 4.7.
Regards,
[1] http://xenbits.xen.org/people/ianc/vits/draftG.html#event-id-event
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 9:49 ` Julien Grall
@ 2015-07-15 10:01 ` Ian Campbell
0 siblings, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-15 10:01 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Wed, 2015-07-15 at 11:49 +0200, Julien Grall wrote:
> Hi Ian,
>
> On 15/07/2015 11:32, Ian Campbell wrote:
> >>>> Why can't we store the event ID in the irq_guest? As said on v3, this is not
> >>>
> >>> Are you referring to irq_desc in above statement?
> >>
> >> Yes sorry.
> >
> > I'm afraid I don't follow your suggestion here, are you suggesting that
> > the vid field added above should be moved to irq_desc?
>
> Yes,
>
> > But the vid _is_ domain specific, it is the virtual event ID which is
> > per-domain (it's the thing looked up in the ITT to get a vLPI to be
> > injected). I think it is a pretty direct analogue of the virq field used
> > for non-LPI irq_guest structs.
>
> No, vid is not specific to a domain but a device. The virtual event ID
> is always the same as the physical event ID (See your design document
> [1]). Furthermore, all the usage of the irq_to_vid in this series are
> for physical command (see lpi_set_config within this patch).
>
> >
> > Your proposal on v3 looks to be around moving the its_device pointer to
> > the irq_desc, which appears to have been done here, along with turning
> > the virq+vid into a union as requested there too.
>
> On v3 I said: "The event ID and
> the its_device assigned are known when the device is added to Xen and
> hence can be set in irq_desc (with a small memory impact, but we have
> plenty of memory on ARM64)."
>
> Sorry if it was confusing.
It was me who was confusing the properties of vid with those of vlpi,
sorry.
Not helped by
http://xenbits.xen.org/people/ianc/vits/draftG.html#virtual-lpi-injection confusingly using "virq" instead of "vid".
Ian.
> >>>> It has been suggested by Ian to move col_id in the its_device in the
> >>>> previous version [4]. Any reason to not doing it?
> >>>
> >>> In round robin fashion each plpi is attached to col_id. So storing
> >>> in its_device is not possible. In linux latest col_id is stored in its_device
> >>> structure for which set_affinity is called.
> >
> > Are you saying that in Linux all Events/LPIs associated with a given ITS
> > device are routed to the same collection?
> >
> >> You could do round robin on its_device... It would be exactly the same
> >
> > Routing all LPIs associated with a given its_device to the same
> > collection is not exactly the same as round robin-ing all LPIs from the
> > device over the collections.
>
> Yes, sorry I was a bit lax on the writing. I wanted to meant that there
> is not much difference to do it.
>
> >> and save 2 byte if not more with the alignment per irq_desc.
> >
> > If this is a concern then I would say we would either want a separate
> > array of per-pLPI information which we do not want in irq_desc because
> > it is irq specific, or do add a pointer to its_desc which points to an
> > array of per-event information.
>
> That would be a good solution. Although, as I said, I don't really care
> for Xen 4.6. It's more an optimization for 4.7.
>
> Regards,
>
> [1] http://xenbits.xen.org/people/ianc/vits/draftG.html#event-id-event
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 9:32 ` Ian Campbell
2015-07-15 9:49 ` Julien Grall
@ 2015-07-15 14:15 ` Vijay Kilari
2015-07-15 14:22 ` Julien Grall
2015-07-15 14:28 ` Ian Campbell
2015-07-18 10:13 ` Julien Grall
2 siblings, 2 replies; 113+ messages in thread
From: Vijay Kilari @ 2015-07-15 14:15 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Wed, Jul 15, 2015 at 3:02 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Wed, 2015-07-15 at 10:26 +0200, Julien Grall wrote:
>
>> >>> @@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned
>> >>> int virq,
>> >>> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>> >>> goto out;
>> >>>
>> >>> - desc->handler = gic_hw_ops->gic_guest_irq_type;
>> >>> + desc->handler = get_guest_hw_irq_controller(desc->irq);
>> >>> set_bit(_IRQ_GUEST, &desc->status);
>> >>>
>> >>> gic_set_irq_properties(desc, cpumask_of(v_target->processor),
>> >>> priority);
>> >>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> >>> index 2dd43ee..ba8528a 100644
>> >>> --- a/xen/arch/arm/irq.c
>> >>> +++ b/xen/arch/arm/irq.c
>> >>> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
>> >>> struct irq_guest
>> >>> {
>> >>> struct domain *d;
>> >>> - unsigned int virq;
>> >>> + union
>> >>> + {
>> >>> + /* virq refer to virtual irq in case of spi */
>> >>> + unsigned int virq;
>> >>> + /* virq refer to event ID in case of lpi */
>> >>> + unsigned int vid;
>> >>
>> >>
>> >> Why can't we store the event ID in the irq_guest? As said on v3, this is not
>> >
>> > Are you referring to irq_desc in above statement?
>>
>> Yes sorry.
>
> I'm afraid I don't follow your suggestion here, are you suggesting that
> the vid field added above should be moved to irq_desc?
>
> But the vid _is_ domain specific, it is the virtual event ID which is
> per-domain (it's the thing looked up in the ITT to get a vLPI to be
> injected). I think it is a pretty direct analogue of the virq field used
> for non-LPI irq_guest structs.
>
> If we had need for the physical event id then that would like belong in
> the irq_desc.
>
> Your proposal on v3 looks to be around moving the its_device pointer to
> the irq_desc, which appears to have been done here, along with turning
> the virq+vid into a union as requested there too.
>
>> >> It has been suggested by Ian to move col_id in the its_device in the
>> >> previous version [4]. Any reason to not doing it?
>> >
>> > In round robin fashion each plpi is attached to col_id. So storing
>> > in its_device is not possible. In linux latest col_id is stored in its_device
>> > structure for which set_affinity is called.
>
> Are you saying that in Linux all Events/LPIs associated with a given ITS
> device are routed to the same collection?
Not associated with same collection. Events are associated with cpu
for which cpu_mask is set and the collection id of that cpu is stored
in the its_device, which is later used for SYNC. So effectively it_device
does not store collection id associated for all Events of that device.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 14:15 ` Vijay Kilari
@ 2015-07-15 14:22 ` Julien Grall
2015-07-15 14:28 ` Ian Campbell
1 sibling, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 14:22 UTC (permalink / raw)
To: Vijay Kilari, Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 15/07/2015 16:15, Vijay Kilari wrote:
> Not associated with same collection. Events are associated with cpu
> for which cpu_mask is set and the collection id of that cpu is stored
> in the its_device, which is later used for SYNC. So effectively it_device
> does not store collection id associated for all Events of that device.
It wouldn't be so bad to decide in Xen that all the LPIs associated to a
specified device are assigned to the same collection. It's kind of round
rounbin but on device rather than LPI.
Anyway, I think it's Xen 4.7 material so you don't have to worry about
it know.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 14:15 ` Vijay Kilari
2015-07-15 14:22 ` Julien Grall
@ 2015-07-15 14:28 ` Ian Campbell
2015-07-15 17:01 ` Vijay Kilari
1 sibling, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-15 14:28 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Wed, 2015-07-15 at 19:45 +0530, Vijay Kilari wrote:
> On Wed, Jul 15, 2015 at 3:02 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> > On Wed, 2015-07-15 at 10:26 +0200, Julien Grall wrote:
> >
> >> >>> @@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned
> >> >>> int virq,
> >> >>> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> >> >>> goto out;
> >> >>>
> >> >>> - desc->handler = gic_hw_ops->gic_guest_irq_type;
> >> >>> + desc->handler = get_guest_hw_irq_controller(desc->irq);
> >> >>> set_bit(_IRQ_GUEST, &desc->status);
> >> >>>
> >> >>> gic_set_irq_properties(desc, cpumask_of(v_target->processor),
> >> >>> priority);
> >> >>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> >> >>> index 2dd43ee..ba8528a 100644
> >> >>> --- a/xen/arch/arm/irq.c
> >> >>> +++ b/xen/arch/arm/irq.c
> >> >>> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
> >> >>> struct irq_guest
> >> >>> {
> >> >>> struct domain *d;
> >> >>> - unsigned int virq;
> >> >>> + union
> >> >>> + {
> >> >>> + /* virq refer to virtual irq in case of spi */
> >> >>> + unsigned int virq;
> >> >>> + /* virq refer to event ID in case of lpi */
> >> >>> + unsigned int vid;
> >> >>
> >> >>
> >> >> Why can't we store the event ID in the irq_guest? As said on v3, this is not
> >> >
> >> > Are you referring to irq_desc in above statement?
> >>
> >> Yes sorry.
> >
> > I'm afraid I don't follow your suggestion here, are you suggesting that
> > the vid field added above should be moved to irq_desc?
> >
> > But the vid _is_ domain specific, it is the virtual event ID which is
> > per-domain (it's the thing looked up in the ITT to get a vLPI to be
> > injected). I think it is a pretty direct analogue of the virq field used
> > for non-LPI irq_guest structs.
> >
> > If we had need for the physical event id then that would like belong in
> > the irq_desc.
> >
> > Your proposal on v3 looks to be around moving the its_device pointer to
> > the irq_desc, which appears to have been done here, along with turning
> > the virq+vid into a union as requested there too.
> >
> >> >> It has been suggested by Ian to move col_id in the its_device in the
> >> >> previous version [4]. Any reason to not doing it?
> >> >
> >> > In round robin fashion each plpi is attached to col_id. So storing
> >> > in its_device is not possible. In linux latest col_id is stored in its_device
> >> > structure for which set_affinity is called.
> >
> > Are you saying that in Linux all Events/LPIs associated with a given ITS
> > device are routed to the same collection?
>
> Not associated with same collection. Events are associated with cpu
> for which cpu_mask is set and the collection id of that cpu is stored
> in the its_device,
Surely storing the collection ID is precisely equivalent to storing the
cpu mask of the one CPU to which that collection ID is routed?
But that is orthogonal to my question anyway, so let me try again:
Are you saying that in Linux all Events/LPIs associated with a given ITS
device are routed to the same CPU?
> which is later used for SYNC.
> So effectively it_device
> does not store collection id associated for all Events of that device.
Right above you said "and the collection id of that cpu is stored in the
its_device", which contradicts what you now say here. (Unless it_device
is not a typo of its_device but a separate thing?)
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 14:28 ` Ian Campbell
@ 2015-07-15 17:01 ` Vijay Kilari
2015-07-16 14:49 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-15 17:01 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Wed, Jul 15, 2015 at 7:58 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Wed, 2015-07-15 at 19:45 +0530, Vijay Kilari wrote:
>> On Wed, Jul 15, 2015 at 3:02 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>> > On Wed, 2015-07-15 at 10:26 +0200, Julien Grall wrote:
>> >
>> >> >>> @@ -149,7 +173,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned
>> >> >>> int virq,
>> >> >>> test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>> >> >>> goto out;
>> >> >>>
>> >> >>> - desc->handler = gic_hw_ops->gic_guest_irq_type;
>> >> >>> + desc->handler = get_guest_hw_irq_controller(desc->irq);
>> >> >>> set_bit(_IRQ_GUEST, &desc->status);
>> >> >>>
>> >> >>> gic_set_irq_properties(desc, cpumask_of(v_target->processor),
>> >> >>> priority);
>> >> >>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> >> >>> index 2dd43ee..ba8528a 100644
>> >> >>> --- a/xen/arch/arm/irq.c
>> >> >>> +++ b/xen/arch/arm/irq.c
>> >> >>> @@ -35,7 +35,13 @@ static DEFINE_SPINLOCK(local_irqs_type_lock);
>> >> >>> struct irq_guest
>> >> >>> {
>> >> >>> struct domain *d;
>> >> >>> - unsigned int virq;
>> >> >>> + union
>> >> >>> + {
>> >> >>> + /* virq refer to virtual irq in case of spi */
>> >> >>> + unsigned int virq;
>> >> >>> + /* virq refer to event ID in case of lpi */
>> >> >>> + unsigned int vid;
>> >> >>
>> >> >>
>> >> >> Why can't we store the event ID in the irq_guest? As said on v3, this is not
>> >> >
>> >> > Are you referring to irq_desc in above statement?
>> >>
>> >> Yes sorry.
>> >
>> > I'm afraid I don't follow your suggestion here, are you suggesting that
>> > the vid field added above should be moved to irq_desc?
>> >
>> > But the vid _is_ domain specific, it is the virtual event ID which is
>> > per-domain (it's the thing looked up in the ITT to get a vLPI to be
>> > injected). I think it is a pretty direct analogue of the virq field used
>> > for non-LPI irq_guest structs.
>> >
>> > If we had need for the physical event id then that would like belong in
>> > the irq_desc.
>> >
>> > Your proposal on v3 looks to be around moving the its_device pointer to
>> > the irq_desc, which appears to have been done here, along with turning
>> > the virq+vid into a union as requested there too.
>> >
>> >> >> It has been suggested by Ian to move col_id in the its_device in the
>> >> >> previous version [4]. Any reason to not doing it?
>> >> >
>> >> > In round robin fashion each plpi is attached to col_id. So storing
>> >> > in its_device is not possible. In linux latest col_id is stored in its_device
>> >> > structure for which set_affinity is called.
>> >
>> > Are you saying that in Linux all Events/LPIs associated with a given ITS
>> > device are routed to the same collection?
>>
>> Not associated with same collection. Events are associated with cpu
>> for which cpu_mask is set and the collection id of that cpu is stored
>> in the its_device,
>
> Surely storing the collection ID is precisely equivalent to storing the
> cpu mask of the one CPU to which that collection ID is routed?
>
> But that is orthogonal to my question anyway, so let me try again:
>
> Are you saying that in Linux all Events/LPIs associated with a given ITS
> device are routed to the same CPU?
>
>> which is later used for SYNC.
>
>
>> So effectively it_device
>> does not store collection id associated for all Events of that device.
>
> Right above you said "and the collection id of that cpu is stored in the
> its_device", which contradicts what you now say here. (Unless it_device
> is not a typo of its_device but a separate thing?)
Sorry. I may not be clear.
In Linux when MSIx is enabled.
Device is created first and its_device->its_collection is set for
first onine cpu
and mapvi is called with collection set in its_create_device() as below.
struct its_device *its_create_device(struct its_node *its, u32 dev_id,
int nvecs)
{
....
/* Bind the device to the first possible CPU */
cpu = cpumask_first(cpu_online_mask);
dev->collection = &its->collections[cpu];
....
}
int its_alloc_device_irq(struct its_device *dev, u32 id,
int *hwirq, unsigned int *irq)
{
...
/* Map the GIC irq ID to the device */
its_send_mapvi(dev, *hwirq, id);
...
}
When affinity is set, movi is sent with collection id selected
for the cpu_mask.
static int its_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
bool force)
{
unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
...
target_col = &its_dev->its->collections[cpu];
....
its_send_movi(its_dev, target_col, id);
its_dev->collection = target_col;
...
}
So, collection id to Event/LPI mapping is not stored.
Regards
Vijay
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 17:01 ` Vijay Kilari
@ 2015-07-16 14:49 ` Ian Campbell
2015-07-16 15:21 ` Marc Zyngier
0 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-16 14:49 UTC (permalink / raw)
To: Vijay Kilari, Marc Zyngier
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Wed, 2015-07-15 at 22:31 +0530, Vijay Kilari wrote:
> Sorry. I may not be clear.
>
> In Linux when MSIx is enabled.
> Device is created first and its_device->its_collection is set for
> first onine cpu
> and mapvi is called with collection set in its_create_device() as below.
[...]
> When affinity is set, movi is sent with collection id selected
> for the cpu_mask.
OK, so at start of day all events for a given device end up mapped to a
single processor, but as individual interrupts are rebalanced they will
then become spread out among different CPUs, that makes sense.
I'm not sure I follow the scheme which Linux is using to achieve that
behaviour though, so I've CC'd Marc.
It seems to me from looking at the Linux code that its_dev->collection
doesn't, as one might expect, contain the collection associated with the
device (and therefore all Events of that device), but rather it contains
the collection corresponding to a single Event which is the one which
most recently had its affinity changed, i.e. the one with an potentially
outstanding INV.
So its_send_movi sends the MOVI command which ends up calling INV on
that collection, which at this point is the _old_ collection. Then it
stores the new collection for that Event in its_dev->collection.
What I don't follow is how set_lpi_config copes with this, since it
always sends the INV for an Event to the corresponding its_dev's
collection, but after a bunch of Events/IRQs have been rebalanced there
doesn't seem to be any guarantee that this would corresponding to the
current collection associated with the interrupt which has been
reconfigured.
Marc, have I misunderstood what the Linux ITS driver is trying to
achieve or how it goes about it?
Perhaps lpi_set_config (and by extension its_(un)mask_irq) is only
called at start of day?
I'm not sure if Xen is going to (want to) follow the way Linux arranges
this stuff, but I would very much like to at least understand it, since
it may have implications...
Cheers,
Ian
>
> struct its_device *its_create_device(struct its_node *its, u32 dev_id,
> int nvecs)
> {
> ....
> /* Bind the device to the first possible CPU */
> cpu = cpumask_first(cpu_online_mask);
> dev->collection = &its->collections[cpu];
> ....
> }
>
> int its_alloc_device_irq(struct its_device *dev, u32 id,
> int *hwirq, unsigned int *irq)
> {
> ...
> /* Map the GIC irq ID to the device */
> its_send_mapvi(dev, *hwirq, id);
> ...
> }
>
> When affinity is set, movi is sent with collection id selected
> for the cpu_mask.
>
> static int its_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
> bool force)
> {
> unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
> ...
> target_col = &its_dev->its->collections[cpu];
> ....
> its_send_movi(its_dev, target_col, id);
> its_dev->collection = target_col;
> ...
> }
>
> So, collection id to Event/LPI mapping is not stored.
>
> Regards
> Vijay
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-16 14:49 ` Ian Campbell
@ 2015-07-16 15:21 ` Marc Zyngier
2015-07-16 16:18 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Marc Zyngier @ 2015-07-16 15:21 UTC (permalink / raw)
To: Ian Campbell, Vijay Kilari
Cc: Stefano Stabellini, Prasun.Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, stefano.stabellini, manish.jaggi
On 16/07/15 15:49, Ian Campbell wrote:
> On Wed, 2015-07-15 at 22:31 +0530, Vijay Kilari wrote:
>> Sorry. I may not be clear.
>>
>> In Linux when MSIx is enabled.
>> Device is created first and its_device->its_collection is set for
>> first onine cpu
>> and mapvi is called with collection set in its_create_device() as below.
> [...]
>> When affinity is set, movi is sent with collection id selected
>> for the cpu_mask.
>
> OK, so at start of day all events for a given device end up mapped to a
> single processor, but as individual interrupts are rebalanced they will
> then become spread out among different CPUs, that makes sense.
>
> I'm not sure I follow the scheme which Linux is using to achieve that
> behaviour though, so I've CC'd Marc.
>
> It seems to me from looking at the Linux code that its_dev->collection
> doesn't, as one might expect, contain the collection associated with the
> device (and therefore all Events of that device), but rather it contains
> the collection corresponding to a single Event which is the one which
> most recently had its affinity changed, i.e. the one with an potentially
> outstanding INV.
>
> So its_send_movi sends the MOVI command which ends up calling INV on
> that collection, which at this point is the _old_ collection. Then it
> stores the new collection for that Event in its_dev->collection.
>
> What I don't follow is how set_lpi_config copes with this, since it
> always sends the INV for an Event to the corresponding its_dev's
> collection, but after a bunch of Events/IRQs have been rebalanced there
> doesn't seem to be any guarantee that this would corresponding to the
> current collection associated with the interrupt which has been
> reconfigured.
>
> Marc, have I misunderstood what the Linux ITS driver is trying to
> achieve or how it goes about it?
Seems like there is a lot of terminology related confusion here, so
let's start from the beginning:
The ITS maps a [DevID, EventID] pair to a [LPI, Collection] pair, where
DevID is the device writing the MSI, EventID is the payload being
written, LPI is the actual interrupt number, and Collection is roughly
equivalent to a target CPU.
The DevID is not directly associated to a Collection; it is the LPI that
is mapped to a Collection (through the MAPVI command). When you perform
a MOVI, you are moving the *result* of the [DevID, EventID] pair
(effectively an LPI) from a Collection to another.
> Perhaps lpi_set_config (and by extension its_(un)mask_irq) is only
> called at start of day?
lpi_set_config is indeed only used when we put the interrupt in service.
It updates the configuration table for the redistributor (the thing
pointed to by the Collection) in charge of this LPI (*LPI*, not device
or event) and issue an INV so that the redistributor (gets the new
value). And yes, this is always done by [DevID, EventID] pair.
When you perform a MOVI, this implicitly contains an INV for the target
redistributor (MOVI will also move the pending state, and the old
redistributor will be left alone).
> I'm not sure if Xen is going to (want to) follow the way Linux arranges
> this stuff, but I would very much like to at least understand it, since
> it may have implications...
Hope this helps,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-16 15:21 ` Marc Zyngier
@ 2015-07-16 16:18 ` Ian Campbell
2015-07-16 16:27 ` Marc Zyngier
0 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-16 16:18 UTC (permalink / raw)
To: Marc Zyngier
Cc: Vijay Kilari, Stefano Stabellini, Prasun.Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Julien Grall, stefano.stabellini,
manish.jaggi
On Thu, 2015-07-16 at 16:21 +0100, Marc Zyngier wrote:
> Hope this helps,
It, plus the chat we had on IRC did, yes, thanks.
In summary:
I was very confusedly talking about INV when I meant SYNC.
There is a real issue with the update of its_dev->collection in
its_set_affinity since it will result in the wrong collection being used
for subsequent affinity operations.
Marc is intending to replace its_dev->collection with an array
(presumably dynamically allocated) in its_dev mapping eventid to a
collection.
I think that is probably the right answer for Xen too.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-16 16:18 ` Ian Campbell
@ 2015-07-16 16:27 ` Marc Zyngier
2015-07-16 16:37 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Marc Zyngier @ 2015-07-16 16:27 UTC (permalink / raw)
To: Ian Campbell
Cc: Vijay Kilari, Stefano Stabellini, Prasun.Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Julien Grall, stefano.stabellini,
manish.jaggi
On 16/07/15 17:18, Ian Campbell wrote:
> On Thu, 2015-07-16 at 16:21 +0100, Marc Zyngier wrote:
>> Hope this helps,
>
> It, plus the chat we had on IRC did, yes, thanks.
>
> In summary:
>
> I was very confusedly talking about INV when I meant SYNC.
>
> There is a real issue with the update of its_dev->collection in
> its_set_affinity since it will result in the wrong collection being used
> for subsequent affinity operations.
Yeah, this is a very stupid bug on my side, and I realize that I never
saw it because the whole ITS driver has been written on a model that can
only generate a single interrupt per device. Lovely.
> Marc is intending to replace its_dev->collection with an array
> (presumably dynamically allocated) in its_dev mapping eventid to a
> collection.
Exactly.
> I think that is probably the right answer for Xen too.
I'd think so.
Thanks so much for putting me right!
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-16 16:27 ` Marc Zyngier
@ 2015-07-16 16:37 ` Ian Campbell
0 siblings, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-16 16:37 UTC (permalink / raw)
To: Marc Zyngier
Cc: Vijay Kilari, Stefano Stabellini, Prasun.Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Julien Grall, stefano.stabellini,
manish.jaggi
On Thu, 2015-07-16 at 17:27 +0100, Marc Zyngier wrote:
> On 16/07/15 17:18, Ian Campbell wrote:
> > On Thu, 2015-07-16 at 16:21 +0100, Marc Zyngier wrote:
> >> Hope this helps,
> >
> > It, plus the chat we had on IRC did, yes, thanks.
> >
> > In summary:
> >
> > I was very confusedly talking about INV when I meant SYNC.
> >
> > There is a real issue with the update of its_dev->collection in
> > its_set_affinity since it will result in the wrong collection being used
> > for subsequent affinity operations.
>
> Yeah, this is a very stupid bug on my side, and I realize that I never
> saw it because the whole ITS driver has been written on a model that can
> only generate a single interrupt per device. Lovely.
Oh dear!
> > Marc is intending to replace its_dev->collection with an array
> > (presumably dynamically allocated) in its_dev mapping eventid to a
> > collection.
>
> Exactly.
>
> > I think that is probably the right answer for Xen too.
>
> I'd think so.
>
> Thanks so much for putting me right!
No problem!
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-15 9:32 ` Ian Campbell
2015-07-15 9:49 ` Julien Grall
2015-07-15 14:15 ` Vijay Kilari
@ 2015-07-18 10:13 ` Julien Grall
2015-07-20 11:52 ` Ian Campbell
2015-07-20 12:22 ` Ian Campbell
2 siblings, 2 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-18 10:13 UTC (permalink / raw)
To: Ian Campbell
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Ian,
On 15/07/2015 10:32, Ian Campbell wrote:
>> and save 2 byte if not more with the alignment per irq_desc.
>
> If this is a concern then I would say we would either want a separate
> array of per-pLPI information which we do not want in irq_desc because
> it is irq specific, or do add a pointer to its_desc which points to an
> array of per-event information.
I noticed that we have a field msi_desc in the structure irq_desc. On
ARM, the structure msi_desc is not defined at all but still waste a 8
byte for the pointer in the irq_desc.
Given that LPI is an MSI, I'm wondering if we could move all LPI related
data in this msi_desc. This would avoid to introduce new field in the
irq_desc structure.
The msi_desc would look like:
struct msi_desc
{
its_device *dev;
unsigned int eventID;
};
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-18 10:13 ` Julien Grall
@ 2015-07-20 11:52 ` Ian Campbell
2015-07-20 12:22 ` Ian Campbell
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-20 11:52 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Sat, 2015-07-18 at 11:13 +0100, Julien Grall wrote:
> Hi Ian,
>
> On 15/07/2015 10:32, Ian Campbell wrote:
> >> and save 2 byte if not more with the alignment per irq_desc.
> >
> > If this is a concern then I would say we would either want a separate
> > array of per-pLPI information which we do not want in irq_desc because
> > it is irq specific, or do add a pointer to its_desc which points to an
> > array of per-event information.
>
> I noticed that we have a field msi_desc in the structure irq_desc. On
> ARM, the structure msi_desc is not defined at all but still waste a 8
> byte for the pointer in the irq_desc.
>
> Given that LPI is an MSI, I'm wondering if we could move all LPI related
> data in this msi_desc. This would avoid to introduce new field in the
> irq_desc structure.
Yes, I think I suggested something similar at some point, although I
seemed to think msi_desc was more similar to its_device.
Looking at the content of the struct, alloc_msi_entry and
msi_capability_init makes me less sure, it looks like msi_entry on x86
is allocated as an array (sized for events) stored per device, which
doesn't directly map onto how ARM is structured.
So what you propose may make more sense:
>
> The msi_desc would look like:
>
> struct msi_desc
> {
> its_device *dev;
> unsigned int eventID;
> };
>
> Regards,
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-18 10:13 ` Julien Grall
2015-07-20 11:52 ` Ian Campbell
@ 2015-07-20 12:22 ` Ian Campbell
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-20 12:22 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Sat, 2015-07-18 at 11:13 +0100, Julien Grall wrote:
> Hi Ian,
>
> On 15/07/2015 10:32, Ian Campbell wrote:
> >> and save 2 byte if not more with the alignment per irq_desc.
> >
> > If this is a concern then I would say we would either want a separate
> > array of per-pLPI information which we do not want in irq_desc because
> > it is irq specific, or do add a pointer to its_desc which points to an
> > array of per-event information.
>
> I noticed that we have a field msi_desc in the structure irq_desc. On
> ARM, the structure msi_desc is not defined at all but still waste a 8
> byte for the pointer in the irq_desc.
>
> Given that LPI is an MSI, I'm wondering if we could move all LPI related
> data in this msi_desc. This would avoid to introduce new field in the
> irq_desc structure.
Yes, I think I suggested something similar at some point, although I
seemed to think msi_desc was more similar to its_device.
Looking at the content of the struct, alloc_msi_entry and
msi_capability_init makes me less sure, it looks like msi_entry on x86
is allocated as an array (sized for events) stored per device, which
doesn't directly map onto how ARM is structured.
So I think you suggestion may make more sense.
>
> The msi_desc would look like:
>
> struct msi_desc
> {
> its_device *dev;
> unsigned int eventID;
> };
>
> Regards,
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-10 7:42 ` [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
2015-07-10 13:46 ` Ian Campbell
2015-07-13 21:18 ` Julien Grall
@ 2015-07-15 18:19 ` Julien Grall
2 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 18:19 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> +static void its_host_irq_end(struct irq_desc *desc)
> +{
> + /* Lower the priority */
> + gicv3_eoi_irq(desc);
> + /* Deactivate */
> + gicv3_dir_irq(desc);
> +}
I though that an LPI doesn't have active state. And therefore,
deactivate is not necessary. Did I miss something?
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (4 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 05/17] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 13:54 ` Ian Campbell
` (2 more replies)
2015-07-10 7:42 ` [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
` (10 subsequent siblings)
16 siblings, 3 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
This patch introduces virtual ITS driver with following
functionality
- Introduces helper functions to manage device table and
ITT table in guest memory
- Helper function to handle virtual ITS devices assigned
to domain
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Rename functions {find,remove,insert}_vits_* to
vits_{find,remove,insert}.
- Add common helper function to map and read/write dt
or vitt table entry.
- Removed unused code
---
xen/arch/arm/vgic-v3-its.c | 212 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/domain.h | 4 +
xen/include/asm-arm/gic-its.h | 45 ++++++++-
3 files changed, 260 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
new file mode 100644
index 0000000..c63f478
--- /dev/null
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -0,0 +1,212 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/bitops.h>
+#include <xen/config.h>
+#include <xen/init.h>
+#include <xen/irq.h>
+#include <xen/list.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+#include <asm/device.h>
+#include <asm/mmio.h>
+#include <asm/io.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/vgic.h>
+#include <asm/gic-its.h>
+#include <xen/log2.h>
+
+static int vits_entry(struct domain *d, paddr_t entry, void *addr,
+ uint32_t size, bool_t set)
+{
+ struct page_info *page;
+ uint64_t offset;
+ p2m_type_t p2mt;
+ void *p;
+
+ page = get_page_from_gfn(d, paddr_to_pfn(entry), &p2mt, P2M_ALLOC);
+ if ( !page )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to get table entry\n",
+ current);
+ return -EINVAL;
+ }
+
+ if ( !p2m_is_ram(p2mt) )
+ {
+ put_page(page);
+ dprintk(XENLOG_G_ERR, "%pv: vITS: with wrong attributes\n", current);
+ return -EINVAL;
+ }
+
+ p = __map_domain_page(page);
+ /* Offset within the mapped page */
+ offset = entry & ~PAGE_MASK;
+
+ if ( set )
+ memcpy(p + offset, addr, size);
+ else
+ memcpy(addr, p + offset, size);
+
+ unmap_domain_page(p);
+ put_page(page);
+
+ return 0;
+}
+
+/* ITS device table helper functions */
+static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
+ struct vdevice_table *entry, bool_t set)
+{
+ uint64_t offset;
+ paddr_t dt_entry;
+
+ BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
+
+ offset = dev_id * sizeof(struct vdevice_table);
+ if ( offset > d->arch.vits->dt_size )
+ {
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: Out of range offset %ld id 0x%x size %ld\n",
+ current, offset, dev_id, d->arch.vits->dt_size);
+ return -EINVAL;
+ }
+
+ dt_entry = d->arch.vits->dt_ipa + offset;
+
+ return vits_entry(d, dt_entry, (void *)entry,
+ sizeof(struct vdevice_table), set);
+}
+
+int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 1);
+}
+
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 0);
+}
+
+static int vits_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry, bool_t set)
+{
+ struct vdevice_table dt_entry;
+ paddr_t vitt_entry;
+ uint64_t offset;
+
+ BUILD_BUG_ON(sizeof(struct vitt) != 8);
+
+ if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Fail to get vdevice for dev 0x%x\n",
+ current, devid);
+ return -EINVAL;
+ }
+
+ /* dt_entry is validated when read */
+ offset = event * sizeof(struct vitt);
+ if ( offset > dt_entry.vitt_size )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: ITT out of range\n", current);
+ return -EINVAL;
+ }
+
+ vitt_entry = dt_entry.vitt_ipa + offset;
+
+ return vits_entry(d, vitt_entry, (void *)entry,
+ sizeof(struct vitt), set);
+}
+
+int vits_set_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 1);
+}
+
+int vits_get_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 0);
+}
+
+/* RB-tree helpers for vits_device attached to a domain */
+struct vits_device *vits_find_device(struct rb_root *root, uint32_t devid)
+{
+ struct rb_node *node = root->rb_node;
+
+ while ( node )
+ {
+ struct vits_device *dev;
+
+ dev = container_of(node, struct vits_device, node);
+
+ if ( devid < dev->vdevid )
+ node = node->rb_left;
+ else if ( devid > dev->vdevid )
+ node = node->rb_right;
+ else
+ return dev;
+ }
+
+ return NULL;
+}
+
+int vits_insert_device(struct rb_root *root, struct vits_device *dev)
+{
+ struct rb_node **new, *parent;
+
+ new = &root->rb_node;
+ parent = NULL;
+ while ( *new )
+ {
+ struct vits_device *this;
+
+ this = container_of(*new, struct vits_device, node);
+
+ parent = *new;
+ if ( dev->vdevid < this->vdevid )
+ new = &((*new)->rb_left);
+ else if ( dev->vdevid > this->vdevid )
+ new = &((*new)->rb_right);
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&dev->node, parent, new);
+ rb_insert_color(&dev->node, root);
+
+ return 0;
+}
+
+void vits_remove_device(struct rb_root *root, struct vits_device *dev)
+{
+ if ( dev )
+ rb_erase(&dev->node, root);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index f1a087e..67e4695 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -115,6 +115,10 @@ struct arch_domain
#endif
} vgic;
+#ifdef CONFIG_ARM_64
+ struct vgic_its *vits;
+#endif
+
struct vuart {
#define VUART_BUF_SIZE 128
char *buf;
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index e8d244f..d21aefe 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -31,7 +31,20 @@ struct its_collection {
u16 col_id;
};
-/* ITS command structure */
+/*
+ * Per domain virtual ITS structure.
+ */
+struct vgic_its
+{
+ /* vITT device table ipa */
+ paddr_t dt_ipa;
+ /* vITT device table size */
+ uint64_t dt_size;
+ /* Radix-tree root of devices attached to this domain */
+ struct rb_root dev_root;
+};
+
+/* ITS command structures */
typedef union {
u64 bits[4];
struct __packed {
@@ -171,11 +184,41 @@ struct its_device {
struct rb_node node;
};
+struct vits_device {
+ uint32_t vdevid;
+ struct its_device *its_dev;
+ struct rb_node node;
+};
+
+struct vdevice_table {
+ uint64_t vitt_ipa;
+ uint32_t vitt_size;
+ uint32_t padding;
+};
+
+struct vitt {
+ uint16_t valid:1;
+ uint16_t pad:15;
+ uint16_t vcollection;
+ uint32_t vlpi;
+};
+
int its_lpi_init(u32 id_bits);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
struct its_device *its_find_device(u32 devid);
int its_insert_device(struct its_device *dev);
+int vits_set_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry);
+int vits_get_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry);
+int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry);
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry);
+struct vits_device *vits_find_device(struct rb_root *root, uint32_t devid);
+int vits_insert_device(struct rb_root *root, struct vits_device *dev);
+void vits_remove_device(struct rb_root *root, struct vits_device *dev);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-10 7:42 ` [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver vijay.kilari
@ 2015-07-10 13:54 ` Ian Campbell
2015-07-11 14:48 ` Vijay Kilari
2015-07-10 14:15 ` Ian Campbell
2015-07-15 12:17 ` Julien Grall
2 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 13:54 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> +/* RB-tree helpers for vits_device attached to a domain */
In the rest of the series I found this used in three places:
* On assignment, to insert the device into the tree
* On deassignment, to remove it again
* In vgic_vcpu_inject_lpi, where the device is looked up and then
never used.
I don't see any other use and therefore I don't think this RB tree
serves any purpose, which is consistent with the design which doesn't
require this lookup anywhere. Please remove it.
If there is some use of it in some future series (e.g. perhaps the PCI
one) then please still remove it and add a patch to that series to
introduce it.
> +struct vits_device *vits_find_device(struct rb_root *root, uint32_t devid)
> +{
> + struct rb_node *node = root->rb_node;
> +
> + while ( node )
> + {
> + struct vits_device *dev;
> +
> + dev = container_of(node, struct vits_device, node);
> +
> + if ( devid < dev->vdevid )
> + node = node->rb_left;
> + else if ( devid > dev->vdevid )
> + node = node->rb_right;
> + else
> + return dev;
> + }
> +
> + return NULL;
> +}
> +
> +int vits_insert_device(struct rb_root *root, struct vits_device *dev)
> +{
> + struct rb_node **new, *parent;
> +
> + new = &root->rb_node;
> + parent = NULL;
> + while ( *new )
> + {
> + struct vits_device *this;
> +
> + this = container_of(*new, struct vits_device, node);
> +
> + parent = *new;
> + if ( dev->vdevid < this->vdevid )
> + new = &((*new)->rb_left);
> + else if ( dev->vdevid > this->vdevid )
> + new = &((*new)->rb_right);
> + else
> + return -EEXIST;
> + }
> +
> + rb_link_node(&dev->node, parent, new);
> + rb_insert_color(&dev->node, root);
> +
> + return 0;
> +}
> +
> +void vits_remove_device(struct rb_root *root, struct vits_device *dev)
> +{
> + if ( dev )
> + rb_erase(&dev->node, root);
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-10 13:54 ` Ian Campbell
@ 2015-07-11 14:48 ` Vijay Kilari
2015-07-13 9:27 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-11 14:48 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
Hi Ian,
On Fri, Jul 10, 2015 at 7:24 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
>> +/* RB-tree helpers for vits_device attached to a domain */
>
> In the rest of the series I found this used in three places:
> * On assignment, to insert the device into the tree
> * On deassignment, to remove it again
> * In vgic_vcpu_inject_lpi, where the device is looked up and then
> never used.
>
> I don't see any other use and therefore I don't think this RB tree
> serves any purpose, which is consistent with the design which doesn't
> require this lookup anywhere. Please remove it.
>
> If there is some use of it in some future series (e.g. perhaps the PCI
> one) then please still remove it and add a patch to that series to
> introduce it.
>
You mean for now we will remove RB-tree for managing devices assigned
to domain and introduce RB-tree and do look up when pci-passthrough is
introduced?.
>
>
>> +struct vits_device *vits_find_device(struct rb_root *root, uint32_t devid)
>> +{
>> + struct rb_node *node = root->rb_node;
>> +
>> + while ( node )
>> + {
>> + struct vits_device *dev;
>> +
>> + dev = container_of(node, struct vits_device, node);
>> +
>> + if ( devid < dev->vdevid )
>> + node = node->rb_left;
>> + else if ( devid > dev->vdevid )
>> + node = node->rb_right;
>> + else
>> + return dev;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +int vits_insert_device(struct rb_root *root, struct vits_device *dev)
>> +{
>> + struct rb_node **new, *parent;
>> +
>> + new = &root->rb_node;
>> + parent = NULL;
>> + while ( *new )
>> + {
>> + struct vits_device *this;
>> +
>> + this = container_of(*new, struct vits_device, node);
>> +
>> + parent = *new;
>> + if ( dev->vdevid < this->vdevid )
>> + new = &((*new)->rb_left);
>> + else if ( dev->vdevid > this->vdevid )
>> + new = &((*new)->rb_right);
>> + else
>> + return -EEXIST;
>> + }
>> +
>> + rb_link_node(&dev->node, parent, new);
>> + rb_insert_color(&dev->node, root);
>> +
>> + return 0;
>> +}
>> +
>> +void vits_remove_device(struct rb_root *root, struct vits_device *dev)
>> +{
>> + if ( dev )
>> + rb_erase(&dev->node, root);
>> +}
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-11 14:48 ` Vijay Kilari
@ 2015-07-13 9:27 ` Ian Campbell
0 siblings, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-13 9:27 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Sat, 2015-07-11 at 20:18 +0530, Vijay Kilari wrote:
> Hi Ian,
>
> On Fri, Jul 10, 2015 at 7:24 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> > On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> >> +/* RB-tree helpers for vits_device attached to a domain */
> >
> > In the rest of the series I found this used in three places:
> > * On assignment, to insert the device into the tree
> > * On deassignment, to remove it again
> > * In vgic_vcpu_inject_lpi, where the device is looked up and then
> > never used.
> >
> > I don't see any other use and therefore I don't think this RB tree
> > serves any purpose, which is consistent with the design which doesn't
> > require this lookup anywhere. Please remove it.
> >
> > If there is some use of it in some future series (e.g. perhaps the PCI
> > one) then please still remove it and add a patch to that series to
> > introduce it.
> >
>
> You mean for now we will remove RB-tree for managing devices assigned
> to domain
Yes, it isn't needed for ITS at all AFAICT and having it around has just
tempted you into using it incorrectly during vpli injection.
> and introduce RB-tree and do look up when pci-passthrough is
> introduced?.
If it is needed then yes.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-10 7:42 ` [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver vijay.kilari
2015-07-10 13:54 ` Ian Campbell
@ 2015-07-10 14:15 ` Ian Campbell
2015-07-11 14:48 ` Vijay Kilari
2015-07-15 12:17 ` Julien Grall
2 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 14:15 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> +static int vits_entry(struct domain *d, paddr_t entry, void *addr,
> + uint32_t size, bool_t set)
> +{
> [...]
> +}
> +
> +/* ITS device table helper functions */
> +static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
> + struct vdevice_table *entry, bool_t set)
> +{
> + uint64_t offset;
> + paddr_t dt_entry;
> +
> + BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
> +
> + offset = dev_id * sizeof(struct vdevice_table);
> + if ( offset > d->arch.vits->dt_size )
> + {
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: Out of range offset %ld id 0x%x size %ld\n",
> + current, offset, dev_id, d->arch.vits->dt_size);
> + return -EINVAL;
> + }
> +
> + dt_entry = d->arch.vits->dt_ipa + offset;
> +
> + return vits_entry(d, dt_entry, (void *)entry,
> + sizeof(struct vdevice_table),
Please drop the (void *) cast here, you can pass a "foo *" to a "void *"
without one.
It took me a little while to work out why this was void * before I
realised that vits_entry was a generic helper used for different types
of table. "vits_access_guest_table" to make it clear what it is doing.
> +static int vits_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry, bool_t set)
> +{
> + struct vdevice_table dt_entry;
> + paddr_t vitt_entry;
> + uint64_t offset;
> +
> + BUILD_BUG_ON(sizeof(struct vitt) != 8);
> +
> + if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Fail to get vdevice for dev 0x%x\n",
> + current, devid);
> + return -EINVAL;
> + }
> +
> + /* dt_entry is validated when read */
Please can you say "is validated by vits_get_vdevice_entry" to be very
clear.
> + offset = event * sizeof(struct vitt);
> + if ( offset > dt_entry.vitt_size )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: ITT out of range\n", current);
> + return -EINVAL;
> + }
> +
> + vitt_entry = dt_entry.vitt_ipa + offset;
> +
> + return vits_entry(d, vitt_entry, (void *)entry,
> + sizeof(struct vitt), set);
Same story WRT the cast.
[...]
> @@ -171,11 +184,41 @@ struct its_device {
> struct rb_node node;
> };
>
> +struct vits_device {
> + uint32_t vdevid;
> + struct its_device *its_dev;
> + struct rb_node node;
> +};
> +
Please add a big comment here:
/*
* struct vdevice_table and struct vitt are typically stored in memory
* which has been provided by the guest out of its own address space
* and which remains accessible to the guest.
*
* Therefore great care _must_ be taken when accessing an entry in
* either table to validate the sanity of any values which are used
*/
> +struct vdevice_table {
> + uint64_t vitt_ipa;
> + uint32_t vitt_size;
> + uint32_t padding;
> +};
> +
> +struct vitt {
> + uint16_t valid:1;
> + uint16_t pad:15;
> + uint16_t vcollection;
> + uint32_t vlpi;
> +};
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-10 14:15 ` Ian Campbell
@ 2015-07-11 14:48 ` Vijay Kilari
2015-07-13 9:25 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-11 14:48 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Fri, Jul 10, 2015 at 7:45 PM, Ian Campbell <ijc@xen.org> wrote:
> On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
>> +static int vits_entry(struct domain *d, paddr_t entry, void *addr,
>> + uint32_t size, bool_t set)
>> +{
>> [...]
>> +}
>> +
>> +/* ITS device table helper functions */
>> +static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
>> + struct vdevice_table *entry, bool_t set)
>> +{
>> + uint64_t offset;
>> + paddr_t dt_entry;
>> +
>> + BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
>> +
>> + offset = dev_id * sizeof(struct vdevice_table);
>> + if ( offset > d->arch.vits->dt_size )
>> + {
>> + dprintk(XENLOG_G_ERR,
>> + "%pv: vITS: Out of range offset %ld id 0x%x size %ld\n",
>> + current, offset, dev_id, d->arch.vits->dt_size);
>> + return -EINVAL;
>> + }
>> +
>> + dt_entry = d->arch.vits->dt_ipa + offset;
>> +
>> + return vits_entry(d, dt_entry, (void *)entry,
>> + sizeof(struct vdevice_table),
>
> Please drop the (void *) cast here, you can pass a "foo *" to a "void *"
> without one.
>
> It took me a little while to work out why this was void * before I
> realised that vits_entry was a generic helper used for different types
> of table. "vits_access_guest_table" to make it clear what it is doing.
This is also used in later patches read virtual ITS command and also
property pending table. I prefer to move it to some generic file like
guestcopy.c/p2m.c?
and should be named as copy_{from|to}guest_gfn()?
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-11 14:48 ` Vijay Kilari
@ 2015-07-13 9:25 ` Ian Campbell
0 siblings, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-13 9:25 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Sat, 2015-07-11 at 20:18 +0530, Vijay Kilari wrote:
> On Fri, Jul 10, 2015 at 7:45 PM, Ian Campbell <ijc@xen.org> wrote:
> > On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> >> +static int vits_entry(struct domain *d, paddr_t entry, void *addr,
> >> + uint32_t size, bool_t set)
> >> +{
> >> [...]
> >> +}
> >> +
> >> +/* ITS device table helper functions */
> >> +static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
> >> + struct vdevice_table *entry, bool_t set)
> >> +{
> >> + uint64_t offset;
> >> + paddr_t dt_entry;
> >> +
> >> + BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
> >> +
> >> + offset = dev_id * sizeof(struct vdevice_table);
> >> + if ( offset > d->arch.vits->dt_size )
> >> + {
> >> + dprintk(XENLOG_G_ERR,
> >> + "%pv: vITS: Out of range offset %ld id 0x%x size %ld\n",
> >> + current, offset, dev_id, d->arch.vits->dt_size);
> >> + return -EINVAL;
> >> + }
> >> +
> >> + dt_entry = d->arch.vits->dt_ipa + offset;
> >> +
> >> + return vits_entry(d, dt_entry, (void *)entry,
> >> + sizeof(struct vdevice_table),
> >
> > Please drop the (void *) cast here, you can pass a "foo *" to a "void *"
> > without one.
> >
> > It took me a little while to work out why this was void * before I
> > realised that vits_entry was a generic helper used for different types
> > of table. "vits_access_guest_table" to make it clear what it is doing.
>
> This is also used in later patches read virtual ITS command and also
> property pending table. I prefer to move it to some generic file like
> guestcopy.c/p2m.c?
> and should be named as copy_{from|to}guest_gfn()?
I nearly suggested using the existing copy to/from guest functions but:
Why do the existing copy to/from guest helpers not check the page has
memory type. If it did they would be closer to being directly usable.
Those functions check for guest read/write access as appropriate, but
those do not apply to this case (which is in effect a privileged DMA
from outside the virtual CPU).
In particular due to the second thing I think we would be best off
keeping this as a specific helper for the VITS, having general helper
functions with lax security checks in them just invites people to use
them inappropriately.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver
2015-07-10 7:42 ` [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver vijay.kilari
2015-07-10 13:54 ` Ian Campbell
2015-07-10 14:15 ` Ian Campbell
@ 2015-07-15 12:17 ` Julien Grall
2 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 12:17 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> +static int vits_entry(struct domain *d, paddr_t entry, void *addr,
> + uint32_t size, bool_t set)
> +{
> + struct page_info *page;
> + uint64_t offset;
> + p2m_type_t p2mt;
> + void *p;
> +
> + page = get_page_from_gfn(d, paddr_to_pfn(entry), &p2mt, P2M_ALLOC);
> + if ( !page )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to get table entry\n",
> + current);
The function vits_entry will be used when an LPI is injected to the
guest. At that time, current may not be a VCPU of the domain d.
Therefore, this log will be very confusing if an error occur. I would
prefer if you only print the domain id using d.
Furthermore, dprintk is a nop on non-debug build. You may want to use
printk here.
Those comments are valid in every dprintk of this patch.
[..]
> +/* ITS device table helper functions */
> +static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
> + struct vdevice_table *entry, bool_t set)
> +{
> + uint64_t offset;
> + paddr_t dt_entry;
> +
> + BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
> +
> + offset = dev_id * sizeof(struct vdevice_table);
> + if ( offset > d->arch.vits->dt_size )
> + {
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: Out of range offset %ld id 0x%x size %ld\n",
> + current, offset, dev_id, d->arch.vits->dt_size);
> + return -EINVAL;
> + }
> +
> + dt_entry = d->arch.vits->dt_ipa + offset;
> +
> + return vits_entry(d, dt_entry, (void *)entry,
> + sizeof(struct vdevice_table), set);
> +}
> +
> +int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
> + struct vdevice_table *entry)
> +{
> + return vits_vdevice_entry(d, devid, entry, 1);
> +}
> +
> +int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
> + struct vdevice_table *entry)
> +{
> + return vits_vdevice_entry(d, devid, entry, 0);
> +}
While I can understand that you export vits_get_vdevice_entry, it's used
in the LPI injection. I don't understand for vits_set_vdevice_entry.
> +
> +static int vits_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry, bool_t set)
> +{
> + struct vdevice_table dt_entry;
> + paddr_t vitt_entry;
> + uint64_t offset;
> +
> + BUILD_BUG_ON(sizeof(struct vitt) != 8);
> +
> + if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Fail to get vdevice for dev 0x%x\n",
s/dev/vdevid/
> + current, devid);
> + return -EINVAL;
> + }
> +
> + /* dt_entry is validated when read */
> + offset = event * sizeof(struct vitt);
> + if ( offset > dt_entry.vitt_size )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: ITT out of range\n", current);
> + return -EINVAL;
> + }
> +
> + vitt_entry = dt_entry.vitt_ipa + offset;
> +
> + return vits_entry(d, vitt_entry, (void *)entry,
> + sizeof(struct vitt), set);
> +}
> +
> +int vits_set_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry)
> +{
> + return vits_vitt_entry(d, devid, event, entry, 1);
> +}
> +
> +int vits_get_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry)
> +{
> + return vits_vitt_entry(d, devid, event, entry, 0);
> +}
Same remark as vits_*_device_entry.
[..]
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index f1a087e..67e4695 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -115,6 +115,10 @@ struct arch_domain
> #endif
> } vgic;
>
> +#ifdef CONFIG_ARM_64
> + struct vgic_its *vits;
> +#endif
> +
I would prefer to see this field part of the vgic structure (see just
above). There is already a #ifdef for GICv3 stuff so it will avoid 2 new
lines.
> struct vuart {
> #define VUART_BUF_SIZE 128
> char *buf;
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index e8d244f..d21aefe 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -31,7 +31,20 @@ struct its_collection {
> u16 col_id;
> };
>
> -/* ITS command structure */
> +/*
> + * Per domain virtual ITS structure.
> + */
> +struct vgic_its
> +{
> + /* vITT device table ipa */
> + paddr_t dt_ipa;
> + /* vITT device table size */
> + uint64_t dt_size;
> + /* Radix-tree root of devices attached to this domain */
> + struct rb_root dev_root;
> +};
> +
> +/* ITS command structures */
Please fix the typo in the patch where you introduced it. Not after.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (5 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 06/17] xen/arm: ITS: Add virtual ITS driver vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 14:35 ` Ian Campbell
2015-07-15 12:57 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
` (9 subsequent siblings)
16 siblings, 2 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add Virtual ITS command processing support to Virtual ITS driver
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Use helper function to read from command queue
- Add MOVALL
- Removed check for entry in device in domain RB-tree
---
xen/arch/arm/gic-v3-its.c | 7 +
xen/arch/arm/vgic-v3-its.c | 391 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 19 ++
xen/include/asm-arm/gic.h | 1 +
4 files changed, 418 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index b98d396..9161053 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -91,6 +91,7 @@ static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
static struct rb_root rb_its_dev;
+static struct gic_its_info its_data;
#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
@@ -102,6 +103,11 @@ void dump_cmd(its_cmd_block *cmd)
}
#endif
+u32 its_get_nr_events(void)
+{
+ return (1 << its_data.id_bits);
+}
+
/* RB-tree helpers for its_device */
struct its_device *its_find_device(u32 devid)
{
@@ -940,6 +946,7 @@ static int its_probe(struct dt_device_node *node)
its->phys_size = its_size;
typer = readl_relaxed(its_base + GITS_TYPER);
its->ite_size = ((typer >> 4) & 0xf) + 1;
+ its_data.id_bits = GITS_TYPER_IDBITS(typer);
its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
if ( !its->cmd_base )
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index c63f478..af2bacd 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -31,6 +31,22 @@
#include <asm/gic-its.h>
#include <xen/log2.h>
+#define DEBUG_ITS
+
+#ifdef DEBUG_ITS
+# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#ifdef DEBUG_ITS
+static void dump_cmd(its_cmd_block *cmd)
+{
+ printk("CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
+}
+#endif
+
static int vits_entry(struct domain *d, paddr_t entry, void *addr,
uint32_t size, bool_t set)
{
@@ -202,6 +218,381 @@ void vits_remove_device(struct rb_root *root, struct vits_device *dev)
rb_erase(&dev->node, root);
}
+static int vgic_its_process_sync(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ DPRINTK("%pv: vITS: SYNC: ta 0x%x \n", v, virt_cmd->sync.ta);
+
+ return 0;
+}
+
+static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint8_t vcol_id, cmd;
+ uint32_t vid, dev_id, event;
+
+ vcol_id = virt_cmd->mapvi.col;
+ vid = virt_cmd->mapvi.phy_id;
+ cmd = virt_cmd->mapvi.cmd;
+ dev_id = virt_cmd->mapvi.devid;
+
+ DPRINTK("%pv: vITS: MAPVI: dev_id 0x%x vcol_id %d vid %d \n",
+ v, dev_id, vcol_id, vid);
+
+ if ( vcol_id > (d->max_vcpus + 1) || vid > its_get_nr_events() )
+ return -EINVAL;
+
+ entry.valid = true;
+ entry.vcollection = vcol_id;
+ entry.vlpi = vid;
+
+ if ( cmd == GITS_CMD_MAPI )
+ vits_set_vitt_entry(d, dev_id, vid, &entry);
+ else
+ {
+ event = virt_cmd->mapvi.event;
+ vits_set_vitt_entry(d, dev_id, event, &entry);
+ }
+
+ return 0;
+}
+
+static int vgic_its_process_movi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint32_t dev_id, event;
+ uint8_t vcol_id;
+
+ vcol_id = virt_cmd->movi.col;
+ event = virt_cmd->movi.event;
+ dev_id = virt_cmd->movi.devid;
+
+ DPRINTK("%pv vITS: MOVI: dev_id 0x%x vcol_id %d event %d\n",
+ v, dev_id, vcol_id, event);
+
+ if ( vcol_id > (d->max_vcpus + 1) || event > its_get_nr_events() )
+ return -EINVAL;
+
+ if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ entry.vcollection = vcol_id;
+
+ if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vgic_its_process_movall(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ return 0;
+}
+
+static int vgic_its_process_discard(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint32_t event, dev_id;
+
+ event = virt_cmd->discard.event;
+ dev_id = virt_cmd->discard.devid;
+
+ DPRINTK("%pv vITS: DISCARD: dev_id 0x%x id %d\n",
+ v, virt_cmd->discard.devid, event);
+
+ if ( event > its_get_nr_events() )
+ return -EINVAL;
+
+ if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ entry.valid = false;
+
+ if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vgic_its_process_inv(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ DPRINTK("%pv vITS: INV: dev_id 0x%x id %d\n",
+ v, virt_cmd->inv.devid, virt_cmd->inv.event);
+
+ return 0;
+}
+
+static int vgic_its_process_clear(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ DPRINTK("%pv: vITS: CLEAR: dev_id 0x%x id %d\n",
+ v, virt_cmd->clear.devid, virt_cmd->clear.event);
+
+ return 0;
+}
+
+static int vgic_its_process_invall(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ DPRINTK("%pv: vITS: INVALL: vCID %d\n", v, virt_cmd->invall.col);
+
+ return 0;
+}
+
+static int vgic_its_process_int(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt vitt_entry;
+ struct domain *d = v->domain;
+ uint32_t event, dev_id, col_id;
+
+ event = virt_cmd->int_cmd.cmd;
+ dev_id = virt_cmd->int_cmd.devid;
+
+ DPRINTK("%pv: vITS: INT: Device 0x%x id %d\n", v, dev_id, event);
+ if ( event > its_get_nr_events() )
+ return -EINVAL;
+
+ if ( vits_get_vitt_entry(d, dev_id, event, &vitt_entry) )
+ return -EINVAL;
+
+ if ( !vitt_entry.valid )
+ {
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: INT CMD invalid event %d for dev 0x%x\n",
+ v, event, dev_id);
+ return -EINVAL;
+ }
+
+ col_id = vitt_entry.vcollection;
+ if ( col_id < d->max_vcpus )
+ {
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: INT CMD invalid col_id %d for dev 0x%x\n",
+ v, col_id, dev_id);
+ return -EINVAL;
+ }
+
+ vgic_vcpu_inject_irq(d->vcpu[col_id], vitt_entry.vlpi);
+
+ return 0;
+}
+
+static int vgic_its_add_device(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct domain *d = v->domain;
+ struct vdevice_table dt_entry;
+ uint32_t dev_id = virt_cmd->mapd.devid;
+
+ DPRINTK("%pv: vITS: Add device dev_id 0x%x vitt_ipa = 0x%lx size %d\n",
+ v, dev_id, (u64)virt_cmd->mapd.itt << 8,
+ virt_cmd->mapd.size);
+
+ if ( virt_cmd->mapd.valid )
+ {
+ /* itt field is 40 bit. extract 48 bit address by shifting */
+ dt_entry.vitt_ipa = virt_cmd->mapd.itt << 8;
+ dt_entry.vitt_size = (1 << (virt_cmd->mapd.size + 1)) *
+ sizeof(struct vitt);
+ }
+ else
+ {
+ dt_entry.vitt_ipa = INVALID_PADDR;
+ dt_entry.vitt_size = 0;
+ }
+
+ if ( vits_set_vdevice_entry(d, dev_id, &dt_entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vgic_its_process_mapc(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct domain *d = v->domain;
+ uint8_t vcol_id;
+ uint64_t vta = 0;
+
+ vcol_id = virt_cmd->mapc.col;
+ vta = virt_cmd->mapc.ta;
+
+ DPRINTK("%pv: vITS: MAPC: vCID %d vTA 0x%lx valid %d \n",
+ v, vcol_id, vta, virt_cmd->mapc.valid);
+
+ if ( vcol_id > (d->max_vcpus + 1) || vta > v->domain->max_vcpus )
+ return -EINVAL;
+
+ if ( virt_cmd->mapc.valid )
+ d->arch.vits->collections[vcol_id].target_address = vta;
+ else
+ d->arch.vits->collections[vcol_id].target_address = ~0UL;
+
+ return 0;
+}
+
+static void vgic_its_update_read_ptr(struct vcpu *v, struct vgic_its *vits)
+{
+ vits->cmd_read = vits->cmd_write_save;
+}
+
+#ifdef DEBUG_ITS
+char *cmd_str[] = {
+ [GITS_CMD_MOVI] = "MOVI",
+ [GITS_CMD_INT] = "INT",
+ [GITS_CMD_CLEAR] = "CLEAR",
+ [GITS_CMD_SYNC] = "SYNC",
+ [GITS_CMD_MAPD] = "MAPD",
+ [GITS_CMD_MAPC] = "MAPC",
+ [GITS_CMD_MAPVI] = "MAPVI",
+ [GITS_CMD_MAPI] = "MAPI",
+ [GITS_CMD_INV] = "INV",
+ [GITS_CMD_INVALL] = "INVALL",
+ [GITS_CMD_MOVALL] = "MOVALL",
+ [GITS_CMD_DISCARD] = "DISCARD",
+ };
+#endif
+
+static int vgic_its_parse_its_command(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint8_t cmd = virt_cmd->hdr.cmd;
+ int ret;
+
+#ifdef DEBUG_ITS
+ DPRINTK("%pv: vITS: Received cmd %s (0x%x)\n", v, cmd_str[cmd], cmd);
+ DPRINTK("Dump Virt cmd: ");
+ dump_cmd(virt_cmd);
+#endif
+
+ switch ( cmd )
+ {
+ case GITS_CMD_MAPD:
+ ret = vgic_its_add_device(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPC:
+ ret = vgic_its_process_mapc(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPI:
+ /* MAPI is same as MAPVI */
+ case GITS_CMD_MAPVI:
+ ret = vgic_its_process_mapvi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MOVI:
+ ret = vgic_its_process_movi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MOVALL:
+ ret = vgic_its_process_movall(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_DISCARD:
+ ret = vgic_its_process_discard(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INV:
+ ret = vgic_its_process_inv(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INVALL:
+ ret = vgic_its_process_invall(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INT:
+ ret = vgic_its_process_int(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_CLEAR:
+ ret = vgic_its_process_clear(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_SYNC:
+ ret = vgic_its_process_sync(v, vits, virt_cmd);
+ break;
+ default:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Unhandled command cmd %d\n", v, cmd);
+ return 1;
+ }
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to handle cmd %d\n", v, cmd);
+ return 1;
+ }
+
+ return 0;
+}
+
+static int vgic_its_read_virt_cmd(struct vcpu *v,
+ struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ paddr_t maddr;
+ struct domain *d = v->domain;
+
+ ASSERT(spin_is_locked(&vits->lock));
+
+ if ( !(vits->cmd_base & GITS_CBASER_VALID) )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Invalid CBASER\n", v);
+ return 0;
+ }
+
+ /* CMD Q can be more than 1 page. Map only page that is required */
+ maddr = (vits->cmd_base & BIT_48_12_MASK) + vits->cmd_write_save;
+
+ DPRINTK("%pv: vITS: Mapping CMD Q maddr 0x%lx write_save 0x%lx\n",
+ v, maddr, vits->cmd_write_save);
+
+ if ( vits_entry(d, maddr, (void *)virt_cmd, sizeof(its_cmd_block), 0) )
+ {
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: Failed to get command page @page 0x%lx\n",
+ v, vits->cmd_write_save);
+ return -EINVAL;
+ }
+
+ /* No command queue is created by vits to check on Q full */
+ vits->cmd_write_save += sizeof(its_cmd_block);
+ if ( vits->cmd_write_save == vits->cmd_qsize )
+ {
+ DPRINTK("%pv: vITS: Reset write_save 0x%lx qsize 0x%lx \n",
+ v, vits->cmd_write_save, vits->cmd_qsize);
+
+ vits->cmd_write_save = 0x0;
+ }
+
+ return 0;
+}
+
+int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
+{
+ its_cmd_block virt_cmd;
+
+ ASSERT(spin_is_locked(&vits->lock));
+
+ do {
+ if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
+ goto err;
+ if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
+ goto err;
+ vgic_its_update_read_ptr(v, vits);
+ } while ( vits->cmd_write != vits->cmd_write_save );
+
+ DPRINTK("%pv: vITS: write_save 0x%lx write 0x%lx\n",
+ v, vits->cmd_write_save,
+ vits->cmd_write);
+
+ return 1;
+err:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to process guest cmd\n", v);
+ /*XXX: Be nice to guest though we cannot process command? */
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index d21aefe..a143003 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -36,12 +36,25 @@ struct its_collection {
*/
struct vgic_its
{
+ spinlock_t lock;
+ /* Command queue base */
+ paddr_t cmd_base;
+ /* Command queue write pointer */
+ paddr_t cmd_write;
+ /* Command queue write saved pointer */
+ paddr_t cmd_write_save;
+ /* Command queue read pointer */
+ paddr_t cmd_read;
+ /* Command queue size */
+ unsigned long cmd_qsize;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
uint64_t dt_size;
/* Radix-tree root of devices attached to this domain */
struct rb_root dev_root;
+ /* collections mapped */
+ struct its_collection *collections;
};
/* ITS command structures */
@@ -203,6 +216,12 @@ struct vitt {
uint32_t vlpi;
};
+struct gic_its_info {
+ uint32_t id_bits;
+ uint32_t dev_bits;
+};
+
+u32 its_get_nr_events(void);
int its_lpi_init(u32 id_bits);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 44c2317..fdd96c8 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -24,6 +24,7 @@
#define NR_GIC_LPI 4096
#define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
#define MAX_RDIST_COUNT 4
+#define BIT_48_12_MASK 0xfffffffff000UL
#define GICD_CTLR (0x000)
#define GICD_TYPER (0x004)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-10 7:42 ` [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-07-10 14:35 ` Ian Campbell
2015-07-11 14:49 ` Vijay Kilari
2015-07-15 13:02 ` Julien Grall
2015-07-15 12:57 ` Julien Grall
1 sibling, 2 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 14:35 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
[...]
> +static int vgic_its_process_int(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> [...]
> +
> + col_id = vitt_entry.vcollection;
> + if ( col_id < d->max_vcpus )
I think the condition here is backwards? And might be missing a + 1?
I think you've ended up open coding this max_vcpus+1 a lot. I think you
should encapsulate it into an valid_vcollection(d, col_id) helper and
use it throughout.
> + {
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: INT CMD invalid col_id %d for dev 0x%x\n",
> + v, col_id, dev_id);
> + return -EINVAL;
> + }
> +
> + vgic_vcpu_inject_irq(d->vcpu[col_id], vitt_entry.vlpi);
> +
> + return 0;
> +}
> +
> +static int vgic_its_add_device(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct domain *d = v->domain;
> + struct vdevice_table dt_entry;
> + uint32_t dev_id = virt_cmd->mapd.devid;
> +
> + DPRINTK("%pv: vITS: Add device dev_id 0x%x vitt_ipa = 0x%lx size %d\n",
> + v, dev_id, (u64)virt_cmd->mapd.itt << 8,
Where you have uintXX_t types being printed please always use %"PRIxXX"
or %"PRIdXX" etc and not just %x or %lx etc. This is good practice even
in code which is only compiled for 64-bit.
Please also avoid uXX types in favour of uintXX_t (i.e. use the latter)
in code that hasn't come from elsewhere.
Both of these comments likely apply to all sort of bits of this series.
> +static int vgic_its_process_mapc(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct domain *d = v->domain;
> + uint8_t vcol_id;
> + uint64_t vta = 0;
> +
> + vcol_id = virt_cmd->mapc.col;
> + vta = virt_cmd->mapc.ta;
> +
> + DPRINTK("%pv: vITS: MAPC: vCID %d vTA 0x%lx valid %d \n",
> + v, vcol_id, vta, virt_cmd->mapc.valid);
> +
> + if ( vcol_id > (d->max_vcpus + 1) || vta > v->domain->max_vcpus )
> + return -EINVAL;
> +
> + if ( virt_cmd->mapc.valid )
> + d->arch.vits->collections[vcol_id].target_address = vta;
> + else
> + d->arch.vits->collections[vcol_id].target_address = ~0UL;
You should use INVALID_PADDR here.
> +int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
> +{
> + its_cmd_block virt_cmd;
> +
> + ASSERT(spin_is_locked(&vits->lock));
> +
> + do {
> + if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
> + goto err;
> + if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
> + goto err;
> + vgic_its_update_read_ptr(v, vits);
> + } while ( vits->cmd_write != vits->cmd_write_save );
I can't find anywhere other than here where vits->cmd_write is touched.
What am I missing?
> + DPRINTK("%pv: vITS: write_save 0x%lx write 0x%lx\n",
> + v, vits->cmd_write_save,
> + vits->cmd_write);
> +
> + return 1;
> +err:
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to process guest cmd\n", v);
> + /*XXX: Be nice to guest though we cannot process command? */
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 44c2317..fdd96c8 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -24,6 +24,7 @@
> #define NR_GIC_LPI 4096
> #define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
> #define MAX_RDIST_COUNT 4
> +#define BIT_48_12_MASK 0xfffffffff000UL
I think you should use ~PAGE_MASK instead of defining this.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-10 14:35 ` Ian Campbell
@ 2015-07-11 14:49 ` Vijay Kilari
2015-07-13 9:22 ` Ian Campbell
2015-07-15 13:02 ` Julien Grall
1 sibling, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-11 14:49 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
>> +int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
>> +{
>> + its_cmd_block virt_cmd;
>> +
>> + ASSERT(spin_is_locked(&vits->lock));
>> +
>> + do {
>> + if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
>> + goto err;
>> + if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
>> + goto err;
>> + vgic_its_update_read_ptr(v, vits);
>> + } while ( vits->cmd_write != vits->cmd_write_save );
>
> I can't find anywhere other than here where vits->cmd_write is touched.
> What am I missing?
It is written by guest by GITS_CWRITER emulation in patch #9
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-11 14:49 ` Vijay Kilari
@ 2015-07-13 9:22 ` Ian Campbell
2015-07-13 11:15 ` Vijay Kilari
0 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-13 9:22 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Sat, 2015-07-11 at 20:19 +0530, Vijay Kilari wrote:
> >> +int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
> >> +{
> >> + its_cmd_block virt_cmd;
> >> +
> >> + ASSERT(spin_is_locked(&vits->lock));
> >> +
> >> + do {
> >> + if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
> >> + goto err;
> >> + if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
> >> + goto err;
> >> + vgic_its_update_read_ptr(v, vits);
> >> + } while ( vits->cmd_write != vits->cmd_write_save );
> >
> > I can't find anywhere other than here where vits->cmd_write is touched.
> > What am I missing?
>
> It is written by guest by GITS_CWRITER emulation in patch #9
Ah, then please reverse the order so that the variable comes first and
the target comes second.
Also I think you need to find a better name that "cmd_write_save".
Something which indicates the progress made perhaps? But why isn't this
just cmd_read? Why the separate progress pointer?
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-13 9:22 ` Ian Campbell
@ 2015-07-13 11:15 ` Vijay Kilari
2015-07-13 11:37 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-13 11:15 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Jul 13, 2015 at 2:52 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Sat, 2015-07-11 at 20:19 +0530, Vijay Kilari wrote:
>> >> +int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
>> >> +{
>> >> + its_cmd_block virt_cmd;
>> >> +
>> >> + ASSERT(spin_is_locked(&vits->lock));
>> >> +
>> >> + do {
>> >> + if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
>> >> + goto err;
>> >> + if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
>> >> + goto err;
>> >> + vgic_its_update_read_ptr(v, vits);
>> >> + } while ( vits->cmd_write != vits->cmd_write_save );
>> >
>> > I can't find anywhere other than here where vits->cmd_write is touched.
>> > What am I missing?
>>
>> It is written by guest by GITS_CWRITER emulation in patch #9
>
> Ah, then please reverse the order so that the variable comes first and
> the target comes second.
>
> Also I think you need to find a better name that "cmd_write_save".
> Something which indicates the progress made perhaps? But why isn't this
> just cmd_read? Why the separate progress pointer?
I will check If I can use cmd_read.
BTW, I want to know if atomic_t supports 64-bit access?.
I have not made cmd_read as atomic_t.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-13 11:15 ` Vijay Kilari
@ 2015-07-13 11:37 ` Ian Campbell
2015-07-17 15:01 ` Vijay Kilari
0 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-13 11:37 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, 2015-07-13 at 16:45 +0530, Vijay Kilari wrote:
> BTW, I want to know if atomic_t supports 64-bit access?.
I don't know off the top of my head. I'm sure it would be apparent in
the code.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-13 11:37 ` Ian Campbell
@ 2015-07-17 15:01 ` Vijay Kilari
0 siblings, 0 replies; 113+ messages in thread
From: Vijay Kilari @ 2015-07-17 15:01 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
Hi Ian,
On Mon, Jul 13, 2015 at 5:07 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Mon, 2015-07-13 at 16:45 +0530, Vijay Kilari wrote:
>
>> BTW, I want to know if atomic_t supports 64-bit access?.
>
> I don't know off the top of my head. I'm sure it would be apparent in
> the code.
atomic_t is 32 bit. But CREADER is 64-bit register.
But only bit[19:0] are valid.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-10 14:35 ` Ian Campbell
2015-07-11 14:49 ` Vijay Kilari
@ 2015-07-15 13:02 ` Julien Grall
2015-07-15 13:56 ` Ian Campbell
1 sibling, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 13:02 UTC (permalink / raw)
To: Ian Campbell, vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
stefano.stabellini, manish.jaggi
Hi Ian,
On 10/07/2015 16:35, Ian Campbell wrote:
>> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
>> index 44c2317..fdd96c8 100644
>> --- a/xen/include/asm-arm/gic.h
>> +++ b/xen/include/asm-arm/gic.h
>> @@ -24,6 +24,7 @@
>> #define NR_GIC_LPI 4096
>> #define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
>> #define MAX_RDIST_COUNT 4
>> +#define BIT_48_12_MASK 0xfffffffff000UL
>
> I think you should use ~PAGE_MASK instead of defining this.
This used to get the physical address in the GITS_CBASER register. This
field is not based on the Xen page granularity neither 4KB granularity.
It's just a field containing the bits [47:12] of the address.
Therefore it would be very strange to use PAGE_MASK and an hypothetical
4K_MASK. Though I would prefer the latter.
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-15 13:02 ` Julien Grall
@ 2015-07-15 13:56 ` Ian Campbell
0 siblings, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-15 13:56 UTC (permalink / raw)
To: Julien Grall
Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
tim, xen-devel, stefano.stabellini, manish.jaggi
On Wed, 2015-07-15 at 15:02 +0200, Julien Grall wrote:
> Hi Ian,
>
> On 10/07/2015 16:35, Ian Campbell wrote:
> >> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> >> index 44c2317..fdd96c8 100644
> >> --- a/xen/include/asm-arm/gic.h
> >> +++ b/xen/include/asm-arm/gic.h
> >> @@ -24,6 +24,7 @@
> >> #define NR_GIC_LPI 4096
> >> #define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
> >> #define MAX_RDIST_COUNT 4
> >> +#define BIT_48_12_MASK 0xfffffffff000UL
> >
> > I think you should use ~PAGE_MASK instead of defining this.
>
> This used to get the physical address in the GITS_CBASER register. This
> field is not based on the Xen page granularity neither 4KB granularity.
> It's just a field containing the bits [47:12] of the address.
Then it should be GITS_CBASER_PA_MASK or something along those lines.
Ian.
>
> Therefore it would be very strange to use PAGE_MASK and an hypothetical
> 4K_MASK. Though I would prefer the latter.
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-10 7:42 ` [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
2015-07-10 14:35 ` Ian Campbell
@ 2015-07-15 12:57 ` Julien Grall
2015-07-17 14:12 ` Vijay Kilari
1 sibling, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 12:57 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add Virtual ITS command processing support to Virtual ITS driver
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Use helper function to read from command queue
> - Add MOVALL
> - Removed check for entry in device in domain RB-tree
> ---
> xen/arch/arm/gic-v3-its.c | 7 +
> xen/arch/arm/vgic-v3-its.c | 391 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 19 ++
> xen/include/asm-arm/gic.h | 1 +
> 4 files changed, 418 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index b98d396..9161053 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -91,6 +91,7 @@ static LIST_HEAD(its_nodes);
> static DEFINE_SPINLOCK(its_lock);
> static struct rdist_prop *gic_rdists;
> static struct rb_root rb_its_dev;
> +static struct gic_its_info its_data;
>
> #define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
>
> @@ -102,6 +103,11 @@ void dump_cmd(its_cmd_block *cmd)
> }
> #endif
>
> +u32 its_get_nr_events(void)
> +{
> + return (1 << its_data.id_bits);
> +}
> +
Please give a look to the new vgic infrastructure in order to avoid
introduced helper to pass data to the vgic.
See for instance vgic_v3_setup_hw.
> /* RB-tree helpers for its_device */
> struct its_device *its_find_device(u32 devid)
> {
> @@ -940,6 +946,7 @@ static int its_probe(struct dt_device_node *node)
> its->phys_size = its_size;
> typer = readl_relaxed(its_base + GITS_TYPER);
> its->ite_size = ((typer >> 4) & 0xf) + 1;
> + its_data.id_bits = GITS_TYPER_IDBITS(typer);
>
> its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
> if ( !its->cmd_base )
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index c63f478..af2bacd 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -31,6 +31,22 @@
> #include <asm/gic-its.h>
> #include <xen/log2.h>
>
> +#define DEBUG_ITS
> +
> +#ifdef DEBUG_ITS
> +# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
> +#else
> +# define DPRINTK(fmt, args...) do {} while ( 0 )
> +#endif
> +
> +#ifdef DEBUG_ITS
> +static void dump_cmd(its_cmd_block *cmd)
> +{
> + printk("CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
> + cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
> +}
> +#endif
> +
> static int vits_entry(struct domain *d, paddr_t entry, void *addr,
> uint32_t size, bool_t set)
> {
> @@ -202,6 +218,381 @@ void vits_remove_device(struct rb_root *root, struct vits_device *dev)
> rb_erase(&dev->node, root);
> }
>
> +static int vgic_its_process_sync(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
While the "XXX" wasn't valid, the comment "ignored" was still valid...
> + DPRINTK("%pv: vITS: SYNC: ta 0x%x \n", v, virt_cmd->sync.ta);
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct vitt entry;
> + struct domain *d = v->domain;
> + uint8_t vcol_id, cmd;
> + uint32_t vid, dev_id, event;
> +
> + vcol_id = virt_cmd->mapvi.col;
> + vid = virt_cmd->mapvi.phy_id;
> + cmd = virt_cmd->mapvi.cmd;
> + dev_id = virt_cmd->mapvi.devid;
> +
> + DPRINTK("%pv: vITS: MAPVI: dev_id 0x%x vcol_id %d vid %d \n",
> + v, dev_id, vcol_id, vid);
> +
> + if ( vcol_id > (d->max_vcpus + 1) || vid > its_get_nr_events() )
> + return -EINVAL;
As said on v3, checking the validity is pointless as a malicious guest
can rewrite the
ITT. We only need to check it when the LPI is effectively injected.
If you think this is necessary please explain why...
Furthermore, its_get_nr_events is for the hardware and not the virtual
ITS. I would prefer to see a field in the vits structure which contains
the number of event ID bits for a given domain.
[...]
> +static int vgic_its_process_movi(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct vitt entry;
> + struct domain *d = v->domain;
> + uint32_t dev_id, event;
> + uint8_t vcol_id;
> +
> + vcol_id = virt_cmd->movi.col;
> + event = virt_cmd->movi.event;
> + dev_id = virt_cmd->movi.devid;
> +
> + DPRINTK("%pv vITS: MOVI: dev_id 0x%x vcol_id %d event %d\n",
> + v, dev_id, vcol_id, event);
> +
> + if ( vcol_id > (d->max_vcpus + 1) || event > its_get_nr_events() )
> + return -EINVAL;
My comment on the check in the previous function is valid here too.
> +
> + if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
> + return -EINVAL;
> +
> + entry.vcollection = vcol_id;
> +
> + if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_movall(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
/* Ignored */
DPRINTK("%pv vITS: MOVALL: ....",...);
> + return 0;
> +}
> +
> +static int vgic_its_process_discard(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct vitt entry;
> + struct domain *d = v->domain;
> + uint32_t event, dev_id;
> +
> + event = virt_cmd->discard.event;
> + dev_id = virt_cmd->discard.devid;
> +
> + DPRINTK("%pv vITS: DISCARD: dev_id 0x%x id %d\n",
> + v, virt_cmd->discard.devid, event);
> +
> + if ( event > its_get_nr_events() )
> + return -EINVAL;
Ditto for the check.
> +
> + if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
> + return -EINVAL;
> +
> + entry.valid = false;
> +
> + if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_inv(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
Please add
/* Ignored */
> + DPRINTK("%pv vITS: INV: dev_id 0x%x id %d\n",
> + v, virt_cmd->inv.devid, virt_cmd->inv.event);
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_clear(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
/* Ignored */
> + DPRINTK("%pv: vITS: CLEAR: dev_id 0x%x id %d\n",
> + v, virt_cmd->clear.devid, virt_cmd->clear.event);
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_invall(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
/* Ignored */
> + DPRINTK("%pv: vITS: INVALL: vCID %d\n", v, virt_cmd->invall.col);
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_int(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct vitt vitt_entry;
> + struct domain *d = v->domain;
> + uint32_t event, dev_id, col_id;
> +
> + event = virt_cmd->int_cmd.cmd;
> + dev_id = virt_cmd->int_cmd.devid;
> +
> + DPRINTK("%pv: vITS: INT: Device 0x%x id %d\n", v, dev_id, event);
> + if ( event > its_get_nr_events() )
> + return -EINVAL;
Ditto for the check.
> +
> + if ( vits_get_vitt_entry(d, dev_id, event, &vitt_entry) )
> + return -EINVAL;
> +
> + if ( !vitt_entry.valid )
> + {
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: INT CMD invalid event %d for dev 0x%x\n",
> + v, event, dev_id);
> + return -EINVAL;
> + }
> +
> + col_id = vitt_entry.vcollection;
> + if ( col_id < d->max_vcpus )
> + {
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: INT CMD invalid col_id %d for dev 0x%x\n",
> + v, col_id, dev_id);
> + return -EINVAL;
> + }
> +
> + vgic_vcpu_inject_irq(d->vcpu[col_id], vitt_entry.vlpi);
As said on v3, the design document [1] suggested to implement the INT
command using vgic_vcpu_inject_lpi. Is there any issue to do it?
Also, you have to translate the col_id into to a VCPU ID.
> +
> + return 0;
> +}
> +
> +static int vgic_its_add_device(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct domain *d = v->domain;
> + struct vdevice_table dt_entry;
> + uint32_t dev_id = virt_cmd->mapd.devid;
> +
> + DPRINTK("%pv: vITS: Add device dev_id 0x%x vitt_ipa = 0x%lx size %d\n",
> + v, dev_id, (u64)virt_cmd->mapd.itt << 8,
> + virt_cmd->mapd.size);
> +
> + if ( virt_cmd->mapd.valid )
> + {
> + /* itt field is 40 bit. extract 48 bit address by shifting */
> + dt_entry.vitt_ipa = virt_cmd->mapd.itt << 8;
Could you introduce a define for the 8? It would be more clear than
open-coding the value twice in the same function.
> + dt_entry.vitt_size = (1 << (virt_cmd->mapd.size + 1)) *
> + sizeof(struct vitt);
> + }
> + else
> + {
> + dt_entry.vitt_ipa = INVALID_PADDR;
> + dt_entry.vitt_size = 0;
> + }
> +
> + if ( vits_set_vdevice_entry(d, dev_id, &dt_entry) )
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_mapc(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct domain *d = v->domain;
> + uint8_t vcol_id;
> + uint64_t vta = 0;
> +
> + vcol_id = virt_cmd->mapc.col;
> + vta = virt_cmd->mapc.ta;
> +
> + DPRINTK("%pv: vITS: MAPC: vCID %d vTA 0x%lx valid %d \n",
> + v, vcol_id, vta, virt_cmd->mapc.valid);
> +
> + if ( vcol_id > (d->max_vcpus + 1) || vta > v->domain->max_vcpus )
> + return -EINVAL;
The target address doesn't have to be valid when the collection is unmapped.
[...]
> +int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
Missing static, the function will never be called outside of this file.
> +{
> + its_cmd_block virt_cmd;
> +
> + ASSERT(spin_is_locked(&vits->lock));
> +
> + do {
> + if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
> + goto err;
> + if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
> + goto err;
> + vgic_its_update_read_ptr(v, vits);
> + } while ( vits->cmd_write != vits->cmd_write_save );
> +
> + DPRINTK("%pv: vITS: write_save 0x%lx write 0x%lx\n",
> + v, vits->cmd_write_save,
> + vits->cmd_write);
> +
> + return 1;
> +err:
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to process guest cmd\n", v);
> + /*XXX: Be nice to guest though we cannot process command? */
/* ... */
It looks like to me we want to crash the guest using
domain_crash_synchronous.
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
[..]
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 44c2317..fdd96c8 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -24,6 +24,7 @@
> #define NR_GIC_LPI 4096
> #define MAX_LPI (FIRST_GIC_LPI + NR_GIC_LPI)
> #define MAX_RDIST_COUNT 4
> +#define BIT_48_12_MASK 0xfffffffff000UL
This shouldn't be part of gic.h but gic-its.h
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-15 12:57 ` Julien Grall
@ 2015-07-17 14:12 ` Vijay Kilari
2015-07-17 15:15 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-17 14:12 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Julien,
On Wed, Jul 15, 2015 at 6:27 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Add Virtual ITS command processing support to Virtual ITS driver
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> ---
>> v4: - Use helper function to read from command queue
>> - Add MOVALL
>> - Removed check for entry in device in domain RB-tree
>> ---
[..]
>> +
>> +static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
>> + its_cmd_block *virt_cmd)
>> +{
>> + struct vitt entry;
>> + struct domain *d = v->domain;
>> + uint8_t vcol_id, cmd;
>> + uint32_t vid, dev_id, event;
>> +
>> + vcol_id = virt_cmd->mapvi.col;
>> + vid = virt_cmd->mapvi.phy_id;
>> + cmd = virt_cmd->mapvi.cmd;
>> + dev_id = virt_cmd->mapvi.devid;
>> +
>> + DPRINTK("%pv: vITS: MAPVI: dev_id 0x%x vcol_id %d vid %d \n",
>> + v, dev_id, vcol_id, vid);
>> +
>> + if ( vcol_id > (d->max_vcpus + 1) || vid > its_get_nr_events() )
>> + return -EINVAL;
>
>
> As said on v3, checking the validity is pointless as a malicious guest can
> rewrite the
> ITT. We only need to check it when the LPI is effectively injected.
>
> If you think this is necessary please explain why...
vcol_id is read from ITS command but not from guest memory.
So command values are validated here instead of doing at time time
of LPI injection.
If not done here, then we still allow malicious guest to run and during
LPI injection if invalid col_id is found in ITT we just drop LPI.
>
> Furthermore, its_get_nr_events is for the hardware and not the virtual ITS.
> I would prefer to see a field in the vits structure which contains the
> number of event ID bits for a given domain.
Why do we need to restrict number of LPIs to domain?
[...]
>> +
>> + if ( vits_get_vitt_entry(d, dev_id, event, &vitt_entry) )
>> + return -EINVAL;
>> +
>> + if ( !vitt_entry.valid )
>> + {
>> + dprintk(XENLOG_G_ERR,
>> + "%pv: vITS: INT CMD invalid event %d for dev 0x%x\n",
>> + v, event, dev_id);
>> + return -EINVAL;
>> + }
>> +
>> + col_id = vitt_entry.vcollection;
>> + if ( col_id < d->max_vcpus )
>> + {
>> + dprintk(XENLOG_G_ERR,
>> + "%pv: vITS: INT CMD invalid col_id %d for dev 0x%x\n",
>> + v, col_id, dev_id);
>> + return -EINVAL;
>> + }
>> +
>> + vgic_vcpu_inject_irq(d->vcpu[col_id], vitt_entry.vlpi);
>
>
> As said on v3, the design document [1] suggested to implement the INT
> command using vgic_vcpu_inject_lpi. Is there any issue to do it?
IIRC, INT command contains vlpi which does not have its_device because irq_desc
is not reserved for this. Hence it is handled similar to event_ch int.
>
> Also, you have to translate the col_id into to a VCPU ID.
This is virtual collection id which itself is vcpu id. isn't it?
Regards
Vijay
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-17 14:12 ` Vijay Kilari
@ 2015-07-17 15:15 ` Julien Grall
2015-07-17 15:34 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-17 15:15 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 17/07/15 15:12, Vijay Kilari wrote:
> Hi Julien,
Hi Vijay,
> On Wed, Jul 15, 2015 at 6:27 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>>
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Add Virtual ITS command processing support to Virtual ITS driver
>>>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> ---
>>> v4: - Use helper function to read from command queue
>>> - Add MOVALL
>>> - Removed check for entry in device in domain RB-tree
>>> ---
> [..]
>>> +
>>> +static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
>>> + its_cmd_block *virt_cmd)
>>> +{
>>> + struct vitt entry;
>>> + struct domain *d = v->domain;
>>> + uint8_t vcol_id, cmd;
>>> + uint32_t vid, dev_id, event;
>>> +
>>> + vcol_id = virt_cmd->mapvi.col;
>>> + vid = virt_cmd->mapvi.phy_id;
>>> + cmd = virt_cmd->mapvi.cmd;
>>> + dev_id = virt_cmd->mapvi.devid;
>>> +
>>> + DPRINTK("%pv: vITS: MAPVI: dev_id 0x%x vcol_id %d vid %d \n",
>>> + v, dev_id, vcol_id, vid);
>>> +
>>> + if ( vcol_id > (d->max_vcpus + 1) || vid > its_get_nr_events() )
>>> + return -EINVAL;
>>
>>
>> As said on v3, checking the validity is pointless as a malicious guest can
>> rewrite the
>> ITT. We only need to check it when the LPI is effectively injected.
>>
>> If you think this is necessary please explain why...
>
> vcol_id is read from ITS command but not from guest memory.
> So command values are validated here instead of doing at time time
> of LPI injection.
>
> If not done here, then we still allow malicious guest to run and during
> LPI injection if invalid col_id is found in ITT we just drop LPI.
You are storing the collection in the ITT. The ITT is part of the guest
memory.
The guest can modify at his convenience the region before we receive an
LPI and handle it. So we *must* check the validity of the collection
(and anything else in the ITT entry) everytime we receveive an LPI.
Therefore this check here is pointless.
>>
>> Furthermore, its_get_nr_events is for the hardware and not the virtual ITS.
>> I would prefer to see a field in the vits structure which contains the
>> number of event ID bits for a given domain.
>
> Why do we need to restrict number of LPIs to domain?
Why do you speak about LPIs here? its_get_nr_events return the maximum
number of eventID and not intID. LPIs are validate against the later.
Although, to answer to your question, the reason are various, one of
them is to save Xen memory, everytime we are creating domain we have to
allocate a pending_irq structure per vLPIs.
It's pointless to allocate all of them if we know that only a few will
be used.
FWIW, we are doing the same for SPIs.
> [...]
>
>>> +
>>> + if ( vits_get_vitt_entry(d, dev_id, event, &vitt_entry) )
>>> + return -EINVAL;
>>> +
>>> + if ( !vitt_entry.valid )
>>> + {
>>> + dprintk(XENLOG_G_ERR,
>>> + "%pv: vITS: INT CMD invalid event %d for dev 0x%x\n",
>>> + v, event, dev_id);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + col_id = vitt_entry.vcollection;
>>> + if ( col_id < d->max_vcpus )
>>> + {
>>> + dprintk(XENLOG_G_ERR,
>>> + "%pv: vITS: INT CMD invalid col_id %d for dev 0x%x\n",
>>> + v, col_id, dev_id);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + vgic_vcpu_inject_irq(d->vcpu[col_id], vitt_entry.vlpi);
>>
>>
>> As said on v3, the design document [1] suggested to implement the INT
>> command using vgic_vcpu_inject_lpi. Is there any issue to do it?
>
> IIRC, INT command contains vlpi which does not have its_device because irq_desc
> is not reserved for this. Hence it is handled similar to event_ch int.
Why do you need an irq_desc in vgic_vcpu_inject_lpi? If you follow what
Ian suggested on patch #12 [1], you can directly call the function and
avoid to duplicate most of the code.
>
>>
>> Also, you have to translate the col_id into to a VCPU ID.
>
> This is virtual collection id which itself is vcpu id. isn't it?
No. A collection can move between CPU, so you will end up to have vCPUID
!= vCID.
Your MAPC command is correctly storing the mapping in collections, so
please use it to retrieve the correct vCPU.
Regards,
[1] http://lists.xen.org/archives/html/xen-devel/2015-07/msg02160.html
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-17 15:15 ` Julien Grall
@ 2015-07-17 15:34 ` Ian Campbell
2015-07-17 15:44 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-17 15:34 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Fri, 2015-07-17 at 16:15 +0100, Julien Grall wrote:
> >>
> >> Also, you have to translate the col_id into to a VCPU ID.
> >
> > This is virtual collection id which itself is vcpu id. isn't it?
>
> No. A collection can move between CPU, so you will end up to have vCPUID
> != vCID.
Specifically it is a TA (TargetAddress) and not a collection which is
equal to a VCPU ID (in our implementation, because we have chosen to
implement GITS_TYPER.PTA==0).
A collection is mapped onto a TA by MAPC command.
Thus some sort of record of which collection is mapped to which
TA(==vcpuid) is required.
> Your MAPC command is correctly storing the mapping in collections, so
> please use it to retrieve the correct vCPU.
Indeed.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support
2015-07-17 15:34 ` Ian Campbell
@ 2015-07-17 15:44 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-17 15:44 UTC (permalink / raw)
To: Ian Campbell
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 17/07/15 16:34, Ian Campbell wrote:
> On Fri, 2015-07-17 at 16:15 +0100, Julien Grall wrote:
>
>>>>
>>>> Also, you have to translate the col_id into to a VCPU ID.
>>>
>>> This is virtual collection id which itself is vcpu id. isn't it?
>>
>> No. A collection can move between CPU, so you will end up to have vCPUID
>> != vCID.
>
> Specifically it is a TA (TargetAddress) and not a collection which is
> equal to a VCPU ID (in our implementation, because we have chosen to
> implement GITS_TYPER.PTA==0).
>
> A collection is mapped onto a TA by MAPC command.
>
> Thus some sort of record of which collection is mapped to which
> TA(==vcpuid) is required.
Correct, I shouldn't have take shortcut while writing my previous mail.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (6 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 07/17] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 14:52 ` Ian Campbell
2015-07-15 14:15 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 09/17] xen/arm: ITS: Add GITS registers emulation vijay.kilari
` (8 subsequent siblings)
16 siblings, 2 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add APIs to add devices to RB-tree, assign and remove
devices to domain.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Introduced helper to populate its_device struct
- Fixed freeing of its_device memory
- its_device struct holds domain id
---
xen/arch/arm/gic-v3-its.c | 362 +++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/irq.c | 8 +
xen/include/asm-arm/gic-its.h | 18 ++
xen/include/asm-arm/irq.h | 1 +
4 files changed, 389 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 9161053..1d2fdde 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -92,6 +92,7 @@ static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
static struct rb_root rb_its_dev;
static struct gic_its_info its_data;
+static DEFINE_SPINLOCK(rb_its_dev_lock);
#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
@@ -108,6 +109,14 @@ u32 its_get_nr_events(void)
return (1 << its_data.id_bits);
}
+static struct its_node * its_get_phys_node(u32 dev_id)
+{
+ /* TODO: For now return ITS0 node.
+ * Need Query PCI helper function to get on which
+ * ITS node the device is attached
+ */
+ return list_first_entry(&its_nodes, struct its_node, entry);
+}
/* RB-tree helpers for its_device */
struct its_device *its_find_device(u32 devid)
{
@@ -314,6 +323,30 @@ static void its_send_inv(struct its_device *dev, struct its_collection *col,
its_send_single_command(dev->its, &cmd, col);
}
+static void its_send_mapd(struct its_device *dev, int valid)
+{
+ its_cmd_block cmd;
+ unsigned long itt_addr;
+ u8 size;
+
+ size = max(ilog2(dev->nr_ites), 1);
+ itt_addr = __pa(dev->itt_addr);
+ itt_addr = ROUNDUP(itt_addr, ITS_ITT_ALIGN);
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapd.cmd = GITS_CMD_MAPD;
+ cmd.mapd.devid = dev->device_id;
+ cmd.mapd.size = size - 1;
+ /*
+ * ITT address field of MAPD command holds bit[48:8] of
+ * itt address. Hence shift by 8.
+ */
+ cmd.mapd.itt = itt_addr >> 8;
+ cmd.mapd.valid = !!valid;
+
+ its_send_single_command(dev->its, &cmd, &dev->its->collections[0]);
+}
+
static void its_send_mapc(struct its_node *its, struct its_collection *col,
int valid)
{
@@ -328,6 +361,21 @@ static void its_send_mapc(struct its_node *its, struct its_collection *col,
its_send_single_command(its, &cmd, col);
}
+static void its_send_mapvi(struct its_device *dev, struct its_collection *col,
+ u32 phys_id, u32 event)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapvi.cmd = GITS_CMD_MAPVI;
+ cmd.mapvi.devid = dev->device_id;
+ cmd.mapvi.event = event;
+ cmd.mapvi.phy_id = phys_id;
+ cmd.mapvi.col = col->col_id;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
static void its_send_invall(struct its_node *its, struct its_collection *col)
{
its_cmd_block cmd;
@@ -473,12 +521,18 @@ static const struct its_hw_operations its_ops = {
static unsigned long *lpi_bitmap;
static u32 lpi_chunks;
+static DEFINE_SPINLOCK(lpi_lock);
static int its_lpi_to_chunk(int lpi)
{
return (lpi - 8192) >> IRQS_PER_CHUNK_SHIFT;
}
+static int its_chunk_to_lpi(int chunk)
+{
+ return (chunk << IRQS_PER_CHUNK_SHIFT) + 8192;
+}
+
int its_lpi_init(u32 id_bits)
{
lpi_chunks = its_lpi_to_chunk(1UL << id_bits);
@@ -495,6 +549,314 @@ int its_lpi_init(u32 id_bits)
return 0;
}
+static unsigned long *its_lpi_alloc_chunks(int nirqs, int *base, int *nr_ids)
+{
+ unsigned long *bitmap = NULL;
+ int chunk_id;
+ int nr_chunks;
+ int i;
+
+ nr_chunks = DIV_ROUND_UP(nirqs, IRQS_PER_CHUNK);
+
+ spin_lock(&lpi_lock);
+
+ do {
+ chunk_id = bitmap_find_next_zero_area(lpi_bitmap, lpi_chunks,
+ 0, nr_chunks, 0);
+ if ( chunk_id < lpi_chunks )
+ break;
+
+ nr_chunks--;
+ } while ( nr_chunks > 0 );
+
+ if ( !nr_chunks )
+ goto out;
+
+ bitmap = xzalloc_bytes(BITS_TO_LONGS(nr_chunks * IRQS_PER_CHUNK) *
+ sizeof (long));
+ if ( !bitmap )
+ goto out;
+
+ for ( i = 0; i < nr_chunks; i++ )
+ set_bit(chunk_id + i, lpi_bitmap);
+
+ *base = its_chunk_to_lpi(chunk_id);
+ *nr_ids = nr_chunks * IRQS_PER_CHUNK;
+
+out:
+ spin_unlock(&lpi_lock);
+
+ return bitmap;
+}
+
+static void its_lpi_free(unsigned long *bitmap, int base, int nr_ids)
+{
+ int lpi;
+
+ spin_lock(&lpi_lock);
+
+ for ( lpi = base; lpi < (base + nr_ids); lpi += IRQS_PER_CHUNK )
+ {
+ int chunk = its_lpi_to_chunk(lpi);
+
+ BUG_ON(chunk > lpi_chunks);
+ if ( test_bit(chunk, lpi_bitmap) )
+ clear_bit(chunk, lpi_bitmap);
+ else
+ its_err("Bad LPI chunk %d\n", chunk);
+ }
+
+ spin_unlock(&lpi_lock);
+
+ xfree(bitmap);
+}
+
+static inline u32 its_get_plpi(struct its_device *dev, u32 event)
+{
+ return dev->lpi_base + event;
+}
+
+static int its_alloc_device_irq(struct its_device *dev, u32 *hwirq)
+{
+ int idx;
+
+ idx = find_first_zero_bit(dev->lpi_map, dev->nr_lpis);
+ if ( idx == dev->nr_lpis )
+ return -ENOSPC;
+
+ *hwirq = its_get_plpi(dev, idx);
+ set_bit(idx, dev->lpi_map);
+
+ return 0;
+}
+
+static void its_free_device(struct its_device *dev)
+{
+ xfree(dev->itt_addr);
+ xfree(dev->lpi_map);
+ xfree(dev);
+}
+
+static struct its_device *its_alloc_device(u32 devid)
+{
+ struct its_device *dev;
+ paddr_t *itt;
+ unsigned long *lpi_map;
+ int lpi_base, nr_lpis, sz;
+ u32 nr_ites;
+
+ dev = xzalloc(struct its_device);
+ if ( dev == NULL )
+ return NULL;
+
+ dev->its = its_get_phys_node(devid);
+ /* TODO: Use pci helper to get nvecs */
+ nr_ites = 64;
+ sz = nr_ites * dev->its->ite_size;
+ sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
+ itt = xzalloc_bytes(sz);
+ if ( !itt )
+ goto err;
+
+ lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base, &nr_lpis);
+ if ( !lpi_map || (nr_lpis < nr_ites) )
+ goto lpi_err;
+
+ dev->itt_addr = itt;
+ dev->nr_ites = nr_ites;
+ dev->lpi_map = lpi_map;
+ dev->lpi_base = lpi_base;
+ dev->nr_lpis = nr_lpis;
+ dev->device_id = devid;
+
+ return dev;
+
+lpi_err:
+ xfree(itt);
+ xfree(lpi_map);
+err:
+ xfree(dev);
+
+ return NULL;
+}
+
+/* Device assignment. Should be called from PHYSDEVOPS_pci_device_add */
+int its_add_device(u32 devid)
+{
+ struct its_device *dev;
+ u32 i, plpi, nr_cpus;
+ struct its_collection *col;
+ struct irq_desc *desc;
+
+ spin_lock(&rb_its_dev_lock);
+ dev = its_find_device(devid);
+ if ( dev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ dprintk(XENLOG_G_ERR, "%pv: ITS: Device already exists dev 0x%x\n",
+ current, dev->device_id);
+ return -EEXIST;
+ }
+
+ DPRINTK("%pv: ITS: Add device devid 0x%x\n", current, devid);
+
+ dev = its_alloc_device(devid);
+ if ( !dev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ return -ENOMEM;
+ }
+
+ if ( its_insert_device(dev) )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ its_free_device(dev);
+ dprintk(XENLOG_G_ERR, "%pv: ITS: failed to insert device 0x%x\n",
+ current, devid);
+ return -EINVAL;
+ }
+
+ DPRINTK("%pv: ITS: Adding Device with id 0x%x nvecs %d lpi_base 0x%x\n",
+ current, dev->device_id, dev->nr_lpis, dev->lpi_base);
+
+ /* Map device to its ITT */
+ its_send_mapd(dev, 1);
+
+ /* TODO: Use nr_cpu_ids? */
+ nr_cpus = num_online_cpus();
+ for ( i = 0; i < dev->nr_lpis; i++ )
+ {
+ /* Reserve pLPI */
+ if ( its_alloc_device_irq(dev, &plpi) )
+ {
+ /* Cannot revert MAPVI */
+ its_send_mapd(dev, 0);
+ its_lpi_free(dev->lpi_map, dev->lpi_base, dev->nr_lpis);
+ its_free_device(dev);
+ dprintk(XENLOG_G_ERR, "%pv: ITS: Cannot add device 0x%x\n",
+ current, devid);
+ return -ENOSPC;
+ }
+
+ /* For each pLPI send MAPVI command */
+ col = &dev->its->collections[(i % nr_cpus)];
+ /* Store collection for this plpi in irq_desc */
+ desc = irq_to_desc(plpi);
+ spin_lock(&desc->lock);
+ gic_set_irq_collection(plpi, col->col_id);
+ spin_unlock(&desc->lock);
+ its_send_mapvi(dev, col, plpi, i);
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ return 0;
+}
+
+int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
+{
+ struct its_device *pdev;
+ struct vits_device *vdev;
+ struct irq_desc *desc;
+ u32 plpi, i;
+
+ DPRINTK("%pv: ITS: Assign request for device 0x%x to domain %d\n",
+ current, vdevid, d->domain_id);
+
+ spin_lock(&d->arch.vits->dev_lock);
+ vdev = vits_find_device(&d->arch.vits->dev_root, vdevid);
+ if ( vdev )
+ {
+ spin_unlock(&d->arch.vits->dev_lock);
+ return -EEXIST;
+ }
+
+ spin_lock(&rb_its_dev_lock);
+ pdev = its_find_device(pdevid);
+ if ( !pdev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ return -ENODEV;
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ vdev = xzalloc_bytes(sizeof(struct vits_device));
+ if ( !pdev )
+ return -ENOMEM;
+
+ vdev->its_dev = pdev;
+ pdev->domain_id = d->domain_id;
+ pdev->virt_device_id = vdevid;
+ vdev->vdevid = vdevid;
+
+ /* Insert to domains' list */
+ if ( vits_insert_device(&d->arch.vits->dev_root, vdev) )
+ {
+ spin_unlock(&d->arch.vits->dev_lock);
+ return -EINVAL;
+ }
+ spin_unlock(&d->arch.vits->dev_lock);
+
+ DPRINTK("%pv: ITS: Assigned pdevid 0x%x with nvecs %d for domain %d\n",
+ current, pdevid, pdev->nr_lpis, d->domain_id);
+
+ for ( i = 0; i < pdev->nr_lpis; i++ )
+ {
+ plpi = its_get_plpi(pdev, i);
+ route_irq_to_guest(d, i, plpi, "LPI");
+ desc = irq_to_desc(plpi);
+ spin_lock(&desc->lock);
+ set_irq_device(desc, pdev);
+ lpi_set_config(desc, 1);
+ spin_unlock(&desc->lock);
+ }
+
+ return 0;
+}
+
+int its_detach_device(struct domain *d, u32 vdevid, u32 pdevid)
+{
+ struct its_device *pdev;
+ struct vits_device *vdev;
+ struct irq_desc *desc;
+ u32 plpi, i;
+
+ DPRINTK("%pv: ITS: Detach request for device 0x%x domain %d\n",
+ current, pdevid, d->domain_id);
+
+ spin_lock(&d->arch.vits->dev_lock);
+ vdev = vits_find_device(&d->arch.vits->dev_root, vdevid);
+ if ( !vdev )
+ {
+ spin_unlock(&d->arch.vits->dev_lock);
+ return -EINVAL;
+ }
+
+ pdev = vdev->its_dev;
+ pdev->domain_id = 0;
+
+ DPRINTK("%pv: ITS: Detach pdevid 0x%x with nvecs %d from domain %d\n",
+ current, pdev->device_id, pdev->nr_lpis, d->domain_id);
+ /* Remove vits_device from domain list */
+ vits_remove_device(&d->arch.vits->dev_root, vdev);
+ spin_unlock(&d->arch.vits->dev_lock);
+
+ for ( i = 0; i < pdev->nr_lpis; i++ )
+ {
+ plpi = its_get_plpi(pdev, i);
+ desc = irq_to_desc(plpi);
+ spin_lock(&desc->lock);
+ lpi_set_config(desc, 0);
+ set_irq_device(desc, NULL);
+ /* TODO: Fix for lpi */
+ release_irq(plpi, d);
+ spin_unlock(&desc->lock);
+ }
+
+ xfree(vdev);
+
+ return 0;
+}
+
/*
* We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
* deal with (one configuration byte per interrupt). PENDBASE has to
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index ba8528a..3806d98 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -181,6 +181,14 @@ u16 gic_get_irq_collection(unsigned int irq)
return desc->arch.col_id;
}
+void gic_set_irq_collection(unsigned int irq, u16 col_id)
+{
+ struct irq_desc *desc = irq_to_desc(irq);
+
+ ASSERT(spin_is_locked(&desc->lock));
+ desc->arch.col_id = col_id;
+}
+
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
{
if ( desc != NULL )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index a143003..f041efc 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -53,6 +53,8 @@ struct vgic_its
uint64_t dt_size;
/* Radix-tree root of devices attached to this domain */
struct rb_root dev_root;
+ /* Lock to manange virtual devices in rb-tree*/
+ spinlock_t dev_lock;
/* collections mapped */
struct its_collection *collections;
};
@@ -189,10 +191,24 @@ typedef union {
struct its_device {
/* Physical ITS */
struct its_node *its;
+ /* Device ITT address */
+ paddr_t *itt_addr;
+ /* Device ITT size */
+ unsigned long itt_size;
+ /* Physical LPI map */
+ unsigned long *lpi_map;
+ /* First Physical LPI number assigned */
+ u32 lpi_base;
/* Number of Physical LPIs assigned */
int nr_lpis;
+ /* Number of ITES entries */
+ u32 nr_ites;
/* Physical Device id */
u32 device_id;
+ /* Virtual Device id */
+ u32 virt_device_id;
+ /* Domain id */
+ int domain_id;
/* RB-tree entry */
struct rb_node node;
};
@@ -227,6 +243,8 @@ int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
struct its_device *its_find_device(u32 devid);
int its_insert_device(struct its_device *dev);
+int its_add_device(u32 devid);
+int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid);
int vits_set_vitt_entry(struct domain *d, uint32_t devid,
uint32_t event, struct vitt *entry);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 55e219f..6bf8fcb 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -57,6 +57,7 @@ unsigned int irq_to_vid(struct irq_desc *desc);
struct its_device *get_irq_device(struct irq_desc *desc);
void set_irq_device(struct irq_desc *desc, struct its_device *dev);
u16 gic_get_irq_collection(unsigned int irq);
+void gic_set_irq_collection(unsigned int irq, u16 col_id);
int platform_get_irq(const struct dt_device_node *device, int index);
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-10 7:42 ` [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-07-10 14:52 ` Ian Campbell
2015-07-15 13:14 ` Julien Grall
2015-07-15 14:15 ` Julien Grall
1 sibling, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 14:52 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add APIs to add devices to RB-tree, assign and remove
> devices to domain.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Introduced helper to populate its_device struct
> - Fixed freeing of its_device memory
> - its_device struct holds domain id
> ---
> xen/arch/arm/gic-v3-its.c | 362 +++++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/irq.c | 8 +
> xen/include/asm-arm/gic-its.h | 18 ++
> xen/include/asm-arm/irq.h | 1 +
> 4 files changed, 389 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 9161053..1d2fdde 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -92,6 +92,7 @@ static DEFINE_SPINLOCK(its_lock);
> static struct rdist_prop *gic_rdists;
> static struct rb_root rb_its_dev;
> static struct gic_its_info its_data;
> +static DEFINE_SPINLOCK(rb_its_dev_lock);
>
> #define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
>
> @@ -108,6 +109,14 @@ u32 its_get_nr_events(void)
> return (1 << its_data.id_bits);
> }
>
> +static struct its_node * its_get_phys_node(u32 dev_id)
> +{
> + /* TODO: For now return ITS0 node.
> + * Need Query PCI helper function to get on which
> + * ITS node the device is attached
> + */
> + return list_first_entry(&its_nodes, struct its_node, entry);
> +}
> /* RB-tree helpers for its_device */
> struct its_device *its_find_device(u32 devid)
> {
> @@ -314,6 +323,30 @@ static void its_send_inv(struct its_device *dev, struct its_collection *col,
> its_send_single_command(dev->its, &cmd, col);
> }
>
> +static void its_send_mapd(struct its_device *dev, int valid)
Some of the decisions made wrt what gets introduced in what patch and in
what order make this series rather hard to follow. i.e. why wasn't this
added along with all the others.
> [...]
> +static int its_chunk_to_lpi(int chunk)
> +{
> + return (chunk << IRQS_PER_CHUNK_SHIFT) + 8192;
> +}
[...]
> +static unsigned long *its_lpi_alloc_chunks(int nirqs, int *base, int *nr_ids)
And why wasn't all this in the earlier patch which included the comments
about the lpi allocation chunking strategy.
Oh well.
> +static struct its_device *its_alloc_device(u32 devid)
> +{
> + struct its_device *dev;
> + paddr_t *itt;
> + unsigned long *lpi_map;
> + int lpi_base, nr_lpis, sz;
> + u32 nr_ites;
> +
> + dev = xzalloc(struct its_device);
> + if ( dev == NULL )
> + return NULL;
> +
> + dev->its = its_get_phys_node(devid);
> + /* TODO: Use pci helper to get nvecs */
> + nr_ites = 64;
Please add nr_ites as a parameter to this function and to
its_add_device, such that this hardcoding can be pushed all the way down
into the final patch which adds the temporary registration code in
xen/arch/arm/platforms/thunderx.c.
> +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
> +{
> + struct its_device *pdev;
> + struct vits_device *vdev;
> + struct irq_desc *desc;
> + u32 plpi, i;
> +
> + DPRINTK("%pv: ITS: Assign request for device 0x%x to domain %d\n",
> + current, vdevid, d->domain_id);
> +
> + spin_lock(&d->arch.vits->dev_lock);
> + vdev = vits_find_device(&d->arch.vits->dev_root, vdevid);
> + if ( vdev )
> + {
> + spin_unlock(&d->arch.vits->dev_lock);
> + return -EEXIST;
> + }
This just checks that it isn't assigned to the current domain AFAICT.
Which is insufficient, since it might be assigned to some other device.
I think you need to check some field of pdev, probably pdev->domain_id,
instead.
BTW, I would expect the struct domain * to be of more use than the domid
in the general case and then NULL would serve as a reasonable "not
assigned" flag.
> +int its_detach_device(struct domain *d, u32 vdevid, u32 pdevid)
> +{
> + struct its_device *pdev;
> + struct vits_device *vdev;
> + struct irq_desc *desc;
> + u32 plpi, i;
> +
> + DPRINTK("%pv: ITS: Detach request for device 0x%x domain %d\n",
> + current, pdevid, d->domain_id);
> +
> + spin_lock(&d->arch.vits->dev_lock);
> + vdev = vits_find_device(&d->arch.vits->dev_root, vdevid);
> + if ( !vdev )
> + {
> + spin_unlock(&d->arch.vits->dev_lock);
> + return -EINVAL;
> + }
> +
> + pdev = vdev->its_dev;
You can lookup pdev via the pdevid, no need for the vdev.
As a sanity check you could ensure that pdev->virt_device_id == vdevid.
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index a143003..f041efc 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -53,6 +53,8 @@ struct vgic_its
> uint64_t dt_size;
> /* Radix-tree root of devices attached to this domain */
> struct rb_root dev_root;
> + /* Lock to manange virtual devices in rb-tree*/
"manage".
Perhaps having gotten rid of the rb-tree this won't be needed any
longer, or maybe it also protects other stuff?
> + spinlock_t dev_lock;
> /* collections mapped */
> struct its_collection *collections;
> };
> @@ -189,10 +191,24 @@ typedef union {
> struct its_device {
> /* Physical ITS */
> struct its_node *its;
> + /* Device ITT address */
> + paddr_t *itt_addr;
> + /* Device ITT size */
> + unsigned long itt_size;
> + /* Physical LPI map */
> + unsigned long *lpi_map;
> + /* First Physical LPI number assigned */
> + u32 lpi_base;
> /* Number of Physical LPIs assigned */
> int nr_lpis;
> + /* Number of ITES entries */
"Number of IT entries" or "Number of ITES" I think?
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-10 14:52 ` Ian Campbell
@ 2015-07-15 13:14 ` Julien Grall
2015-07-16 13:40 ` Vijay Kilari
0 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 13:14 UTC (permalink / raw)
To: Ian Campbell, vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
stefano.stabellini, manish.jaggi
Hi Ian,
On 10/07/2015 16:52, Ian Campbell wrote:
>> +static struct its_device *its_alloc_device(u32 devid)
>> +{
>> + struct its_device *dev;
>> + paddr_t *itt;
>> + unsigned long *lpi_map;
>> + int lpi_base, nr_lpis, sz;
>> + u32 nr_ites;
>> +
>> + dev = xzalloc(struct its_device);
>> + if ( dev == NULL )
>> + return NULL;
>> +
>> + dev->its = its_get_phys_node(devid);
>> + /* TODO: Use pci helper to get nvecs */
>> + nr_ites = 64;
>
> Please add nr_ites as a parameter to this function and to
> its_add_device, such that this hardcoding can be pushed all the way down
> into the final patch which adds the temporary registration code in
> xen/arch/arm/platforms/thunderx.c.
+1, I would also add the physical ITS in parameter as it's not possible
to get it based on the devID (see its_get_phys_node).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-15 13:14 ` Julien Grall
@ 2015-07-16 13:40 ` Vijay Kilari
2015-07-16 14:38 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-16 13:40 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Wed, Jul 15, 2015 at 6:44 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Ian,
>
> On 10/07/2015 16:52, Ian Campbell wrote:
>>>
>>> +static struct its_device *its_alloc_device(u32 devid)
>>> +{
>>> + struct its_device *dev;
>>> + paddr_t *itt;
>>> + unsigned long *lpi_map;
>>> + int lpi_base, nr_lpis, sz;
>>> + u32 nr_ites;
>>> +
>>> + dev = xzalloc(struct its_device);
>>> + if ( dev == NULL )
>>> + return NULL;
>>> +
>>> + dev->its = its_get_phys_node(devid);
>>> + /* TODO: Use pci helper to get nvecs */
>>> + nr_ites = 64;
>>
>>
>> Please add nr_ites as a parameter to this function and to
>> its_add_device, such that this hardcoding can be pushed all the way down
>> into the final patch which adds the temporary registration code in
>> xen/arch/arm/platforms/thunderx.c.
>
>
> +1, I would also add the physical ITS in parameter as it's not possible to
> get it based on the devID (see its_get_phys_node).
thunderx.c does not have physical ITS to pass as parameter.
>
> Regards,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-16 13:40 ` Vijay Kilari
@ 2015-07-16 14:38 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-16 14:38 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Vijay,
On 16/07/2015 15:40, Vijay Kilari wrote:
> On Wed, Jul 15, 2015 at 6:44 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Ian,
>>
>> On 10/07/2015 16:52, Ian Campbell wrote:
>>>>
>>>> +static struct its_device *its_alloc_device(u32 devid)
>>>> +{
>>>> + struct its_device *dev;
>>>> + paddr_t *itt;
>>>> + unsigned long *lpi_map;
>>>> + int lpi_base, nr_lpis, sz;
>>>> + u32 nr_ites;
>>>> +
>>>> + dev = xzalloc(struct its_device);
>>>> + if ( dev == NULL )
>>>> + return NULL;
>>>> +
>>>> + dev->its = its_get_phys_node(devid);
>>>> + /* TODO: Use pci helper to get nvecs */
>>>> + nr_ites = 64;
>>>
>>>
>>> Please add nr_ites as a parameter to this function and to
>>> its_add_device, such that this hardcoding can be pushed all the way down
>>> into the final patch which adds the temporary registration code in
>>> xen/arch/arm/platforms/thunderx.c.
>>
>>
>> +1, I would also add the physical ITS in parameter as it's not possible to
>> get it based on the devID (see its_get_phys_node).
>
> thunderx.c does not have physical ITS to pass as parameter.
That's not true. You are able to get the PCI root controller which
should have a phandle to the ITS device tree node.
With this dt node in hand you can get the correct physical ITS.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-10 7:42 ` [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
2015-07-10 14:52 ` Ian Campbell
@ 2015-07-15 14:15 ` Julien Grall
2015-07-18 9:44 ` Vijay Kilari
1 sibling, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 14:15 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> +static struct its_device *its_alloc_device(u32 devid)
> +{
> + struct its_device *dev;
> + paddr_t *itt;
> + unsigned long *lpi_map;
> + int lpi_base, nr_lpis, sz;
> + u32 nr_ites;
> +
> + dev = xzalloc(struct its_device);
> + if ( dev == NULL )
> + return NULL;
> +
> + dev->its = its_get_phys_node(devid);
> + /* TODO: Use pci helper to get nvecs */
> + nr_ites = 64;
> + sz = nr_ites * dev->its->ite_size;
> + sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
> + itt = xzalloc_bytes(sz);
> + if ( !itt )
> + goto err;
> +
> + lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base, &nr_lpis);
> + if ( !lpi_map || (nr_lpis < nr_ites) )
> + goto lpi_err;
The check "nr_lpis != nr_ites" is here to ensure that
its_lpi_alloc_chuncks allocate the right number of LPI, right? If so,
given this is the only place you call its_lpi_alloc_chunks, why don't
you put the check within the function? It would avoid one parameter.
> +
> + dev->itt_addr = itt;
> + dev->nr_ites = nr_ites;
> + dev->lpi_map = lpi_map;
> + dev->lpi_base = lpi_base;
> + dev->nr_lpis = nr_lpis;
> + dev->device_id = devid;
> +
> + return dev;
> +
> +lpi_err:
> + xfree(itt);
> + xfree(lpi_map);
> +err:
> + xfree(dev);
> +
> + return NULL;
> +}
> +
> +/* Device assignment. Should be called from PHYSDEVOPS_pci_device_add */
This comment is wrong. It's not device assignment but making aware Xen
that this device exists. Also, I would drop the second part of the
comment as it may be call from other place than PHYSDEVOP_pci_device_add.
> +int its_add_device(u32 devid)
[...]
> + /* Map device to its ITT */
> + its_send_mapd(dev, 1);
> +
> + /* TODO: Use nr_cpu_ids? */
I though you fixed nr_cpu_ids? If not can you please do it and use this
value here.
> + nr_cpus = num_online_cpus();
> + for ( i = 0; i < dev->nr_lpis; i++ )
> + {
> + /* Reserve pLPI */
> + if ( its_alloc_device_irq(dev, &plpi) )
> + {
> + /* Cannot revert MAPVI */
Why not? The invert of MAPVI is DISCARD.
[...]
> +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
[...]
> + for ( i = 0; i < pdev->nr_lpis; i++ )
> + {
> + plpi = its_get_plpi(pdev, i);
> + route_irq_to_guest(d, i, plpi, "LPI");
> + desc = irq_to_desc(plpi);
> + spin_lock(&desc->lock);
> + set_irq_device(desc, pdev);
This should be part of its_add_device and not its_assign_device.
> + lpi_set_config(desc, 1);
The goal of route_irq_to_guest is to setup everything in order to route
correctly the IRQ (here LPI) to the guest.
As said on v3, if the current function doesn't fit you need, please
introduce a new one.
In this case, lpi_set_config is used to configure the IRQ (i.e set
property ...). It's likely the same as gic_set_irq_properties.
Implementing there would avoid the is_lpi you added in this function.
Furthermore, enabling the interrupt here is very fragile. As soon as the
LPI is enabled for the given device, he can send an interrupt. See the
thread on the draft F [1] for more details.
I don't think this is really important to have it until PCI passthrough
is added. But at least a comment would be nice.
> + spin_unlock(&desc->lock);
> + }
> +
> + return 0;
> +}
> +
> +int its_detach_device(struct domain *d, u32 vdevid, u32 pdevid)
> +{
I would prefer if you don't introduce its_detach_device until you used
when PCI passthrough will be added.
> + for ( i = 0; i < pdev->nr_lpis; i++ )
> + {
> + plpi = its_get_plpi(pdev, i);
> + desc = irq_to_desc(plpi);
> + spin_lock(&desc->lock);
> + lpi_set_config(desc, 0);
> + set_irq_device(desc, NULL);
> + /* TODO: Fix for lpi */
This would avoid take spend time on reviewing if everything is correctly
done when you release an IRQ. Which doesn't seem to be the case based on
the TODO.
> + release_irq(plpi, d);
release_irq is for IRQ assigned to Xen and not guest IRQ. You would have
to use release_guest_irq here. Furthermore, there will be some extra
care to do when a domain crashed...
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index ba8528a..3806d98 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -181,6 +181,14 @@ u16 gic_get_irq_collection(unsigned int irq)
> return desc->arch.col_id;
> }
>
> +void gic_set_irq_collection(unsigned int irq, u16 col_id)
> +{
> + struct irq_desc *desc = irq_to_desc(irq);
> +
> + ASSERT(spin_is_locked(&desc->lock));
> + desc->arch.col_id = col_id;
> +}
> +
This function is not a GIC function but a function which handle
irq_desc. Please name accordingly i.e irqdesc_set_collection.
Furthermore, given the size of the function, an inline function would
have been better.
Finally, I think the function can directly get an irq_desc in parameter.
My remarks are the same for gic_get_irq_collection (from patch #5) which
I didn't spot before.
Regards,
[1] http://lists.xen.org/archives/html/xen-devel/2015-06/msg02591.html
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-15 14:15 ` Julien Grall
@ 2015-07-18 9:44 ` Vijay Kilari
2015-07-18 10:06 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-18 9:44 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Julien
On Wed, Jul 15, 2015 at 7:45 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
>
> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>
[...]
>> +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
>
>
> [...]
>
>> + for ( i = 0; i < pdev->nr_lpis; i++ )
>> + {
>> + plpi = its_get_plpi(pdev, i);
>> + route_irq_to_guest(d, i, plpi, "LPI");
>> + desc = irq_to_desc(plpi);
>> + spin_lock(&desc->lock);
>> + set_irq_device(desc, pdev);
>
>
> This should be part of its_add_device and not its_assign_device.
The domain to which irq should be routed is known only in assign_device.
So route_irq_to_guest is called from assign_device.
I can move set_irq_device to add_device
Regards
Vijay
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device
2015-07-18 9:44 ` Vijay Kilari
@ 2015-07-18 10:06 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-18 10:06 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 18/07/2015 10:44, Vijay Kilari wrote:
> Hi Julien
>
> On Wed, Jul 15, 2015 at 7:45 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>>
>> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>>
> [...]
>>> +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
>>
>>
>> [...]
>>
>>> + for ( i = 0; i < pdev->nr_lpis; i++ )
>>> + {
>>> + plpi = its_get_plpi(pdev, i);
>>> + route_irq_to_guest(d, i, plpi, "LPI");
>>> + desc = irq_to_desc(plpi);
>>> + spin_lock(&desc->lock);
>>> + set_irq_device(desc, pdev);
>>
>>
>> This should be part of its_add_device and not its_assign_device.
>
> The domain to which irq should be routed is known only in assign_device.
> So route_irq_to_guest is called from assign_device.
I know ... I was speaking about set_irq_device.
> I can move set_irq_device to add_device
Yes please, this would be the right place.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 09/17] xen/arm: ITS: Add GITS registers emulation
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (7 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 08/17] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 14:56 ` Ian Campbell
2015-07-15 16:13 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 10/17] xen/arm: ITS: Enable physical and virtual ITS driver compilation vijay.kilari
` (7 subsequent siblings)
16 siblings, 2 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate GITS* registers
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Removed GICR register emulation
---
xen/arch/arm/gic-v3-its.c | 11 ++
xen/arch/arm/vgic-v3-its.c | 319 ++++++++++++++++++++++++++++++++++++-
xen/include/asm-arm/gic-its.h | 10 ++
xen/include/asm-arm/gic_v3_defs.h | 11 ++
4 files changed, 349 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 1d2fdde..5e6c7f2 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -109,6 +109,16 @@ u32 its_get_nr_events(void)
return (1 << its_data.id_bits);
}
+u32 its_get_id_bits(void)
+{
+ return its_data.id_bits;
+}
+
+u32 its_get_dev_bits(void)
+{
+ return its_data.dev_bits;
+}
+
static struct its_node * its_get_phys_node(u32 dev_id)
{
/* TODO: For now return ITS0 node.
@@ -1309,6 +1319,7 @@ static int its_probe(struct dt_device_node *node)
typer = readl_relaxed(its_base + GITS_TYPER);
its->ite_size = ((typer >> 4) & 0xf) + 1;
its_data.id_bits = GITS_TYPER_IDBITS(typer);
+ its_data.dev_bits = GITS_TYPER_DEVBITS(typer);
its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
if ( !its->cmd_base )
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index af2bacd..abf60e2 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -31,7 +31,9 @@
#include <asm/gic-its.h>
#include <xen/log2.h>
-#define DEBUG_ITS
+// #define DEBUG_ITS
+#define VITS_GITS_ITT_SIZE (0x7U << GITS_TYPER_ITT_SIZE_SHIFT)
+#define VITS_GITS_PLPIS (0x1U)
#ifdef DEBUG_ITS
# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
@@ -568,7 +570,7 @@ static int vgic_its_read_virt_cmd(struct vcpu *v,
return 0;
}
-int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
+static int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
{
its_cmd_block virt_cmd;
@@ -593,6 +595,319 @@ err:
return 0;
}
+static inline void vits_spin_lock(struct vgic_its *vits)
+{
+ spin_lock(&vits->lock);
+}
+
+static inline void vits_spin_unlock(struct vgic_its *vits)
+{
+ spin_unlock(&vits->lock);
+}
+
+static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits = v->domain->arch.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ uint64_t val = 0;
+ uint32_t gits_reg;
+
+ gits_reg = info->gpa - vits->gits_base;
+
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vits_spin_lock(vits);
+ *r = vits->ctrl & GITS_CTLR_ENABLE;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_IIDR:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = GICV3_GICD_IIDR_VAL;
+ return 1;
+ case GITS_TYPER:
+ case GITS_TYPER + 4:
+ vits_spin_lock(vits);
+ val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
+ (its_get_dev_bits() - 1) << GITS_TYPER_DEVBITS_SHIFT |
+ (its_get_id_bits() - 1) << GITS_TYPER_IDBITS_SHIFT |
+ VITS_GITS_ITT_SIZE | VITS_GITS_PLPIS);
+ if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
+ *r = val;
+ else if ( dabt.size == DABT_WORD )
+ {
+ if ( (gits_reg % 8) == 0 )
+ *r = (u32)(val >> 32);
+ else
+ *r = (u32)val;
+ }
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- read ignored */
+ goto read_as_zero;
+ case GITS_CBASER:
+ case GITS_CBASER + 4:
+ /* Only read support 32/64-bit access */
+ vits_spin_lock(vits);
+ if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
+ *r = vits->cmd_base;
+ else if ( dabt.size == DABT_WORD )
+ {
+ if ( (gits_reg % 8) == 0 )
+ *r = (u32)vits->cmd_base;
+ else
+ *r = (u32)(vits->cmd_base >> 32);
+ }
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER:
+ case GITS_CWRITER + 4:
+ /* Only read support 32/64-bit access */
+ vits_spin_lock(vits);
+ if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
+ *r = vits->cmd_write;
+ else if ( dabt.size == DABT_WORD )
+ {
+ if ( (gits_reg % 8) == 0 )
+ *r = (u32)vits->cmd_write;
+ else
+ *r = (u32)(vits->cmd_write >> 32);
+ }
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CREADR:
+ case GITS_CREADR + 4:
+ /* Only read support 32/64-bit access */
+ vits_spin_lock(vits);
+ if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
+ *r = vits->cmd_read;
+ else if ( dabt.size == DABT_WORD )
+ {
+ if ( (gits_reg % 8) == 0 )
+ *r = (u32)vits->cmd_read;
+ else
+ *r = (u32)(vits->cmd_read >> 32);
+ }
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- read ignored */
+ goto read_as_zero;
+ case GITS_BASER:
+ /* XXX: Support only 32-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD ||
+ (gits_reg % 8) != 0 )
+ goto bad_width;
+ vits_spin_lock(vits);
+ *r = vits->baser;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_BASER1 ... GITS_BASERN:
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ *r = 0;
+ return 1;
+ case GITS_PIDR0:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR0_VAL;
+ return 1;
+ case GITS_PIDR1:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR1_VAL;
+ return 1;
+ case GITS_PIDR2:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR2_VAL;
+ return 1;
+ case GITS_PIDR3:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR3_VAL;
+ return 1;
+ case GITS_PIDR4:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR4_VAL;
+ return 1;
+ case GITS_PIDR5 ... GITS_PIDR7:
+ goto read_as_zero_32;
+ default:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: unhandled read r%d offset %#08x\n",
+ v, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: bad read width %d r%d offset %#08x\n",
+ v, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+read_as_zero_32:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+read_as_zero:
+ *r = 0;
+ return 1;
+}
+
+/*
+ * GITS_BASER.Type[58:56],GITS_BASER.Entry_size[55:48]
+ * and GITS_BASER.Shareability[11:10] are read-only.
+ * Mask those fields while emulating GITS_BASER reg.
+ */
+#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
+ (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
+ (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
+
+static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits = v->domain->arch.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ int ret;
+ uint32_t gits_reg, sz, psz;
+
+ gits_reg = info->gpa - vits->gits_base;
+
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ vits->ctrl = *r & GITS_CTLR_ENABLE;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_IIDR:
+ /* R0 -- write ignored */
+ goto write_ignore;
+ case GITS_TYPER:
+ case GITS_TYPER + 4:
+ /* R0 -- write ignored */
+ goto write_ignore;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- write ignored */
+ goto write_ignore;
+ case GITS_CBASER:
+ /* XXX: support 32-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ vits->cmd_base = *r;
+ vits->cmd_qsize = SZ_4K * ((*r & GITS_BASER_PAGES_MASK_VAL) + 1);
+ if ( vits->cmd_qsize & GITS_BASER_VALID )
+ vits->cmd_read = 0;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CBASER + 4:
+ /* XXX: Does not support word write */
+ goto bad_width;
+ case GITS_CWRITER:
+ if ( dabt.size == DABT_BYTE ) goto bad_width;
+ /* XXX: Validate val */
+ vits_spin_lock(vits);
+ vits->cmd_write = *r & 0xfffe0;
+ if ( !(vits->ctrl & GITS_CTLR_ENABLE) )
+ return 1;
+ ret = vgic_its_process_cmd(v, vits);
+ vits_spin_unlock(vits);
+ return ret;
+ case GITS_CWRITER + 4:
+ if (dabt.size != DABT_WORD ) goto bad_width;
+ return 1;
+ case GITS_CREADR:
+ /* R0 -- write ignored */
+ goto write_ignore;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- write ignored */
+ goto write_ignore;
+ case GITS_BASER0:
+ /* XXX: Support 32-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ vits->baser = vits->baser | (*r & GITS_BASER_MASK);
+ vits->dt_ipa = vits->baser & BIT_48_12_MASK;
+ psz = (vits->baser >> GITS_BASER_PAGE_SIZE_SHIFT) &
+ GITS_BASER_PAGE_SIZE_MASK_VAL;
+ if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
+ sz = 4;
+ else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
+ sz = 16;
+ else
+ sz = 64;
+
+ vits->dt_size = (vits->baser & GITS_BASER_PAGES_MASK_VAL)
+ * sz * SZ_1K;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_BASER1 ... GITS_BASERN:
+ goto write_ignore_64;
+ case GITS_PIDR7 ... GITS_PIDR0:
+ /* R0 -- write ignored */
+ goto write_ignore_32;
+ default:
+ dprintk(XENLOG_G_ERR, "%pv vITS: unhandled write r%d offset %#08x\n",
+ v, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: bad write width %d r%d offset %#08x\n",
+ v, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+write_ignore_64:
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ return 1;
+write_ignore_32:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ return 1;
+write_ignore:
+ *r = 0;
+ return 1;
+}
+
+
+static const struct mmio_handler_ops vgic_gits_mmio_handler = {
+ .read_handler = vgic_v3_gits_mmio_read,
+ .write_handler = vgic_v3_gits_mmio_write,
+};
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index f041efc..9c004c2 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -37,6 +37,8 @@ struct its_collection {
struct vgic_its
{
spinlock_t lock;
+ /* Emulation of BASER0 */
+ paddr_t baser;
/* Command queue base */
paddr_t cmd_base;
/* Command queue write pointer */
@@ -47,6 +49,12 @@ struct vgic_its
paddr_t cmd_read;
/* Command queue size */
unsigned long cmd_qsize;
+ /* ITS mmio physical base */
+ paddr_t gits_base;
+ /* ITS mmio physical size */
+ unsigned long gits_size;
+ /* GICR ctrl register */
+ uint32_t ctrl;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
@@ -237,6 +245,8 @@ struct gic_its_info {
uint32_t dev_bits;
};
+u32 its_get_id_bits(void);
+u32 its_get_dev_bits(void);
u32 its_get_nr_events(void);
int its_lpi_init(u32 id_bits);
int its_init(struct rdist_prop *rdists);
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 0443ae7..84366df 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -202,6 +202,7 @@
#define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
#define GITS_TYPER_PTA (1UL << 19)
#define GITS_TYPER_HCC_SHIFT (24)
+#define GITS_TYPER_ITT_SIZE_SHIFT (4)
#define GITS_CBASER_VALID (1UL << 63)
#define GITS_CBASER_nC (1UL << 59)
@@ -228,6 +229,10 @@
#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_4K_VAL (0)
+#define GITS_BASER_PAGE_SIZE_16K_VAL (1)
+#define GITS_BASER_PAGE_SIZE_MASK_VAL (0x3)
+#define GITS_BASER_PAGES_MASK_VAL (0xff)
#define GITS_BASER_TYPE_NONE 0
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_VCPU 2
@@ -236,6 +241,12 @@
#define GITS_BASER_TYPE_RESERVED5 5
#define GITS_BASER_TYPE_RESERVED6 6
#define GITS_BASER_TYPE_RESERVED7 7
+/* GITS_PIDRn register values for ARM implementations */
+#define GITS_PIDR0_VAL (0x94)
+#define GITS_PIDR1_VAL (0xb4)
+#define GITS_PIDR2_VAL (0x3b)
+#define GITS_PIDR3_VAL (0x00)
+#define GITS_PIDR4_VAL (0x04)
/*
* ITS commands
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 09/17] xen/arm: ITS: Add GITS registers emulation
2015-07-10 7:42 ` [PATCH v4 09/17] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-07-10 14:56 ` Ian Campbell
2015-07-15 16:13 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 14:56 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> + case GITS_CWRITER:
> + if ( dabt.size == DABT_BYTE ) goto bad_width;
> + /* XXX: Validate val */
> + vits_spin_lock(vits);
> + vits->cmd_write = *r & 0xfffe0;
> + if ( !(vits->ctrl & GITS_CTLR_ENABLE) )
> + return 1;
You are returning with the lock held here.
I suggest
if ( vits->ctrl & GITS_CTLR_ENABLE )
ret = vits_its_process_cmd(...)
else
return 1;
unlock
return ret;
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 09/17] xen/arm: ITS: Add GITS registers emulation
2015-07-10 7:42 ` [PATCH v4 09/17] xen/arm: ITS: Add GITS registers emulation vijay.kilari
2015-07-10 14:56 ` Ian Campbell
@ 2015-07-15 16:13 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 16:13 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Emulate GITS* registers
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Removed GICR register emulation
> ---
> xen/arch/arm/gic-v3-its.c | 11 ++
> xen/arch/arm/vgic-v3-its.c | 319 ++++++++++++++++++++++++++++++++++++-
> xen/include/asm-arm/gic-its.h | 10 ++
> xen/include/asm-arm/gic_v3_defs.h | 11 ++
> 4 files changed, 349 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 1d2fdde..5e6c7f2 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -109,6 +109,16 @@ u32 its_get_nr_events(void)
> return (1 << its_data.id_bits);
> }
>
> +u32 its_get_id_bits(void)
> +{
> + return its_data.id_bits;
> +}
> +
> +u32 its_get_dev_bits(void)
> +{
> + return its_data.dev_bits;
> +}
> +
Please give a look to the new vgic infrastructure in order to avoid
introduced helper to pass data to the vgic.
See for instance vgic_v3_setup_hw.
> static struct its_node * its_get_phys_node(u32 dev_id)
> {
> /* TODO: For now return ITS0 node.
> @@ -1309,6 +1319,7 @@ static int its_probe(struct dt_device_node *node)
> typer = readl_relaxed(its_base + GITS_TYPER);
> its->ite_size = ((typer >> 4) & 0xf) + 1;
> its_data.id_bits = GITS_TYPER_IDBITS(typer);
> + its_data.dev_bits = GITS_TYPER_DEVBITS(typer);
>
> its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
> if ( !its->cmd_base )
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index af2bacd..abf60e2 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -31,7 +31,9 @@
> #include <asm/gic-its.h>
> #include <xen/log2.h>
>
> -#define DEBUG_ITS
> +// #define DEBUG_ITS
This changes should be merged in the patch which add #define DEBUG_ITS
(i.e #7).
> +#define VITS_GITS_ITT_SIZE (0x7U << GITS_TYPER_ITT_SIZE_SHIFT)
As said on v3, this name is misleading, this is not the size of the ITT
but the value to store in the GITS_TYPER register.
In any case, you should not hardcode the size of the entry and use
sizeof(struct vitt) as you did well for the device and other things.
> +#define VITS_GITS_PLPIS (0x1U)
Please introduce GITS_TYPER_PHYSICAL rather than hardcoding the value
for the vITS.
>
> #ifdef DEBUG_ITS
> # define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
> @@ -568,7 +570,7 @@ static int vgic_its_read_virt_cmd(struct vcpu *v,
> return 0;
> }
>
> -int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
> +static int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
The static belongs to the patch where vgic_its_process_cmd has been
added (i.e patch #7).
> {
> its_cmd_block virt_cmd;
>
> @@ -593,6 +595,319 @@ err:
> return 0;
> }
>
> +static inline void vits_spin_lock(struct vgic_its *vits)
> +{
> + spin_lock(&vits->lock);
> +}
> +
> +static inline void vits_spin_unlock(struct vgic_its *vits)
> +{
> + spin_unlock(&vits->lock);
> +}
> +
> +static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits = v->domain->arch.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + uint64_t val = 0;
> + uint32_t gits_reg;
> +
> + gits_reg = info->gpa - vits->gits_base;
> +
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + vits_spin_lock(vits);
> + *r = vits->ctrl & GITS_CTLR_ENABLE;
The & GITS_CTLR_ENABLE is not necessary. You already ensure in the write
emulation that only this bit saved. So
*r = vits->ctlr;
> + vits_spin_unlock(vits);
> + return 1;
[..]
> + case GITS_TYPER:
> + case GITS_TYPER + 4:
> + vits_spin_lock(vits);
Why the lock? AFAICT you don't touch the vITS structure.
Also, as said on v3, please add a comment explaining the configuration
of this field. It would avoid to scratch his head trying to understand
why PTA is not mentioned...
> + val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
I would prefer to see a macro computing the number of collection.
Although, HCC is only able to encore 16-bit of collection ID (i.e 256
collection => 255 VCPUs). Please make a note and add a BUILD_ON_BUG or
anything else to ensure that it will never happen within this register
emulation (i.e GITS_TYPER).
> + (its_get_dev_bits() - 1) << GITS_TYPER_DEVBITS_SHIFT |
> + (its_get_id_bits() - 1) << GITS_TYPER_IDBITS_SHIFT |
Please introduce fields in the vits structure to store the number of
devID and eventID bits.
> + VITS_GITS_ITT_SIZE | VITS_GITS_PLPIS);
> + if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
Why the gits_reg % 8 ? AFAICT, 64-bit access should always be aligned
when trapped in Xen. I did some test and wasn't able to proof the invert.
If it's not true we have to fix for every IO handler and not only this
one as all the emulation assume a such things. So it has to be done
generically.
> + *r = val;
> + else if ( dabt.size == DABT_WORD )
> + {
> + if ( (gits_reg % 8) == 0 )
> + *r = (u32)(val >> 32);
> + else
> + *r = (u32)val;
> + }
You duplicate this code in a lot of register emulation and this code is
quite hard to read. This is a call to introduce an helper.
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- read ignored */
> + goto read_as_zero;
> + case GITS_CBASER:
> + case GITS_CBASER + 4:
> + /* Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_base;
> + else if ( dabt.size == DABT_WORD )
> + {
> + if ( (gits_reg % 8) == 0 )
> + *r = (u32)vits->cmd_base;
> + else
> + *r = (u32)(vits->cmd_base >> 32);
> + }
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
You could have avoid the double unlock by saving the vits->cmd_base in a
temporary value i.e
vits_spin_lock(vits)
val = vits->cmd_base;
vits_spin_unlock(vits)
if ( .... )
...
> + return 1;
> + case GITS_CWRITER:
> + case GITS_CWRITER + 4:
> + /* Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_write;
> + else if ( dabt.size == DABT_WORD )
> + {
> + if ( (gits_reg % 8) == 0 )
> + *r = (u32)vits->cmd_write;
> + else
> + *r = (u32)(vits->cmd_write >> 32);
> + }
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
Ditto
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CREADR:
> + case GITS_CREADR + 4:
> + /* Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( (gits_reg % 8) == 0 && dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_read;
> + else if ( dabt.size == DABT_WORD )
> + {
> + if ( (gits_reg % 8) == 0 )
> + *r = (u32)vits->cmd_read;
> + else
> + *r = (u32)(vits->cmd_read >> 32);
> + }
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
ditto
> + vits_spin_unlock(vits);
> + return 1;
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- read ignored */
> + goto read_as_zero;
> + case GITS_BASER:
Please use GITS_BASER0 rather than GITS_RATHER. It's more meaningful.
> + /* XXX: Support only 32-bit access */
This comment is wrong. You only support 64-bit access.
If you add the helper suggested above to read either 32-bit or 64-bit,
it would make the 32-bit support for GITS_BASER free.
> + if ( dabt.size != DABT_DOUBLE_WORD ||
> + (gits_reg % 8) != 0 )
The second part of the condition is not useful.
> + goto bad_width;
> + vits_spin_lock(vits);
> + *r = vits->baser;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_BASER1 ... GITS_BASERN:
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> + *r = 0;
> + return 1;
The *r = 0 ... return 1 could be replaced by goto read_as_zero;
[...]
> +/*
> + * GITS_BASER.Type[58:56],GITS_BASER.Entry_size[55:48]
> + * and GITS_BASER.Shareability[11:10] are read-only.
Implemented Shareability as fixed (i.e read-only) is deprecated. I don't
mind if we do it for now. Although I'd like to see a comment about it here.
> + * Mask those fields while emulating GITS_BASER reg.
> + */
Other fields are (or could be RO) in GITS_BASER:
- Indirect: we only support flat table
- Page_Size: it's fine to only support 4KB granularity. It also means
less code.
I don't mind if you don't do the later.
> +#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
> + (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
> + (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
> +
> +static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits = v->domain->arch.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + int ret;
> + uint32_t gits_reg, sz, psz;
> +
> + gits_reg = info->gpa - vits->gits_base;
> +
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->ctrl = *r & GITS_CTLR_ENABLE;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_IIDR:
> + /* R0 -- write ignored */
The field is read-only not read-as-zero. Please use RO rather than R0.
> + goto write_ignore;
> + case GITS_TYPER:
> + case GITS_TYPER + 4:
> + /* R0 -- write ignored */
s/R0/RO/
> + goto write_ignore;
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- write ignored */
> + goto write_ignore;
> + case GITS_CBASER:
GITS_CBASER is read-only when GITS_CTLR.Enable is Zero or
GITS_CTLR.Quiescent is zero.
Please handle this case.
> + /* XXX: support 32-bit access */
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->cmd_base = *r;
> + vits->cmd_qsize = SZ_4K * ((*r & GITS_BASER_PAGES_MASK_VAL) + 1);
> + if ( vits->cmd_qsize & GITS_BASER_VALID )
> + vits->cmd_read = 0;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CBASER + 4:
> + /* XXX: Does not support word write */
> + goto bad_width;
You can drop this case, it's not necessary.
> + case GITS_CWRITER:
> + if ( dabt.size == DABT_BYTE ) goto bad_width;
> + /* XXX: Validate val */
It's not so hard to check the validity of the offset... Please do it.
> + vits_spin_lock(vits);
> + vits->cmd_write = *r & 0xfffe0;
Can you add a comment explaining why the 0xfffe0? i.e the only bit
[19:5] are writeable.
> + if ( !(vits->ctrl & GITS_CTLR_ENABLE) )
> + return 1;
> + ret = vgic_its_process_cmd(v, vits);
> + vits_spin_unlock(vits);
> + return ret;
> + case GITS_CWRITER + 4:
> + if (dabt.size != DABT_WORD ) goto bad_width;
I had to go through the previous review in order to understand why you
do that. Please add a comment to say that the top half of the register
is read-only.
> + return 1;
> + case GITS_CREADR:
> + /* R0 -- write ignored */
s/R0/RO/
> + goto write_ignore;
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- write ignored */
> + goto write_ignore;
> + case GITS_BASER0:
GITS_CBASER is read-only when GITS_CTLR.Enable is Zero or
GITS_CTLR.Quiescent is zero.
It's quite important to do it as, AFAICT, you don't have any protection
on the usage of dt_ipa in vits_get_vdevice_entry.
Although this won't protect everything as it may be possible to receive
an LPI before the GITS is effectively setup (see patch #8) or because
the guest decided to clear GITS_CRTL.Enable.
> + /* XXX: Support 32-bit access */
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->baser = vits->baser | (*r & GITS_BASER_MASK);
As said on v3, this doesn't do what you want. It's an or operation so if
bit is 0 in the *r it won't be clear in the resulting vits->baser.
The correct solution would be:
vits->baser &= ~GITS_BASER_MASK;
vits->baser |= *r | GITS_BASER_MASK;
> + vits->dt_ipa = vits->baser & BIT_48_12_MASK;
> + psz = (vits->baser >> GITS_BASER_PAGE_SIZE_SHIFT) &
> + GITS_BASER_PAGE_SIZE_MASK_VAL;
> + if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
> + sz = 4;
> + else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
> + sz = 16;
> + else
Please add a comment to explain that 0x11 is treated as 0x10 i.e 64KB.
> + sz = 64;
> +
> + vits->dt_size = (vits->baser & GITS_BASER_PAGES_MASK_VAL)
> + * sz * SZ_1K;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_BASER1 ... GITS_BASERN:
Please handle the 32-bit access here. I.e smth like
if ( !DOUBLE_WORD || !WORD )
goto bad_write
return 1;
> + goto write_ignore_64;
> + case GITS_PIDR7 ... GITS_PIDR0:
> + /* R0 -- write ignored */
> + goto write_ignore_32;
> + default:
> + dprintk(XENLOG_G_ERR, "%pv vITS: unhandled write r%d offset %#08x\n",
> + v, dabt.reg, gits_reg);
> + return 0;
> + }
> +
> +bad_width:
> + dprintk(XENLOG_G_ERR, "%pv: vITS: bad write width %d r%d offset %#08x\n",
> + v, dabt.size, dabt.reg, gits_reg);
> + domain_crash_synchronous();
> + return 0;
> +
> +write_ignore_64:
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> + return 1;
> +write_ignore_32:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + return 1;
> +write_ignore:
> + *r = 0;
> + return 1;
> +}
> +
> +
> +static const struct mmio_handler_ops vgic_gits_mmio_handler = {
> + .read_handler = vgic_v3_gits_mmio_read,
> + .write_handler = vgic_v3_gits_mmio_write,
> +};
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index f041efc..9c004c2 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -37,6 +37,8 @@ struct its_collection {
> struct vgic_its
> {
> spinlock_t lock;
> + /* Emulation of BASER0 */
> + paddr_t baser;
Please rename it to baser0.
> /* Command queue base */
> paddr_t cmd_base;
> /* Command queue write pointer */
> @@ -47,6 +49,12 @@ struct vgic_its
> paddr_t cmd_read;
> /* Command queue size */
> unsigned long cmd_qsize;
> + /* ITS mmio physical base */
> + paddr_t gits_base;
> + /* ITS mmio physical size */
> + unsigned long gits_size;
I don't see any usage of gits_size.
> + /* GICR ctrl register */
> + uint32_t ctrl;
> /* vITT device table ipa */
> paddr_t dt_ipa;
> /* vITT device table size */
> @@ -237,6 +245,8 @@ struct gic_its_info {
> uint32_t dev_bits;
> };
[..]
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 0443ae7..84366df 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -202,6 +202,7 @@
> #define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
> #define GITS_TYPER_PTA (1UL << 19)
> #define GITS_TYPER_HCC_SHIFT (24)
> +#define GITS_TYPER_ITT_SIZE_SHIFT (4)
>
> #define GITS_CBASER_VALID (1UL << 63)
> #define GITS_CBASER_nC (1UL << 59)
> @@ -228,6 +229,10 @@
> #define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
> #define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
> #define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_4K_VAL (0)
> +#define GITS_BASER_PAGE_SIZE_16K_VAL (1)
> +#define GITS_BASER_PAGE_SIZE_MASK_VAL (0x3)
> +#define GITS_BASER_PAGES_MASK_VAL (0xff)
> #define GITS_BASER_TYPE_NONE 0
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_VCPU 2
> @@ -236,6 +241,12 @@
> #define GITS_BASER_TYPE_RESERVED5 5
> #define GITS_BASER_TYPE_RESERVED6 6
> #define GITS_BASER_TYPE_RESERVED7 7
> +/* GITS_PIDRn register values for ARM implementations */
> +#define GITS_PIDR0_VAL (0x94)
> +#define GITS_PIDR1_VAL (0xb4)
> +#define GITS_PIDR2_VAL (0x3b)
> +#define GITS_PIDR3_VAL (0x00)
> +#define GITS_PIDR4_VAL (0x04)
Those new defines are part of the vITS implementation and should not be
exposed ouside of vgic-v3-its.c.
FWIW, this is what we do for GICv3 and GICv2.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 10/17] xen/arm: ITS: Enable physical and virtual ITS driver compilation
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (8 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 09/17] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-15 16:16 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation vijay.kilari
` (6 subsequent siblings)
16 siblings, 1 reply; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Compilation is delayed till this patch.
>From now on functions in physical ITS and virtual ITS
driver are required. So enable compilation
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 935999e..048a762 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -14,6 +14,7 @@ obj-y += domain_build.o
obj-y += gic.o gic-v2.o
obj-$(CONFIG_ARM_32) += gic-hip04.o
obj-$(CONFIG_ARM_64) += gic-v3.o
+obj-$(CONFIG_ARM_64) += gic-v3-its.o
obj-y += io.o
obj-y += irq.o
obj-y += kernel.o
@@ -32,6 +33,7 @@ obj-y += shutdown.o
obj-y += traps.o
obj-y += vgic.o vgic-v2.o
obj-$(CONFIG_ARM_64) += vgic-v3.o
+obj-$(CONFIG_ARM_64) += vgic-v3-its.o
obj-y += vtimer.o
obj-y += vuart.o
obj-y += hvm.o
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 10/17] xen/arm: ITS: Enable physical and virtual ITS driver compilation
2015-07-10 7:42 ` [PATCH v4 10/17] xen/arm: ITS: Enable physical and virtual ITS driver compilation vijay.kilari
@ 2015-07-15 16:16 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 16:16 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Compilation is delayed till this patch.
> From now on functions in physical ITS and virtual ITS
What do you mean by "on"?
> driver are required. So enable compilation
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (9 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 10/17] xen/arm: ITS: Enable physical and virtual ITS driver compilation vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 15:10 ` Ian Campbell
` (2 more replies)
2015-07-10 7:42 ` [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route vijay.kilari
` (5 subsequent siblings)
16 siblings, 3 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate LPI related changes to GICR registers
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Added LPI configuration table emulation
- Rename function inline with vits
- Copied guest lpi configuration table to xen
---
xen/arch/arm/gic-v3.c | 15 ++++
xen/arch/arm/gic.c | 10 +++
xen/arch/arm/vgic-v3-its.c | 165 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/vgic-v3.c | 85 +++++++++++++++++--
xen/arch/arm/vgic.c | 4 +
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/gic-its.h | 11 +++
xen/include/asm-arm/gic.h | 9 ++
xen/include/asm-arm/gic_v3_defs.h | 3 +
9 files changed, 295 insertions(+), 8 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 904fe57..e6004d2 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -677,6 +677,11 @@ static int __init gicv3_populate_rdist(void)
return -ENODEV;
}
+static int gicv3_dist_supports_lpis(void)
+{
+ return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
+}
+
static int __cpuinit gicv3_cpu_init(void)
{
int i;
@@ -1293,10 +1298,20 @@ static int __init gicv3_init(void)
gicv3.rdist_regions[0].size, gicv3.rdist_regions[0].map_base,
gicv3_info.maintenance_irq);
+ reg = readl_relaxed(GICD + GICD_TYPER);
+
+ gicv3.rdist_data.id_bits = ((reg >> 19) & 0x1f) + 1;
+ gicv3_info.nr_id_bits = gicv3.rdist_data.id_bits;
+
spin_lock_init(&gicv3.lock);
spin_lock(&gicv3.lock);
+ if ( gicv3_dist_supports_lpis() )
+ gicv3_info.lpi_supported = 1;
+ else
+ gicv3_info.lpi_supported = 0;
+
gicv3_dist_init();
res = gicv3_cpu_init();
gicv3_hyp_init();
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 4f3801b..3ebadcf 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -73,6 +73,16 @@ unsigned int gic_number_lines(void)
return gic_hw_ops->info->nr_lines;
}
+unsigned int gic_nr_id_bits(void)
+{
+ return gic_hw_ops->info->nr_id_bits;
+}
+
+bool_t gic_lpi_supported(void)
+{
+ return gic_hw_ops->info->lpi_supported;
+}
+
void gic_save_state(struct vcpu *v)
{
ASSERT(!local_irq_is_enabled());
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index abf60e2..bbcc7bb 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -49,6 +49,36 @@ static void dump_cmd(its_cmd_block *cmd)
}
#endif
+static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
+{
+ struct pending_irq *p;
+
+ p = irq_to_pending(v, vlpi);
+ clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+ gic_remove_from_queues(v, vlpi);
+}
+
+static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
+{
+ struct pending_irq *p;
+ unsigned long flags;
+
+ /* Get plpi for the given vlpi */
+ p = irq_to_pending(v, vlpi);
+
+ set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+
+ spin_lock_irqsave(&v->arch.vgic.lock, flags);
+
+ /*XXX: raise on right vcpu */
+ if ( !list_empty(&p->inflight) &&
+ !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
+ gic_raise_guest_irq(v, irq_to_virq(p->desc), p->priority);
+
+ spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+}
+
+/* ITS device table helper functions */
static int vits_entry(struct domain *d, paddr_t entry, void *addr,
uint32_t size, bool_t set)
{
@@ -595,6 +625,141 @@ err:
return 0;
}
+static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+
+ offset = info->gpa -
+ (v->domain->arch.vits->propbase & BIT_48_12_MASK);
+
+ if ( offset < v->domain->arch.vits->prop_size )
+ {
+ DPRINTK("%pv: vITS: LPI Table read offset 0x%x\n", v, offset);
+ spin_lock(&v->domain->arch.vits->prop_lock);
+ *r = *((u8*)v->domain->arch.vits->prop_page + offset);
+ spin_unlock(&v->domain->arch.vits->prop_lock);
+ return 1;
+ }
+ else
+ dprintk(XENLOG_G_ERR, "%pv: vITS: LPI Table read with wrong offset 0x%x\n",
+ v, offset);
+
+ return 0;
+}
+
+static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset;
+ uint32_t vid;
+ uint8_t cfg, *p;
+ bool_t enable;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+
+ offset = info->gpa -
+ (v->domain->arch.vits->propbase & BIT_48_12_MASK);
+
+ vid = offset + NR_GIC_LPI;
+ if ( offset < v->domain->arch.vits->prop_size )
+ {
+ DPRINTK("%pv: vITS: LPI Table write offset 0x%x\n", v, offset);
+
+ spin_lock(&v->domain->arch.vits->prop_lock);
+ p = ((u8*)v->domain->arch.vits->prop_page + offset);
+ cfg = *p;
+ enable = (cfg & *r) & 0x1;
+
+ if ( !enable )
+ vits_enable_lpi(v, vid, (*r & LPI_PRIORITY_MASK));
+ else
+ vits_disable_lpi(v, vid);
+
+ /* Update virtual prop page */
+ *p = (*r & 0xff);
+ spin_unlock(&v->domain->arch.vits->prop_lock);
+ return 1;
+ }
+ else
+ dprintk(XENLOG_G_ERR, "%pv: vITS: LPI Table invalid write @ 0x%x\n",
+ v, offset);
+
+ return 0;
+}
+
+static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
+ .read_handler = vgic_v3_gits_lpi_mmio_read,
+ .write_handler = vgic_v3_gits_lpi_mmio_write,
+};
+
+int vits_unmap_lpi_prop(struct vcpu *v)
+{
+ paddr_t maddr, addr;
+ unsigned long mfn;
+ uint32_t lpi_size, id_bits;
+ int i;
+
+ maddr = v->domain->arch.vits->propbase & BIT_48_12_MASK;
+ id_bits = ((v->domain->arch.vits->propbase & GICR_PROPBASER_IDBITS_MASK)+1);
+
+ DPRINTK("%pv: vITS: Unmap guest LPI conf table maddr 0x%lx lpi_size 0x%x\n",
+ v, maddr, lpi_size);
+
+ spin_lock(&v->domain->arch.vits->prop_lock);
+ if ( id_bits > gic_nr_id_bits() )
+ id_bits = gic_nr_id_bits();
+
+ lpi_size = 1UL << id_bits;
+
+ v->domain->arch.vits->prop_size = lpi_size;
+ /* Allocate Virtual LPI Property table */
+ /* TODO: To re-use guest property table? */
+ v->domain->arch.vits->prop_page =
+ alloc_xenheap_pages(get_order_from_bytes(lpi_size), 0);
+ if ( !v->domain->arch.vits->prop_page )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Fail to allocate LPI Prop page\n", v);
+ return 0;
+ }
+
+ addr = maddr;
+ for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
+ {
+ vits_entry(v->domain, addr,
+ (void *)(v->domain->arch.vits->prop_page + i * PAGE_SIZE),
+ PAGE_SIZE, 0);
+ addr += PAGE_SIZE;
+ }
+
+ /*
+ * Each re-distributor shares a common LPI configuration table
+ * So one set of mmio handlers to manage configuration table is enough
+ */
+ addr = maddr;
+ for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
+ {
+ mfn = gmfn_to_mfn(v->domain, paddr_to_pfn(addr));
+ if ( unlikely(!mfn_valid(mfn)) )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Wrong propbaser address\n", v);
+ return 0;
+ }
+ guest_physmap_remove_page(v->domain, paddr_to_pfn(addr), mfn, 0);
+ addr += PAGE_SIZE;
+ }
+
+ /* Register mmio handlers for this region */
+ register_mmio_handler(v->domain, &vgic_gits_lpi_mmio_handler,
+ maddr, lpi_size);
+
+ spin_unlock(&v->domain->arch.vits->prop_lock);
+
+ return 1;
+}
+
static inline void vits_spin_lock(struct vgic_its *vits)
{
spin_lock(&vits->lock);
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 4af5a84..25b69a0 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -30,6 +30,7 @@
#include <asm/mmio.h>
#include <asm/gic_v3_defs.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
/* GICD_PIDRn register values for ARM implementations */
@@ -93,7 +94,18 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case GICR_CTLR:
- /* We have not implemented LPI's, read zero */
+ /*
+ * Enable LPI's for ITS. Direct injection of LPI
+ * by writing to GICR_{SET,CLR}LPIR are not supported
+ */
+ if ( gic_lpi_supported() )
+ {
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vgic_lock(v);
+ *r = v->domain->arch.vgic.gicr_ctlr;
+ vgic_unlock(v);
+ return 1;
+ }
goto read_as_zero_32;
case GICR_IIDR:
if ( dabt.size != DABT_WORD ) goto bad_width;
@@ -106,11 +118,16 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
+ if ( gic_lpi_supported() )
+ {
+ /* Set LPI support */
+ aff |= GICR_TYPER_PLPIS;
+ /* GITS_TYPER.PTA is 0. Provice vcpu number as ta */
+ aff |= (v->vcpu_id << GICR_TYPER_PROCESSOR_SHIFT);
+ }
*r = aff;
-
if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
*r |= GICR_TYPER_LAST;
-
return 1;
case GICR_STATUSR:
/* Not implemented */
@@ -125,10 +142,21 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
/* WO. Read as zero */
goto read_as_zero_64;
case GICR_PROPBASER:
- /* LPI's not implemented */
+ if ( gic_lpi_supported() )
+ {
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ /* Remove shareability attribute we don't want dom to flush */
+ *r = v->domain->arch.vits->propbase;
+ return 1;
+ }
goto read_as_zero_64;
case GICR_PENDBASER:
- /* LPI's not implemented */
+ if ( gic_lpi_supported() )
+ {
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ *r = v->domain->arch.vits->pendbase[v->vcpu_id];
+ return 1;
+ }
goto read_as_zero_64;
case GICR_INVLPIR:
/* WO. Read as zero */
@@ -203,7 +231,18 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case GICR_CTLR:
- /* LPI's not implemented */
+ if ( gic_lpi_supported() )
+ {
+ /*
+ * Enable LPI's for ITS. Direct injection of LPI
+ * by writing to GICR_{SET,CLR}LPIR are not supported
+ */
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vgic_lock(v);
+ v->domain->arch.vgic.gicr_ctlr = (*r) & GICR_CTLR_ENABLE_LPIS;
+ vgic_unlock(v);
+ return 1;
+ }
goto write_ignore_32;
case GICR_IIDR:
/* RO */
@@ -224,10 +263,32 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
/* LPI is not implemented */
goto write_ignore_64;
case GICR_PROPBASER:
- /* LPI is not implemented */
+ if ( gic_lpi_supported() )
+ {
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ vgic_lock(v);
+ /* LPI configuration tables are shared across cpus. Should be same */
+ /* TODO: Manage change in property table */
+ if ( v->domain->arch.vits->propbase != 0 )
+ {
+ vgic_unlock(v);
+ return 1;
+ }
+ v->domain->arch.vits->propbase = *r;
+ vgic_unlock(v);
+ return vits_unmap_lpi_prop(v);
+ }
goto write_ignore_64;
case GICR_PENDBASER:
- /* LPI is not implemented */
+ if ( gic_lpi_supported() )
+ {
+ /* Just hold pendbaser value for guest read */
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ vgic_lock(v);
+ v->domain->arch.vits->pendbase[v->vcpu_id] = *r;
+ vgic_unlock(v);
+ return 1;
+ }
goto write_ignore_64;
case GICR_INVLPIR:
/* LPI is not implemented */
@@ -694,6 +755,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
*r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
+ if ( gic_lpi_supported() )
+ {
+ irq_bits = gic_nr_id_bits();
+ *r |= GICD_TYPE_LPIS;
+ }
+ else
+ irq_bits = get_count_order(vgic_num_irqs(v->domain));
+
*r |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
return 1;
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 73a6f7e..a5f66f6 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -154,6 +154,10 @@ void domain_vgic_free(struct domain *d)
xfree(d->arch.vgic.shared_irqs);
xfree(d->arch.vgic.pending_irqs);
xfree(d->arch.vgic.allocated_irqs);
+#ifdef CONFIG_ARM_64
+ free_xenheap_pages(d->arch.vits->prop_page,
+ get_order_from_bytes(d->arch.vits->prop_size));
+#endif
}
int vcpu_vgic_init(struct vcpu *v)
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 67e4695..49db7f0 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -102,6 +102,7 @@ struct arch_domain
paddr_t dbase; /* Distributor base address */
paddr_t cbase; /* CPU base address */
#ifdef CONFIG_ARM_64
+ int gicr_ctlr;
/* GIC V3 addressing */
paddr_t dbase_size; /* Distributor base size */
/* List of contiguous occupied by the redistributors */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 9c004c2..a79b70f 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -55,6 +55,16 @@ struct vgic_its
unsigned long gits_size;
/* GICR ctrl register */
uint32_t ctrl;
+ /* LPI propbase */
+ paddr_t propbase;
+ /* percpu pendbase */
+ paddr_t pendbase[MAX_VIRT_CPUS];
+ /* Virtual LPI property table */
+ void *prop_page;
+ /* Virtual LPI property size */
+ uint64_t prop_size;
+ /* spinlock to protect lpi property table */
+ spinlock_t prop_lock;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
@@ -266,6 +276,7 @@ int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
struct vits_device *vits_find_device(struct rb_root *root, uint32_t devid);
int vits_insert_device(struct rb_root *root, struct vits_device *dev);
void vits_remove_device(struct rb_root *root, struct vits_device *dev);
+int vits_unmap_lpi_prop(struct vcpu *v);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index fdd96c8..69bf1ff 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -100,6 +100,7 @@
#define GICD_TYPE_CPUS_SHIFT 5
#define GICD_TYPE_CPUS 0x0e0
#define GICD_TYPE_SEC 0x400
+#define GICD_TYPE_LPIS (0x1UL << 17)
#define GICC_CTL_ENABLE 0x1
#define GICC_CTL_EOI (0x1 << 9)
@@ -283,6 +284,10 @@ extern void gic_dump_info(struct vcpu *v);
/* Number of interrupt lines */
extern unsigned int gic_number_lines(void);
+/* Number of interrupt id bits supported */
+extern unsigned int gic_nr_id_bits(void);
+/* LPI support info */
+bool_t gic_lpi_supported(void);
void gicv3_eoi_irq(struct irq_desc *irqd);
void gicv3_dir_irq(struct irq_desc *irqd);
@@ -302,6 +307,10 @@ struct gic_info {
unsigned int maintenance_irq;
/* Pointer to the device tree node representing the interrupt controller */
const struct dt_device_node *node;
+ /* Number of IRQ ID bits supported */
+ uint32_t nr_id_bits;
+ /* LPIs are support information */
+ bool_t lpi_supported;
};
struct gic_hw_operations {
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 84366df..368ebb3 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -127,6 +127,7 @@
#define GICR_TYPER_PLPIS (1U << 0)
#define GICR_TYPER_VLPIS (1U << 1)
#define GICR_TYPER_LAST (1U << 4)
+#define GICR_TYPER_PROCESSOR_SHIFT (8)
#define GICR_PENDBASER_InnerShareable (1U << 10)
#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
@@ -170,6 +171,7 @@
#define ICH_SGI_TARGETLIST_MASK 0xffff
#define LPI_PROP_GROUP1 (1 << 1)
#define LPI_PROP_ENABLED (1 << 0)
+#define LPI_PRIORITY_MASK (0xfc)
/*
* ITS registers, offsets from ITS_base
@@ -272,6 +274,7 @@ struct rdist {
struct rdist_prop {
void *prop_page;
+ int id_bits;
uint64_t flags;
};
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-10 7:42 ` [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation vijay.kilari
@ 2015-07-10 15:10 ` Ian Campbell
2015-07-11 18:25 ` Julien Grall
2015-07-13 16:53 ` Stefano Stabellini
2015-07-15 17:32 ` Julien Grall
2 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 15:10 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> [...]
> +int vits_unmap_lpi_prop(struct vcpu *v)
Why is this function called "unmap"?
> + /* Register mmio handlers for this region */
> + register_mmio_handler(v->domain, &vgic_gits_lpi_mmio_handler,
> + maddr, lpi_size);
Something, somewhere, must ensure that we never register this MMIO
handler for any guest which does not have a VITS associated with it.
As it stands after this series I believe that means that the GITS_*
register handling, the GITS LPI MMIO region and the GITS_TRANSLATER
mapping should be setup for dom0 if-and-only-if a physical ITS is
present in the system.
In particular they should never, as it stands today, be mapped for a
domU.
I am unable to spot the code which arranges all that. There is some if's
in the GICR_* handlers, but I don't think that covers everything.
Obviously this will change with the addition of PCI passthrough later
on, but that change should be done in that series and not preemptively
here.
> case GICR_IIDR:
> if ( dabt.size != DABT_WORD ) goto bad_width;
> @@ -106,11 +118,16 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
> + if ( gic_lpi_supported() )
> + {
> + /* Set LPI support */
> + aff |= GICR_TYPER_PLPIS;
> + /* GITS_TYPER.PTA is 0. Provice vcpu number as ta */
"Provide" and an extra space before "0".
> + aff |= (v->vcpu_id << GICR_TYPER_PROCESSOR_SHIFT);
> + }
> *r = aff;
> -
> if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
> *r |= GICR_TYPER_LAST;
> -
Spurious whitespace changes.
> return 1;
> case GICR_STATUSR:
> /* Not implemented */
> @@ -125,10 +142,21 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> /* WO. Read as zero */
> goto read_as_zero_64;
> case GICR_PROPBASER:
> - /* LPI's not implemented */
> + if ( gic_lpi_supported() )
> + {
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> + /* Remove shareability attribute we don't want dom to flush */
This code doesn't appear to match the code, nothing is removing an
attribute here. Perhaps it belongs next to the place which initialises,
or handles writes to, v->domain->arch.vits->propbase?
> @@ -203,7 +231,18 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> switch ( gicr_reg )
> {
> case GICR_CTLR:
> - /* LPI's not implemented */
> + if ( gic_lpi_supported() )
> + {
> + /*
> + * Enable LPI's for ITS. Direct injection of LPI
> + * by writing to GICR_{SET,CLR}LPIR are not supported
"is not supported"
> + */
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + vgic_lock(v);
> + v->domain->arch.vgic.gicr_ctlr = (*r) & GICR_CTLR_ENABLE_LPIS;
Extra space after the &.
> @@ -694,6 +755,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
> *r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
>
> + if ( gic_lpi_supported() )
> + {
> + irq_bits = gic_nr_id_bits();
> + *r |= GICD_TYPE_LPIS;
> + }
> + else
> + irq_bits = get_count_order(vgic_num_irqs(v->domain));
I think gic_nr_id_bits should return the correct thing whether or not
LPIs are supported, i.e.
if ( gic_lpi_supported() )
*r |= GICD_TYPE_LPIS;
irq_bits = gic_nr_id_bits();
should be sufficient.
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 67e4695..49db7f0 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -102,6 +102,7 @@ struct arch_domain
> paddr_t dbase; /* Distributor base address */
> paddr_t cbase; /* CPU base address */
> #ifdef CONFIG_ARM_64
> + int gicr_ctlr;
Wrong indentation
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-10 15:10 ` Ian Campbell
@ 2015-07-11 18:25 ` Julien Grall
2015-07-13 9:28 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-11 18:25 UTC (permalink / raw)
To: Ian Campbell, vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
stefano.stabellini, manish.jaggi
Hi,
On 10/07/2015 17:10, Ian Campbell wrote:
> Extra space after the &.
>> @@ -694,6 +755,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
>> *r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
>> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
>>
>> + if ( gic_lpi_supported() )
>> + {
>> + irq_bits = gic_nr_id_bits();
>> + *r |= GICD_TYPE_LPIS;
>> + }
>> + else
>> + irq_bits = get_count_order(vgic_num_irqs(v->domain));
>
> I think gic_nr_id_bits should return the correct thing whether or not
> LPIs are supported, i.e.
>
> if ( gic_lpi_supported() )
> *r |= GICD_TYPE_LPIS;
> irq_bits = gic_nr_id_bits();
>
> should be sufficient.
Well no. The field GICD_TYPER.IDbits represents the number of bits
supported for the interrupt identifier.
The guest may have a different number of IDbits than the hardware which
could be higher (for instance a guest where emulated SPI is supported).
Furthermore, I'm against in principle to call directly GIC function in
the vGIC. We spent time to introduced various vGIC structure per domain
which stores the specific information for the emulated GIC (such as the
number of SPIs). It's not time to break it, even though we'd like to see
the ITS code in Xen 4.6.
I don't think it's too hard to do and will avoid to forget it when guest
support will be added...
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-11 18:25 ` Julien Grall
@ 2015-07-13 9:28 ` Ian Campbell
2015-07-13 9:53 ` Ian Campbell
0 siblings, 1 reply; 113+ messages in thread
From: Ian Campbell @ 2015-07-13 9:28 UTC (permalink / raw)
To: Julien Grall
Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
tim, xen-devel, stefano.stabellini, manish.jaggi
On Sat, 2015-07-11 at 20:25 +0200, Julien Grall wrote:
> Hi,
>
> On 10/07/2015 17:10, Ian Campbell wrote:
> > Extra space after the &.
> >> @@ -694,6 +755,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
> >> *r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
> >> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
> >>
> >> + if ( gic_lpi_supported() )
> >> + {
> >> + irq_bits = gic_nr_id_bits();
> >> + *r |= GICD_TYPE_LPIS;
> >> + }
> >> + else
> >> + irq_bits = get_count_order(vgic_num_irqs(v->domain));
> >
> > I think gic_nr_id_bits should return the correct thing whether or not
> > LPIs are supported, i.e.
> >
> > if ( gic_lpi_supported() )
> > *r |= GICD_TYPE_LPIS;
> > irq_bits = gic_nr_id_bits();
> >
> > should be sufficient.
>
> Well no. The field GICD_TYPER.IDbits represents the number of bits
> supported for the interrupt identifier.
>
> The guest may have a different number of IDbits than the hardware which
> could be higher (for instance a guest where emulated SPI is supported).
Yes, I really meant vgic_nr_id_bits(), which might for the dom0 case end
up returning something related to the h/w value from the appropriate
vgic hw cfg struct.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-13 9:28 ` Ian Campbell
@ 2015-07-13 9:53 ` Ian Campbell
0 siblings, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-13 9:53 UTC (permalink / raw)
To: Julien Grall
Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, vijaya.kumar,
tim, xen-devel, stefano.stabellini, manish.jaggi
On Mon, 2015-07-13 at 10:28 +0100, Ian Campbell wrote:
> On Sat, 2015-07-11 at 20:25 +0200, Julien Grall wrote:
> > Hi,
> >
> > On 10/07/2015 17:10, Ian Campbell wrote:
> > > Extra space after the &.
> > >> @@ -694,6 +755,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
> > >> *r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
> > >> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
> > >>
> > >> + if ( gic_lpi_supported() )
> > >> + {
> > >> + irq_bits = gic_nr_id_bits();
> > >> + *r |= GICD_TYPE_LPIS;
> > >> + }
> > >> + else
> > >> + irq_bits = get_count_order(vgic_num_irqs(v->domain));
> > >
> > > I think gic_nr_id_bits should return the correct thing whether or not
> > > LPIs are supported, i.e.
> > >
> > > if ( gic_lpi_supported() )
> > > *r |= GICD_TYPE_LPIS;
> > > irq_bits = gic_nr_id_bits();
> > >
> > > should be sufficient.
> >
> > Well no. The field GICD_TYPER.IDbits represents the number of bits
> > supported for the interrupt identifier.
> >
> > The guest may have a different number of IDbits than the hardware which
> > could be higher (for instance a guest where emulated SPI is supported).
>
> Yes, I really meant vgic_nr_id_bits(), which might for the dom0 case end
> up returning something related to the h/w value from the appropriate
> vgic hw cfg struct.
Vijay, to be more specific, the number of idbits should be added to
xen/arch/arm/vgic-v3.c:vgic_v3_hw and as a new argument to
vgic_v3_setup_hw to initialise it.
Then vgic_v3_domain_init() should consult vgic_v3_hw in the
is_hardware_domain case to initialise a new field
d->arch.vgic.nr_id_bits.
For the !is_hardware_domain case I suppose it ought to be some hardcoded
value corresponding to whatever the right value is when LPIs are not
supported.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-10 7:42 ` [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation vijay.kilari
2015-07-10 15:10 ` Ian Campbell
@ 2015-07-13 16:53 ` Stefano Stabellini
2015-07-15 17:32 ` Julien Grall
2 siblings, 0 replies; 113+ messages in thread
From: Stefano Stabellini @ 2015-07-13 16:53 UTC (permalink / raw)
To: vijay.kilari
Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Fri, 10 Jul 2015, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Emulate LPI related changes to GICR registers
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Added LPI configuration table emulation
> - Rename function inline with vits
> - Copied guest lpi configuration table to xen
> ---
> xen/arch/arm/gic-v3.c | 15 ++++
> xen/arch/arm/gic.c | 10 +++
> xen/arch/arm/vgic-v3-its.c | 165 +++++++++++++++++++++++++++++++++++++
> xen/arch/arm/vgic-v3.c | 85 +++++++++++++++++--
> xen/arch/arm/vgic.c | 4 +
> xen/include/asm-arm/domain.h | 1 +
> xen/include/asm-arm/gic-its.h | 11 +++
> xen/include/asm-arm/gic.h | 9 ++
> xen/include/asm-arm/gic_v3_defs.h | 3 +
> 9 files changed, 295 insertions(+), 8 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 904fe57..e6004d2 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -677,6 +677,11 @@ static int __init gicv3_populate_rdist(void)
> return -ENODEV;
> }
>
> +static int gicv3_dist_supports_lpis(void)
> +{
> + return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
> +}
>
> static int __cpuinit gicv3_cpu_init(void)
> {
> int i;
> @@ -1293,10 +1298,20 @@ static int __init gicv3_init(void)
> gicv3.rdist_regions[0].size, gicv3.rdist_regions[0].map_base,
> gicv3_info.maintenance_irq);
>
> + reg = readl_relaxed(GICD + GICD_TYPER);
> +
> + gicv3.rdist_data.id_bits = ((reg >> 19) & 0x1f) + 1;
> + gicv3_info.nr_id_bits = gicv3.rdist_data.id_bits;
> +
> spin_lock_init(&gicv3.lock);
>
> spin_lock(&gicv3.lock);
>
> + if ( gicv3_dist_supports_lpis() )
> + gicv3_info.lpi_supported = 1;
> + else
> + gicv3_info.lpi_supported = 0;
> +
Is it possible that a GICD returns "LPIs supported", without actually
having an ITS? I don't think we want to support that scenario, right?
To stay on the safe side we could turn this into:
if ( gicv3_dist_supports_lpis() && its_enabled )
where its_enabled could be set to true at boot time by its_init, if the
initialization is successful.
> gicv3_dist_init();
> res = gicv3_cpu_init();
> gicv3_hyp_init();
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-10 7:42 ` [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation vijay.kilari
2015-07-10 15:10 ` Ian Campbell
2015-07-13 16:53 ` Stefano Stabellini
@ 2015-07-15 17:32 ` Julien Grall
2015-07-16 14:15 ` Vijay Kilari
2 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 17:32 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Emulate LPI related changes to GICR registers
It looks like to me that the LPI configuration table emulation doesn't
contain any LPI specific code. So it looks like to me that this code
should live in GICv3.
Anyway, I don't mind to defer this for Xen 4.7. But please add a TODO.
[...]
> +static int gicv3_dist_supports_lpis(void)
> +{
> + return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
> +}
> +
> static int __cpuinit gicv3_cpu_init(void)
> {
> int i;
> @@ -1293,10 +1298,20 @@ static int __init gicv3_init(void)
> gicv3.rdist_regions[0].size, gicv3.rdist_regions[0].map_base,
> gicv3_info.maintenance_irq);
>
> + reg = readl_relaxed(GICD + GICD_TYPER);
> +
> + gicv3.rdist_data.id_bits = ((reg >> 19) & 0x1f) + 1;
Can you please introduce define for value 19 and 0x1f rather than
hardcoding them?
> + gicv3_info.nr_id_bits = gicv3.rdist_data.id_bits;
> +
> spin_lock_init(&gicv3.lock);
>
> spin_lock(&gicv3.lock);
>
> + if ( gicv3_dist_supports_lpis() )
> + gicv3_info.lpi_supported = 1;
> + else
> + gicv3_info.lpi_supported = 0;
> +
gicv3_info.lpi_supported = !!gicv3_gist_supports_lpis();
> gicv3_dist_init();
> res = gicv3_cpu_init();
> gicv3_hyp_init();
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 4f3801b..3ebadcf 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -73,6 +73,16 @@ unsigned int gic_number_lines(void)
> return gic_hw_ops->info->nr_lines;
> }
>
> +unsigned int gic_nr_id_bits(void)
> +{
> + return gic_hw_ops->info->nr_id_bits;
> +}
> +
> +bool_t gic_lpi_supported(void)
> +{
> + return gic_hw_ops->info->lpi_supported;
> +}
> +
> void gic_save_state(struct vcpu *v)
> {
> ASSERT(!local_irq_is_enabled());
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index abf60e2..bbcc7bb 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -49,6 +49,36 @@ static void dump_cmd(its_cmd_block *cmd)
> }
> #endif
>
> +static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
> +{
> + struct pending_irq *p;
> +
> + p = irq_to_pending(v, vlpi);
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + gic_remove_from_queues(v, vlpi);
> +}
> +
> +static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
The argument priority is never used. Please drop it.
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> +
> + /* Get plpi for the given vlpi */
This comment seems wrong.
> + p = irq_to_pending(v, vlpi);
> +
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
> +
> + /*XXX: raise on right vcpu */
/* XXX: ... */
I guess you added this comment because I mentioned possible locking
problem here?
If so, did you think how this would affect the vLPI handling to not do
the correct locking for Xen 4.6?
Note for the others: AFAICT the locking of the pending_irq structure is
done using the lock of the vCPU where the IRQ is routed. Stefano, please
confirm if it's true.
> + if ( !list_empty(&p->inflight) &&
> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> + gic_raise_guest_irq(v, irq_to_virq(p->desc), p->priority);
> +
> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> +}
> +
> +/* ITS device table helper functions */
This comment doesn't belong to this patch but patch #6.
> static int vits_entry(struct domain *d, paddr_t entry, void *addr,
> uint32_t size, bool_t set)
> {
> @@ -595,6 +625,141 @@ err:
> return 0;
> }
>
> +static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa -
> + (v->domain->arch.vits->propbase & BIT_48_12_MASK);
> +
> + if ( offset < v->domain->arch.vits->prop_size )
As said on v3, this check is not necessary. You already register the
handler on a valid range. So please drop this check.
> + {
> + DPRINTK("%pv: vITS: LPI Table read offset 0x%x\n", v, offset);
> + spin_lock(&v->domain->arch.vits->prop_lock);
> + *r = *((u8*)v->domain->arch.vits->prop_page + offset);
You didn't answer my question on v3... I.e "what about other access? a
64/32/16 bits access are valid and will return the wrong value."
> + spin_unlock(&v->domain->arch.vits->prop_lock);
> + return 1;
> + }
> + else
> + dprintk(XENLOG_G_ERR, "%pv: vITS: LPI Table read with wrong offset 0x%x\n",
> + v, offset);
> +
> + return 0;
> +}
> +
> +static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + uint32_t vid;
> + uint8_t cfg, *p;
> + bool_t enable;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa -
> + (v->domain->arch.vits->propbase & BIT_48_12_MASK);
> +
> + vid = offset + NR_GIC_LPI;
I'm not sure how you test this series... Based on patch #6, NR_GIC_LPI
(which is defined as 4096) represents the number of LPIs and not the
BASE of LPIs (i.e FIRST_GIC_LPI = 8196)
> + if ( offset < v->domain->arch.vits->prop_size )
This check is not necessary. Please drop it.
> + {
> + DPRINTK("%pv: vITS: LPI Table write offset 0x%x\n", v, offset);
> +
> + spin_lock(&v->domain->arch.vits->prop_lock);
> + p = ((u8*)v->domain->arch.vits->prop_page + offset);
> + cfg = *p;
> + enable = (cfg & *r) & 0x1;
Please use a define rather than 0x1. It would have been more clear that
we are checking the enable bit.
> +
> + if ( !enable )
> + vits_enable_lpi(v, vid, (*r & LPI_PRIORITY_MASK));
> + else
> + vits_disable_lpi(v, vid);
> +
> + /* Update virtual prop page */
> + *p = (*r & 0xff);
> + spin_unlock(&v->domain->arch.vits->prop_lock);
What about other access than 8-bit (i.e 16-bit, 32-bit, 64-bit)?
> + return 1;
> + }
> + else
> + dprintk(XENLOG_G_ERR, "%pv: vITS: LPI Table invalid write @ 0x%x\n",
> + v, offset);
> +
> + return 0;
> +}
> +
> +static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
> + .read_handler = vgic_v3_gits_lpi_mmio_read,
> + .write_handler = vgic_v3_gits_lpi_mmio_write,
> +};
> +
> +int vits_unmap_lpi_prop(struct vcpu *v)
It looks like to me that the function should take a domain rather than a
vcpu.
> +{
> + paddr_t maddr, addr;
> + unsigned long mfn;
> + uint32_t lpi_size, id_bits;
> + int i;
> +
> + maddr = v->domain->arch.vits->propbase & BIT_48_12_MASK;
maddr means machine address which doesn't seem to be the case here.
Please use the right name.
> + id_bits = ((v->domain->arch.vits->propbase & GICR_PROPBASER_IDBITS_MASK)+1);
> +
> + DPRINTK("%pv: vITS: Unmap guest LPI conf table maddr 0x%lx lpi_size 0x%x\n",
> + v, maddr, lpi_size);
> +
> + spin_lock(&v->domain->arch.vits->prop_lock);
I don't understand why you need to take the prop_lock here. You only use
them in the handler which you don't have yet registered.
> + if ( id_bits > gic_nr_id_bits() )
> + id_bits = gic_nr_id_bits();
What happen if the property configuration table is smaller?
> +
> + lpi_size = 1UL << id_bits;
> +
> + v->domain->arch.vits->prop_size = lpi_size;
> + /* Allocate Virtual LPI Property table */
> + /* TODO: To re-use guest property table? */
> + v->domain->arch.vits->prop_page =
> + alloc_xenheap_pages(get_order_from_bytes(lpi_size), 0);
> + if ( !v->domain->arch.vits->prop_page )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Fail to allocate LPI Prop page\n", v);
> + return 0;
> + }
> +
> + addr = maddr;
> + for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
> + {
> + vits_entry(v->domain, addr,
> + (void *)(v->domain->arch.vits->prop_page + i * PAGE_SIZE),
> + PAGE_SIZE, 0);
> + addr += PAGE_SIZE;
It's good to copy the value from the guest memory to our copy. It would
be better to also replicate the value in the vITS structure. I.e
enable/disable the LPIs when necessary.
Though, it may means a long process in this function.
[...]
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 4af5a84..25b69a0 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -30,6 +30,7 @@
> #include <asm/mmio.h>
> #include <asm/gic_v3_defs.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <asm/vgic.h>
>
> /* GICD_PIDRn register values for ARM implementations */
> @@ -93,7 +94,18 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> switch ( gicr_reg )
> {
> case GICR_CTLR:
> - /* We have not implemented LPI's, read zero */
> + /*
> + * Enable LPI's for ITS. Direct injection of LPI
> + * by writing to GICR_{SET,CLR}LPIR are not supported
> + */
This comment should be on write but not read. The read function only
return the value stored in gicr_ctlr.
> + if ( gic_lpi_supported() )
> + {
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + vgic_lock(v);
> + *r = v->domain->arch.vgic.gicr_ctlr;
> + vgic_unlock(v);
> + return 1;
> + }
I think this could be simplified to
if ( dabt.size != DABT_WORD ) goto bad_width;
vgic_lock(v);
*r = v->domain->arch.vgic.gicr_ctlr;
vgic_unlock(v);
return 0;
This is because the write emulation will ensure the validity of gicr_ctlr.
> goto read_as_zero_32;
> case GICR_IIDR:
> if ( dabt.size != DABT_WORD ) goto bad_width;
> @@ -106,11 +118,16 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
> + if ( gic_lpi_supported() )
> + {
> + /* Set LPI support */
> + aff |= GICR_TYPER_PLPIS;
> + /* GITS_TYPER.PTA is 0. Provice vcpu number as ta */
s/Provice/Provide/
And please use "Target Address" rather than "ta". Easier to read.
> + aff |= (v->vcpu_id << GICR_TYPER_PROCESSOR_SHIFT);
> + }
The variable aff is used to store the affinity nothing else.
Please use *r instead of aff. Note that you will have to move the code
after *r = aff;
> *r = aff;
> -
> if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
> *r |= GICR_TYPER_LAST;
> -
> return 1;
[...]
> @@ -224,10 +263,32 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> /* LPI is not implemented */
> goto write_ignore_64;
> case GICR_PROPBASER:
> - /* LPI is not implemented */
> + if ( gic_lpi_supported() )
> + {
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> + vgic_lock(v);
> + /* LPI configuration tables are shared across cpus. Should be same */
> + /* TODO: Manage change in property table */
> + if ( v->domain->arch.vits->propbase != 0 )
Well it's potentially valid to have propbase equals to 0. You may want
to add explain this restriction in the comment as a new TODO.
[..]
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 73a6f7e..a5f66f6 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -154,6 +154,10 @@ void domain_vgic_free(struct domain *d)
> xfree(d->arch.vgic.shared_irqs);
> xfree(d->arch.vgic.pending_irqs);
> xfree(d->arch.vgic.allocated_irqs);
> +#ifdef CONFIG_ARM_64
> + free_xenheap_pages(d->arch.vits->prop_page,
> + get_order_from_bytes(d->arch.vits->prop_size));
> +#endif
Anything allocated for the vITS should be deallocated in the vITS code
not in the common vGIC code.
A new callback in the vgic structure should be added in order to free
vgic specific code.
I would be nice if you don't do it for xen 4.6, but you need at least to
add a /* TODO: Move in vits code */.
> }
>
> int vcpu_vgic_init(struct vcpu *v)
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 67e4695..49db7f0 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -102,6 +102,7 @@ struct arch_domain
> paddr_t dbase; /* Distributor base address */
> paddr_t cbase; /* CPU base address */
> #ifdef CONFIG_ARM_64
> + int gicr_ctlr;
The indentation look wrong here.
> /* GIC V3 addressing */
> paddr_t dbase_size; /* Distributor base size */
> /* List of contiguous occupied by the redistributors */
[...]
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index fdd96c8..69bf1ff 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -100,6 +100,7 @@
> #define GICD_TYPE_CPUS_SHIFT 5
> #define GICD_TYPE_CPUS 0x0e0
> #define GICD_TYPE_SEC 0x400
> +#define GICD_TYPE_LPIS (0x1UL << 17)
>
> #define GICC_CTL_ENABLE 0x1
> #define GICC_CTL_EOI (0x1 << 9)
> @@ -283,6 +284,10 @@ extern void gic_dump_info(struct vcpu *v);
>
> /* Number of interrupt lines */
> extern unsigned int gic_number_lines(void);
> +/* Number of interrupt id bits supported */
> +extern unsigned int gic_nr_id_bits(void);
> +/* LPI support info */
> +bool_t gic_lpi_supported(void);
>
> void gicv3_eoi_irq(struct irq_desc *irqd);
> void gicv3_dir_irq(struct irq_desc *irqd);
> @@ -302,6 +307,10 @@ struct gic_info {
> unsigned int maintenance_irq;
> /* Pointer to the device tree node representing the interrupt controller */
> const struct dt_device_node *node;
> + /* Number of IRQ ID bits supported */
> + uint32_t nr_id_bits;
> + /* LPIs are support information */
> + bool_t lpi_supported;
[..]
> struct rdist_prop {
> void *prop_page;
> + int id_bits;
The nr_id_bits (in gic_info) and id_bits here looks the same. Please
remove one of them.
IHMO, the former (i.e in gic_info) should be removed.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-15 17:32 ` Julien Grall
@ 2015-07-16 14:15 ` Vijay Kilari
2015-07-16 14:41 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-16 14:15 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Wed, Jul 15, 2015 at 11:02 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Emulate LPI related changes to GICR registers
[..]
>
>> + p = irq_to_pending(v, vlpi);
>> +
>> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>> +
>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>> +
>> + /*XXX: raise on right vcpu */
>
>
> /* XXX: ... */
>
> I guess you added this comment because I mentioned possible locking problem
> here?
I meant to find out Collection (VCPU) for this vlpi and inject on that VCPU
>
> If so, did you think how this would affect the vLPI handling to not do the
> correct locking for Xen 4.6?
>
> Note for the others: AFAICT the locking of the pending_irq structure is done
> using the lock of the vCPU where the IRQ is routed. Stefano, please confirm
> if it's true.
>
>> + if ( !list_empty(&p->inflight) &&
>> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
>> + gic_raise_guest_irq(v, irq_to_virq(p->desc), p->priority);
>> +
>> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>> +}
>> +
[...]
>
>> + {
>> + DPRINTK("%pv: vITS: LPI Table read offset 0x%x\n", v, offset);
>> + spin_lock(&v->domain->arch.vits->prop_lock);
>> + *r = *((u8*)v->domain->arch.vits->prop_page + offset);
>
>
> You didn't answer my question on v3... I.e "what about other access? a
> 64/32/16 bits access are valid and will return the wrong value."
One byte represents one LPI in the configuration table. This table being in
memory, guest can access 16/32/64. I will make a check on access size and
handle accordingly.
>> +
>> + spin_lock(&v->domain->arch.vits->prop_lock);
>
>
> I don't understand why you need to take the prop_lock here. You only use
> them in the handler which you don't have yet registered.
>
lock is taken as property table can be changed. However as I am
not handling change in property table in this series, It can be dropped.
>> + if ( id_bits > gic_nr_id_bits() )
>> + id_bits = gic_nr_id_bits();
>
>
> What happen if the property configuration table is smaller?
Spec does not say anything about it
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-16 14:15 ` Vijay Kilari
@ 2015-07-16 14:41 ` Julien Grall
2015-07-16 14:46 ` Vijay Kilari
0 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-16 14:41 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Vijay,
On 16/07/2015 16:15, Vijay Kilari wrote:
> On Wed, Jul 15, 2015 at 11:02 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>>
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Emulate LPI related changes to GICR registers
> [..]
>>
>>> + p = irq_to_pending(v, vlpi);
>>> +
>>> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>>> +
>>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>> +
>>> + /*XXX: raise on right vcpu */
>>
>>
>> /* XXX: ... */
>>
>> I guess you added this comment because I mentioned possible locking problem
>> here?
>
> I meant to find out Collection (VCPU) for this vlpi and inject on that VCPU
I'm nearly sure there is locking problem if you don't do it know.
>>
>> If so, did you think how this would affect the vLPI handling to not do the
>> correct locking for Xen 4.6?
>>
>> Note for the others: AFAICT the locking of the pending_irq structure is done
>> using the lock of the vCPU where the IRQ is routed. Stefano, please confirm
>> if it's true.
[...]
>>> +
>>> + spin_lock(&v->domain->arch.vits->prop_lock);
>>
>>
>> I don't understand why you need to take the prop_lock here. You only use
>> them in the handler which you don't have yet registered.
>>
> lock is taken as property table can be changed. However as I am
> not handling change in property table in this series, It can be dropped.
>
>>> + if ( id_bits > gic_nr_id_bits() )
>>> + id_bits = gic_nr_id_bits();
>>
>>
>> What happen if the property configuration table is smaller?
>
> Spec does not say anything about it
So you have to handle it properly to avoid the helper reading out of the
LPI configuration table.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-16 14:41 ` Julien Grall
@ 2015-07-16 14:46 ` Vijay Kilari
2015-07-16 14:58 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-16 14:46 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Thu, Jul 16, 2015 at 8:11 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 16/07/2015 16:15, Vijay Kilari wrote:
>>
>> On Wed, Jul 15, 2015 at 11:02 PM, Julien Grall <julien.grall@citrix.com>
>> wrote:
>>>
>>> On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
>>>>
>>>>
>>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>>
>>>> Emulate LPI related changes to GICR registers
>>
>> [..]
>>>
>>>
>>>> + p = irq_to_pending(v, vlpi);
>>>> +
>>>> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>>>> +
>>>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>>> +
>>>> + /*XXX: raise on right vcpu */
>>>
>>>
>>>
>>> /* XXX: ... */
>>>
>>> I guess you added this comment because I mentioned possible locking
>>> problem
>>> here?
>>
>>
>> I meant to find out Collection (VCPU) for this vlpi and inject on that
>> VCPU
>
>
> I'm nearly sure there is locking problem if you don't do it know.
>
>>>
>>> If so, did you think how this would affect the vLPI handling to not do
>>> the
>>> correct locking for Xen 4.6?
>>>
>>> Note for the others: AFAICT the locking of the pending_irq structure is
>>> done
>>> using the lock of the vCPU where the IRQ is routed. Stefano, please
>>> confirm
>>> if it's true.
>
>
> [...]
>
>
>>>> +
>>>> + spin_lock(&v->domain->arch.vits->prop_lock);
>>>
>>>
>>>
>>> I don't understand why you need to take the prop_lock here. You only use
>>> them in the handler which you don't have yet registered.
>>>
>> lock is taken as property table can be changed. However as I am
>> not handling change in property table in this series, It can be dropped.
>>
>>>> + if ( id_bits > gic_nr_id_bits() )
>>>> + id_bits = gic_nr_id_bits();
>>>
>>>
>>>
>>> What happen if the property configuration table is smaller?
>>
>>
>> Spec does not say anything about it
>
>
> So you have to handle it properly to avoid the helper reading out of the LPI
> configuration table.
the check on size lpi_size manages this.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation
2015-07-16 14:46 ` Vijay Kilari
@ 2015-07-16 14:58 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-16 14:58 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 16/07/2015 16:46, Vijay Kilari wrote:
>> So you have to handle it properly to avoid the helper reading out of the LPI
>> configuration table.
>
> the check on size lpi_size manages this.
I don't see any check on the lpi_size... While the code in this function
looks ok, there is some usage of the LPI configuration where you don't
check the validity of the offset (I'm thinking of vgic_its_get_priority
in patch #12).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (10 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 11/17] xen/arm: ITS: Add GICR register emulation vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 15:30 ` Ian Campbell
` (4 more replies)
2015-07-10 7:42 ` [PATCH v4 13/17] xen/arm: ITS: Initialize physical ITS vijay.kilari
` (4 subsequent siblings)
16 siblings, 5 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Allocate and initialize irq descriptor for LPIs and
route LPIs to guest
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Merge patch #16
- Changed commit message
---
xen/arch/arm/gic-v3.c | 2 +-
xen/arch/arm/gic.c | 24 +++++++--
xen/arch/arm/irq.c | 44 +++++++++++++----
xen/arch/arm/vgic-v3-its.c | 9 ++++
xen/arch/arm/vgic-v3.c | 15 ++++--
xen/arch/arm/vgic.c | 110 ++++++++++++++++++++++++++++++++++++++---
xen/include/asm-arm/domain.h | 3 ++
xen/include/asm-arm/gic-its.h | 1 +
xen/include/asm-arm/gic.h | 7 ++-
xen/include/asm-arm/vgic.h | 1 +
10 files changed, 192 insertions(+), 24 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index e6004d2..53554e6 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -895,7 +895,7 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
- if ( p->desc != NULL )
+ if ( p->desc != NULL && !(is_lpi(p->irq)) )
val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
<< GICH_LR_PHYSICAL_SHIFT);
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 3ebadcf..92d2be9 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -68,11 +68,18 @@ enum gic_version gic_hw_version(void)
return gic_hw_ops->info->hw_version;
}
+/* Only validates PPIs/SGIs/SPIs supported */
unsigned int gic_number_lines(void)
{
return gic_hw_ops->info->nr_lines;
}
+/* Validates PPIs/SGIs/SPIs/LPIs supported */
+bool_t gic_is_valid_irq(unsigned int irq)
+{
+ return ((irq < gic_hw_ops->info->nr_lines) && is_lpi(irq));
+}
+
unsigned int gic_nr_id_bits(void)
{
return gic_hw_ops->info->nr_id_bits;
@@ -148,7 +155,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
{
ASSERT(priority <= 0xff); /* Only 8 bits of priority */
/* Can't route interrupts that don't exist */
- ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
+ ASSERT(gic_is_valid_irq(desc->irq));
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
@@ -160,6 +167,17 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
/* Program the GIC to route an interrupt to a guest
* - desc.lock must be held
*/
+int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc, unsigned int priority)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ desc->handler = get_guest_hw_irq_controller(desc->irq);
+ set_bit(_IRQ_GUEST, &desc->status);
+
+ return 0;
+}
+
int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
struct irq_desc *desc, unsigned int priority)
{
@@ -454,7 +472,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
{
- if ( p->desc == NULL )
+ if ( p->desc == NULL || is_lpi(irq) )
{
lr_val.state |= GICH_LR_PENDING;
gic_hw_ops->write_lr(i, &lr_val);
@@ -677,7 +695,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
/* Reading IRQ will ACK it */
irq = gic_hw_ops->read_irq();
- if ( likely(irq >= 16 && irq < 1020) )
+ if ( (likely(irq >= 16 && irq < 1020)) || is_lpi(irq) )
{
local_irq_enable();
do_IRQ(regs, irq, is_fiq);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 3806d98..c8ea627 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -62,12 +62,21 @@ hw_irq_controller no_irq_type = {
};
static irq_desc_t irq_desc[NR_IRQS];
+#ifdef CONFIG_ARM_64
+static irq_desc_t irq_desc_lpi[NR_GIC_LPI];
+#endif
static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
irq_desc_t *__irq_to_desc(int irq)
{
if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
- return &irq_desc[irq-NR_LOCAL_IRQS];
+ else if ( irq >= NR_LOCAL_IRQS && irq < NR_IRQS)
+ return &irq_desc[irq-NR_LOCAL_IRQS];
+#ifdef CONFIG_ARM_64
+ else if ( is_lpi(irq) )
+ return &irq_desc_lpi[irq - NR_GIC_LPI];
+#endif
+ return NULL;
}
int __init arch_init_one_irq_desc(struct irq_desc *desc)
@@ -88,6 +97,15 @@ static int __init init_irq_data(void)
desc->action = NULL;
}
+#ifdef CONFIG_ARM_64
+ for ( irq = NR_GIC_LPI; irq < MAX_LPI; irq++ )
+ {
+ struct irq_desc *desc = irq_to_desc(irq);
+ init_one_irq_desc(desc);
+ desc->irq = irq;
+ desc->action = NULL;
+ }
+#endif
return 0;
}
@@ -208,7 +226,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
* which interrupt is which (messes up the interrupt freeing
* logic etc).
*/
- if ( irq >= nr_irqs )
+ if ( irq >= nr_irqs && !is_lpi(irq) )
return -EINVAL;
if ( !handler )
return -EINVAL;
@@ -267,9 +285,14 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
set_bit(_IRQ_INPROGRESS, &desc->status);
desc->arch.eoi_cpu = smp_processor_id();
+#ifdef CONFIG_ARM_64
+ if ( is_lpi(irq) )
+ vgic_vcpu_inject_lpi(info->d, irq);
+ else
+#endif
/* the irq cannot be a PPI, we only support delivery of SPIs to
* guests */
- vgic_vcpu_inject_spi(info->d, info->virq);
+ vgic_vcpu_inject_spi(info->d, info->virq);
goto out_no_end;
}
@@ -436,7 +459,8 @@ err:
bool_t is_assignable_irq(unsigned int irq)
{
/* For now, we can only route SPIs to the guest */
- return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
+ return (((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines())) ||
+ is_lpi(irq));
}
/*
@@ -452,7 +476,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned long flags;
int retval = 0;
- if ( virq >= vgic_num_irqs(d) )
+ if ( virq >= vgic_num_irqs(d) && !is_lpi(irq) )
{
printk(XENLOG_G_ERR
"the vIRQ number %u is too high for domain %u (max = %u)\n",
@@ -460,10 +484,10 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
return -EINVAL;
}
- /* Only routing to virtual SPIs is supported */
+ /* Only routing to virtual SPIs/LPIs is supported */
if ( virq < NR_LOCAL_IRQS )
{
- printk(XENLOG_G_ERR "IRQ can only be routed to an SPI\n");
+ printk(XENLOG_G_ERR "IRQ can only be routed to an SPI/LPI\n");
return -EINVAL;
}
@@ -537,8 +561,10 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
retval = __setup_irq(desc, 0, action);
if ( retval )
goto out;
-
- retval = gic_route_irq_to_guest(d, virq, desc, GIC_PRI_IRQ);
+ if ( is_lpi(irq) )
+ retval = gic_route_lpi_to_guest(d, virq, desc, GIC_PRI_IRQ);
+ else
+ retval = gic_route_irq_to_guest(d, virq, desc, GIC_PRI_IRQ);
spin_unlock_irqrestore(&desc->lock, flags);
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index bbcc7bb..4649b07 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -625,6 +625,15 @@ err:
return 0;
}
+uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid)
+{
+ uint8_t priority;
+
+ priority = readb_relaxed(v->domain->arch.vits->prop_page + pid);
+ priority &= LPI_PRIORITY_MASK;
+
+ return priority;
+}
static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
{
uint32_t offset;
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 25b69a0..4e14439 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1111,12 +1111,19 @@ static const struct mmio_handler_ops vgic_distr_mmio_handler = {
static int vgic_v3_get_irq_priority(struct vcpu *v, unsigned int irq)
{
- int priority;
- struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
+ int priority = 0;
+ struct vgic_irq_rank *rank;
- ASSERT(spin_is_locked(&rank->lock));
- priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
+ if ( !is_lpi(irq) )
+ {
+ rank = vgic_rank_irq(v, irq);
+
+ ASSERT(spin_is_locked(&rank->lock));
+ priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
irq, DABT_WORD)], 0, irq & 0x3);
+ }
+ if ( is_lpi(irq) && gic_lpi_supported() )
+ priority = vgic_its_get_priority(v, irq);
return priority;
}
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index a5f66f6..8190a46 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -30,6 +30,7 @@
#include <asm/mmio.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
@@ -111,6 +112,15 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
for (i=0; i<d->arch.vgic.nr_spis; i++)
vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
+#ifdef CONFIG_ARM_64
+ d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq, NR_GIC_LPI);
+ if ( d->arch.vgic.pending_lpis == NULL )
+ return -ENOMEM;
+
+ for ( i = 0; i < NR_GIC_LPI; i++ )
+ vgic_init_pending_irq(&d->arch.vgic.pending_lpis[i], i);
+#endif
+
for (i=0; i<DOMAIN_NR_RANKS(d); i++)
spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
@@ -157,6 +167,7 @@ void domain_vgic_free(struct domain *d)
#ifdef CONFIG_ARM_64
free_xenheap_pages(d->arch.vits->prop_page,
get_order_from_bytes(d->arch.vits->prop_size));
+ xfree(d->arch.vgic.pending_lpis);
#endif
}
@@ -381,13 +392,17 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
{
- struct pending_irq *n;
+ struct pending_irq *n = NULL;
/* Pending irqs allocation strategy: the first vgic.nr_spis irqs
* are used for SPIs; the rests are used for per cpu irqs */
if ( irq < 32 )
n = &v->arch.vgic.pending_irqs[irq];
- else
+ else if ( irq < NR_IRQS )
n = &v->domain->arch.vgic.pending_irqs[irq - 32];
+#ifdef CONFIG_ARM_64
+ else if ( is_lpi(irq) )
+ n = &v->domain->arch.vgic.pending_lpis[irq - FIRST_GIC_LPI];
+#endif
return n;
}
@@ -413,14 +428,20 @@ void vgic_clear_pending_irqs(struct vcpu *v)
void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
{
uint8_t priority;
- struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
+ struct vgic_irq_rank *rank;
struct pending_irq *iter, *n = irq_to_pending(v, virq);
unsigned long flags;
bool_t running;
- vgic_lock_rank(v, rank, flags);
- priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
- vgic_unlock_rank(v, rank, flags);
+ if ( virq < NR_GIC_LPI )
+ {
+ rank = vgic_rank_irq(v, virq);
+ vgic_lock_rank(v, rank, flags);
+ priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
+ vgic_unlock_rank(v, rank, flags);
+ }
+ else
+ priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
spin_lock_irqsave(&v->arch.vgic.lock, flags);
@@ -477,6 +498,83 @@ void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
vgic_vcpu_inject_irq(v, virq);
}
+#ifdef CONFIG_ARM_64
+void vgic_vcpu_inject_lpi(struct domain *d, unsigned int irq)
+{
+ struct irq_desc *desc;
+ struct pending_irq *p;
+ struct its_device *dev;
+ struct vits_device *vdev;
+ struct vdevice_table dt_entry;
+ struct vitt vitt_entry;
+ uint32_t devid, col_id;
+ int event;
+
+ desc = irq_to_desc(irq);
+ event = irq_to_virq(desc);
+
+ dev = get_irq_device(desc);
+ devid = dev->virt_device_id;
+ event = irq - dev->lpi_base;
+ if ( (event < 0 && event > dev->nr_lpis) ||
+ (dev->domain_id != d->domain_id) )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x is not assigned..dropping\n",
+ irq, devid);
+ return;
+ }
+
+ vdev = vits_find_device(&d->arch.vits->dev_root, devid);
+ if ( !vdev )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x not assigned..dropping\n",
+ irq, devid);
+ return;
+ }
+
+ if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
+ {
+ dprintk(XENLOG_WARNING,
+ "Failed to read device table entry for dev 0x%x ..dropping\n",
+ devid);
+ return;
+ }
+ if ( dt_entry.vitt_ipa == INVALID_PADDR )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x which is disabled..dropping\n",
+ irq, devid);
+ return;
+ }
+
+ if ( vits_get_vitt_entry(d, devid, event, &vitt_entry) )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x with invalid entry..dropping\n",
+ irq, devid);
+ return;
+ }
+
+ col_id = vitt_entry.vcollection;
+
+ if ( !vitt_entry.valid || col_id > d->max_vcpus ||
+ vitt_entry.vlpi > its_get_nr_events() )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x is not valid..dropping\n",
+ irq, devid);
+ return;
+ }
+
+ p = irq_to_pending(d->vcpu[0], vitt_entry.vlpi);
+ p->desc = desc;
+
+ vgic_vcpu_inject_irq(d->vcpu[col_id], vitt_entry.vlpi);
+}
+#endif
+
void arch_evtchn_inject(struct vcpu *v)
{
vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 49db7f0..a4bf8f6 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -98,6 +98,9 @@ struct arch_domain
* struct arch_vcpu.
*/
struct pending_irq *pending_irqs;
+#ifdef CONFIG_ARM_64
+ struct pending_irq *pending_lpis;
+#endif
/* Base address for guest GIC */
paddr_t dbase; /* Distributor base address */
paddr_t cbase; /* CPU base address */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index a79b70f..fbed905 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -277,6 +277,7 @@ struct vits_device *vits_find_device(struct rb_root *root, uint32_t devid);
int vits_insert_device(struct rb_root *root, struct vits_device *dev);
void vits_remove_device(struct rb_root *root, struct vits_device *dev);
int vits_unmap_lpi_prop(struct vcpu *v);
+uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 69bf1ff..537ed3d 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -226,6 +226,9 @@ extern void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mas
extern int gic_route_irq_to_guest(struct domain *, unsigned int virq,
struct irq_desc *desc,
unsigned int priority);
+extern int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc,
+ unsigned int priority);
/* Remove an IRQ passthrough to a guest */
int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
@@ -282,8 +285,10 @@ extern void send_SGI_allbutself(enum gic_sgi sgi);
/* print useful debug info */
extern void gic_dump_info(struct vcpu *v);
-/* Number of interrupt lines */
+/* Number of interrupt lines (SPIs)*/
extern unsigned int gic_number_lines(void);
+/* Check if irq is valid SPI or LPI */
+bool_t gic_is_valid_irq(unsigned int irq);
/* Number of interrupt id bits supported */
extern unsigned int gic_nr_id_bits(void);
/* LPI support info */
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 8d22532..f8928ab 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -183,6 +183,7 @@ extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
+extern void vgic_vcpu_inject_lpi(struct domain *d, unsigned int virq);
extern void vgic_clear_pending_irqs(struct vcpu *v);
extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
extern struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 7:42 ` [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route vijay.kilari
@ 2015-07-10 15:30 ` Ian Campbell
2015-07-20 13:07 ` Vijay Kilari
2015-07-22 13:31 ` Vijay Kilari
2015-07-13 17:03 ` Stefano Stabellini
` (3 subsequent siblings)
4 siblings, 2 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 15:30 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 3ebadcf..92d2be9 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -68,11 +68,18 @@ enum gic_version gic_hw_version(void)
> return gic_hw_ops->info->hw_version;
> }
>
> +/* Only validates PPIs/SGIs/SPIs supported */
> unsigned int gic_number_lines(void)
> {
> return gic_hw_ops->info->nr_lines;
> }
>
> +/* Validates PPIs/SGIs/SPIs/LPIs supported */
> +bool_t gic_is_valid_irq(unsigned int irq)
> +{
> + return ((irq < gic_hw_ops->info->nr_lines) && is_lpi(irq));
> +}
> +
> unsigned int gic_nr_id_bits(void)
> {
> return gic_hw_ops->info->nr_id_bits;
> @@ -148,7 +155,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> {
> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> /* Can't route interrupts that don't exist */
> - ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
> + ASSERT(gic_is_valid_irq(desc->irq));
The fact that I commented on this earlier is another artefact in the way
this series presents functionality in an essentially arbitrary order.
I notice that you appear to have reintroduced the logic but when you
moved this code from here into gic_is_valid_irq. AFAICT that function
can never return true, but then I'm rather confused about how this
series can have been tested.
> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
> ASSERT(spin_is_locked(&desc->lock));
>
> @@ -160,6 +167,17 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> /* Program the GIC to route an interrupt to a guest
> * - desc.lock must be held
> */
> +int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
> + struct irq_desc *desc, unsigned int priority)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
ASSERT(virq >= SOME_DEFINE_FOR_MIN_LPI);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 3806d98..c8ea627 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -62,12 +62,21 @@ hw_irq_controller no_irq_type = {
> };
>
> static irq_desc_t irq_desc[NR_IRQS];
> +#ifdef CONFIG_ARM_64
> +static irq_desc_t irq_desc_lpi[NR_GIC_LPI];
http://xenbits.xen.org/people/ianc/vits/draftG.html#irq-descriptors
contains: "Therefore a second dynamically allocated array will be added
to cover the range 8192..nr_lpis"
IOW this should be dynamically allocated and therefore NR_GIC_LPI (which
I think I already commented on earlier) should go away, since the limit
should come from the h/w.
> @@ -267,9 +285,14 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
> set_bit(_IRQ_INPROGRESS, &desc->status);
> desc->arch.eoi_cpu = smp_processor_id();
>
> +#ifdef CONFIG_ARM_64
> + if ( is_lpi(irq) )
> + vgic_vcpu_inject_lpi(info->d, irq);
> + else
> +#endif
> /* the irq cannot be a PPI, we only support delivery of SPIs to
> * guests */
> - vgic_vcpu_inject_spi(info->d, info->virq);
> + vgic_vcpu_inject_spi(info->d, info->virq);
Comment should be reindented too.
> + if ( !is_lpi(irq) )
> + {
> + rank = vgic_rank_irq(v, irq);
> +
> + ASSERT(spin_is_locked(&rank->lock));
> + priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
> irq, DABT_WORD)], 0, irq & 0x3);
> + }
> + if ( is_lpi(irq) && gic_lpi_supported() )
Should be "else if ( gic_lpi_supported() )"
> + vdev = vits_find_device(&d->arch.vits->dev_root, devid);
> + if ( !vdev )
> + {
> + dprintk(XENLOG_WARNING,
> + "LPI %d received for dev 0x%x not assigned..dropping\n",
> + irq, devid);
> + return;
> + }
This isn't used again and the whole R-B tree should go away.
> + p = irq_to_pending(d->vcpu[0], vitt_entry.vlpi);
Perhaps given that we know this is a vlpi, we could skipping going via
vcpu[0] and just go right to the entry in the d->pending_lpi array (via
a suitable accessor of course)
> + p->desc = desc;
This should have happened during routing, not now.
Oh. I see what is going on, your vgic_vcpu_inject_lpi appears to be
taking an IRQ, and not a vpli, or at least to be confused about what it
should do with the number which it calls irq.
Therefore you find yourself needing to lookup the irqdesc, and look in
the vitt etc, none of which belongs here.
For interrupts coming from a plpi all of that lookup should happen based
on the its_device which is contained in the irq_guest and other info
from there, before calling this function. You can get the irq_guest
which you can get because you have a PLPI in your hand and can look up
the irq_guest via the irq_desc which you get using the plpi number.
For interrupts coming from the VITS INT command the command itself
contains a vDevice and vEvent, which can then be looked up in the ITT to
get a vpli which can be passed to vgic_vcpu_inject_lpi.
Thus this function has no need to lookup an irq_desc, nor a dev id, nor
to do an rb tree lookup.
I think both of these cases are adequately explained, with pseudocode,
in the draftG vits design doc. Please ask if you are unsure.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 15:30 ` Ian Campbell
@ 2015-07-20 13:07 ` Vijay Kilari
2015-07-20 13:25 ` Julien Grall
2015-07-22 13:31 ` Vijay Kilari
1 sibling, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-20 13:07 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
Hi Ian,
On Fri, Jul 10, 2015 at 9:00 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index 3ebadcf..92d2be9 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -68,11 +68,18 @@ enum gic_version gic_hw_version(void)
>> return gic_hw_ops->info->hw_version;
>> }
>>
>> +/* Only validates PPIs/SGIs/SPIs supported */
>> unsigned int gic_number_lines(void)
>> {
>> return gic_hw_ops->info->nr_lines;
>> }
>>
>> +/* Validates PPIs/SGIs/SPIs/LPIs supported */
>> +bool_t gic_is_valid_irq(unsigned int irq)
>> +{
>> + return ((irq < gic_hw_ops->info->nr_lines) && is_lpi(irq));
>> +}
>> +
>> unsigned int gic_nr_id_bits(void)
>> {
>> return gic_hw_ops->info->nr_id_bits;
>> @@ -148,7 +155,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>> {
>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>> /* Can't route interrupts that don't exist */
>> - ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
>> + ASSERT(gic_is_valid_irq(desc->irq));
>
>
> The fact that I commented on this earlier is another artefact in the way
> this series presents functionality in an essentially arbitrary order.
>
> I notice that you appear to have reintroduced the logic but when you
> moved this code from here into gic_is_valid_irq. AFAICT that function
> can never return true, but then I'm rather confused about how this
> series can have been tested.
>
>> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
>> ASSERT(spin_is_locked(&desc->lock));
>>
>> @@ -160,6 +167,17 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>> /* Program the GIC to route an interrupt to a guest
>> * - desc.lock must be held
>> */
>> +int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
>> + struct irq_desc *desc, unsigned int priority)
>> +{
>> + ASSERT(spin_is_locked(&desc->lock));
>
> ASSERT(virq >= SOME_DEFINE_FOR_MIN_LPI);
>
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 3806d98..c8ea627 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -62,12 +62,21 @@ hw_irq_controller no_irq_type = {
>> };
>>
>> static irq_desc_t irq_desc[NR_IRQS];
>> +#ifdef CONFIG_ARM_64
>> +static irq_desc_t irq_desc_lpi[NR_GIC_LPI];
>
> http://xenbits.xen.org/people/ianc/vits/draftG.html#irq-descriptors
> contains: "Therefore a second dynamically allocated array will be added
> to cover the range 8192..nr_lpis"
>
> IOW this should be dynamically allocated and therefore NR_GIC_LPI (which
> I think I already commented on earlier) should go away, since the limit
> should come from the h/w.
>> @@ -267,9 +285,14 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>> set_bit(_IRQ_INPROGRESS, &desc->status);
>> desc->arch.eoi_cpu = smp_processor_id();
>>
>> +#ifdef CONFIG_ARM_64
>> + if ( is_lpi(irq) )
>> + vgic_vcpu_inject_lpi(info->d, irq);
>> + else
>> +#endif
>> /* the irq cannot be a PPI, we only support delivery of SPIs to
>> * guests */
>> - vgic_vcpu_inject_spi(info->d, info->virq);
>> + vgic_vcpu_inject_spi(info->d, info->virq);
>
> Comment should be reindented too.
>
>> + if ( !is_lpi(irq) )
>> + {
>> + rank = vgic_rank_irq(v, irq);
>> +
>> + ASSERT(spin_is_locked(&rank->lock));
>> + priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
>> irq, DABT_WORD)], 0, irq & 0x3);
>> + }
>> + if ( is_lpi(irq) && gic_lpi_supported() )
>
> Should be "else if ( gic_lpi_supported() )"
>
>> + vdev = vits_find_device(&d->arch.vits->dev_root, devid);
>> + if ( !vdev )
>> + {
>> + dprintk(XENLOG_WARNING,
>> + "LPI %d received for dev 0x%x not assigned..dropping\n",
>> + irq, devid);
>> + return;
>> + }
>
> This isn't used again and the whole R-B tree should go away.
>
>> + p = irq_to_pending(d->vcpu[0], vitt_entry.vlpi);
>
> Perhaps given that we know this is a vlpi, we could skipping going via
> vcpu[0] and just go right to the entry in the d->pending_lpi array (via
> a suitable accessor of course)
>
>> + p->desc = desc;
>
> This should have happened during routing, not now.
>
> Oh. I see what is going on, your vgic_vcpu_inject_lpi appears to be
> taking an IRQ, and not a vpli, or at least to be confused about what it
> should do with the number which it calls irq.
>
> Therefore you find yourself needing to lookup the irqdesc, and look in
> the vitt etc, none of which belongs here.
>
> For interrupts coming from a plpi all of that lookup should happen based
> on the its_device which is contained in the irq_guest and other info
> from there, before calling this function. You can get the irq_guest
> which you can get because you have a PLPI in your hand and can look up
> the irq_guest via the irq_desc which you get using the plpi number.
>
AUI, you are refering to
http://xenbits.xen.org/people/ianc/vits/draftG.html#virtual-lpi-injection
If we want to extract, its_device, and VITT information before calling
vgic_vcpu_inject_lpi(),
it cannot be done in do_IRQ() as irq.c cannot have vITS related code.
So one option is to call vits_foo() function which will validated plpi and get
its_device / VITT info and call vgic_vcpu_inject_lpi(). So vITS will
call back vgic function.
OR
Introduce new vgic function for doing this and call vgic_vcpu_inject_lpi().
> For interrupts coming from the VITS INT command the command itself
> contains a vDevice and vEvent, which can then be looked up in the ITT to
> get a vpli which can be passed to vgic_vcpu_inject_lpi.
>
> Thus this function has no need to lookup an irq_desc, nor a dev id, nor
> to do an rb tree lookup.
>
> I think both of these cases are adequately explained, with pseudocode,
> in the draftG vits design doc. Please ask if you are unsure.
>
> Ian.
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-20 13:07 ` Vijay Kilari
@ 2015-07-20 13:25 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-20 13:25 UTC (permalink / raw)
To: Vijay Kilari, Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 20/07/15 14:07, Vijay Kilari wrote:
> AUI, you are refering to
>
> http://xenbits.xen.org/people/ianc/vits/draftG.html#virtual-lpi-injection
>
> If we want to extract, its_device, and VITT information before calling
> vgic_vcpu_inject_lpi(),
> it cannot be done in do_IRQ() as irq.c cannot have vITS related code.
IHMO it can't be done in vgic.c too...
>
> So one option is to call vits_foo() function which will validated plpi and get
> its_device / VITT info and call vgic_vcpu_inject_lpi(). So vITS will
> call back vgic function.
> OR
> Introduce new vgic function for doing this and call vgic_vcpu_inject_lpi().
I don't understand why you need to validate the plpi. It's already done
when we receive the IRQ.
Furthermore, you will need to retrieve again the VITT entry in
vgic_vcpu_inject_lpi in order to get the collection associated to the vLPI.
Given that the irq_desc as the vDevID and the eventID and the INT
command will have the same parameters in hand it looks like to me that
the prototype of vgic_vcpu_inject_lpi should be
vgic_vcpu_inject_lpi(struct domain *d, uint32_t vdevid, uint32_t eventid)
And the vgic_vcpu_inject_lpi will take care of retrieving the vLPI and
the collection associated.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 15:30 ` Ian Campbell
2015-07-20 13:07 ` Vijay Kilari
@ 2015-07-22 13:31 ` Vijay Kilari
2015-07-22 13:39 ` Julien Grall
1 sibling, 1 reply; 113+ messages in thread
From: Vijay Kilari @ 2015-07-22 13:31 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Fri, Jul 10, 2015 at 9:00 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index 3ebadcf..92d2be9 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -68,11 +68,18 @@ enum gic_version gic_hw_version(void)
>> return gic_hw_ops->info->hw_version;
>> }
>>
>> +/* Only validates PPIs/SGIs/SPIs supported */
>> unsigned int gic_number_lines(void)
>> {
>> return gic_hw_ops->info->nr_lines;
>> }
>>
>> +/* Validates PPIs/SGIs/SPIs/LPIs supported */
>> +bool_t gic_is_valid_irq(unsigned int irq)
>> +{
>> + return ((irq < gic_hw_ops->info->nr_lines) && is_lpi(irq));
>> +}
>> +
>> unsigned int gic_nr_id_bits(void)
>> {
>> return gic_hw_ops->info->nr_id_bits;
>> @@ -148,7 +155,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>> {
>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>> /* Can't route interrupts that don't exist */
>> - ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
>> + ASSERT(gic_is_valid_irq(desc->irq));
>
>
> The fact that I commented on this earlier is another artefact in the way
> this series presents functionality in an essentially arbitrary order.
>
> I notice that you appear to have reintroduced the logic but when you
> moved this code from here into gic_is_valid_irq. AFAICT that function
> can never return true, but then I'm rather confused about how this
> series can have been tested.
>
>> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
>> ASSERT(spin_is_locked(&desc->lock));
>>
>> @@ -160,6 +167,17 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>> /* Program the GIC to route an interrupt to a guest
>> * - desc.lock must be held
>> */
>> +int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
>> + struct irq_desc *desc, unsigned int priority)
>> +{
>> + ASSERT(spin_is_locked(&desc->lock));
>
> ASSERT(virq >= SOME_DEFINE_FOR_MIN_LPI);
>
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 3806d98..c8ea627 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -62,12 +62,21 @@ hw_irq_controller no_irq_type = {
>> };
>>
>> static irq_desc_t irq_desc[NR_IRQS];
>> +#ifdef CONFIG_ARM_64
>> +static irq_desc_t irq_desc_lpi[NR_GIC_LPI];
>
> http://xenbits.xen.org/people/ianc/vits/draftG.html#irq-descriptors
> contains: "Therefore a second dynamically allocated array will be added
> to cover the range 8192..nr_lpis"
>
> IOW this should be dynamically allocated and therefore NR_GIC_LPI (which
> I think I already commented on earlier) should go away, since the limit
> should come from the h/w.
>> @@ -267,9 +285,14 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>> set_bit(_IRQ_INPROGRESS, &desc->status);
>> desc->arch.eoi_cpu = smp_processor_id();
>>
>> +#ifdef CONFIG_ARM_64
>> + if ( is_lpi(irq) )
>> + vgic_vcpu_inject_lpi(info->d, irq);
>> + else
>> +#endif
>> /* the irq cannot be a PPI, we only support delivery of SPIs to
>> * guests */
>> - vgic_vcpu_inject_spi(info->d, info->virq);
>> + vgic_vcpu_inject_spi(info->d, info->virq);
>
> Comment should be reindented too.
>
>> + if ( !is_lpi(irq) )
>> + {
>> + rank = vgic_rank_irq(v, irq);
>> +
>> + ASSERT(spin_is_locked(&rank->lock));
>> + priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
>> irq, DABT_WORD)], 0, irq & 0x3);
>> + }
>> + if ( is_lpi(irq) && gic_lpi_supported() )
>
> Should be "else if ( gic_lpi_supported() )"
>
>> + vdev = vits_find_device(&d->arch.vits->dev_root, devid);
>> + if ( !vdev )
>> + {
>> + dprintk(XENLOG_WARNING,
>> + "LPI %d received for dev 0x%x not assigned..dropping\n",
>> + irq, devid);
>> + return;
>> + }
>
> This isn't used again and the whole R-B tree should go away.
>
>> + p = irq_to_pending(d->vcpu[0], vitt_entry.vlpi);
>
> Perhaps given that we know this is a vlpi, we could skipping going via
> vcpu[0] and just go right to the entry in the d->pending_lpi array (via
> a suitable accessor of course)
Why can't we use irq_to_pending(d->vcpu[target], vitt_entry.vlpi) ?.
>
>> + p->desc = desc;
>
> This should have happened during routing, not now.
While routing we don't have vlpi to update p->desc.
>
> Oh. I see what is going on, your vgic_vcpu_inject_lpi appears to be
> taking an IRQ, and not a vpli, or at least to be confused about what it
> should do with the number which it calls irq.
>
> Therefore you find yourself needing to lookup the irqdesc, and look in
> the vitt etc, none of which belongs here.
>
> For interrupts coming from a plpi all of that lookup should happen based
> on the its_device which is contained in the irq_guest and other info
> from there, before calling this function. You can get the irq_guest
> which you can get because you have a PLPI in your hand and can look up
> the irq_guest via the irq_desc which you get using the plpi number.
>
> For interrupts coming from the VITS INT command the command itself
> contains a vDevice and vEvent, which can then be looked up in the ITT to
> get a vpli which can be passed to vgic_vcpu_inject_lpi.
>
> Thus this function has no need to lookup an irq_desc, nor a dev id, nor
> to do an rb tree lookup.
>
> I think both of these cases are adequately explained, with pseudocode,
> in the draftG vits design doc. Please ask if you are unsure.
>
> Ian.
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-22 13:31 ` Vijay Kilari
@ 2015-07-22 13:39 ` Julien Grall
2015-07-22 14:17 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-22 13:39 UTC (permalink / raw)
To: Vijay Kilari, Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 22/07/15 14:31, Vijay Kilari wrote:
>>> + p->desc = desc;
>>
>> This should have happened during routing, not now.
>
> While routing we don't have vlpi to update p->desc.
Why do you need the p->desc? You don't use at all the p->desc during the
injection because you replaced all the
if ( p->desc ) by if ( p->desc && !is_lpi(p->irq) )
or similar.
regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-22 13:39 ` Julien Grall
@ 2015-07-22 14:17 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-22 14:17 UTC (permalink / raw)
To: Vijay Kilari, Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 22/07/15 14:39, Julien Grall wrote:
> On 22/07/15 14:31, Vijay Kilari wrote:
>>>> + p->desc = desc;
>>>
>>> This should have happened during routing, not now.
>>
>> While routing we don't have vlpi to update p->desc.
>
> Why do you need the p->desc? You don't use at all the p->desc during the
> injection because you replaced all the
> if ( p->desc ) by if ( p->desc && !is_lpi(p->irq) )
Looking to the spec (ARM IHI 0069A section 5.4):
"Because an LPI does not have an active state, it is not possible to
associate a virtual LPI with a physical interrupt."
So it looks like to me that we never need the IRQ desc for LPI.
Furthermore, it may need some care in gic_update_one_lr given that the
active bit is never set.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 7:42 ` [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route vijay.kilari
2015-07-10 15:30 ` Ian Campbell
@ 2015-07-13 17:03 ` Stefano Stabellini
2015-07-13 17:13 ` Stefano Stabellini
` (2 subsequent siblings)
4 siblings, 0 replies; 113+ messages in thread
From: Stefano Stabellini @ 2015-07-13 17:03 UTC (permalink / raw)
To: vijay.kilari
Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Fri, 10 Jul 2015, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Allocate and initialize irq descriptor for LPIs and
> route LPIs to guest
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Merge patch #16
> - Changed commit message
> ---
> xen/arch/arm/gic-v3.c | 2 +-
> xen/arch/arm/gic.c | 24 +++++++--
> xen/arch/arm/irq.c | 44 +++++++++++++----
> xen/arch/arm/vgic-v3-its.c | 9 ++++
> xen/arch/arm/vgic-v3.c | 15 ++++--
> xen/arch/arm/vgic.c | 110 ++++++++++++++++++++++++++++++++++++++---
> xen/include/asm-arm/domain.h | 3 ++
> xen/include/asm-arm/gic-its.h | 1 +
> xen/include/asm-arm/gic.h | 7 ++-
> xen/include/asm-arm/vgic.h | 1 +
> 10 files changed, 192 insertions(+), 24 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index e6004d2..53554e6 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -895,7 +895,7 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
> val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>
> - if ( p->desc != NULL )
> + if ( p->desc != NULL && !(is_lpi(p->irq)) )
> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
> << GICH_LR_PHYSICAL_SHIFT);
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 3ebadcf..92d2be9 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -68,11 +68,18 @@ enum gic_version gic_hw_version(void)
> return gic_hw_ops->info->hw_version;
> }
>
> +/* Only validates PPIs/SGIs/SPIs supported */
> unsigned int gic_number_lines(void)
> {
> return gic_hw_ops->info->nr_lines;
> }
>
> +/* Validates PPIs/SGIs/SPIs/LPIs supported */
> +bool_t gic_is_valid_irq(unsigned int irq)
> +{
> + return ((irq < gic_hw_ops->info->nr_lines) && is_lpi(irq));
> +}
shouldn't it be ((irq < gic_hw_ops->info->nr_lines) || is_lpi(irq)) ?
> unsigned int gic_nr_id_bits(void)
> {
> return gic_hw_ops->info->nr_id_bits;
> @@ -148,7 +155,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> {
> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> /* Can't route interrupts that don't exist */
> - ASSERT(desc->irq < gic_number_lines() || is_lpi(desc->irq));
> + ASSERT(gic_is_valid_irq(desc->irq));
> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
> ASSERT(spin_is_locked(&desc->lock));
>
> @@ -160,6 +167,17 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> /* Program the GIC to route an interrupt to a guest
> * - desc.lock must be held
> */
> +int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
> + struct irq_desc *desc, unsigned int priority)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> +
> + desc->handler = get_guest_hw_irq_controller(desc->irq);
> + set_bit(_IRQ_GUEST, &desc->status);
> +
> + return 0;
> +}
> +
> int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> struct irq_desc *desc, unsigned int priority)
> {
> @@ -454,7 +472,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
> if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
> test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
> {
> - if ( p->desc == NULL )
> + if ( p->desc == NULL || is_lpi(irq) )
> {
> lr_val.state |= GICH_LR_PENDING;
> gic_hw_ops->write_lr(i, &lr_val);
> @@ -677,7 +695,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
> /* Reading IRQ will ACK it */
> irq = gic_hw_ops->read_irq();
>
> - if ( likely(irq >= 16 && irq < 1020) )
> + if ( (likely(irq >= 16 && irq < 1020)) || is_lpi(irq) )
> {
> local_irq_enable();
> do_IRQ(regs, irq, is_fiq);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 3806d98..c8ea627 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -62,12 +62,21 @@ hw_irq_controller no_irq_type = {
> };
>
> static irq_desc_t irq_desc[NR_IRQS];
> +#ifdef CONFIG_ARM_64
> +static irq_desc_t irq_desc_lpi[NR_GIC_LPI];
> +#endif
> static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
>
> irq_desc_t *__irq_to_desc(int irq)
> {
> if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
> - return &irq_desc[irq-NR_LOCAL_IRQS];
> + else if ( irq >= NR_LOCAL_IRQS && irq < NR_IRQS)
> + return &irq_desc[irq-NR_LOCAL_IRQS];
> +#ifdef CONFIG_ARM_64
> + else if ( is_lpi(irq) )
> + return &irq_desc_lpi[irq - NR_GIC_LPI];
> +#endif
> + return NULL;
> }
>
> int __init arch_init_one_irq_desc(struct irq_desc *desc)
> @@ -88,6 +97,15 @@ static int __init init_irq_data(void)
> desc->action = NULL;
> }
>
> +#ifdef CONFIG_ARM_64
> + for ( irq = NR_GIC_LPI; irq < MAX_LPI; irq++ )
> + {
> + struct irq_desc *desc = irq_to_desc(irq);
> + init_one_irq_desc(desc);
> + desc->irq = irq;
> + desc->action = NULL;
> + }
> +#endif
> return 0;
> }
>
> @@ -208,7 +226,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
> * which interrupt is which (messes up the interrupt freeing
> * logic etc).
> */
> - if ( irq >= nr_irqs )
> + if ( irq >= nr_irqs && !is_lpi(irq) )
> return -EINVAL;
> if ( !handler )
> return -EINVAL;
> @@ -267,9 +285,14 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
> set_bit(_IRQ_INPROGRESS, &desc->status);
> desc->arch.eoi_cpu = smp_processor_id();
>
> +#ifdef CONFIG_ARM_64
> + if ( is_lpi(irq) )
> + vgic_vcpu_inject_lpi(info->d, irq);
> + else
> +#endif
> /* the irq cannot be a PPI, we only support delivery of SPIs to
> * guests */
> - vgic_vcpu_inject_spi(info->d, info->virq);
> + vgic_vcpu_inject_spi(info->d, info->virq);
> goto out_no_end;
> }
>
> @@ -436,7 +459,8 @@ err:
> bool_t is_assignable_irq(unsigned int irq)
> {
> /* For now, we can only route SPIs to the guest */
> - return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
> + return (((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines())) ||
> + is_lpi(irq));
> }
>
> /*
> @@ -452,7 +476,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
> unsigned long flags;
> int retval = 0;
>
> - if ( virq >= vgic_num_irqs(d) )
> + if ( virq >= vgic_num_irqs(d) && !is_lpi(irq) )
> {
> printk(XENLOG_G_ERR
> "the vIRQ number %u is too high for domain %u (max = %u)\n",
> @@ -460,10 +484,10 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
> return -EINVAL;
> }
>
> - /* Only routing to virtual SPIs is supported */
> + /* Only routing to virtual SPIs/LPIs is supported */
> if ( virq < NR_LOCAL_IRQS )
> {
> - printk(XENLOG_G_ERR "IRQ can only be routed to an SPI\n");
> + printk(XENLOG_G_ERR "IRQ can only be routed to an SPI/LPI\n");
> return -EINVAL;
> }
>
> @@ -537,8 +561,10 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
> retval = __setup_irq(desc, 0, action);
> if ( retval )
> goto out;
> -
> - retval = gic_route_irq_to_guest(d, virq, desc, GIC_PRI_IRQ);
> + if ( is_lpi(irq) )
> + retval = gic_route_lpi_to_guest(d, virq, desc, GIC_PRI_IRQ);
> + else
> + retval = gic_route_irq_to_guest(d, virq, desc, GIC_PRI_IRQ);
>
> spin_unlock_irqrestore(&desc->lock, flags);
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index bbcc7bb..4649b07 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -625,6 +625,15 @@ err:
> return 0;
> }
>
> +uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid)
> +{
> + uint8_t priority;
> +
> + priority = readb_relaxed(v->domain->arch.vits->prop_page + pid);
> + priority &= LPI_PRIORITY_MASK;
> +
> + return priority;
> +}
> static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
> {
> uint32_t offset;
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 25b69a0..4e14439 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1111,12 +1111,19 @@ static const struct mmio_handler_ops vgic_distr_mmio_handler = {
>
> static int vgic_v3_get_irq_priority(struct vcpu *v, unsigned int irq)
> {
> - int priority;
> - struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
> + int priority = 0;
> + struct vgic_irq_rank *rank;
>
> - ASSERT(spin_is_locked(&rank->lock));
> - priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
> + if ( !is_lpi(irq) )
> + {
> + rank = vgic_rank_irq(v, irq);
> +
> + ASSERT(spin_is_locked(&rank->lock));
> + priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
> irq, DABT_WORD)], 0, irq & 0x3);
> + }
> + if ( is_lpi(irq) && gic_lpi_supported() )
> + priority = vgic_its_get_priority(v, irq);
>
> return priority;
> }
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index a5f66f6..8190a46 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -30,6 +30,7 @@
>
> #include <asm/mmio.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <asm/vgic.h>
>
> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
> @@ -111,6 +112,15 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> for (i=0; i<d->arch.vgic.nr_spis; i++)
> vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
>
> +#ifdef CONFIG_ARM_64
> + d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq, NR_GIC_LPI);
> + if ( d->arch.vgic.pending_lpis == NULL )
> + return -ENOMEM;
> +
> + for ( i = 0; i < NR_GIC_LPI; i++ )
> + vgic_init_pending_irq(&d->arch.vgic.pending_lpis[i], i);
> +#endif
> +
> for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
>
> @@ -157,6 +167,7 @@ void domain_vgic_free(struct domain *d)
> #ifdef CONFIG_ARM_64
> free_xenheap_pages(d->arch.vits->prop_page,
> get_order_from_bytes(d->arch.vits->prop_size));
> + xfree(d->arch.vgic.pending_lpis);
> #endif
> }
>
> @@ -381,13 +392,17 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
>
> struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
> {
> - struct pending_irq *n;
> + struct pending_irq *n = NULL;
> /* Pending irqs allocation strategy: the first vgic.nr_spis irqs
> * are used for SPIs; the rests are used for per cpu irqs */
> if ( irq < 32 )
> n = &v->arch.vgic.pending_irqs[irq];
> - else
> + else if ( irq < NR_IRQS )
> n = &v->domain->arch.vgic.pending_irqs[irq - 32];
> +#ifdef CONFIG_ARM_64
> + else if ( is_lpi(irq) )
> + n = &v->domain->arch.vgic.pending_lpis[irq - FIRST_GIC_LPI];
> +#endif
> return n;
> }
>
> @@ -413,14 +428,20 @@ void vgic_clear_pending_irqs(struct vcpu *v)
> void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
> {
> uint8_t priority;
> - struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
> + struct vgic_irq_rank *rank;
> struct pending_irq *iter, *n = irq_to_pending(v, virq);
> unsigned long flags;
> bool_t running;
>
> - vgic_lock_rank(v, rank, flags);
> - priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
> - vgic_unlock_rank(v, rank, flags);
> + if ( virq < NR_GIC_LPI )
> + {
> + rank = vgic_rank_irq(v, virq);
> + vgic_lock_rank(v, rank, flags);
> + priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
> + vgic_unlock_rank(v, rank, flags);
> + }
> + else
> + priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
>
> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 7:42 ` [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route vijay.kilari
2015-07-10 15:30 ` Ian Campbell
2015-07-13 17:03 ` Stefano Stabellini
@ 2015-07-13 17:13 ` Stefano Stabellini
2015-07-13 17:36 ` Julien Grall
2015-07-15 18:13 ` Julien Grall
2015-07-16 8:37 ` Julien Grall
4 siblings, 1 reply; 113+ messages in thread
From: Stefano Stabellini @ 2015-07-13 17:13 UTC (permalink / raw)
To: vijay.kilari
Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Fri, 10 Jul 2015, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Allocate and initialize irq descriptor for LPIs and
> route LPIs to guest
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index a5f66f6..8190a46 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -30,6 +30,7 @@
>
> #include <asm/mmio.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <asm/vgic.h>
>
> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
> @@ -111,6 +112,15 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> for (i=0; i<d->arch.vgic.nr_spis; i++)
> vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
>
> +#ifdef CONFIG_ARM_64
> + d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq, NR_GIC_LPI);
Would it be possible to only allocate pending_lpis if lpi_supported?
> + if ( d->arch.vgic.pending_lpis == NULL )
> + return -ENOMEM;
> +
> + for ( i = 0; i < NR_GIC_LPI; i++ )
> + vgic_init_pending_irq(&d->arch.vgic.pending_lpis[i], i);
> +#endif
> +
> for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-13 17:13 ` Stefano Stabellini
@ 2015-07-13 17:36 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-13 17:36 UTC (permalink / raw)
To: Stefano Stabellini, vijay.kilari
Cc: Ian.Campbell, Prasun.Kapoor, Vijaya Kumar K, tim, xen-devel,
stefano.stabellini, manish.jaggi
Hi Stefano,
On 13/07/2015 19:13, Stefano Stabellini wrote:
> On Fri, 10 Jul 2015, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Allocate and initialize irq descriptor for LPIs and
>> route LPIs to guest
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index a5f66f6..8190a46 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -30,6 +30,7 @@
>>
>> #include <asm/mmio.h>
>> #include <asm/gic.h>
>> +#include <asm/gic-its.h>
>> #include <asm/vgic.h>
>>
>> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
>> @@ -111,6 +112,15 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>> for (i=0; i<d->arch.vgic.nr_spis; i++)
>> vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
>>
>> +#ifdef CONFIG_ARM_64
>> + d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq, NR_GIC_LPI);
>
> Would it be possible to only allocate pending_lpis if lpi_supported?
This should be called in the vITS code and not in the common vgic...
Therefore no need to check if lpi are supported.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 7:42 ` [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route vijay.kilari
` (2 preceding siblings ...)
2015-07-13 17:13 ` Stefano Stabellini
@ 2015-07-15 18:13 ` Julien Grall
2015-07-16 8:06 ` Julien Grall
2015-07-16 8:37 ` Julien Grall
4 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-15 18:13 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index e6004d2..53554e6 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -895,7 +895,7 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
> val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>
> - if ( p->desc != NULL )
> + if ( p->desc != NULL && !(is_lpi(p->irq)) )
It seems that you replaced all the p->desc != NULL by "p->desc != NULL
&& !is_lpi(p->irq).
Why don't you avoid to set p->desc in this case?
You may also want to put some explanation in the commit message to
explain why you don't have to set the GICH_LR.HW bit for LPIs.
> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
> << GICH_LR_PHYSICAL_SHIFT);
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 3ebadcf..92d2be9 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -68,11 +68,18 @@ enum gic_version gic_hw_version(void)
> return gic_hw_ops->info->hw_version;
> }
>
> +/* Only validates PPIs/SGIs/SPIs supported */
This comment seems wrong. The function doesn't validate the IRQ but
return the number of Lines (i.e PPIs/SGIs/SPIs).
> unsigned int gic_number_lines(void)
> {
> return gic_hw_ops->info->nr_lines;
> }
[...]
> int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> struct irq_desc *desc, unsigned int priority)
> {
> @@ -454,7 +472,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
> if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
> test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
> {
> - if ( p->desc == NULL )
> + if ( p->desc == NULL || is_lpi(irq) )
> {
> lr_val.state |= GICH_LR_PENDING;
> gic_hw_ops->write_lr(i, &lr_val);
> @@ -677,7 +695,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
> /* Reading IRQ will ACK it */
> irq = gic_hw_ops->read_irq();
>
> - if ( likely(irq >= 16 && irq < 1020) )
> + if ( (likely(irq >= 16 && irq < 1020)) || is_lpi(irq) )
Please move the is_lpi(irq) in likely.
> {
> local_irq_enable();
> do_IRQ(regs, irq, is_fiq);
[...]
> @@ -208,7 +226,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
> * which interrupt is which (messes up the interrupt freeing
> * logic etc).
> */
> - if ( irq >= nr_irqs )
> + if ( irq >= nr_irqs && !is_lpi(irq) )
Technically nr_irqs should contain the total number of IRQ and not only
the number of SPI/PPI/SGI.
Either modify nr_irqs or use gic_is_valid_irq which would do the same here.
> return -EINVAL;
> if ( !handler )
> return -EINVAL;
> @@ -267,9 +285,14 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
> set_bit(_IRQ_INPROGRESS, &desc->status);
> desc->arch.eoi_cpu = smp_processor_id();
>
> +#ifdef CONFIG_ARM_64
> + if ( is_lpi(irq) )
> + vgic_vcpu_inject_lpi(info->d, irq);
> + else
> +#endif
> /* the irq cannot be a PPI, we only support delivery of SPIs to
> * guests */
> - vgic_vcpu_inject_spi(info->d, info->virq);
> + vgic_vcpu_inject_spi(info->d, info->virq);
> goto out_no_end;
> }
>
> @@ -436,7 +459,8 @@ err:
> bool_t is_assignable_irq(unsigned int irq)
> {
> /* For now, we can only route SPIs to the guest */
> - return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
> + return (((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines())) ||
> + is_lpi(irq));
If you modify the function, please also modify the comment which become
invalid now.
[...]
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index bbcc7bb..4649b07 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -625,6 +625,15 @@ err:
> return 0;
> }
>
> +uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid)
> +{
> + uint8_t priority;
> +
> + priority = readb_relaxed(v->domain->arch.vits->prop_page + pid);
Why do you use readb_relaxed here? This should only be used for Device MMIO.
Although, you need to ensure that the value will be correctly
synchronize if another CPU is writing in prop_page which is protected by
prop_lock.
> + priority &= LPI_PRIORITY_MASK;
> +
> + return priority;
> +}
> static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
> {
> uint32_t offset;
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 69bf1ff..537ed3d 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -226,6 +226,9 @@ extern void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mas
> extern int gic_route_irq_to_guest(struct domain *, unsigned int virq,
> struct irq_desc *desc,
> unsigned int priority);
> +extern int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
> + struct irq_desc *desc,
> + unsigned int priority);
>
> /* Remove an IRQ passthrough to a guest */
> int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
> @@ -282,8 +285,10 @@ extern void send_SGI_allbutself(enum gic_sgi sgi);
> /* print useful debug info */
> extern void gic_dump_info(struct vcpu *v);
>
> -/* Number of interrupt lines */
> +/* Number of interrupt lines (SPIs)*/
This is not really true. The interrupts lines is equals to PPIs + SGIs +
SPIs.
> extern unsigned int gic_number_lines(void);
> +/* Check if irq is valid SPI or LPI */
This comment is not true. This function is used to check that an IRQ is
valid in general, not only for SPI or LPI.
> +bool_t gic_is_valid_irq(unsigned int irq);
> /* Number of interrupt id bits supported */
> extern unsigned int gic_nr_id_bits(void);
> /* LPI support info */
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-15 18:13 ` Julien Grall
@ 2015-07-16 8:06 ` Julien Grall
0 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-16 8:06 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 15/07/2015 20:13, Julien Grall wrote:
>> +uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid)
>> +{
>> + uint8_t priority;
>> +
>> + priority = readb_relaxed(v->domain->arch.vits->prop_page + pid);
>
> Why do you use readb_relaxed here? This should only be used for Device
> MMIO.
>
> Although, you need to ensure that the value will be correctly
> synchronize if another CPU is writing in prop_page which is protected by
> prop_lock.
I though a bit more during the night about this function. On patch #11,
where you allocate prop_page, you allow to have a smaller table than the
number of LPIs. If the pid is too high, even though valid, you may read
Xen memory or even crash Xen.
Although, what does mean pid? Should not it be vlpi?
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route
2015-07-10 7:42 ` [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route vijay.kilari
` (3 preceding siblings ...)
2015-07-15 18:13 ` Julien Grall
@ 2015-07-16 8:37 ` Julien Grall
4 siblings, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-16 8:37 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 3806d98..c8ea627 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
[...]
> int __init arch_init_one_irq_desc(struct irq_desc *desc)
> @@ -88,6 +97,15 @@ static int __init init_irq_data(void)
> desc->action = NULL;
> }
>
> +#ifdef CONFIG_ARM_64
> + for ( irq = NR_GIC_LPI; irq < MAX_LPI; irq++ )
NR_GIC_LPI = 4096 which is the maximum number of LPIs supported you've
hardcoded.
Here you want to use FIRST_GIC_LPI.
> + {
> + struct irq_desc *desc = irq_to_desc(irq);
> + init_one_irq_desc(desc);
> + desc->irq = irq;
> + desc->action = NULL;
> + }
> +#endif
> return 0;
> }
>
> @@ -208,7 +226,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
> * which interrupt is which (messes up the interrupt freeing
> * logic etc).
> */
> - if ( irq >= nr_irqs )
> + if ( irq >= nr_irqs && !is_lpi(irq) )
It's expecting that nr_irqs will encompass SPIs, LPIs...
In this particular case, you want to use gic_is_valid_irq.
> @@ -436,7 +459,8 @@ err:
> bool_t is_assignable_irq(unsigned int irq)
> {
> /* For now, we can only route SPIs to the guest */
> - return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
> + return (((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines())) ||
> + is_lpi(irq));
> }
>
> /*
> @@ -452,7 +476,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
It's not right to re-use route_irq_to_guest for a completely different
meaning. The virq stands for virtual IRQ and not eventID.
For more details, please see my remarks on patch #8, #5 and the feature
freeze thread.
IHMO, the prototype to route an LPI to a guest should be
route_lpi_to_guest(struct domain *d, unsigned int lpi);
[...]
> spin_unlock_irqrestore(&desc->lock, flags);
>
[...]
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 25b69a0..4e14439 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1111,12 +1111,19 @@ static const struct mmio_handler_ops vgic_distr_mmio_handler = {
>
> static int vgic_v3_get_irq_priority(struct vcpu *v, unsigned int irq)
> {
> - int priority;
> - struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
> + int priority = 0;
> + struct vgic_irq_rank *rank;
>
> - ASSERT(spin_is_locked(&rank->lock));
> - priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
> + if ( !is_lpi(irq) )
is_lpi is checking the validity of the host LPI not the guest LPI.
> + {
> + rank = vgic_rank_irq(v, irq);
> +
> + ASSERT(spin_is_locked(&rank->lock));
> + priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
> irq, DABT_WORD)], 0, irq & 0x3);
> + }
> + if ( is_lpi(irq) && gic_lpi_supported() )
> + priority = vgic_its_get_priority(v, irq);
I'm wondering how you tested the series... To get the priority in the
LPI configuration table, you have to use substract 8192 before reading
in the config table. But, here you are directly using the value to read
in the table.
>
> return priority;
> }
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index a5f66f6..8190a46 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -30,6 +30,7 @@
>
> #include <asm/mmio.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
I'm not in favor to see any include of the gic-its.h in the common code.
It will likely means that the build on arm32 is broken...
> #include <asm/vgic.h>
>
> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
> @@ -111,6 +112,15 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> for (i=0; i<d->arch.vgic.nr_spis; i++)
> vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
>
> +#ifdef CONFIG_ARM_64
> + d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq, NR_GIC_LPI);
> + if ( d->arch.vgic.pending_lpis == NULL )
> + return -ENOMEM;
> +
> + for ( i = 0; i < NR_GIC_LPI; i++ )
> + vgic_init_pending_irq(&d->arch.vgic.pending_lpis[i], i);
> +#endif
> +
This should go in the vITS code.
> for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
>
> @@ -157,6 +167,7 @@ void domain_vgic_free(struct domain *d)
> #ifdef CONFIG_ARM_64
> free_xenheap_pages(d->arch.vits->prop_page,
> get_order_from_bytes(d->arch.vits->prop_size));
> + xfree(d->arch.vgic.pending_lpis);
> #endif
> }
>
> @@ -381,13 +392,17 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
>
> struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
> {
> - struct pending_irq *n;
> + struct pending_irq *n = NULL;
> /* Pending irqs allocation strategy: the first vgic.nr_spis irqs
> * are used for SPIs; the rests are used for per cpu irqs */
Please update this comment.
> if ( irq < 32 )
> n = &v->arch.vgic.pending_irqs[irq];
> - else
> + else if ( irq < NR_IRQS )
You should use vgic_num_irqs and not NR_IRQS which is Xen specific.
> n = &v->domain->arch.vgic.pending_irqs[irq - 32];
> +#ifdef CONFIG_ARM_64
> + else if ( is_lpi(irq) )
You helper is_lpi is checking that the irq is valid based of the number
of LPIs supported by the host.
In the case of the guest, the number of LPIs may be different. Please
check this irq against the virtual gic.
> + n = &v->domain->arch.vgic.pending_lpis[irq - FIRST_GIC_LPI];
> +#endif
Please add ASSERT(n != NULL) given that with your change, it may be
possible to return NULL.
> return n;
> }
>
> @@ -413,14 +428,20 @@ void vgic_clear_pending_irqs(struct vcpu *v)
> void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
> {
> uint8_t priority;
> - struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
> + struct vgic_irq_rank *rank;
> struct pending_irq *iter, *n = irq_to_pending(v, virq);
> unsigned long flags;
> bool_t running;
>
> - vgic_lock_rank(v, rank, flags);
> - priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
> - vgic_unlock_rank(v, rank, flags);
> + if ( virq < NR_GIC_LPI )
> + {
> + rank = vgic_rank_irq(v, virq);
> + vgic_lock_rank(v, rank, flags);
> + priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
> + vgic_unlock_rank(v, rank, flags);
> + }
> + else
> + priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
Why didn't you push vgic_rank_* call in get_irq_priority?
IHMO, a function should be called the same way for all the arguments
rather than taking a lock depending on the value of the argument.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 13/17] xen/arm: ITS: Initialize physical ITS
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (11 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 12/17] xen/arm: ITS: Initialize LPI irq descriptors and route vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-13 17:06 ` Stefano Stabellini
2015-07-10 7:42 ` [PATCH v4 14/17] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
` (3 subsequent siblings)
16 siblings, 1 reply; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Initialize physical ITS driver from GIC v3 driver
if LPIs are supported by hardware
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 53554e6..f4881d7 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -694,6 +694,10 @@ static int __cpuinit gicv3_cpu_init(void)
if ( gicv3_enable_redist() )
return -ENODEV;
+ /* Give LPIs a spin */
+ if ( gicv3_info.lpi_supported )
+ its_cpu_init();
+
/* Set priority on PPI and SGI interrupts */
priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
GIC_PRI_IPI);
@@ -1312,6 +1316,9 @@ static int __init gicv3_init(void)
else
gicv3_info.lpi_supported = 0;
+ if ( gicv3_info.lpi_supported )
+ its_init(&gicv3.rdist_data);
+
gicv3_dist_init();
res = gicv3_cpu_init();
gicv3_hyp_init();
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 13/17] xen/arm: ITS: Initialize physical ITS
2015-07-10 7:42 ` [PATCH v4 13/17] xen/arm: ITS: Initialize physical ITS vijay.kilari
@ 2015-07-13 17:06 ` Stefano Stabellini
0 siblings, 0 replies; 113+ messages in thread
From: Stefano Stabellini @ 2015-07-13 17:06 UTC (permalink / raw)
To: vijay.kilari
Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Fri, 10 Jul 2015, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Initialize physical ITS driver from GIC v3 driver
> if LPIs are supported by hardware
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 53554e6..f4881d7 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -694,6 +694,10 @@ static int __cpuinit gicv3_cpu_init(void)
> if ( gicv3_enable_redist() )
> return -ENODEV;
>
> + /* Give LPIs a spin */
> + if ( gicv3_info.lpi_supported )
> + its_cpu_init();
I see that actually is the other way around compared to what I
previously suggested. That is OK if we can count on the fact that if
lpis are supported by the gic, the presence of an its is guaranteed, but
I am not sure that is the case.
Otherwise we also need to check that an its is actually present.
> /* Set priority on PPI and SGI interrupts */
> priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
> GIC_PRI_IPI);
> @@ -1312,6 +1316,9 @@ static int __init gicv3_init(void)
> else
> gicv3_info.lpi_supported = 0;
>
> + if ( gicv3_info.lpi_supported )
> + its_init(&gicv3.rdist_data);
> +
> gicv3_dist_init();
> res = gicv3_cpu_init();
> gicv3_hyp_init();
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 14/17] xen/arm: ITS: Add domain specific ITS initialization
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (12 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 13/17] xen/arm: ITS: Initialize physical ITS vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 15:41 ` Ian Campbell
2015-07-15 17:41 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 15/17] xen/arm: ITS: Map ITS translation space vijay.kilari
` (2 subsequent siblings)
16 siblings, 2 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add Domain and vcpu specific ITS initialization
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 17 ++++++++++++++++
xen/arch/arm/setup.c | 4 +++-
xen/arch/arm/vgic-v3-its.c | 39 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/vgic-v3.c | 3 +++
xen/include/asm-arm/gic-its.h | 2 ++
xen/include/asm-arm/gic_v3_defs.h | 2 ++
xen/include/asm-arm/vgic.h | 1 +
7 files changed, 67 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 5e6c7f2..b159b0b 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -1256,6 +1256,23 @@ static int its_force_quiescent(void __iomem *base)
}
}
+void its_domain_init(struct domain *d)
+{
+ struct its_node *its;
+
+ if ( is_hardware_domain(d) )
+ {
+ /*
+ * Only one virtual ITS is provided to domain.
+ * Assign first physical ITS address to Dom0 virtual ITS.
+ */
+ its = list_first_entry(&its_nodes, struct its_node, entry);
+ d->arch.vits->gits_base = its->phys_base;
+ d->arch.vits->gits_size = its->phys_size;
+ }
+ /* TODO: DomU */
+}
+
static int its_probe(struct dt_device_node *node)
{
paddr_t its_addr, its_size;
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 06f8e54..3bd1f2e 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -776,7 +776,9 @@ void __init start_xen(unsigned long boot_phys_offset,
init_xen_time();
gic_init();
-
+#ifdef CONFIG_ARM_64
+ vgic_its_init();
+#endif
p2m_vmid_allocator_init();
softirq_init();
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 4649b07..74e6ee7 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -1082,6 +1082,45 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
.write_handler = vgic_v3_gits_mmio_write,
};
+int vits_domain_init(struct domain *d)
+{
+ int i;
+
+ d->arch.vits = xzalloc(struct vgic_its);
+ if ( !d->arch.vits )
+ return -ENOMEM;
+
+ spin_lock_init(&d->arch.vits->lock);
+ spin_lock_init(&d->arch.vits->dev_lock);
+ spin_lock_init(&d->arch.vits->prop_lock);
+
+ d->arch.vits->collections = xzalloc_array(struct its_collection,
+ nr_cpu_ids);
+ if ( !d->arch.vits->collections )
+ return -ENOMEM;
+
+ for ( i = 0; i < nr_cpu_ids; i++ )
+ d->arch.vits->collections[i].target_address = ~0UL;
+
+ d->arch.vits->baser = GITS_BASER_INIT_VAL;
+ d->arch.vits->dev_root = RB_ROOT;
+
+ spin_lock_init(&d->arch.vits->lock);
+
+ its_domain_init(d);
+ register_mmio_handler(d, &vgic_gits_mmio_handler,
+ d->arch.vits->gits_base,
+ SZ_64K);
+
+ return 0;
+}
+
+void vgic_its_init(void)
+{
+ if ( gic_lpi_supported() )
+ its_lpi_init(gic_nr_id_bits());
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 4e14439..79dbfc4 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1217,6 +1217,9 @@ static int vgic_v3_domain_init(struct domain *d)
d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
+ if ( gic_lpi_supported() )
+ vits_domain_init(d);
+
return 0;
}
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index fbed905..cbe7596 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -265,6 +265,8 @@ struct its_device *its_find_device(u32 devid);
int its_insert_device(struct its_device *dev);
int its_add_device(u32 devid);
int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid);
+void its_domain_init(struct domain *d);
+int vits_domain_init(struct domain *d);
int vits_set_vitt_entry(struct domain *d, uint32_t devid,
uint32_t event, struct vitt *entry);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 368ebb3..24ef547 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -243,6 +243,8 @@
#define GITS_BASER_TYPE_RESERVED5 5
#define GITS_BASER_TYPE_RESERVED6 6
#define GITS_BASER_TYPE_RESERVED7 7
+#define GITS_BASER_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
+ (0x7UL << GITS_BASER_ENTRY_SIZE_SHIFT))
/* GITS_PIDRn register values for ARM implementations */
#define GITS_PIDR0_VAL (0x94)
#define GITS_PIDR1_VAL (0xb4)
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index f8928ab..4509e9a 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -178,6 +178,7 @@ enum gic_sgi_mode;
#define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
+extern void vgic_its_init(void);
extern void domain_vgic_free(struct domain *d);
extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 14/17] xen/arm: ITS: Add domain specific ITS initialization
2015-07-10 7:42 ` [PATCH v4 14/17] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
@ 2015-07-10 15:41 ` Ian Campbell
2015-07-15 17:41 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 15:41 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> +void vgic_its_init(void)
> +{
> + if ( gic_lpi_supported() )
> + its_lpi_init(gic_nr_id_bits());
Ah, here is some code which tries to only enable this stuff when it is
needed.
I think for the time being this also needs to be gated on dom0, which in
practice means is_hardware_domain(d).
When PCI passthrough lands then this will be changed to gate on
is_hardware_domain(d) || some_option instead.
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 4e14439..79dbfc4 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1217,6 +1217,9 @@ static int vgic_v3_domain_init(struct domain *d)
>
> d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
>
> + if ( gic_lpi_supported() )
> + vits_domain_init(d);
and here as well. I suspect there will be other places too.
Ian.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 14/17] xen/arm: ITS: Add domain specific ITS initialization
2015-07-10 7:42 ` [PATCH v4 14/17] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
2015-07-10 15:41 ` Ian Campbell
@ 2015-07-15 17:41 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 17:41 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> +void its_domain_init(struct domain *d)
> +{
> + struct its_node *its;
> +
> + if ( is_hardware_domain(d) )
> + {
> + /*
> + * Only one virtual ITS is provided to domain.
> + * Assign first physical ITS address to Dom0 virtual ITS.
> + */
> + its = list_first_entry(&its_nodes, struct its_node, entry);
> + d->arch.vits->gits_base = its->phys_base;
> + d->arch.vits->gits_size = its->phys_size;
> + }
> + /* TODO: DomU */
> +}
> +
With the new vGIC infrastructure, anything related to domain
initialization is done in the vGIC.
> static int its_probe(struct dt_device_node *node)
> {
> paddr_t its_addr, its_size;
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 06f8e54..3bd1f2e 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -776,7 +776,9 @@ void __init start_xen(unsigned long boot_phys_offset,
> init_xen_time();
>
> gic_init();
> -
> +#ifdef CONFIG_ARM_64
> + vgic_its_init();
> +#endif
> p2m_vmid_allocator_init();
>
> softirq_init();
[...]
> +void vgic_its_init(void)
> +{
> + if ( gic_lpi_supported() )
> + its_lpi_init(gic_nr_id_bits());
> +}
> +
As said on patch #3 its_lpi_init can be called in its_init.
In any anycase, it's not the goal of the vGIC to initialize the ITS driver.
[...]
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 368ebb3..24ef547 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -243,6 +243,8 @@
> #define GITS_BASER_TYPE_RESERVED5 5
> #define GITS_BASER_TYPE_RESERVED6 6
> #define GITS_BASER_TYPE_RESERVED7 7
> +#define GITS_BASER_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
> + (0x7UL << GITS_BASER_ENTRY_SIZE_SHIFT))
Please keep this define within the vgic ITS code.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 15/17] xen/arm: ITS: Map ITS translation space
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (13 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 14/17] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 15:43 ` Ian Campbell
2015-07-15 9:01 ` Julien Grall
2015-07-10 7:42 ` [PATCH v4 16/17] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
2015-07-10 7:42 ` [PATCH v4 17/17] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
16 siblings, 2 replies; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
ITS translation space contains GITS_TRANSLATOR
register which is written by device to raise
LPI. This space needs to mapped to every domain
address space for all physical ITS available,
so that device can access GITS_TRANSLATOR
register using SMMU.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/vgic-v3-its.c | 31 ++++++++++++++++++++++++++++++-
1 file changed, 30 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 74e6ee7..301f065 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -1082,6 +1082,35 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
.write_handler = vgic_v3_gits_mmio_write,
};
+/*
+ * Map the 64K ITS translation space in guest.
+ * This is required purely for device smmu writes.
+*/
+
+static int vits_map_translation_space(struct domain *d)
+{
+ uint64_t addr, size;
+ int ret;
+
+ addr = d->arch.vits->gits_base + SZ_64K;
+ size = SZ_64K;
+
+ ret = map_mmio_regions(d,
+ paddr_to_pfn(addr & PAGE_MASK),
+ DIV_ROUND_UP(size, PAGE_SIZE),
+ paddr_to_pfn(addr & PAGE_MASK));
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "vITS: Unable to map to dom%d access to"
+ " 0x%"PRIx64" - 0x%"PRIx64"\n",
+ d->domain_id,
+ addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
+ }
+
+ return ret;
+}
+
int vits_domain_init(struct domain *d)
{
int i;
@@ -1112,7 +1141,7 @@ int vits_domain_init(struct domain *d)
d->arch.vits->gits_base,
SZ_64K);
- return 0;
+ return vits_map_translation_space(d);
}
void vgic_its_init(void)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 15/17] xen/arm: ITS: Map ITS translation space
2015-07-10 7:42 ` [PATCH v4 15/17] xen/arm: ITS: Map ITS translation space vijay.kilari
@ 2015-07-10 15:43 ` Ian Campbell
2015-07-15 9:01 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 15:43 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> ITS translation space contains GITS_TRANSLATOR
> register which is written by device to raise
> LPI. This space needs to mapped to every domain
> address space for all physical ITS available,
> so that device can access GITS_TRANSLATOR
> register using SMMU.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 15/17] xen/arm: ITS: Map ITS translation space
2015-07-10 7:42 ` [PATCH v4 15/17] xen/arm: ITS: Map ITS translation space vijay.kilari
2015-07-10 15:43 ` Ian Campbell
@ 2015-07-15 9:01 ` Julien Grall
1 sibling, 0 replies; 113+ messages in thread
From: Julien Grall @ 2015-07-15 9:01 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 10/07/2015 09:42, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> ITS translation space contains GITS_TRANSLATOR
> register which is written by device to raise
> LPI. This space needs to mapped to every domain
> address space for all physical ITS available,
> so that device can access GITS_TRANSLATOR
> register using SMMU.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 31 ++++++++++++++++++++++++++++++-
> 1 file changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 74e6ee7..301f065 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -1082,6 +1082,35 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
> .write_handler = vgic_v3_gits_mmio_write,
> };
>
> +/*
> + * Map the 64K ITS translation space in guest.
> + * This is required purely for device smmu writes.
> +*/
> +
> +static int vits_map_translation_space(struct domain *d)
> +{
> + uint64_t addr, size;
> + int ret;
> +
> + addr = d->arch.vits->gits_base + SZ_64K;
> + size = SZ_64K;
> +
> + ret = map_mmio_regions(d,
> + paddr_to_pfn(addr & PAGE_MASK),
> + DIV_ROUND_UP(size, PAGE_SIZE),
> + paddr_to_pfn(addr & PAGE_MASK));
You are assuming a direct mapping in the guest memory for the ITS
translation space.
While this may be true for dom0, it won't work for guests.
I'm fine if you don't handle this case for 4.6. Although I'd like to at
least see a comment stating that we are using 1:1 mapping and an assert
to check if the domain is using direct mapping (i.e
is_domain_direct_mapped(d)).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 16/17] xen/arm: ITS: Generate ITS node for Dom0
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (14 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 15/17] xen/arm: ITS: Map ITS translation space vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-13 16:32 ` Stefano Stabellini
2015-07-10 7:42 ` [PATCH v4 17/17] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
16 siblings, 1 reply; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Parse host dt and generate ITS node for Dom0.
ITS node resides inside GIC node so when GIC node
is encountered look for ITS node.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Generate only one ITS node for Dom0
- Replace msi-parent references to single its phandle
---
xen/arch/arm/domain_build.c | 78 +++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3-its.c | 57 ++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 2 ++
3 files changed, 137 insertions(+)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index e9cb8a9..5c62437 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -20,6 +20,7 @@
#include <asm/cpufeature.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <xen/irq.h>
#include "kernel.h"
@@ -61,6 +62,9 @@ custom_param("dom0_mem", parse_dom0_mem);
*/
#define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
+#ifdef CONFIG_ARM_64
+static fdt32_t its_phandle;
+#endif
struct vcpu *__init alloc_dom0_vcpu0(struct domain *dom0)
{
if ( opt_dom0_max_vcpus == 0 )
@@ -468,6 +472,18 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
continue;
}
+#ifdef CONFIG_ARM_64
+ /*
+ * Replace all msi-parent phandle references to single ITS node
+ * generated for Dom0
+ */
+ if ( dt_property_name_is_equal(prop, "msi-parent") )
+ {
+ fdt_property(kinfo->fdt, prop->name, (void *)&its_phandle,
+ sizeof(its_phandle));
+ continue;
+ }
+#endif
res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
xfree(new_data);
@@ -803,6 +819,38 @@ static int make_cpus_node(const struct domain *d, void *fdt,
return res;
}
+#ifdef CONFIG_ARM_64
+static int make_its_node(const struct domain *d, void *fdt,
+ const struct dt_device_node *node)
+{
+ int res = 0;
+
+ DPRINT("Create GIC ITS node\n");
+
+ res = its_make_dt_node(d, node, fdt);
+ if ( res )
+ return res;
+
+ /*
+ * The value of the property "phandle" in the property "interrupts"
+ * to know on which interrupt controller the interrupt is wired.
+ */
+ if ( node->phandle )
+ {
+ DPRINT(" Set phandle = 0x%x\n", node->phandle);
+ res = fdt_property_cell(fdt, "phandle", node->phandle);
+ if ( res )
+ return res;
+ }
+
+ its_phandle = cpu_to_fdt32(node->phandle);
+
+ res = fdt_end_node(fdt);
+
+ return res;
+}
+#endif
+
static int make_gic_node(const struct domain *d, void *fdt,
const struct dt_device_node *node)
{
@@ -1119,6 +1167,14 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
DT_MATCH_TIMER,
{ /* sentinel */ },
};
+#ifdef CONFIG_ARM_64
+ static const struct dt_device_match gits_matches[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+ struct dt_device_node *gic_child;
+#endif
struct dt_device_node *child;
int res;
const char *name;
@@ -1143,7 +1199,29 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
/* Replace these nodes with our own. Note that the original may be
* used_by DOMID_XEN so this check comes first. */
if ( device_get_class(node) == DEVICE_GIC )
+ {
+#ifdef CONFIG_ARM_64
+ if ( !make_gic_node(d, kinfo->fdt, node) )
+ {
+ res = 0;
+ dt_for_each_child_node(node, gic_child)
+ {
+ if ( gic_child != NULL )
+ {
+ if ( dt_match_node(gits_matches, gic_child) )
+ {
+ res = make_its_node(d, kinfo->fdt, gic_child);
+ break;
+ }
+ }
+ }
+ return res;
+ }
+ return 0;
+#else
return make_gic_node(d, kinfo->fdt, node);
+#endif
+ }
if ( dt_match_node(timer_matches, node) )
return make_timer_node(d, kinfo->fdt, node);
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index b159b0b..4193624 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -27,6 +27,8 @@
#include <xen/sched.h>
#include <xen/errno.h>
#include <xen/delay.h>
+#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
#include <xen/list.h>
#include <xen/sizes.h>
#include <xen/vmap.h>
@@ -1227,6 +1229,61 @@ static void its_cpu_init_collection(void)
spin_unlock(&its_lock);
}
+int its_make_dt_node(const struct domain *d,
+ const struct dt_device_node *node, void *fdt)
+{
+ struct its_node *its;
+ const struct dt_device_node *gic;
+ const void *compatible = NULL;
+ u32 len;
+ __be32 *new_cells, *tmp;
+ int res = 0;
+
+ /* Will pass only first ITS node info */
+ /* TODO: Handle multi node */
+ its = list_first_entry(&its_nodes, struct its_node, entry);
+ if ( !its )
+ {
+ dprintk(XENLOG_ERR, "ITS node not found\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ gic = its->dt_node;
+
+ compatible = dt_get_property(gic, "compatible", &len);
+ if ( !compatible )
+ {
+ dprintk(XENLOG_ERR, "Can't find compatible property for the its node\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ res = fdt_begin_node(fdt, "gic-its");
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "compatible", compatible, len);
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "msi-controller", NULL, 0);
+ if ( res )
+ return res;
+
+ len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+
+ new_cells = xzalloc_bytes(len);
+ if ( new_cells == NULL )
+ return -FDT_ERR_XEN(ENOMEM);
+ tmp = new_cells;
+
+ dt_set_range(&tmp, node, its->phys_base, its->phys_size);
+
+ res = fdt_property(fdt, "reg", new_cells, len);
+ xfree(new_cells);
+
+ return res;
+}
+
static int its_force_quiescent(void __iomem *base)
{
u32 count = 1000000; /* 1s */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index cbe7596..1bb9825 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -258,6 +258,8 @@ struct gic_its_info {
u32 its_get_id_bits(void);
u32 its_get_dev_bits(void);
u32 its_get_nr_events(void);
+int its_make_dt_node(const struct domain *d,
+ const struct dt_device_node *node, void *fdt);
int its_lpi_init(u32 id_bits);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 16/17] xen/arm: ITS: Generate ITS node for Dom0
2015-07-10 7:42 ` [PATCH v4 16/17] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
@ 2015-07-13 16:32 ` Stefano Stabellini
2015-07-13 17:31 ` Julien Grall
0 siblings, 1 reply; 113+ messages in thread
From: Stefano Stabellini @ 2015-07-13 16:32 UTC (permalink / raw)
To: vijay.kilari
Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Fri, 10 Jul 2015, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Parse host dt and generate ITS node for Dom0.
> ITS node resides inside GIC node so when GIC node
> is encountered look for ITS node.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Generate only one ITS node for Dom0
> - Replace msi-parent references to single its phandle
> ---
> xen/arch/arm/domain_build.c | 78 +++++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/gic-v3-its.c | 57 ++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 2 ++
> 3 files changed, 137 insertions(+)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index e9cb8a9..5c62437 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -20,6 +20,7 @@
> #include <asm/cpufeature.h>
>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <xen/irq.h>
> #include "kernel.h"
>
> @@ -61,6 +62,9 @@ custom_param("dom0_mem", parse_dom0_mem);
> */
> #define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
>
> +#ifdef CONFIG_ARM_64
> +static fdt32_t its_phandle;
> +#endif
> struct vcpu *__init alloc_dom0_vcpu0(struct domain *dom0)
> {
> if ( opt_dom0_max_vcpus == 0 )
> @@ -468,6 +472,18 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
> continue;
> }
>
> +#ifdef CONFIG_ARM_64
> + /*
> + * Replace all msi-parent phandle references to single ITS node
> + * generated for Dom0
> + */
> + if ( dt_property_name_is_equal(prop, "msi-parent") )
> + {
> + fdt_property(kinfo->fdt, prop->name, (void *)&its_phandle,
> + sizeof(its_phandle));
> + continue;
> + }
> +#endif
> res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
>
> xfree(new_data);
> @@ -803,6 +819,38 @@ static int make_cpus_node(const struct domain *d, void *fdt,
> return res;
> }
>
> +#ifdef CONFIG_ARM_64
> +static int make_its_node(const struct domain *d, void *fdt,
> + const struct dt_device_node *node)
> +{
> + int res = 0;
> +
> + DPRINT("Create GIC ITS node\n");
> +
> + res = its_make_dt_node(d, node, fdt);
> + if ( res )
> + return res;
> +
> + /*
> + * The value of the property "phandle" in the property "interrupts"
> + * to know on which interrupt controller the interrupt is wired.
> + */
> + if ( node->phandle )
> + {
> + DPRINT(" Set phandle = 0x%x\n", node->phandle);
> + res = fdt_property_cell(fdt, "phandle", node->phandle);
> + if ( res )
> + return res;
> + }
> +
> + its_phandle = cpu_to_fdt32(node->phandle);
> +
> + res = fdt_end_node(fdt);
> +
> + return res;
> +}
> +#endif
> +
> static int make_gic_node(const struct domain *d, void *fdt,
> const struct dt_device_node *node)
> {
> @@ -1119,6 +1167,14 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
> DT_MATCH_TIMER,
> { /* sentinel */ },
> };
> +#ifdef CONFIG_ARM_64
> + static const struct dt_device_match gits_matches[] __initconst =
> + {
> + DT_MATCH_GIC_ITS,
> + { /* sentinel */ },
> + };
> + struct dt_device_node *gic_child;
> +#endif
> struct dt_device_node *child;
> int res;
> const char *name;
> @@ -1143,7 +1199,29 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
> /* Replace these nodes with our own. Note that the original may be
> * used_by DOMID_XEN so this check comes first. */
> if ( device_get_class(node) == DEVICE_GIC )
> + {
> +#ifdef CONFIG_ARM_64
> + if ( !make_gic_node(d, kinfo->fdt, node) )
> + {
> + res = 0;
> + dt_for_each_child_node(node, gic_child)
> + {
> + if ( gic_child != NULL )
> + {
> + if ( dt_match_node(gits_matches, gic_child) )
> + {
> + res = make_its_node(d, kinfo->fdt, gic_child);
> + break;
> + }
> + }
> + }
> + return res;
> + }
> + return 0;
> +#else
> return make_gic_node(d, kinfo->fdt, node);
> +#endif
All these #ifdefs are a bit ugly. Couldn't we just build this code
always, even on arm32, relying on dt_match_node not to match in that
case?
> + }
> if ( dt_match_node(timer_matches, node) )
> return make_timer_node(d, kinfo->fdt, node);
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index b159b0b..4193624 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -27,6 +27,8 @@
> #include <xen/sched.h>
> #include <xen/errno.h>
> #include <xen/delay.h>
> +#include <xen/device_tree.h>
> +#include <xen/libfdt/libfdt.h>
> #include <xen/list.h>
> #include <xen/sizes.h>
> #include <xen/vmap.h>
> @@ -1227,6 +1229,61 @@ static void its_cpu_init_collection(void)
> spin_unlock(&its_lock);
> }
>
> +int its_make_dt_node(const struct domain *d,
> + const struct dt_device_node *node, void *fdt)
> +{
> + struct its_node *its;
> + const struct dt_device_node *gic;
> + const void *compatible = NULL;
> + u32 len;
> + __be32 *new_cells, *tmp;
> + int res = 0;
> +
> + /* Will pass only first ITS node info */
> + /* TODO: Handle multi node */
> + its = list_first_entry(&its_nodes, struct its_node, entry);
> + if ( !its )
> + {
> + dprintk(XENLOG_ERR, "ITS node not found\n");
> + return -FDT_ERR_XEN(ENOENT);
> + }
> +
> + gic = its->dt_node;
> +
> + compatible = dt_get_property(gic, "compatible", &len);
> + if ( !compatible )
> + {
> + dprintk(XENLOG_ERR, "Can't find compatible property for the its node\n");
> + return -FDT_ERR_XEN(ENOENT);
> + }
> +
> + res = fdt_begin_node(fdt, "gic-its");
> + if ( res )
> + return res;
> +
> + res = fdt_property(fdt, "compatible", compatible, len);
> + if ( res )
> + return res;
> +
> + res = fdt_property(fdt, "msi-controller", NULL, 0);
> + if ( res )
> + return res;
> +
> + len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
> +
> + new_cells = xzalloc_bytes(len);
> + if ( new_cells == NULL )
> + return -FDT_ERR_XEN(ENOMEM);
> + tmp = new_cells;
> +
> + dt_set_range(&tmp, node, its->phys_base, its->phys_size);
> +
> + res = fdt_property(fdt, "reg", new_cells, len);
> + xfree(new_cells);
> +
> + return res;
> +}
> +
> static int its_force_quiescent(void __iomem *base)
> {
> u32 count = 1000000; /* 1s */
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index cbe7596..1bb9825 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -258,6 +258,8 @@ struct gic_its_info {
> u32 its_get_id_bits(void);
> u32 its_get_dev_bits(void);
> u32 its_get_nr_events(void);
> +int its_make_dt_node(const struct domain *d,
> + const struct dt_device_node *node, void *fdt);
> int its_lpi_init(u32 id_bits);
> int its_init(struct rdist_prop *rdists);
> int its_cpu_init(void);
> --
> 1.7.9.5
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH v4 16/17] xen/arm: ITS: Generate ITS node for Dom0
2015-07-13 16:32 ` Stefano Stabellini
@ 2015-07-13 17:31 ` Julien Grall
2015-07-13 17:36 ` Stefano Stabellini
0 siblings, 1 reply; 113+ messages in thread
From: Julien Grall @ 2015-07-13 17:31 UTC (permalink / raw)
To: Stefano Stabellini, vijay.kilari
Cc: Ian.Campbell, Prasun.Kapoor, Vijaya Kumar K, tim, xen-devel,
stefano.stabellini, manish.jaggi
Hi Stefano,
On 13/07/2015 18:32, Stefano Stabellini wrote:
>
> All these #ifdefs are a bit ugly. Couldn't we just build this code
> always, even on arm32, relying on dt_match_node not to match in that
> case?
I'm not in favor of any ITS code in the common code unless it's strictly
necessary.
In this specific case, the number of ITS between DOM0 and the hardware
may be different. Per the design document, we only support 1 virtual ITS.
Furthermore, on the version 3 of this patch [1], I said that it's
possible to do it in the hwdom_dt_callback and avoid all this mess in a
function which only browse the device tree.
Regards,
[1] http://lists.xen.org/archives/html/xen-devel/2015-07/msg01092.html
--
Julien Grall
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH v4 16/17] xen/arm: ITS: Generate ITS node for Dom0
2015-07-13 17:31 ` Julien Grall
@ 2015-07-13 17:36 ` Stefano Stabellini
0 siblings, 0 replies; 113+ messages in thread
From: Stefano Stabellini @ 2015-07-13 17:36 UTC (permalink / raw)
To: Julien Grall
Cc: Ian.Campbell, vijay.kilari, Stefano Stabellini, Prasun.Kapoor,
Vijaya Kumar K, tim, xen-devel, stefano.stabellini, manish.jaggi
On Mon, 13 Jul 2015, Julien Grall wrote:
> Hi Stefano,
>
> On 13/07/2015 18:32, Stefano Stabellini wrote:
> >
> > All these #ifdefs are a bit ugly. Couldn't we just build this code
> > always, even on arm32, relying on dt_match_node not to match in that
> > case?
>
> I'm not in favor of any ITS code in the common code unless it's strictly
> necessary.
>
> In this specific case, the number of ITS between DOM0 and the hardware may be
> different. Per the design document, we only support 1 virtual ITS.
>
> Furthermore, on the version 3 of this patch [1], I said that it's possible to
> do it in the hwdom_dt_callback and avoid all this mess in a function which
> only browse the device tree.
That would be OK by me
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH v4 17/17] xen/arm: ITS: Add pci devices in ThunderX
2015-07-10 7:42 [PATCH v4 00/17] Add ITS support vijay.kilari
` (15 preceding siblings ...)
2015-07-10 7:42 ` [PATCH v4 16/17] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
@ 2015-07-10 7:42 ` vijay.kilari
2015-07-10 15:45 ` Ian Campbell
16 siblings, 1 reply; 113+ messages in thread
From: vijay.kilari @ 2015-07-10 7:42 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
ITS initialization required for all PCI devices in
ThunderX platform are done by calling from specific
mapping function.
This patch can be reverted once XEN PCI passthrough
framework for arm64 is in available.
For now all the PCI devices are assigned to Dom0
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/platforms/thunderx.c | 77 +++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
diff --git a/xen/arch/arm/platforms/thunderx.c b/xen/arch/arm/platforms/thunderx.c
index be6f24f..6bd21d2 100644
--- a/xen/arch/arm/platforms/thunderx.c
+++ b/xen/arch/arm/platforms/thunderx.c
@@ -18,6 +18,82 @@
*/
#include <asm/platform.h>
+#include <asm/gic-its.h>
+
+struct pci_dev_list
+{
+ uint32_t seg;
+ uint32_t bus;
+ uint32_t dev;
+ uint32_t func;
+};
+
+#define NUM_DEVIDS 39
+
+static struct pci_dev_list bdf[NUM_DEVIDS] =
+{
+ {0, 0, 2, 0}, /* 1 */
+ {0, 0, 6, 0},
+ {0, 0, 7, 0},
+ {0, 0, 10, 0},
+ {0, 0, 11, 0},
+ {0, 1, 0, 0},
+ {0, 1, 0, 1},
+ {0, 1, 0, 5},
+ {0, 1, 1, 4},
+ {0, 1, 9, 0}, /* 10 */
+ {0, 1, 9, 1},
+ {0, 1, 9, 2},
+ {0, 1, 9, 3},
+ {0, 1, 9, 4},
+ {0, 1, 9, 5},
+ {0, 1, 10, 0},
+ {0, 1, 10, 1},
+ {0, 1, 10, 2},
+ {0, 1, 10, 3},
+ {0, 1, 14, 0}, /* 20 */
+ {0, 1, 14, 2},
+ {0, 1, 14, 4},
+ {0, 1, 16, 0},
+ {0, 1, 16, 1},
+ {0, 2, 0, 0},
+ {0, 3, 0, 0},
+ {0, 4, 0, 0},
+ {1, 0, 8, 0},
+ {1, 0, 9, 0},
+ {1, 0, 10, 0}, /* 30 */
+ {1, 0, 11, 0},
+ {2, 0, 1, 0},
+ {2, 0, 2, 0},
+ {2, 0, 3, 0},
+ {2, 1, 0, 0},
+ {2, 1, 0, 1},
+ {2, 1, 0, 2},
+ {2, 1, 0, 3},
+ {3, 0, 1, 0}, /* 39 */
+};
+
+#define BDF_TO_DEVID(seg, bus, dev, func) (seg << 16 | bus << 8 | dev << 3| func)
+
+/* TODO: add and assign devices using PCI framework */
+static int thunderx_specific_mapping(struct domain *d)
+{
+ uint32_t devid, i;
+ int res;
+
+ for ( i = 0; i < NUM_DEVIDS; i++ )
+ {
+ devid = BDF_TO_DEVID(bdf[i].seg, bdf[i].bus,bdf[i].dev, bdf[i].func);
+ res = its_add_device(devid);
+ if ( res )
+ return res;
+ res = its_assign_device(d, devid, devid);
+ if ( res )
+ return res;
+ }
+
+ return 0;
+}
static const char * const thunderx_dt_compat[] __initconst =
{
@@ -27,6 +103,7 @@ static const char * const thunderx_dt_compat[] __initconst =
PLATFORM_START(thunderx, "THUNDERX")
.compatible = thunderx_dt_compat,
+ .specific_mapping = thunderx_specific_mapping,
.dom0_gnttab_start = 0x40000000000,
.dom0_gnttab_size = 0x20000,
PLATFORM_END
--
1.7.9.5
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH v4 17/17] xen/arm: ITS: Add pci devices in ThunderX
2015-07-10 7:42 ` [PATCH v4 17/17] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
@ 2015-07-10 15:45 ` Ian Campbell
0 siblings, 0 replies; 113+ messages in thread
From: Ian Campbell @ 2015-07-10 15:45 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Fri, 2015-07-10 at 13:12 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> ITS initialization required for all PCI devices in
> ThunderX platform are done by calling from specific
> mapping function.
>
> This patch can be reverted once XEN PCI passthrough
> framework for arm64 is in available.
>
> For now all the PCI devices are assigned to Dom0
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
^ permalink raw reply [flat|nested] 113+ messages in thread